January 27, 2020

3287 words 16 mins read

Paper Group ANR 1076

Functional Correlations in the Pursuit of Performance Assessment of Classifiers. New nonasymptotic convergence rates of stochastic proximal pointalgorithm for convex optimization problems with many constraints. Robby is Not a Robber (anymore): On the Use of Institutions for Learning Normative Behavior. Using theoretical ROC curves for analysing mac …

Functional Correlations in the Pursuit of Performance Assessment of Classifiers


Title	Functional Correlations in the Pursuit of Performance Assessment of Classifiers
Authors	Nadezhda Gribkova, Ričardas Zitikis
Abstract	In statistical classification and machine learning, as well as in social and other sciences, a number of measures of association have been proposed for assessing and comparing individual classifiers, raters, as well as their groups. In this paper, we introduce, justify, and explore several new measures of association, which we call CO-, ANTI- and COANTI-correlation coefficients, that we demonstrate to be powerful tools for classifying confusion matrices. We illustrate the performance of these new coefficients using a number of examples, from which we also conclude that the coefficients are new objects in the sense that they differ from those already in the literature.
Tasks
Published	2019-05-12
URL	https://arxiv.org/abs/1905.04667v3
PDF	https://arxiv.org/pdf/1905.04667v3.pdf
PWC	https://paperswithcode.com/paper/functional-correlations-in-the-pursuit-of
Repo
Framework

New nonasymptotic convergence rates of stochastic proximal pointalgorithm for convex optimization problems with many constraints


Title	New nonasymptotic convergence rates of stochastic proximal pointalgorithm for convex optimization problems with many constraints
Authors	Andrei Patrascu
Abstract	\noindent Significant parts of the recent stochastic optimization literature focused on analyzing the theoretical and practical behaviour of stochastic first order schemes under various convexity properties. Due to its simplicity, the traditional method of choice for most supervised machine learning problems is the stochastic gradient descent (SGD) method, which is known to have a relatively slow convergence. Many iteration improvements and accelerations have been added to the pure SGD in order to boost its convergence under different (strong) convexity conditions when constraints are present. However, full projections on complicated feasible set, smoothness or strong convexity assumptions are an essential requirement for these improved stochastic first-order schemes. In this paper novel convergence results are presented for the stochastic proximal point (SPP) algorithm for (non-)strongly convex optimization with many constraints. We show that a prox-quadratic growth assumption is sufficient to guarantee for SPP $\mathcal{O}\left(\frac{1}{k}\right)$ convergence rate, in terms of the distance to the optimal set, using only projections onto a simple component set. Furthermore, linear convergence is obtained for interpolation setting, when the optimal set of the expected cost is included into the optimal sets of each functional component.
Tasks	Stochastic Optimization
Published	2019-01-22
URL	https://arxiv.org/abs/1901.08663v3
PDF	https://arxiv.org/pdf/1901.08663v3.pdf
PWC	https://paperswithcode.com/paper/on-the-convergence-rate-of-stochastic
Repo
Framework

Robby is Not a Robber (anymore): On the Use of Institutions for Learning Normative Behavior


Title	Robby is Not a Robber (anymore): On the Use of Institutions for Learning Normative Behavior
Authors	Stevan Tomic, Federico Pecora, Alessandro Saffiotti
Abstract	Future robots should follow human social norms in order to be useful and accepted in human society. In this paper, we leverage already existing social knowledge in human societies by capturing it in our framework through the notion of social norms. We show how norms can be used to guide a reinforcement learning agent towards achieving normative behavior and apply the same set of norms over different domains. Thus, we are able to: (1) provide a way to intuitively encode social knowledge (through norms); (2) guide learning towards normative behaviors (through an automatic norm reward system); and (3) achieve a transfer of learning by abstracting policies; Finally, (4) the method is not dependent on a particular RL algorithm. We show how our approach can be seen as a means to achieve abstract representation and learn procedural knowledge based on the declarative semantics of norms and discuss possible implications of this in some areas of cognitive science.
Tasks
Published	2019-08-01
URL	https://arxiv.org/abs/1908.02138v1
PDF	https://arxiv.org/pdf/1908.02138v1.pdf
PWC	https://paperswithcode.com/paper/robby-is-not-a-robber-anymore-on-the-use-of
Repo
Framework

Using theoretical ROC curves for analysing machine learning binary classifiers


Title	Using theoretical ROC curves for analysing machine learning binary classifiers
Authors	Luma Omar, Ioannis Ivrissimtzis
Abstract	Most binary classifiers work by processing the input to produce a scalar response and comparing it to a threshold value. The various measures of classifier performance assume, explicitly or implicitly, probability distributions $P_s$ and $P_n$ of the response belonging to either class, probability distributions for the cost of each type of misclassification, and compute a performance score from the expected cost. In machine learning, classifier responses are obtained experimentally and performance scores are computed directly from them, without any assumptions on $P_s$ and $P_n$. Here, we argue that the omitted step of estimating theoretical distributions for $P_s$ and $P_n$ can be useful. In a biometric security example, we fit beta distributions to the responses of two classifiers, one based on logistic regression and one on ANNs, and use them to establish a categorisation into a small number of classes with different extremal behaviours at the ends of the ROC curves.
Tasks
Published	2019-09-21
URL	https://arxiv.org/abs/1909.09816v1
PDF	https://arxiv.org/pdf/1909.09816v1.pdf
PWC	https://paperswithcode.com/paper/190909816
Repo
Framework

Breast Ultrasound Computer-Aided Diagnosis Using Structure-Aware Triplet Path Networks


Title	Breast Ultrasound Computer-Aided Diagnosis Using Structure-Aware Triplet Path Networks
Authors	Erlei Zhang, Zi Yang, Stephen Seiler, Mingli Chen, Weiguo Lu, Xuejun Gu
Abstract	Breast ultrasound (US) is an effective imaging modality for breast cancer detec-tion and diagnosis. The structural characteristics of breast lesion play an im-portant role in Computer-Aided Diagnosis (CAD). In this paper, a novel struc-ture-aware triplet path networks (SATPN) was designed to integrate classifica-tion and two image reconstruction tasks to achieve accurate diagnosis on US im-ages with small training dataset. Specifically, we enhance clinically-approved breast lesion structure characteristics though converting original breast US imag-es to BIRADS-oriented feature maps (BFMs) with a distance-transformation coupled Gaussian filter. Then, the converted BFMs were used as the inputs of SATPN, which performed lesion classification task and two unsupervised stacked convolutional Auto-Encoder (SCAE) networks for benign and malignant image reconstruction tasks, independently. We trained the SATPN with an alter-native learning strategy by balancing image reconstruction error and classification label prediction error. At the test stage, the lesion label was determined by the weighted voting with reconstruction error and label prediction error. We com-pared the performance of the SATPN with TPN using original image as input and our previous developed semi-supervised deep learning methods using BFMs as inputs. Experimental results on two breast US datasets showed that SATPN ranked the best among the three networks, with classification accuracy around 93.5%. These findings indicated that SATPN is promising for effective breast US lesion CAD using small datasets.
Tasks	Image Reconstruction
Published	2019-08-09
URL	https://arxiv.org/abs/1908.09825v1
PDF	https://arxiv.org/pdf/1908.09825v1.pdf
PWC	https://paperswithcode.com/paper/breast-ultrasound-computer-aided-diagnosis
Repo
Framework

Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency


Title	Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency
Authors	Jia Li, Kui Fu, Shengwei Zhao, Shiming Ge
Abstract	The performance of video saliency estimation techniques has achieved significant advances along with the rapid development of Convolutional Neural Networks (CNNs). However, devices like cameras and drones may have limited computational capability and storage space so that the direct deployment of complex deep saliency models becomes infeasible. To address this problem, this paper proposes a dynamic saliency estimation approach for aerial videos via spatiotemporal knowledge distillation. In this approach, five components are involved, including two teachers, two students and the desired spatiotemporal model. The knowledge of spatial and temporal saliency is first separately transferred from the two complex and redundant teachers to their simple and compact students, and the input scenes are also degraded from high-resolution to low-resolution to remove the probable data redundancy so as to greatly speed up the feature extraction process. After that, the desired spatiotemporal model is further trained by distilling and encoding the spatial and temporal saliency knowledge of two students into a unified network. In this manner, the inter-model redundancy can be further removed for the effective estimation of dynamic saliency on aerial videos. Experimental results show that the proposed approach outperforms ten state-of-the-art models in estimating visual saliency on aerial videos, while its speed reaches up to 28,738 FPS on the GPU platform.
Tasks	Saliency Prediction
Published	2019-04-10
URL	http://arxiv.org/abs/1904.04992v1
PDF	http://arxiv.org/pdf/1904.04992v1.pdf
PWC	https://paperswithcode.com/paper/spatiotemporal-knowledge-distillation-for
Repo
Framework

Smell Pittsburgh: Engaging Community Citizen Science for Air Quality


Title	Smell Pittsburgh: Engaging Community Citizen Science for Air Quality
Authors	Yen-Chia Hsu, Jennifer Cross, Paul Dille, Michael Tasota, Beatrice Dias, Randy Sargent, Ting-Hao ‘Kenneth’ Huang, Illah Nourbakhsh
Abstract	Urban air pollution has been linked to various human health concerns, including cardiopulmonary diseases. Communities who suffer from poor air quality often rely on experts to identify pollution sources due to the lack of accessible tools. Taking this into account, we developed Smell Pittsburgh, a system that enables community members to report odors and track where these odors are frequently concentrated. All smell report data are publicly accessible online. These reports are also sent to the local health department and visualized on a map along with air quality data from monitoring stations. This visualization provides a comprehensive overview of the local pollution landscape. Additionally, with these reports and air quality data, we developed a model to predict upcoming smell events and send push notifications to inform communities. We also applied regression analysis to identify statistically significant effects of push notifications on user engagement. Our evaluation of this system demonstrates that engaging residents in documenting their experiences with pollution odors can help identify local air pollution patterns, and can empower communities to advocate for better air quality. All citizen-contributed smell data are publicly accessible and can be downloaded from https://smellpgh.org.
Tasks
Published	2019-12-26
URL	https://arxiv.org/abs/1912.11936v2
PDF	https://arxiv.org/pdf/1912.11936v2.pdf
PWC	https://paperswithcode.com/paper/smell-pittsburgh-engaging-community-citizen
Repo
Framework

Fast classification rates without standard margin assumptions


Title	Fast classification rates without standard margin assumptions
Authors	Olivier Bousquet, Nikita Zhivotovskiy
Abstract	We consider the classical problem of learning rates for classes with finite VC dimension. It is well known that fast learning rates are achievable by the empirical risk minimization algorithm (ERM) if one of the low noise/margin assumptions such as Tsybakov’s and Massart’s condition is satisfied. In this paper, we consider an alternative way of obtaining fast learning rates in classification if none of these conditions are met. We first consider Chow’s reject option model and show that by lowering the impact of a small fraction of hard instances, fast learning rate is achievable in an agnostic model by a specific learning algorithm. Similar results were only known under special versions of margin assumptions. We also show that the learning algorithm achieving these rates is adaptive to standard margin assumptions and always satisfies the risk bounds achieved by ERM. Based on our results on Chow’s model, we then analyze a particular family of VC classes, namely classes with finite combinatorial diameter. Using their special structure, we show that there is an improper learning algorithm that provides fast rates of convergence even in the (poorly understood) situations where ERM is suboptimal. This provides the first setup in which an improper learning algorithm may significantly improve the learning rates for non-convex losses. Finally, we discuss some implications of our techniques to the analysis of ERM.
Tasks
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12756v1
PDF	https://arxiv.org/pdf/1910.12756v1.pdf
PWC	https://paperswithcode.com/paper/fast-classification-rates-without-standard
Repo
Framework

Dynamic Kernel Distillation for Efficient Pose Estimation in Videos


Title	Dynamic Kernel Distillation for Efficient Pose Estimation in Videos
Authors	Xuecheng Nie, Yuncheng Li, Linjie Luo, Ning Zhang, Jiashi Feng
Abstract	Existing video-based human pose estimation methods extensively apply large networks onto every frame in the video to localize body joints, which suffer high computational cost and hardly meet the low-latency requirement in realistic applications. To address this issue, we propose a novel Dynamic Kernel Distillation (DKD) model to facilitate small networks for estimating human poses in videos, thus significantly lifting the efficiency. In particular, DKD introduces a light-weight distillator to online distill pose kernels via leveraging temporal cues from the previous frame in a one-shot feed-forward manner. Then, DKD simplifies body joint localization into a matching procedure between the pose kernels and the current frame, which can be efficiently computed via simple convolution. In this way, DKD fast transfers pose knowledge from one frame to provide compact guidance for body joint localization in the following frame, which enables utilization of small networks in video-based pose estimation. To facilitate the training process, DKD exploits a temporally adversarial training strategy that introduces a temporal discriminator to help generate temporally coherent pose kernels and pose estimation results within a long range. Experiments on Penn Action and Sub-JHMDB benchmarks demonstrate outperforming efficiency of DKD, specifically, 10x flops reduction and 2x speedup over previous best model, and its state-of-the-art accuracy.
Tasks	Pose Estimation
Published	2019-08-24
URL	https://arxiv.org/abs/1908.09216v1
PDF	https://arxiv.org/pdf/1908.09216v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-kernel-distillation-for-efficient
Repo
Framework

A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise


Title	A Universal Algorithm for Variational Inequalities Adaptive to Smoothness and Noise
Authors	Francis Bach, Kfir Y. Levy
Abstract	We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the standard optimality criteria in the aforementioned subcases. We present a universal algorithm for these inequalities based on the Mirror-Prox algorithm. Concretely, our algorithm simultaneously achieves the optimal rates for the smooth/non-smooth, and noisy/noiseless settings. This is done without any prior knowledge of these properties, and in the general set-up of arbitrary norms and compatible Bregman divergences. For convex minimization and convex-concave saddle-point problems, this leads to new adaptive algorithms. Our method relies on a novel yet simple adaptive choice of the step-size, which can be seen as the appropriate extension of AdaGrad to handle constrained problems.
Tasks
Published	2019-02-05
URL	http://arxiv.org/abs/1902.01637v1
PDF	http://arxiv.org/pdf/1902.01637v1.pdf
PWC	https://paperswithcode.com/paper/a-universal-algorithm-for-variational
Repo
Framework

Evolutionary Optimisation of Real-Time Systems and Networks


Title	Evolutionary Optimisation of Real-Time Systems and Networks
Authors	Leandro Soares Indrusiak, Robert I. Davis, Piotr Dziurzanski
Abstract	The design space of networked embedded systems is very large, posing challenges to the optimisation of such platforms when it comes to support applications with real-time guarantees. Recent research has shown that a number of inter-related optimisation problems have a critical influence over the schedulability of a system, i.e. whether all its application components can execute and communicate by their respective deadlines. Examples of such optimization problems include task allocation and scheduling, communication routing and arbitration, memory allocation, and voltage and frequency scaling. In this paper, we advocate the use of evolutionary approaches to address such optimization problems, aiming to evolve individuals of increased fitness over multiple generations of potential solutions. We refer to plentiful evidence that existing real-time schedulability tests can be used effectively to guide evolutionary optimisation, either by themselves or in combination with other metrics such as energy dissipation or hardware overheads. We then push that concept one step further and consider the possibility of using evolutionary techniques to evolve the schedulability tests themselves, aiming to support the verification and optimisation of systems which are too complex for state-of-the-art (manual) derivation of schedulability tests.
Tasks
Published	2019-05-06
URL	https://arxiv.org/abs/1905.01888v2
PDF	https://arxiv.org/pdf/1905.01888v2.pdf
PWC	https://paperswithcode.com/paper/evolutionary-optimisation-of-real-time
Repo
Framework


Title	Multi-Modal Attention-based Fusion Model for Semantic Segmentation of RGB-Depth Images
Authors	Fahimeh Fooladgar, Shohreh Kasaei
Abstract	The 3D scene understanding is mainly considered as a crucial requirement in computer vision and robotics applications. One of the high-level tasks in 3D scene understanding is semantic segmentation of RGB-Depth images. With the availability of RGB-D cameras, it is desired to improve the accuracy of the scene understanding process by exploiting the depth features along with the appearance features. As depth images are independent of illumination, they can improve the quality of semantic labeling alongside RGB images. Consideration of both common and specific features of these two modalities improves the performance of semantic segmentation. One of the main problems in RGB-Depth semantic segmentation is how to fuse or combine these two modalities to achieve more advantages of each modality while being computationally efficient. Recently, the methods that encounter deep convolutional neural networks have reached the state-of-the-art results by early, late, and middle fusion strategies. In this paper, an efficient encoder-decoder model with the attention-based fusion block is proposed to integrate mutual influences between feature maps of these two modalities. This block explicitly extracts the interdependences among concatenated feature maps of these modalities to exploit more powerful feature maps from RGB-Depth images. The extensive experimental results on three main challenging datasets of NYU-V2, SUN RGB-D, and Stanford 2D-3D-Semantic show that the proposed network outperforms the state-of-the-art models with respect to computational cost as well as model size. Experimental results also illustrate the effectiveness of the proposed lightweight attention-based fusion model in terms of accuracy.
Tasks	Scene Understanding, Semantic Segmentation
Published	2019-12-25
URL	https://arxiv.org/abs/1912.11691v1
PDF	https://arxiv.org/pdf/1912.11691v1.pdf
PWC	https://paperswithcode.com/paper/multi-modal-attention-based-fusion-model-for
Repo
Framework

TextSLAM: Visual SLAM with Planar Text Features


Title	TextSLAM: Visual SLAM with Planar Text Features
Authors	Boying Li, Danping Zou, Daniele Sartori, Ling Pei, Wenxian Yu
Abstract	We propose to integrate text objects in man-made scenes tightly into the visual SLAM pipeline. The key idea of our novel text-based visual SLAM is to treat each detected text as a planar feature which is rich of textures and semantic meanings. The text feature is compactly represented by three parameters and integrated into visual SLAM by adopting the illumination-invariant photometric error. We also describe important details involved in implementing a full pipeline of text-based visual SLAM. To our best knowledge, this is the first visual SLAM method tightly coupled with the text features. We tested our method in both indoor and outdoor environments. The results show that with text features, the visual SLAM system becomes more robust and produces much more accurate 3D text maps that could be useful for navigation and scene understanding in robotic or augmented reality applications.
Tasks	Scene Understanding
Published	2019-11-26
URL	https://arxiv.org/abs/1912.05002v1
PDF	https://arxiv.org/pdf/1912.05002v1.pdf
PWC	https://paperswithcode.com/paper/textslam-visual-slam-with-planar-text
Repo
Framework

Theory of Spectral Method for Union of Subspaces-Based Random Geometry Graph


Title	Theory of Spectral Method for Union of Subspaces-Based Random Geometry Graph
Authors	Gen Li, Yuantao Gu
Abstract	Spectral Method is a commonly used scheme to cluster data points lying close to Union of Subspaces by first constructing a Random Geometry Graph, called Subspace Clustering. This paper establishes a theory to analyze this method. Based on this theory, we demonstrate the efficiency of Subspace Clustering in fairly broad conditions. The insights and analysis techniques developed in this paper might also have implications for other random graph problems. Numerical experiments demonstrate the effectiveness of our theoretical study.
Tasks
Published	2019-07-25
URL	https://arxiv.org/abs/1907.10906v1
PDF	https://arxiv.org/pdf/1907.10906v1.pdf
PWC	https://paperswithcode.com/paper/theory-of-spectral-method-for-union-of
Repo
Framework

Adversarial Speaker Adaptation


Title	Adversarial Speaker Adaptation
Authors	Zhong Meng, Jinyu Li, Yifan Gong
Abstract	We propose a novel adversarial speaker adaptation (ASA) scheme, in which adversarial learning is applied to regularize the distribution of deep hidden features in a speaker-dependent (SD) deep neural network (DNN) acoustic model to be close to that of a fixed speaker-independent (SI) DNN acoustic model during adaptation. An additional discriminator network is introduced to distinguish the deep features generated by the SD model from those produced by the SI model. In ASA, with a fixed SI model as the reference, an SD model is jointly optimized with the discriminator network to minimize the senone classification loss, and simultaneously to mini-maximize the SI/SD discrimination loss on the adaptation data. With ASA, a senone-discriminative deep feature is learned in the SD model with a similar distribution to that of the SI model. With such a regularized and adapted deep feature, the SD model can perform improved automatic speech recognition on the target speaker’s speech. Evaluated on the Microsoft short message dictation dataset, ASA achieves 14.4% and 7.9% relative word error rate improvements for supervised and unsupervised adaptation, respectively, over an SI model trained from 2600 hours data, with 200 adaptation utterances per speaker.
Tasks	Speech Recognition
Published	2019-04-29
URL	http://arxiv.org/abs/1904.12407v1
PDF	http://arxiv.org/pdf/1904.12407v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-speaker-adaptation
Repo
Framework