January 26, 2020

3330 words 16 mins read

Paper Group ANR 1500

Expressiveness and Learning of Hidden Quantum Markov Models. Multi-person Articulated Tracking with Spatial and Temporal Embeddings. A Pseudo-Likelihood Approach to Linear Regression with Partially Shuffled Data. Data Proxy Generation for Fast and Efficient Neural Architecture Search. The False Positive Control Lasso. An In-Depth Study on Open-Set …

Expressiveness and Learning of Hidden Quantum Markov Models


Title	Expressiveness and Learning of Hidden Quantum Markov Models
Authors	Sandesh Adhikary, Siddarth Srinivasan, Geoff Gordon, Byron Boots
Abstract	Extending classical probabilistic reasoning using the quantum mechanical view of probability has been of recent interest, particularly in the development of hidden quantum Markov models (HQMMs) to model stochastic processes. However, there has been little progress in characterizing the expressiveness of such models and learning them from data. We tackle these problems by showing that HQMMs are a special subclass of the general class of observable operator models (OOMs) that do not suffer from the \emph{negative probability problem} by design. We also provide a feasible retraction-based learning algorithm for HQMMs using constrained gradient descent on the Stiefel manifold of model parameters. We demonstrate that this approach is faster and scales to larger models than previous learning algorithms.
Tasks
Published	2019-12-02
URL	https://arxiv.org/abs/1912.02098v1
PDF	https://arxiv.org/pdf/1912.02098v1.pdf
PWC	https://paperswithcode.com/paper/expressiveness-and-learning-of-hidden-quantum
Repo
Framework

Multi-person Articulated Tracking with Spatial and Temporal Embeddings


Title	Multi-person Articulated Tracking with Spatial and Temporal Embeddings
Authors	Sheng Jin, Wentao Liu, Wanli Ouyang, Chen Qian
Abstract	We propose a unified framework for multi-person pose estimation and tracking. Our framework consists of two main components,~\ie~SpatialNet and TemporalNet. The SpatialNet accomplishes body part detection and part-level data association in a single frame, while the TemporalNet groups human instances in consecutive frames into trajectories. Specifically, besides body part detection heatmaps, SpatialNet also predicts the Keypoint Embedding (KE) and Spatial Instance Embedding (SIE) for body part association. We model the grouping procedure into a differentiable Pose-Guided Grouping (PGG) module to make the whole part detection and grouping pipeline fully end-to-end trainable. TemporalNet extends spatial grouping of keypoints to temporal grouping of human instances. Given human proposals from two consecutive frames, TemporalNet exploits both appearance features encoded in Human Embedding (HE) and temporally consistent geometric features embodied in Temporal Instance Embedding (TIE) for robust tracking. Extensive experiments demonstrate the effectiveness of our proposed model. Remarkably, we demonstrate substantial improvements over the state-of-the-art pose tracking method from 65.4% to 71.8% Multi-Object Tracking Accuracy (MOTA) on the ICCV’17 PoseTrack Dataset.
Tasks	Multi-Object Tracking, Multi-Person Pose Estimation, Multi-Person Pose Estimation and Tracking, Object Tracking, Pose Estimation, Pose Tracking
Published	2019-03-21
URL	http://arxiv.org/abs/1903.09214v1
PDF	http://arxiv.org/pdf/1903.09214v1.pdf
PWC	https://paperswithcode.com/paper/multi-person-articulated-tracking-with
Repo
Framework

A Pseudo-Likelihood Approach to Linear Regression with Partially Shuffled Data


Title	A Pseudo-Likelihood Approach to Linear Regression with Partially Shuffled Data
Authors	Martin Slawski, Guoqing Diao, Emanuel Ben-David
Abstract	Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the model fit and disrupt the estimation of regression parameters. In this paper, we present a method to adjust for such mismatches under ``partial shuffling” in which a sufficiently large fraction of (predictors, response)-pairs are observed in their correct correspondence. The proposed approach is based on a pseudo-likelihood in which each term takes the form of a two-component mixture density. Expectation-Maximization schemes are proposed for optimization, which (i) scale favorably in the number of samples, and (ii) achieve excellent statistical performance relative to an oracle that has access to the correct pairings as certified by simulations and case studies. In particular, the proposed approach can tolerate considerably larger fraction of mismatches than existing approaches, and enables estimation of the noise level as well as the fraction of mismatches. Inference for the resulting estimator (standard errors, confidence intervals) can be based on established theory for composite likelihood estimation. Along the way, we also propose a statistical test for the presence of mismatches and establish its consistency under suitable conditions. \|
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01623v1
PDF	https://arxiv.org/pdf/1910.01623v1.pdf
PWC	https://paperswithcode.com/paper/a-pseudo-likelihood-approach-to-linear
Repo
Framework

Data Proxy Generation for Fast and Efficient Neural Architecture Search


Title	Data Proxy Generation for Fast and Efficient Neural Architecture Search
Authors	Minje Park
Abstract	Due to the recent advances on Neural Architecture Search (NAS), it gains popularity in designing best networks for specific tasks. Although it shows promising results on many benchmarks and competitions, NAS still suffers from its demanding computation cost for searching high dimensional architectural design space, and this problem becomes even worse when we want to use a large-scale dataset. If we can make a reliable data proxy for NAS, the efficiency of NAS approaches increase accordingly. Our basic observation for making a data proxy is that each example in a specific dataset has a different impact on NAS process and most of examples are redundant from a relative accuracy ranking perspective, which we should preserve when making a data proxy. We propose a systematic approach to measure the importance of each example from this relative accuracy ranking point of view, and make a reliable data proxy based on the statistics of training and testing examples. Our experiment shows that we can preserve the almost same relative accuracy ranking between all possible network configurations even with 10-20$\times$ smaller data proxy.
Tasks	Neural Architecture Search
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09322v1
PDF	https://arxiv.org/pdf/1911.09322v1.pdf
PWC	https://paperswithcode.com/paper/data-proxy-generation-for-fast-and-efficient
Repo
Framework

The False Positive Control Lasso


Title	The False Positive Control Lasso
Authors	Erik Drysdale, Yingwei Peng, Timothy P. Hanna, Paul Nguyen, Anna Goldenberg
Abstract	In high dimensional settings where a small number of regressors are expected to be important, the Lasso estimator can be used to obtain a sparse solution vector with the expectation that most of the non-zero coefficients are associated with true signals. While several approaches have been developed to control the inclusion of false predictors with the Lasso, these approaches are limited by relying on asymptotic theory, having to empirically estimate terms based on theoretical quantities, assuming a continuous response class with Gaussian noise and design matrices, or high computation costs. In this paper we show how: (1) an existing model (the SQRT-Lasso) can be recast as a method of controlling the number of expected false positives, (2) how a similar estimator can used for all other generalized linear model classes, and (3) this approach can be fit with existing fast Lasso optimization solvers. Our justification for false positive control using randomly weighted self-normalized sum theory is to our knowledge novel. Moreover, our estimator’s properties hold in finite samples up to some approximation error which we find in practical settings to be negligible under a strict mutual incoherence condition.
Tasks
Published	2019-03-29
URL	http://arxiv.org/abs/1903.12584v1
PDF	http://arxiv.org/pdf/1903.12584v1.pdf
PWC	https://paperswithcode.com/paper/the-false-positive-control-lasso
Repo
Framework

An In-Depth Study on Open-Set Camera Model Identification


Title	An In-Depth Study on Open-Set Camera Model Identification
Authors	Pedro Ribeiro Mendes Júnior, Luca Bondi, Paolo Bestagini, Stefano Tubaro, Anderson Rocha
Abstract	Camera model identification refers to the problem of linking a picture to the camera model used to shoot it. As this might be an enabling factor in different forensic applications to single out possible suspects (e.g., detecting the author of child abuse or terrorist propaganda material), many accurate camera model attribution methods have been developed in the literature. One of their main drawbacks, however, is the typical closed-set assumption of the problem. This means that an investigated photograph is always assigned to one camera model within a set of known ones present during investigation, i.e., training time, and the fact that the picture can come from a completely unrelated camera model during actual testing is usually ignored. Under realistic conditions, it is not possible to assume that every picture under analysis belongs to one of the available camera models. To deal with this issue, in this paper, we present the first in-depth study on the possibility of solving the camera model identification problem in open-set scenarios. Given a photograph, we aim at detecting whether it comes from one of the known camera models of interest or from an unknown one. We compare different feature extraction algorithms and classifiers specially targeting open-set recognition. We also evaluate possible open-set training protocols that can be applied along with any open-set classifier, observing that a simple of those alternatives obtains best results. Thorough testing on independent datasets shows that it is possible to leverage a recently proposed convolutional neural network as feature extractor paired with a properly trained open-set classifier aiming at solving the open-set camera model attribution problem even to small-scale image patches, improving over state-of-the-art available solutions.
Tasks	Open Set Learning
Published	2019-04-11
URL	https://arxiv.org/abs/1904.08497v2
PDF	https://arxiv.org/pdf/1904.08497v2.pdf
PWC	https://paperswithcode.com/paper/190408497
Repo
Framework


Title	Successive Refinement of Images with Deep Joint Source-Channel Coding
Authors	David Burth Kurka, Deniz Gunduz
Abstract	We introduce deep learning based communication methods for successive refinement of images over wireless channels. We present three different strategies for progressive image transmission with deep JSCC, with different complexity-performance trade-offs, all based on convolutional autoencoders. Numerical results show that deep JSCC not only provides graceful degradation with channel signal-to-noise ratio (SNR) and improved performance in low SNR and low bandwidth regimes compared to state-of-the-art digital communication techniques, but can also successfully learn a layered representation, achieving performance close to a single-layer scheme. These results suggest that natural images encoded with deep JSCC over Gaussian channels are almost successively refinable.
Tasks
Published	2019-03-15
URL	https://arxiv.org/abs/1903.06333v2
PDF	https://arxiv.org/pdf/1903.06333v2.pdf
PWC	https://paperswithcode.com/paper/successive-refinement-of-images-with-deep
Repo
Framework

Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations


Title	Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations
Authors	Debraj Basu, Deepesh Data, Can Karakus, Suhas Diggavi
Abstract	Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper, we propose \emph{Qsparse-local-SGD} algorithm, which combines aggressive sparsification with quantization and local computation along with error compensation, by keeping track of the difference between the true and compressed gradients. We propose both synchronous and asynchronous implementations of \emph{Qsparse-local-SGD}. We analyze convergence for \emph{Qsparse-local-SGD} in the \emph{distributed} setting for smooth non-convex and convex objective functions. We demonstrate that \emph{Qsparse-local-SGD} converges at the same rate as vanilla distributed SGD for many important classes of sparsifiers and quantizers. We use \emph{Qsparse-local-SGD} to train ResNet-50 on ImageNet and show that it results in significant savings over the state-of-the-art, in the number of bits transmitted to reach target accuracy.
Tasks	Distributed Optimization, Quantization
Published	2019-06-06
URL	https://arxiv.org/abs/1906.02367v2
PDF	https://arxiv.org/pdf/1906.02367v2.pdf
PWC	https://paperswithcode.com/paper/qsparse-local-sgd-distributed-sgd-with
Repo
Framework

A Deep, Information-theoretic Framework for Robust Biometric Recognition


Title	A Deep, Information-theoretic Framework for Robust Biometric Recognition
Authors	Renjie Xie, Yanzhi Chen, Yan Wo, Qiao Wang
Abstract	Deep neural networks (DNN) have been a de facto standard for nowadays biometric recognition solutions. A serious, but still overlooked problem in these DNN-based recognition systems is their vulnerability against adversarial attacks. Adversarial attacks can easily cause the output of a DNN system to greatly distort with only tiny changes in its input. Such distortions can potentially lead to an unexpected match between a valid biometric and a synthetic one constructed by a strategic attacker, raising security issue. In this work, we show how this issue can be resolved by learning robust biometric features through a deep, information-theoretic framework, which builds upon the recent deep variational information bottleneck method but is carefully adapted to biometric recognition tasks. Empirical evaluation demonstrates that our method not only offers stronger robustness against adversarial attacks but also provides better recognition performance over state-of-the-art approaches.
Tasks
Published	2019-02-23
URL	http://arxiv.org/abs/1902.08785v1
PDF	http://arxiv.org/pdf/1902.08785v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-information-theoretic-framework-for
Repo
Framework

EnsembleNet: End-to-End Optimization of Multi-headed Models


Title	EnsembleNet: End-to-End Optimization of Multi-headed Models
Authors	Hanhan Li, Joe Yue-Hei Ng, Paul Natsev
Abstract	Ensembling is a universally useful approach to boost the performance of machine learning models. However, individual models in an ensemble were traditionally trained independently in separate stages without information access about the overall ensemble. Many co-distillation approaches were proposed in order to treat model ensembling as first-class citizens. In this paper, we reveal a deeper connection between ensembling and distillation, and come up with a simpler yet more effective co-distillation architecture. On large-scale datasets including ImageNet, YouTube-8M, and Kinetics, we demonstrate a general procedure that can convert a single deep neural network to a multi-headed model that has not only a smaller size but also better performance. The model can be optimized end-to-end with our proposed co-distillation loss in a single stage without human intervention.
Tasks
Published	2019-05-24
URL	https://arxiv.org/abs/1905.09979v2
PDF	https://arxiv.org/pdf/1905.09979v2.pdf
PWC	https://paperswithcode.com/paper/ensemblenet-end-to-end-optimization-of-multi
Repo
Framework

Automatic Design of CNNs via Differentiable Neural Architecture Search for PolSAR Image Classification


Title	Automatic Design of CNNs via Differentiable Neural Architecture Search for PolSAR Image Classification
Authors	Hongwei Dong, Siyu Zhang, Bin Zou, Lamei Zhang
Abstract	Convolutional neural networks (CNNs) have shown good performance in polarimetric synthetic aperture radar (PolSAR) image classification due to the automation of feature engineering. Excellent hand-crafted architectures of CNNs incorporated the wisdom of human experts, which is an important reason for CNN’s success. However, the design of the architectures is a difficult problem, which needs a lot of professional knowledge as well as computational resources. Moreover, the architecture designed by hand might be suboptimal, because it is only one of thousands of unobserved but objective existed paths. Considering that the success of deep learning is largely due to its automation of the feature engineering process, how to design automatic architecture searching methods to replace the hand-crafted ones is an interesting topic. In this paper, we explore the application of neural architecture search (NAS) in PolSAR area for the first time. Different from the utilization of existing NAS methods, we propose a differentiable architecture search (DAS) method which is customized for PolSAR classification. The proposed DAS is equipped with a PolSAR tailored search space and an improved one-shot search strategy. By DAS, the weights parameters and architecture parameters (corresponds to the hyperparameters but not the topologies) can be optimized by stochastic gradient descent method during the training. The optimized architecture parameters should be transformed into corresponding CNN architecture and re-train to achieve high-precision PolSAR classification. In addition, complex-valued DAS is developed to take into account the characteristics of PolSAR images so as to further improve the performance. Experiments on three PolSAR benchmark datasets show that the CNNs obtained by searching have better classification performance than the hand-crafted ones.
Tasks	Feature Engineering, Image Classification, Neural Architecture Search
Published	2019-11-16
URL	https://arxiv.org/abs/1911.06993v2
PDF	https://arxiv.org/pdf/1911.06993v2.pdf
PWC	https://paperswithcode.com/paper/optimized-cnn-for-polsar-image-classification
Repo
Framework

RSM-GAN: A Convolutional Recurrent GAN for Anomaly Detection in Contaminated Seasonal Multivariate Time Series


Title	RSM-GAN: A Convolutional Recurrent GAN for Anomaly Detection in Contaminated Seasonal Multivariate Time Series
Authors	Farzaneh Khoshnevisan, Zhewen Fan
Abstract	Robust anomaly detection is a requirement for monitoring complex modern systems with applications such as cyber-security, fraud prevention, and maintenance. These systems generate multiple correlated time series that are highly seasonal and noisy. This paper presents a novel unsupervised deep learning architecture for multivariate time series anomaly detection, called Robust Seasonal Multivariate Generative Adversarial Network (RSM-GAN). It extends recent advancements in GANs with adoption of convolutional-LSTM layers and an attention mechanism to produce state-of-the-art performance. We conduct extensive experiments to demonstrate the strength of our architecture in adjusting for complex seasonality patterns and handling severe levels of training data contamination. We also propose a novel anomaly score assignment and causal inference framework. We compare RSM-GAN with existing classical and deep-learning based anomaly detection models, and the results show that our architecture is associated with the lowest false positive rate and improves precision by 30% and 16% in real-world and synthetic data, respectively. Furthermore, we report the superiority of RSM-GAN regarding accurate root cause identification and NAB scores in all data settings.
Tasks	Anomaly Detection, Causal Inference, Time Series
Published	2019-11-16
URL	https://arxiv.org/abs/1911.07104v1
PDF	https://arxiv.org/pdf/1911.07104v1.pdf
PWC	https://paperswithcode.com/paper/rsm-gan-a-convolutional-recurrent-gan-for
Repo
Framework

Enabling Efficient Privacy-Assured Outlier Detection over Encrypted Incremental Datasets


Title	Enabling Efficient Privacy-Assured Outlier Detection over Encrypted Incremental Datasets
Authors	Shangqi Lai, Xingliang Yuan, Amin Sakzad, Mahsa Salehi, Joseph K. Liu, Dongxi Liu
Abstract	Outlier detection is widely used in practice to track the anomaly on incremental datasets such as network traffic and system logs. However, these datasets often involve sensitive information, and sharing the data to third parties for anomaly detection raises privacy concerns. In this paper, we present a privacy-preserving outlier detection protocol (PPOD) for incremental datasets. The protocol decomposes the outlier detection algorithm into several phases and recognises the necessary cryptographic operations in each phase. It realises several cryptographic modules via efficient and interchangeable protocols to support the above cryptographic operations and composes them in the overall protocol to enable outlier detection over encrypted datasets. To support efficient updates, it integrates the sliding window model to periodically evict the expired data in order to maintain a constant update time. We build a prototype of PPOD and systematically evaluates the cryptographic modules and the overall protocols under various parameter settings. Our results show that PPOD can handle encrypted incremental datasets with a moderate computation and communication cost.
Tasks	Anomaly Detection, Outlier Detection
Published	2019-11-14
URL	https://arxiv.org/abs/1911.05927v1
PDF	https://arxiv.org/pdf/1911.05927v1.pdf
PWC	https://paperswithcode.com/paper/enabling-efficient-privacy-assured-outlier
Repo
Framework

Prediction of GNSS Phase Scintillations: A Machine Learning Approach


Title	Prediction of GNSS Phase Scintillations: A Machine Learning Approach
Authors	Kara Lamb, Garima Malhotra, Athanasios Vlontzos, Edward Wagstaff, Atılım Günes Baydin, Anahita Bhiwandiwalla, Yarin Gal, Alfredo Kalaitzis, Anthony Reina, Asti Bhatt
Abstract	A Global Navigation Satellite System (GNSS) uses a constellation of satellites around the earth for accurate navigation, timing, and positioning. Natural phenomena like space weather introduce irregularities in the Earth’s ionosphere, disrupting the propagation of the radio signals that GNSS relies upon. Such disruptions affect both the amplitude and the phase of the propagated waves. No physics-based model currently exists to predict the time and location of these disruptions with sufficient accuracy and at relevant scales. In this paper, we focus on predicting the phase fluctuations of GNSS radio waves, known as phase scintillations. We propose a novel architecture and loss function to predict 1 hour in advance the magnitude of phase scintillations within a time window of plus-minus 5 minutes with state-of-the-art performance.
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01570v1
PDF	https://arxiv.org/pdf/1910.01570v1.pdf
PWC	https://paperswithcode.com/paper/prediction-of-gnss-phase-scintillations-a
Repo
Framework

Real-time ultra-low power ECG anomaly detection using an event-driven neuromorphic processor


Title	Real-time ultra-low power ECG anomaly detection using an event-driven neuromorphic processor
Authors	Felix Christian Bauer, Dylan Richard Muir, Giacomo Indiveri
Abstract	Accurate detection of pathological conditions in human subjects can be achieved through off-line analysis of recorded biological signals such as electrocardiograms (ECGs). However, human diagnosis is time-consuming and expensive, as it requires the time of medical professionals. This is especially inefficient when indicative patterns in the biological signals are infrequent. Moreover, patients with suspected pathologies are often monitored for extended periods, requiring the storage and examination of large amounts of non-pathological data, and entailing a difficult visual search task for diagnosing professionals. In this work we propose a compact and sub-mW low power neural processing system that can be used to perform on-line and real-time preliminary diagnosis of pathological conditions, to raise warnings for the existence of possible pathological conditions, or to trigger an off-line data recording system for further analysis by a medical professional. We apply the system to real-time classification of ECG data for distinguishing between healthy heartbeats and pathological rhythms. Multi-channel analog ECG traces are encoded as asynchronous streams of binary events and processed using a spiking recurrent neural network operated in a reservoir computing paradigm. An event-driven neuron output layer is then trained to recognize one of several pathologies. Finally, the filtered activity of this output layer is used to generate a binary trigger signal indicating the presence or absence of a pathological pattern. We validate the approach proposed using a Dynamic Neuromorphic Asynchronous Processor (DYNAP) chip, implemented using a standard 180 nm CMOS VLSI process, and present experimental results measured from the chip.
Tasks	Anomaly Detection
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05521v1
PDF	https://arxiv.org/pdf/1911.05521v1.pdf
PWC	https://paperswithcode.com/paper/real-time-ultra-low-power-ecg-anomaly
Repo
Framework