October 17, 2019

2889 words 14 mins read

Paper Group ANR 757

“Dependency Bottleneck” in Auto-encoding Architectures: an Empirical Study. Accuracy-Reliability Cost Function for Empirical Variance Estimation. Multivariate Bayesian Structural Time Series Model. Oversight of Unsafe Systems via Dynamic Safety Envelopes. Fine-tuning of Language Models with Discriminator. Efficient Multi-level Correlating for Visua …

“Dependency Bottleneck” in Auto-encoding Architectures: an Empirical Study


Title	“Dependency Bottleneck” in Auto-encoding Architectures: an Empirical Study
Authors	Denny Wu, Yixiu Zhao, Yao-Hung Hubert Tsai, Makoto Yamada, Ruslan Salakhutdinov
Abstract	Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the mea- surement of the mutual information (MI) is often inaccurate due to the density estimation. To address this issue, we propose to measure the dependency instead of MI between layers in DNNs. Specifically, we propose to use Hilbert-Schmidt Independence Criterion (HSIC) as the dependency measure, which can measure the dependence of two random variables without estimating probability densities. Moreover, HSIC is a special case of the Squared-loss Mutual Information (SMI). In the experiment, we empirically evaluate the generalization property using HSIC in both the reconstruction and prediction auto-encoding (AE) architectures.
Tasks	Density Estimation
Published	2018-02-15
URL	http://arxiv.org/abs/1802.05408v1
PDF	http://arxiv.org/pdf/1802.05408v1.pdf
PWC	https://paperswithcode.com/paper/dependency-bottleneck-in-auto-encoding
Repo
Framework

Accuracy-Reliability Cost Function for Empirical Variance Estimation


Title	Accuracy-Reliability Cost Function for Empirical Variance Estimation
Authors	Enrico Camporeale
Abstract	In this paper we focus on the problem of assigning uncertainties to single-point predictions. We introduce a cost function that encodes the trade-off between accuracy and reliability in probabilistic forecast. We derive analytic formula for the case of forecasts of continuous scalar variables expressed in terms of Gaussian distributions. The Accuracy-Reliability cost function can be used to empirically estimate the variance in heteroskedastic regression problems (input dependent noise), by solving a two-objective optimization problem. The simple philosophy behind this strategy is that predictions based on the estimated variances should be both accurate and reliable (i.e. statistical consistent with observations). We show several examples with synthetic data, where the underlying hidden noise function can be accurately recovered, both in one and multi-dimensional problems. The practical implementation of the method has been done using a Neural Network and, in the one-dimensional case, with a simple polynomial fit.
Tasks
Published	2018-03-12
URL	http://arxiv.org/abs/1803.04475v1
PDF	http://arxiv.org/pdf/1803.04475v1.pdf
PWC	https://paperswithcode.com/paper/accuracy-reliability-cost-function-for
Repo
Framework

Multivariate Bayesian Structural Time Series Model


Title	Multivariate Bayesian Structural Time Series Model
Authors	S. Rao Jammalamadaka, Jinwen Qiu, Ning Ning
Abstract	This paper deals with inference and prediction for multiple correlated time series, where one has also the choice of using a candidate pool of contemporaneous predictors for each target series. Starting with a structural model for the time-series, Bayesian tools are used for model fitting, prediction, and feature selection, thus extending some recent work along these lines for the univariate case. The Bayesian paradigm in this multivariate setting helps the model avoid overfitting as well as capture correlations among the multiple time series with the various state components. The model provides needed flexibility to choose a different set of components and available predictors for each target series. The cyclical component in the model can handle large variations in the short term, which may be caused by external shocks. We run extensive simulations to investigate properties such as estimation accuracy and performance in forecasting. We then run an empirical study with one-step-ahead prediction on the max log return of a portfolio of stocks that involve four leading financial institutions. Both the simulation studies and the extensive empirical study confirm that this multivariate model outperforms three other benchmark models, viz. a model that treats each target series as independent, the autoregressive integrated moving average model with regression (ARIMAX), and the multivariate ARIMAX (MARIMAX) model.
Tasks	Feature Selection, Time Series
Published	2018-01-10
URL	http://arxiv.org/abs/1801.03222v2
PDF	http://arxiv.org/pdf/1801.03222v2.pdf
PWC	https://paperswithcode.com/paper/multivariate-bayesian-structural-time-series
Repo
Framework

Oversight of Unsafe Systems via Dynamic Safety Envelopes


Title	Oversight of Unsafe Systems via Dynamic Safety Envelopes
Authors	David Manheim
Abstract	This paper reviews the reasons that Human-in-the-Loop is both critical for preventing widely-understood failure modes for machine learning, and not a practical solution. Following this, we review two current heuristic methods for addressing this. The first is provable safety envelopes, which are possible only when the dynamics of the system are fully known, but can be useful safety guarantees when optimal behavior is based on machine learning with poorly-understood safety characteristics. The second is the simpler circuit breaker model, which can forestall or prevent catastrophic outcomes by stopping the system, without any specific model of the system. This paper proposes using heuristic, dynamic safety envelopes, which are a plausible halfway point between these approaches that allows human oversight without some of the more difficult problems faced by Human-in-the-Loop systems. Finally, the paper concludes with how this approach can be used for governance of systems where otherwise unsafe systems are deployed.
Tasks
Published	2018-11-22
URL	http://arxiv.org/abs/1811.09246v1
PDF	http://arxiv.org/pdf/1811.09246v1.pdf
PWC	https://paperswithcode.com/paper/oversight-of-unsafe-systems-via-dynamic
Repo
Framework

Fine-tuning of Language Models with Discriminator


Title	Fine-tuning of Language Models with Discriminator
Authors	Vadim Popov, Mikhail Kudinov
Abstract	Cross-entropy loss is a common choice when it comes to multiclass classification tasks and language modeling in particular. Minimizing this loss results in language models of very good quality. We show that it is possible to fine-tune these models and make them perform even better if they are fine-tuned with sum of cross-entropy loss and reverse Kullback-Leibler divergence. The latter is estimated using discriminator network that we train in advance. During fine-tuning probabilities of rare words that are usually underestimated by language models become bigger. The novel approach that we propose allows us to reach state-of-the-art quality on Penn Treebank: perplexity decreases from 52.4 to 52.1. Our fine-tuning algorithm is rather fast, scales well to different architectures and datasets and requires almost no hyperparameter tuning: the only hyperparameter that needs to be tuned is learning rate.
Tasks	Language Modelling
Published	2018-11-12
URL	http://arxiv.org/abs/1811.04623v2
PDF	http://arxiv.org/pdf/1811.04623v2.pdf
PWC	https://paperswithcode.com/paper/fine-tuning-of-language-models-with
Repo
Framework

Efficient Multi-level Correlating for Visual Tracking


Title	Efficient Multi-level Correlating for Visual Tracking
Authors	Yipeng Ma, Chun Yuan, Peng Gao, Fei Wang
Abstract	Correlation filter (CF) based tracking algorithms have demonstrated favorable performance recently. Nevertheless, the top performance trackers always employ complicated optimization methods which constraint their real-time applications. How to accelerate the tracking speed while retaining the tracking accuracy is a significant issue. In this paper, we propose a multi-level CF-based tracking approach named MLCFT which further explores the potential capacity of CF with two-stage detection: primal detection and oriented re-detection. The cascaded detection scheme is simple but competent to prevent model drift and accelerate the speed. An effective fusion method based on relative entropy is introduced to combine the complementary features extracted from deep and shallow layers of convolutional neural networks (CNN). Moreover, a novel online model update strategy is utilized in our tracker, which enhances the tracking performance further. Experimental results demonstrate that our proposed approach outperforms the most state-of-the-art trackers while tracking at speed of exceeded 16 frames per second on challenging benchmarks.
Tasks	Visual Tracking
Published	2018-10-13
URL	http://arxiv.org/abs/1810.05810v1
PDF	http://arxiv.org/pdf/1810.05810v1.pdf
PWC	https://paperswithcode.com/paper/efficient-multi-level-correlating-for-visual
Repo
Framework

Learnable Histogram: Statistical Context Features for Deep Neural Networks


Title	Learnable Histogram: Statistical Context Features for Deep Neural Networks
Authors	Zhe Wang, Hongsheng Li, Wanli Ouyang, Xiaogang Wang
Abstract	Statistical features, such as histogram, Bag-of-Words (BoW) and Fisher Vector, were commonly used with hand-crafted features in conventional classification methods, but attract less attention since the popularity of deep learning methods. In this paper, we propose a learnable histogram layer, which learns histogram features within deep neural networks in end-to-end training. Such a layer is able to back-propagate (BP) errors, learn optimal bin centers and bin widths, and be jointly optimized with other layers in deep networks during training. Two vision problems, semantic segmentation and object detection, are explored by integrating the learnable histogram layer into deep networks, which show that the proposed layer could be well generalized to different applications. In-depth investigations are conducted to provide insights on the newly introduced layer.
Tasks	Object Detection, Semantic Segmentation
Published	2018-04-25
URL	http://arxiv.org/abs/1804.09398v3
PDF	http://arxiv.org/pdf/1804.09398v3.pdf
PWC	https://paperswithcode.com/paper/learnable-histogram-statistical-context
Repo
Framework

FPGA-based Acceleration System for Visual Tracking


Title	FPGA-based Acceleration System for Visual Tracking
Authors	Ke Song, Chun Yuan, Peng Gao, Yunxu Sun
Abstract	Visual tracking is one of the most important application areas of computer vision. At present, most algorithms are mainly implemented on PCs, and it is difficult to ensure real-time performance when applied in the real scenario. In order to improve the tracking speed and reduce the overall power consumption of visual tracking, this paper proposes a real-time visual tracking algorithm based on DSST(Discriminative Scale Space Tracking) approach. We implement a hardware system on Xilinx XC7K325T FPGA platform based on our proposed visual tracking algorithm. Our hardware system can run at more than 153 frames per second. In order to reduce the resource occupation, our system adopts the batch processing method in the feature extraction module. In the filter processing module, the FFT IP core is time-division multiplexed. Therefore, our hardware system utilizes LUTs and storage blocks of 33% and 40%, respectively. Test results show that the proposed visual tracking hardware system has excellent performance.
Tasks	Real-Time Visual Tracking, Visual Tracking
Published	2018-10-12
URL	http://arxiv.org/abs/1810.05367v2
PDF	http://arxiv.org/pdf/1810.05367v2.pdf
PWC	https://paperswithcode.com/paper/fpga-based-acceleration-system-for-visual
Repo
Framework

Deep Attentive Tracking via Reciprocative Learning


Title	Deep Attentive Tracking via Reciprocative Learning
Authors	Shi Pu, Yibing Song, Chao Ma, Honggang Zhang, Ming-Hsuan Yang
Abstract	Visual attention, derived from cognitive neuroscience, facilitates human perception on the most pertinent subset of the sensory data. Recently, significant efforts have been made to exploit attention schemes to advance computer vision systems. For visual tracking, it is often challenging to track target objects undergoing large appearance changes. Attention maps facilitate visual tracking by selectively paying attention to temporal robust features. Existing tracking-by-detection approaches mainly use additional attention modules to generate feature weights as the classifiers are not equipped with such mechanisms. In this paper, we propose a reciprocative learning algorithm to exploit visual attention for training deep classifiers. The proposed algorithm consists of feed-forward and backward operations to generate attention maps, which serve as regularization terms coupled with the original classification loss function for training. The deep classifier learns to attend to the regions of target objects robust to appearance changes. Extensive experiments on large-scale benchmark datasets show that the proposed attentive tracking method performs favorably against the state-of-the-art approaches.
Tasks	Visual Tracking
Published	2018-10-09
URL	http://arxiv.org/abs/1810.03851v2
PDF	http://arxiv.org/pdf/1810.03851v2.pdf
PWC	https://paperswithcode.com/paper/deep-attentive-tracking-via-reciprocative
Repo
Framework

Adversarial Feature Sampling Learning for Efficient Visual Tracking


Title	Adversarial Feature Sampling Learning for Efficient Visual Tracking
Authors	Yingjie Yin, Lei Zhang, De Xu, Xingang Wang
Abstract	The tracking-by-detection framework usually consist of two stages: drawing samples around the target object in the first stage and classifying each sample as the target object or background in the second stage. Current popular trackers based on tracking-by-detection framework typically draw samples in the raw image as the inputs of deep convolution networks in the first stage, which usually results in high computational burden and low running speed. In this paper, we propose a new visual tracking method using sampling deep convolutional features to address this problem. Only one cropped image around the target object is input into the designed deep convolution network and the samples is sampled on the feature maps of the network by spatial bilinear resampling. In addition, a generative adversarial network is integrated into our network framework to augment positive samples and improve the tracking performance. Extensive experiments on benchmark datasets demonstrate that the proposed method achieves a comparable performance to state-of-the-art trackers and accelerates tracking-by-detection trackers based on raw-image samples effectively.
Tasks	Visual Tracking
Published	2018-09-13
URL	http://arxiv.org/abs/1809.04741v2
PDF	http://arxiv.org/pdf/1809.04741v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-feature-sampling-learning-for
Repo
Framework

Approximating the Void: Learning Stochastic Channel Models from Observation with Variational Generative Adversarial Networks


Title	Approximating the Void: Learning Stochastic Channel Models from Observation with Variational Generative Adversarial Networks
Authors	Timothy J. O’Shea, Tamoghna Roy, Nathan West
Abstract	Channel modeling is a critical topic when considering designing, learning, or evaluating the performance of any communications system. Most prior work in designing or learning new modulation schemes has focused on using highly simplified analytic channel models such as additive white Gaussian noise (AWGN), Rayleigh fading channels or similar. Recently, we proposed the usage of a generative adversarial networks (GANs) to jointly approximate a wireless channel response model (e.g. from real black box measurements) and optimize for an efficient modulation scheme over it using machine learning. This approach worked to some degree, but was unable to produce accurate probability distribution functions (PDFs) representing the stochastic channel response. In this paper, we focus specifically on the problem of accurately learning a channel PDF using a variational GAN, introducing an architecture and loss function which can accurately capture stochastic behavior. We illustrate where our prior method failed and share results capturing the performance of such as system over a range of realistic channel distributions.
Tasks
Published	2018-05-16
URL	http://arxiv.org/abs/1805.06350v2
PDF	http://arxiv.org/pdf/1805.06350v2.pdf
PWC	https://paperswithcode.com/paper/approximating-the-void-learning-stochastic
Repo
Framework

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications


Title	Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications
Authors	Beatriz Blanco-Filgueira, Daniel García-Lesta, Mauro Fernández-Sanjurjo, Víctor M. Brea, Paula López
Abstract	Compute and memory demands of state-of-the-art deep learning methods are still a shortcoming that must be addressed to make them useful at IoT end-nodes. In particular, recent results depict a hopeful prospect for image processing using Convolutional Neural Netwoks, CNNs, but the gap between software and hardware implementations is already considerable for IoT and mobile edge computing applications due to their high power consumption. This proposal performs low-power and real time deep learning-based multiple object visual tracking implemented on an NVIDIA Jetson TX2 development kit. It includes a camera and wireless connection capability and it is battery powered for mobile and outdoor applications. A collection of representative sequences captured with the on-board camera, dETRUSC video dataset, is used to exemplify the performance of the proposed algorithm and to facilitate benchmarking. The results in terms of power consumption and frame rate demonstrate the feasibility of deep learning algorithms on embedded platforms although more effort to joint algorithm and hardware design of CNNs is needed.
Tasks	Visual Tracking
Published	2018-07-31
URL	http://arxiv.org/abs/1808.01356v1
PDF	http://arxiv.org/pdf/1808.01356v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-based-multiple-object-visual
Repo
Framework

An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition


Title	An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition
Authors	Devesh Walawalkar, Yihui He, Rohit Pillai
Abstract	In this project, we worked on speech recognition, specifically predicting individual words based on both the video frames and audio. Empowered by convolutional neural networks, the recent speech recognition and lip reading models are comparable to human level performance. We re-implemented and made derivations of the state-of-the-art model. Then, we conducted rich experiments including the effectiveness of attention mechanism, more accurate residual network as the backbone with pre-trained weights and the sensitivity of our model with respect to audio input with/without noise.
Tasks	Speech Recognition
Published	2018-12-21
URL	http://arxiv.org/abs/1812.09336v1
PDF	http://arxiv.org/pdf/1812.09336v1.pdf
PWC	https://paperswithcode.com/paper/an-empirical-analysis-of-deep-audio-visual
Repo
Framework

Unique Metric for Health Analysis with Optimization of Clustering Activity and Cross Comparison of Results from Different Approach


Title	Unique Metric for Health Analysis with Optimization of Clustering Activity and Cross Comparison of Results from Different Approach
Authors	Kumarjit Pathak, Jitin Kapila
Abstract	In machine learning and data mining, Cluster analysis is one of the most widely used unsupervised learning technique. Philosophy of this algorithm is to find similar data items and group them together based on any distance function in multidimensional space. These methods are suitable for finding groups of data that behave in a coherent fashion. The perspective may vary for clustering i.e. the way we want to find similarity, some methods are based on distance such as K-Means technique and some are probability based, like GMM. Understanding prominent segment of data is always challenging as multidimension space does not allow us to have a look and feel of the distance or any visual context on the health of the clustering. While explaining data using clusters, the major problem is to tell how many cluster are good enough to explain the data. Generally basic descriptive statistics are used to estimate cluster behaviour like scree plot, dendrogram etc. We propose a novel method to understand the cluster behaviour which can be used not only to find right number of clusters but can also be used to access the difference of health between different clustering methods on same data. Our technique would also help to also eliminate the noisy variables and optimize the clustering result. keywords - Clustering, Metric, K-means, hierarchical clustering, silhoutte, clustering index, measures
Tasks
Published	2018-10-08
URL	http://arxiv.org/abs/1810.03419v1
PDF	http://arxiv.org/pdf/1810.03419v1.pdf
PWC	https://paperswithcode.com/paper/unique-metric-for-health-analysis-with
Repo
Framework

Learning Causally-Generated Stationary Time Series


Title	Learning Causally-Generated Stationary Time Series
Authors	Wessel Bruinsma, Richard E. Turner
Abstract	We present the Causal Gaussian Process Convolution Model (CGPCM), a doubly nonparametric model for causal, spectrally complex dynamical phenomena. The CGPCM is a generative model in which white noise is passed through a causal, nonparametric-window moving-average filter, a construction that we show to be equivalent to a Gaussian process with a nonparametric kernel that is biased towards causally-generated signals. We develop enhanced variational inference and learning schemes for the CGPCM and its previous acausal variant, the GPCM (Tobar et al., 2015b), that significantly improve statistical accuracy. These modelling and inferential contributions are demonstrated on a range of synthetic and real-world signals.
Tasks	Time Series
Published	2018-02-22
URL	http://arxiv.org/abs/1802.08167v1
PDF	http://arxiv.org/pdf/1802.08167v1.pdf
PWC	https://paperswithcode.com/paper/learning-causally-generated-stationary-time
Repo
Framework