October 16, 2019

3115 words 15 mins read

Paper Group ANR 1052

A Correlation Maximization Approach for Cross Domain Co-Embeddings. Anaconda: A Non-Adaptive Conditional Sampling Algorithm for Distribution Testing. Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates. Ignition: An End-to-End Supervised Model for Training Simulated Self-Driving Vehicles. Data-driven cortical clust …

A Correlation Maximization Approach for Cross Domain Co-Embeddings


Title	A Correlation Maximization Approach for Cross Domain Co-Embeddings
Authors	Dan Shiebler
Abstract	Although modern recommendation systems can exploit the structure in users’ item feedback, most are powerless in the face of new users who provide no structure for them to exploit. In this paper we introduce ImplicitCE, an algorithm for recommending items to new users during their sign-up flow. ImplicitCE works by transforming users’ implicit feedback towards auxiliary domain items into an embedding in the target domain item embedding space. ImplicitCE learns these embedding spaces and transformation function in an end-to-end fashion and can co-embed users and items with any differentiable similarity function. To train ImplicitCE we explore methods for maximizing the correlations between model predictions and users’ affinities and introduce Sample Correlation Update, a novel and extremely simple training strategy. Finally, we show that ImplicitCE trained with Sample Correlation Update outperforms a variety of state of the art algorithms and loss functions on both a large scale Twitter dataset and the DBLP dataset.
Tasks	Recommendation Systems
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03497v1
PDF	http://arxiv.org/pdf/1809.03497v1.pdf
PWC	https://paperswithcode.com/paper/a-correlation-maximization-approach-for-cross
Repo
Framework

Anaconda: A Non-Adaptive Conditional Sampling Algorithm for Distribution Testing


Title	Anaconda: A Non-Adaptive Conditional Sampling Algorithm for Distribution Testing
Authors	Gautam Kamath, Christos Tzamos
Abstract	We investigate distribution testing with access to non-adaptive conditional samples. In the conditional sampling model, the algorithm is given the following access to a distribution: it submits a query set $S$ to an oracle, which returns a sample from the distribution conditioned on being from $S$. In the non-adaptive setting, all query sets must be specified in advance of viewing the outcomes. Our main result is the first polylogarithmic-query algorithm for equivalence testing, deciding whether two unknown distributions are equal to or far from each other. This is an exponential improvement over the previous best upper bound, and demonstrates that the complexity of the problem in this model is intermediate to the the complexity of the problem in the standard sampling model and the adaptive conditional sampling model. We also significantly improve the sample complexity for the easier problems of uniformity and identity testing. For the former, our algorithm requires only $\tilde O(\log n)$ queries, matching the information-theoretic lower bound up to a $O(\log \log n)$-factor. Our algorithm works by reducing the problem from $\ell_1$-testing to $\ell_\infty$-testing, which enjoys a much cheaper sample complexity. Necessitated by the limited power of the non-adaptive model, our algorithm is very simple to state. However, there are significant challenges in the analysis, due to the complex structure of how two arbitrary distributions may differ.
Tasks
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06168v2
PDF	http://arxiv.org/pdf/1807.06168v2.pdf
PWC	https://paperswithcode.com/paper/anaconda-a-non-adaptive-conditional-sampling
Repo
Framework

Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates


Title	Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates
Authors	Lucas Cassano, Kun Yuan, Ali H. Sayed
Abstract	This work develops a fully decentralized multi-agent algorithm for policy evaluation. The proposed scheme can be applied to two distinct scenarios. In the first scenario, a collection of agents have distinct datasets gathered following different behavior policies (none of which is required to explore the full state space) in different instances of the same environment and they all collaborate to evaluate a common target policy. The network approach allows for efficient exploration of the state space and allows all agents to converge to the optimal solution even in situations where neither agent can converge on its own without cooperation. The second scenario is that of multi-agent games, in which the state is global and rewards are local. In this scenario, agents collaborate to estimate the value function of a target team policy. The proposed algorithm combines off-policy learning, eligibility traces and linear function approximation. The proposed algorithm is of the variance-reduced kind and achieves linear convergence with $O(1)$ memory requirements. The linear convergence of the algorithm is established analytically, and simulations are used to illustrate the effectiveness of the method.
Tasks	Efficient Exploration
Published	2018-10-17
URL	https://arxiv.org/abs/1810.07792v5
PDF	https://arxiv.org/pdf/1810.07792v5.pdf
PWC	https://paperswithcode.com/paper/multi-agent-fully-decentralized-value
Repo
Framework

Ignition: An End-to-End Supervised Model for Training Simulated Self-Driving Vehicles


Title	Ignition: An End-to-End Supervised Model for Training Simulated Self-Driving Vehicles
Authors	Rooz Mahdavian, Richard Diehl Martinez
Abstract	We introduce Ignition: an end-to-end neural network architecture for training unconstrained self-driving vehicles in simulated environments. The model is a ResNet-18 variant, which is fed in images from the front of a simulated F1 car, and outputs optimal labels for steering, throttle, braking. Importantly, we never explicitly train the model to detect road features like the outline of a track or distance to other cars; instead, we illustrate that these latent features can be automatically encapsulated by the network.
Tasks
Published	2018-06-29
URL	http://arxiv.org/abs/1806.11349v1
PDF	http://arxiv.org/pdf/1806.11349v1.pdf
PWC	https://paperswithcode.com/paper/ignition-an-end-to-end-supervised-model-for
Repo
Framework

Data-driven cortical clustering to provide a family of plausible solutions to M/EEG inverse problem


Title	Data-driven cortical clustering to provide a family of plausible solutions to M/EEG inverse problem
Authors	Kostiantyn Maksymenko, Maureen Clerc, Théodore Papadopoulo
Abstract	The M/EEG inverse problem is ill-posed. Thus additional hypotheses are needed to constrain the solution space. In this work, we consider that brain activity which generates an M/EEG signal is a connected cortical region. We study the case when only one region is active at once. We show that even in this simple case several configurations can explain the data. As opposed to methods based on convex optimization which are forced to select one possible solution, we propose an approach which is able to find several “good” candidates - regions which are different in term of their sizes and/or positions but fit the data with similar accuracy.
Tasks	EEG
Published	2018-12-07
URL	http://arxiv.org/abs/1812.04110v1
PDF	http://arxiv.org/pdf/1812.04110v1.pdf
PWC	https://paperswithcode.com/paper/data-driven-cortical-clustering-to-provide-a
Repo
Framework

Monge blunts Bayes: Hardness Results for Adversarial Training


Title	Monge blunts Bayes: Hardness Results for Adversarial Training
Authors	Zac Cranko, Aditya Krishna Menon, Richard Nock, Cheng Soon Ong, Zhan Shi, Christian Walder
Abstract	The last few years have seen a staggering number of empirical studies of the robustness of neural networks in a model of adversarial perturbations of their inputs. Most rely on an adversary which carries out local modifications within prescribed balls. None however has so far questioned the broader picture: how to frame a resource-bounded adversary so that it can be severely detrimental to learning, a non-trivial problem which entails at a minimum the choice of loss and classifiers. We suggest a formal answer for losses that satisfy the minimal statistical requirement of being proper. We pin down a simple sufficient property for any given class of adversaries to be detrimental to learning, involving a central measure of “harmfulness” which generalizes the well-known class of integral probability metrics. A key feature of our result is that it holds for all proper losses, and for a popular subset of these, the optimisation of this central measure appears to be independent of the loss. When classifiers are Lipschitz – a now popular approach in adversarial training –, this optimisation resorts to optimal transport to make a low-budget compression of class marginals. Toy experiments reveal a finding recently separately observed: training against a sufficiently budgeted adversary of this kind improves generalization.
Tasks
Published	2018-06-08
URL	https://arxiv.org/abs/1806.02977v4
PDF	https://arxiv.org/pdf/1806.02977v4.pdf
PWC	https://paperswithcode.com/paper/monge-beats-bayes-hardness-results-for
Repo
Framework

Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis


Title	Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis
Authors	Sourabh Vora, Akshay Rangesh, Mohan M. Trivedi
Abstract	Driver gaze has been shown to be an excellent surrogate for driver attention in intelligent vehicles. With the recent surge of highly autonomous vehicles, driver gaze can be useful for determining the handoff time to a human driver. While there has been significant improvement in personalized driver gaze zone estimation systems, a generalized system which is invariant to different subjects, perspectives and scales is still lacking. We take a step towards this generalized system using Convolutional Neural Networks (CNNs). We finetune 4 popular CNN architectures for this task, and provide extensive comparisons of their outputs. We additionally experiment with different input image patches, and also examine how image size affects performance. For training and testing the networks, we collect a large naturalistic driving dataset comprising of 11 long drives, driven by 10 subjects in two different cars. Our best performing model achieves an accuracy of 95.18% during cross-subject testing, outperforming current state of the art techniques for this task. Finally, we evaluate our best performing model on the publicly available Columbia Gaze Dataset comprising of images from 56 subjects with varying head pose and gaze directions. Without any training, our model successfully encodes the different gaze directions on this diverse dataset, demonstrating good generalization capabilities.
Tasks	Autonomous Vehicles
Published	2018-02-08
URL	http://arxiv.org/abs/1802.02690v2
PDF	http://arxiv.org/pdf/1802.02690v2.pdf
PWC	https://paperswithcode.com/paper/driver-gaze-zone-estimation-using
Repo
Framework

Modeling Treatment Delays for Patients using Feature Label Pairs in a Time Series


Title	Modeling Treatment Delays for Patients using Feature Label Pairs in a Time Series
Authors	Weiyu Huang, Yunlong Wang, Li Zhou, Emily Zhao, Yilian Yuan, Alejandro Ribero
Abstract	Pharmaceutical targeting is one of key inputs for making sales and marketing strategy planning. Targeting list is built on predicting physician’s sales potential of certain type of patient. In this paper, we present a time-sensitive targeting framework leveraging time series model to predict patient’s disease and treatment progression. We create time features by extracting service history within a certain period, and record whether the event happens in a look-forward period. Such feature-label pairs are examined across all time periods and all patients to train a model. It keeps the inherent order of services and evaluates features associated to the imminent future, which contribute to improved accuracy.
Tasks	Time Series
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00554v1
PDF	http://arxiv.org/pdf/1812.00554v1.pdf
PWC	https://paperswithcode.com/paper/modeling-treatment-delays-for-patients-using
Repo
Framework

Metric-Optimized Example Weights


Title	Metric-Optimized Example Weights
Authors	Sen Zhao, Mahdi Milani Fard, Harikrishna Narasimhan, Maya Gupta
Abstract	Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard loss, where the weights on the training examples are learned to optimize the test metric on a validation set. These metric-optimized example weights can be learned for any test metric, including black box and customized ones for specific applications. We illustrate the performance of the proposed method on diverse public benchmark datasets and real-world applications. We also provide a generalization bound for the method.
Tasks
Published	2018-05-27
URL	https://arxiv.org/abs/1805.10582v3
PDF	https://arxiv.org/pdf/1805.10582v3.pdf
PWC	https://paperswithcode.com/paper/metric-optimized-example-weights
Repo
Framework

Transfer Learning and Organic Computing for Autonomous Vehicles


Title	Transfer Learning and Organic Computing for Autonomous Vehicles
Authors	Christofer Fellicious
Abstract	Autonomous Vehicles(AV) are one of the brightest promises of the future which would help cut down fatalities and improve travel time while working in harmony. Autonomous vehicles will face with challenging situations and experiences not seen before. These experiences should be converted to knowledge and help the vehicle prepare better in the future. Online Transfer Learning will help transferring prior knowledge to a new task and also keep the knowledge updated as the task evolves. This paper presents the different methods of transfer learning, online transfer learning and organic computing that could be adapted to the domain of autonomous vehicles.
Tasks	Autonomous Vehicles, Transfer Learning
Published	2018-08-16
URL	http://arxiv.org/abs/1808.05443v1
PDF	http://arxiv.org/pdf/1808.05443v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-and-organic-computing-for
Repo
Framework

Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision


Title	Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision
Authors	Varun Ravi Kumar, Stefan Milz, Martin Simon, Christian Witt, Karl Amende, Johannes Petzold, Senthil Yogamani, Timo Pech
Abstract	Near field depth estimation around a self driving car is an important function that can be achieved by four wide angle fisheye cameras having a field of view of over 180. Depth estimation based on convolutional neural networks (CNNs) produce state of the art results, but progress is hindered because depth annotation cannot be obtained manually. Synthetic datasets are commonly used but they have limitations. For instance, they do not capture the extensive variability in the appearance of objects like vehicles present in real datasets. There is also a domain shift while performing inference on natural images illustrated by many attempts to handle the domain adaptation explicitly. In this work, we explore an alternate approach of training using sparse LiDAR data as ground truth for depth estimation for fisheye camera. We built our own dataset using our self driving car setup which has a 64 beam Velodyne LiDAR and four wide angle fisheye cameras. To handle the difference in view points of LiDAR and fisheye camera, an occlusion resolution mechanism was implemented. We started with Eigen’s multiscale convolutional network architecture and improved by modifying activation function and optimizer. We obtained promising results on our dataset with RMSE errors comparable to the state of the art results obtained on KITTI.
Tasks	Depth Estimation, Domain Adaptation
Published	2018-03-16
URL	http://arxiv.org/abs/1803.06192v3
PDF	http://arxiv.org/pdf/1803.06192v3.pdf
PWC	https://paperswithcode.com/paper/monocular-fisheye-camera-depth-estimation
Repo
Framework

Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity


Title	Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity
Authors	Chulhee Yun, Suvrit Sra, Ali Jadbabaie
Abstract	We study finite sample expressivity, i.e., memorization power of ReLU networks. Recent results require $N$ hidden nodes to memorize/interpolate arbitrary $N$ data points. In contrast, by exploiting depth, we show that 3-layer ReLU networks with $\Omega(\sqrt{N})$ hidden nodes can perfectly memorize most datasets with $N$ points. We also prove that width $\Theta(\sqrt{N})$ is necessary and sufficient for memorizing $N$ data points, proving tight bounds on memorization capacity. The sufficiency result can be extended to deeper networks; we show that an $L$-layer network with $W$ parameters in the hidden layers can memorize $N$ data points if $W = \Omega(N)$. Combined with a recent upper bound $O(WL\log W)$ on VC dimension, our construction is nearly tight for any fixed $L$. Subsequently, we analyze memorization capacity of residual networks under a general position assumption; we prove results that substantially reduce the known requirement of $N$ hidden nodes. Finally, we study the dynamics of stochastic gradient descent (SGD), and show that when initialized near a memorizing global minimum of the empirical risk, SGD quickly finds a nearby point with much smaller empirical risk.
Tasks
Published	2018-10-17
URL	https://arxiv.org/abs/1810.07770v3
PDF	https://arxiv.org/pdf/1810.07770v3.pdf
PWC	https://paperswithcode.com/paper/finite-sample-expressive-power-of-small-width
Repo
Framework

Memorization in Overparameterized Autoencoders


Title	Memorization in Overparameterized Autoencoders
Authors	Adityanarayanan Radhakrishnan, Karren Yang, Mikhail Belkin, Caroline Uhler
Abstract	The ability of deep neural networks to generalize well in the overparameterized regime has become a subject of significant research interest. We show that overparameterized autoencoders exhibit memorization, a form of inductive bias that constrains the functions learned through the optimization process to concentrate around the training examples, although the network could in principle represent a much larger function class. In particular, we prove that single-layer fully-connected autoencoders project data onto the (nonlinear) span of the training examples. In addition, we show that deep fully-connected autoencoders learn a map that is locally contractive at the training examples, and hence iterating the autoencoder results in convergence to the training examples. Finally, we prove that depth is necessary and provide empirical evidence that it is also sufficient for memorization in convolutional autoencoders. Understanding this inductive bias may shed light on the generalization properties of overparametrized deep neural networks that are currently unexplained by classical statistical theory.
Tasks
Published	2018-10-16
URL	https://arxiv.org/abs/1810.10333v3
PDF	https://arxiv.org/pdf/1810.10333v3.pdf
PWC	https://paperswithcode.com/paper/memorization-in-overparameterized
Repo
Framework

Device-directed Utterance Detection


Title	Device-directed Utterance Detection
Authors	Sri Harish Mallidi, Roland Maas, Kyle Goehner, Ariya Rastrow, Spyros Matsoukas, Björn Hoffmeister
Abstract	In this work, we propose a classifier for distinguishing device-directed queries from background speech in the context of interactions with voice assistants. Applications include rejection of false wake-ups or unintended interactions as well as enabling wake-word free follow-up queries. Consider the example interaction: $"Computer,~play~music”, “Computer,~reduce~the~volume"$. In this interaction, the user needs to repeat the wake-word ($Computer$) for the second query. To allow for more natural interactions, the device could immediately re-enter listening state after the first query (without wake-word repetition) and accept or reject a potential follow-up as device-directed or background speech. The proposed model consists of two long short-term memory (LSTM) neural networks trained on acoustic features and automatic speech recognition (ASR) 1-best hypotheses, respectively. A feed-forward deep neural network (DNN) is then trained to combine the acoustic and 1-best embeddings, derived from the LSTMs, with features from the ASR decoder. Experimental results show that ASR decoder, acoustic embeddings, and 1-best embeddings yield an equal-error-rate (EER) of $9.3~%$, $10.9~%$ and $20.1~%$, respectively. Combination of the features resulted in a $44~%$ relative improvement and a final EER of $5.2~%$.
Tasks	Speech Recognition
Published	2018-08-07
URL	http://arxiv.org/abs/1808.02504v1
PDF	http://arxiv.org/pdf/1808.02504v1.pdf
PWC	https://paperswithcode.com/paper/device-directed-utterance-detection
Repo
Framework

Generative Adversarial Networks with Decoder-Encoder Output Noise


Title	Generative Adversarial Networks with Decoder-Encoder Output Noise
Authors	Guoqiang Zhong, Wei Gao, Yongbin Liu, Youzhao Yang
Abstract	In recent years, research on image generation methods has been developing fast. The auto-encoding variational Bayes method (VAEs) was proposed in 2013, which uses variational inference to learn a latent space from the image database and then generates images using the decoder. The generative adversarial networks (GANs) came out as a promising framework, which uses adversarial training to improve the generative ability of the generator. However, the generated pictures by GANs are generally blurry. The deep convolutional generative adversarial networks (DCGANs) were then proposed to leverage the quality of generated images. Since the input noise vectors are randomly sampled from a Gaussian distribution, the generator has to map from a whole normal distribution to the images. This makes DCGANs unable to reflect the inherent structure of the training data. In this paper, we propose a novel deep model, called generative adversarial networks with decoder-encoder output noise (DE-GANs), which takes advantage of both the adversarial training and the variational Bayesain inference to improve the performance of image generation. DE-GANs use a pre-trained decoder-encoder architecture to map the random Gaussian noise vectors to informative ones and pass them to the generator of the adversarial networks. Since the decoder-encoder architecture is trained by the same images as the generators, the output vectors could carry the intrinsic distribution information of the original images. Moreover, the loss function of DE-GANs is different from GANs and DCGANs. A hidden-space loss function is added to the adversarial loss function to enhance the robustness of the model. Extensive empirical results show that DE-GANs can accelerate the convergence of the adversarial training process and improve the quality of the generated images.
Tasks	Image Generation
Published	2018-07-11
URL	http://arxiv.org/abs/1807.03923v1
PDF	http://arxiv.org/pdf/1807.03923v1.pdf
PWC	https://paperswithcode.com/paper/generative-adversarial-networks-with-decoder
Repo
Framework