Paper Group ANR 730
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems. A Network Perspective on Stratification of Multi-Label Data. Infrastructure for Usable Machine Learning: The Stanford DAWN Project. Exploring Convolutional Networks for End-to-End Visual Servoing. How big is big enough? Unsupervised word sense dis …
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
| Title | BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems |
| Authors | Zachary Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng |
| Abstract | We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems. Our agents explore via Thompson sampling, drawing Monte Carlo samples from a Bayes-by-Backprop neural network. Our algorithm learns much faster than common exploration strategies such as \epsilon-greedy, Boltzmann, bootstrapping, and intrinsic-reward-based ones. Additionally, we show that spiking the replay buffer with experiences from just a few successful episodes can make Q-learning feasible when it might otherwise fail. |
| Tasks | Efficient Exploration, Q-Learning, Task-Oriented Dialogue Systems |
| Published | 2017-11-15 |
| URL | http://arxiv.org/abs/1711.05715v2 |
| http://arxiv.org/pdf/1711.05715v2.pdf | |
| PWC | https://paperswithcode.com/paper/bbq-networks-efficient-exploration-in-deep |
| Repo | |
| Framework | |
A Network Perspective on Stratification of Multi-Label Data
| Title | A Network Perspective on Stratification of Multi-Label Data |
| Authors | Piotr Szymański, Tomasz Kajdanowicz |
| Abstract | In the recent years, we have witnessed the development of multi-label classification methods which utilize the structure of the label space in a divide and conquer approach to improve classification performance and allow large data sets to be classified efficiently. Yet most of the available data sets have been provided in train/test splits that did not account for maintaining a distribution of higher-order relationships between labels among splits or folds. We present a new approach to stratifying multi-label data for classification purposes based on the iterative stratification approach proposed by Sechidis et. al. in an ECML PKDD 2011 paper. Our method extends the iterative approach to take into account second-order relationships between labels. Obtained results are evaluated using statistical properties of obtained strata as presented by Sechidis. We also propose new statistical measures relevant to second-order quality: label pairs distribution, the percentage of label pairs without positive evidence in folds and label pair - fold pairs that have no positive evidence for the label pair. We verify the impact of new methods on classification performance of Binary Relevance, Label Powerset and a fast greedy community detection based label space partitioning classifier. Random Forests serve as base classifiers. We check the variation of the number of communities obtained per fold, and the stability of their modularity score. Second-Order Iterative Stratification is compared to standard k-fold, label set, and iterative stratification. The proposed approach lowers the variance of classification quality, improves label pair oriented measures and example distribution while maintaining a competitive quality in label-oriented measures. We also witness an increase in stability of network characteristics. |
| Tasks | Community Detection, Multi-Label Classification |
| Published | 2017-04-27 |
| URL | http://arxiv.org/abs/1704.08756v1 |
| http://arxiv.org/pdf/1704.08756v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-network-perspective-on-stratification-of |
| Repo | |
| Framework | |
Infrastructure for Usable Machine Learning: The Stanford DAWN Project
| Title | Infrastructure for Usable Machine Learning: The Stanford DAWN Project |
| Authors | Peter Bailis, Kunle Olukotun, Christopher Re, Matei Zaharia |
| Abstract | Despite incredible recent advances in machine learning, building machine learning applications remains prohibitively time-consuming and expensive for all but the best-trained, best-funded engineering organizations. This expense comes not from a need for new and improved statistical models but instead from a lack of systems and tools for supporting end-to-end machine learning application development, from data preparation and labeling to productionization and monitoring. In this document, we outline opportunities for infrastructure supporting usable, end-to-end machine learning applications in the context of the nascent DAWN (Data Analytics for What’s Next) project at Stanford. |
| Tasks | |
| Published | 2017-05-22 |
| URL | http://arxiv.org/abs/1705.07538v2 |
| http://arxiv.org/pdf/1705.07538v2.pdf | |
| PWC | https://paperswithcode.com/paper/infrastructure-for-usable-machine-learning |
| Repo | |
| Framework | |
Exploring Convolutional Networks for End-to-End Visual Servoing
| Title | Exploring Convolutional Networks for End-to-End Visual Servoing |
| Authors | Aseem Saxena, Harit Pandya, Gourav Kumar, Ayush Gaud, K. Madhava Krishna |
| Abstract | Present image based visual servoing approaches rely on extracting hand crafted visual features from an image. Choosing the right set of features is important as it directly affects the performance of any approach. Motivated by recent breakthroughs in performance of data driven methods on recognition and localization tasks, we aim to learn visual feature representations suitable for servoing tasks in unstructured and unknown environments. In this paper, we present an end-to-end learning based approach for visual servoing in diverse scenes where the knowledge of camera parameters and scene geometry is not available a priori. This is achieved by training a convolutional neural network over color images with synchronised camera poses. Through experiments performed in simulation and on a quadrotor, we demonstrate the efficacy and robustness of our approach for a wide range of camera poses in both indoor as well as outdoor environments. |
| Tasks | |
| Published | 2017-06-10 |
| URL | http://arxiv.org/abs/1706.03220v1 |
| http://arxiv.org/pdf/1706.03220v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-convolutional-networks-for-end-to |
| Repo | |
| Framework | |
How big is big enough? Unsupervised word sense disambiguation using a very large corpus
| Title | How big is big enough? Unsupervised word sense disambiguation using a very large corpus |
| Authors | Piotr Przybyła |
| Abstract | In this paper, the problem of disambiguating a target word for Polish is approached by searching for related words with known meaning. These relatives are used to build a training corpus from unannotated text. This technique is improved by proposing new rich sources of replacements that substitute the traditional requirement of monosemy with heuristics based on wordnet relations. The na"ive Bayesian classifier has been modified to account for an unknown distribution of senses. A corpus of 600 million web documents (594 billion tokens), gathered by the NEKST search engine allows us to assess the relationship between training set size and disambiguation accuracy. The classifier is evaluated using both a wordnet baseline and a corpus with 17,314 manually annotated occurrences of 54 ambiguous words. |
| Tasks | Word Sense Disambiguation |
| Published | 2017-10-22 |
| URL | http://arxiv.org/abs/1710.07960v1 |
| http://arxiv.org/pdf/1710.07960v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-big-is-big-enough-unsupervised-word-sense |
| Repo | |
| Framework | |
Recurrent and Contextual Models for Visual Question Answering
| Title | Recurrent and Contextual Models for Visual Question Answering |
| Authors | Abhijit Sharang, Eric Lau |
| Abstract | We propose a series of recurrent and contextual neural network models for multiple choice visual question answering on the Visual7W dataset. Motivated by divergent trends in model complexities in the literature, we explore the balance between model expressiveness and simplicity by studying incrementally more complex architectures. We start with LSTM-encoding of input questions and answers; build on this with context generation by LSTM-encodings of neural image and question representations and attention over images; and evaluate the diversity and predictive power of our models and the ensemble thereof. All models are evaluated against a simple baseline inspired by the current state-of-the-art, consisting of involving simple concatenation of bag-of-words and CNN representations for the text and images, respectively. Generally, we observe marked variation in image-reasoning performance between our models not obvious from their overall performance, as well as evidence of dataset bias. Our standalone models achieve accuracies up to $64.6%$, while the ensemble of all models achieves the best accuracy of $66.67%$, within $0.5%$ of the current state-of-the-art for Visual7W. |
| Tasks | Question Answering, Visual Question Answering |
| Published | 2017-03-23 |
| URL | http://arxiv.org/abs/1703.08120v1 |
| http://arxiv.org/pdf/1703.08120v1.pdf | |
| PWC | https://paperswithcode.com/paper/recurrent-and-contextual-models-for-visual |
| Repo | |
| Framework | |
How Noisy Data Affects Geometric Semantic Genetic Programming
| Title | How Noisy Data Affects Geometric Semantic Genetic Programming |
| Authors | Luis F. Miranda, Luiz Otavio V. B. Oliveira, Joao Francisco B. S. Martins, Gisele L. Pappa |
| Abstract | Noise is a consequence of acquiring and pre-processing data from the environment, and shows fluctuations from different sources—e.g., from sensors, signal processing technology or even human error. As a machine learning technique, Genetic Programming (GP) is not immune to this problem, which the field has frequently addressed. Recently, Geometric Semantic Genetic Programming (GSGP), a semantic-aware branch of GP, has shown robustness and high generalization capability. Researchers believe these characteristics may be associated with a lower sensibility to noisy data. However, there is no systematic study on this matter. This paper performs a deep analysis of the GSGP performance over the presence of noise. Using 15 synthetic datasets where noise can be controlled, we added different ratios of noise to the data and compared the results obtained with those of a canonical GP. The results show that, as we increase the percentage of noisy instances, the generalization performance degradation is more pronounced in GSGP than GP. However, in general, GSGP is more robust to noise than GP in the presence of up to 10% of noise, and presents no statistical difference for values higher than that in the test bed. |
| Tasks | |
| Published | 2017-07-04 |
| URL | http://arxiv.org/abs/1707.01046v1 |
| http://arxiv.org/pdf/1707.01046v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-noisy-data-affects-geometric-semantic |
| Repo | |
| Framework | |
Vertical Landing for Micro Air Vehicles using Event-Based Optical Flow
| Title | Vertical Landing for Micro Air Vehicles using Event-Based Optical Flow |
| Authors | Bas J. Pijnacker Hordijk, Kirk Y. W. Scheper, Guido C. H. E. de Croon |
| Abstract | Small flying robots can perform landing maneuvers using bio-inspired optical flow by maintaining a constant divergence. However, optical flow is typically estimated from frame sequences recorded by standard miniature cameras. This requires processing full images on-board, limiting the update rate of divergence measurements, and thus the speed of the control loop and the robot. Event-based cameras overcome these limitations by only measuring pixel-level brightness changes at microsecond temporal accuracy, hence providing an efficient mechanism for optical flow estimation. This paper presents, to the best of our knowledge, the first work integrating event-based optical flow estimation into the control loop of a flying robot. We extend an existing ‘local plane fitting’ algorithm to obtain an improved and more computationally efficient optical flow estimation method, valid for a wide range of optical flow velocities. This method is validated for real event sequences. In addition, a method for estimating the divergence from event-based optical flow is introduced, which accounts for the aperture problem. The developed algorithms are implemented in a constant divergence landing controller on-board of a quadrotor. Experiments show that, using event-based optical flow, accurate divergence estimates can be obtained over a wide range of speeds. This enables the quadrotor to perform very fast landing maneuvers. |
| Tasks | Optical Flow Estimation |
| Published | 2017-01-31 |
| URL | http://arxiv.org/abs/1702.00061v3 |
| http://arxiv.org/pdf/1702.00061v3.pdf | |
| PWC | https://paperswithcode.com/paper/vertical-landing-for-micro-air-vehicles-using |
| Repo | |
| Framework | |
How is Distributed ADMM Affected by Network Topology?
| Title | How is Distributed ADMM Affected by Network Topology? |
| Authors | Guilherme França, José Bento |
| Abstract | When solving consensus optimization problems over a graph, there is often an explicit characterization of the convergence rate of Gradient Descent (GD) using the spectrum of the graph Laplacian. The same type of problems under the Alternating Direction Method of Multipliers (ADMM) are, however, poorly understood. For instance, simple but important non-strongly-convex consensus problems have not yet being analyzed, especially concerning the dependency of the convergence rate on the graph topology. Recently, for a non-strongly-convex consensus problem, a connection between distributed ADMM and lifted Markov chains was proposed, followed by a conjecture that ADMM is faster than GD by a square root factor in its convergence time, in close analogy to the mixing speedup achieved by lifting several Markov chains. Nevertheless, a proof of such a claim is is still lacking. Here we provide a full characterization of the convergence of distributed over-relaxed ADMM for the same type of consensus problem in terms of the topology of the underlying graph. Our results provide explicit formulas for optimal parameter selection in terms of the second largest eigenvalue of the transition matrix of the graph’s random walk. Another consequence of our results is a proof of the aforementioned conjecture, which interestingly, we show it is valid for any graph, even the ones whose random walks cannot be accelerated via Markov chain lifting. |
| Tasks | |
| Published | 2017-10-02 |
| URL | http://arxiv.org/abs/1710.00889v1 |
| http://arxiv.org/pdf/1710.00889v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-is-distributed-admm-affected-by-network |
| Repo | |
| Framework | |
Pairing an arbitrary regressor with an artificial neural network estimating aleatoric uncertainty
| Title | Pairing an arbitrary regressor with an artificial neural network estimating aleatoric uncertainty |
| Authors | Pavel Gurevich, Hannes Stuke |
| Abstract | We suggest a general approach to quantification of different forms of aleatoric uncertainty in regression tasks performed by artificial neural networks. It is based on the simultaneous training of two neural networks with a joint loss function and a specific hyperparameter $\lambda>0$ that allows for automatically detecting noisy and clean regions in the input space and controlling their {\em relative contribution} to the loss and its gradients. After the model has been trained, one of the networks performs predictions and the other quantifies the uncertainty of these predictions by estimating the locally averaged loss of the first one. Unlike in many classical uncertainty quantification methods, we do not assume any a priori knowledge of the ground truth probability distribution, neither do we, in general, maximize the likelihood of a chosen parametric family of distributions. We analyze the learning process and the influence of clean and noisy regions of the input space on the loss surface, depending on $\lambda$. In particular, we show that small values of $\lambda$ increase the relative contribution of clean regions to the loss and its gradients. This explains why choosing small $\lambda$ allows for better predictions compared with neural networks without uncertainty counterparts and those based on classical likelihood maximization. Finally, we demonstrate that one can naturally form ensembles of pairs of our networks and thus capture both aleatoric and epistemic uncertainty and avoid overfitting. |
| Tasks | |
| Published | 2017-07-23 |
| URL | http://arxiv.org/abs/1707.07287v3 |
| http://arxiv.org/pdf/1707.07287v3.pdf | |
| PWC | https://paperswithcode.com/paper/pairing-an-arbitrary-regressor-with-an |
| Repo | |
| Framework | |
Online Adaptive Machine Learning Based Algorithm for Implied Volatility Surface Modeling
| Title | Online Adaptive Machine Learning Based Algorithm for Implied Volatility Surface Modeling |
| Authors | Yaxiong Zeng, Diego Klabjan |
| Abstract | In this work, we design a machine learning based method, online adaptive primal support vector regression (SVR), to model the implied volatility surface (IVS). The algorithm proposed is the first derivation and implementation of an online primal kernel SVR. It features enhancements that allow efficient online adaptive learning by embedding the idea of local fitness and budget maintenance to dynamically update support vectors upon pattern drifts. For algorithm acceleration, we implement its most computationally intensive parts in a Field Programmable Gate Arrays hardware, where a 132x speedup over CPU is achieved during online prediction. Using intraday tick data from the E-mini S&P 500 options market, we show that the Gaussian kernel outperforms the linear kernel in regulating the size of support vectors, and that our empirical IVS algorithm beats two competing online methods with regards to model complexity and regression errors (the mean absolute percentage error of our algorithm is up to 13%). Best results are obtained at the center of the IVS grid due to its larger number of adjacent support vectors than the edges of the grid. Sensitivity analysis is also presented to demonstrate how hyper parameters affect the error rates and model complexity. |
| Tasks | |
| Published | 2017-06-06 |
| URL | http://arxiv.org/abs/1706.01833v2 |
| http://arxiv.org/pdf/1706.01833v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-adaptive-machine-learning-based |
| Repo | |
| Framework | |
Hindsight policy gradients
| Title | Hindsight policy gradients |
| Authors | Paulo Rauber, Avinash Ummadisingu, Filipe Mutz, Juergen Schmidhuber |
| Abstract | A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may also enable higher-level planning based on subgoals. In sparse-reward environments, the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended appears crucial to enable sample efficient learning. However, reinforcement learning agents have only recently been endowed with such capacity for hindsight. In this paper, we demonstrate how hindsight can be introduced to policy gradient methods, generalizing this idea to a broad class of successful algorithms. Our experiments on a diverse selection of sparse-reward environments show that hindsight leads to a remarkable increase in sample efficiency. |
| Tasks | Policy Gradient Methods |
| Published | 2017-11-16 |
| URL | http://arxiv.org/abs/1711.06006v3 |
| http://arxiv.org/pdf/1711.06006v3.pdf | |
| PWC | https://paperswithcode.com/paper/hindsight-policy-gradients |
| Repo | |
| Framework | |
Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality
| Title | Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality |
| Authors | Song Mei, Theodor Misiakiewicz, Andrea Montanari, Roberto I. Oliveira |
| Abstract | A number of statistical estimation problems can be addressed by semidefinite programs (SDP). While SDPs are solvable in polynomial time using interior point methods, in practice generic SDP solvers do not scale well to high-dimensional problems. In order to cope with this problem, Burer and Monteiro proposed a non-convex rank-constrained formulation, which has good performance in practice but is still poorly understood theoretically. In this paper we study the rank-constrained version of SDPs arising in MaxCut and in synchronization problems. We establish a Grothendieck-type inequality that proves that all the local maxima and dangerous saddle points are within a small multiplicative gap from the global maximum. We use this structural information to prove that SDPs can be solved within a known accuracy, by applying the Riemannian trust-region method to this non-convex problem, while constraining the rank to be of order one. For the MaxCut problem, our inequality implies that any local maximizer of the rank-constrained SDP provides a $ (1 - 1/(k-1)) \times 0.878$ approximation of the MaxCut, when the rank is fixed to $k$. We then apply our results to data matrices generated according to the Gaussian ${\mathbb Z}_2$ synchronization problem, and the two-groups stochastic block model with large bounded degree. We prove that the error achieved by local maximizers undergoes a phase transition at the same threshold as for information-theoretically optimal methods. |
| Tasks | |
| Published | 2017-03-25 |
| URL | http://arxiv.org/abs/1703.08729v2 |
| http://arxiv.org/pdf/1703.08729v2.pdf | |
| PWC | https://paperswithcode.com/paper/solving-sdps-for-synchronization-and-maxcut |
| Repo | |
| Framework | |
Generating Steganographic Text with LSTMs
| Title | Generating Steganographic Text with LSTMs |
| Authors | Tina Fang, Martin Jaggi, Katerina Argyraki |
| Abstract | Motivated by concerns for user privacy, we design a steganographic system (“stegosystem”) that enables two users to exchange encrypted messages without an adversary detecting that such an exchange is taking place. We propose a new linguistic stegosystem based on a Long Short-Term Memory (LSTM) neural network. We demonstrate our approach on the Twitter and Enron email datasets and show that it yields high-quality steganographic text while significantly improving capacity (encrypted bits per word) relative to the state-of-the-art. |
| Tasks | |
| Published | 2017-05-30 |
| URL | http://arxiv.org/abs/1705.10742v1 |
| http://arxiv.org/pdf/1705.10742v1.pdf | |
| PWC | https://paperswithcode.com/paper/generating-steganographic-text-with-lstms |
| Repo | |
| Framework | |
Blind Gain and Phase Calibration via Sparse Spectral Methods
| Title | Blind Gain and Phase Calibration via Sparse Spectral Methods |
| Authors | Yanjun Li, Kiryung Lee, Yoram Bresler |
| Abstract | Blind gain and phase calibration (BGPC) is a bilinear inverse problem involving the determination of unknown gains and phases of the sensing system, and the unknown signal, jointly. BGPC arises in numerous applications, e.g., blind albedo estimation in inverse rendering, synthetic aperture radar autofocus, and sensor array auto-calibration. In some cases, sparse structure in the unknown signal alleviates the ill-posedness of BGPC. Recently there has been renewed interest in solutions to BGPC with careful analysis of error bounds. In this paper, we formulate BGPC as an eigenvalue/eigenvector problem, and propose to solve it via power iteration, or in the sparsity or joint sparsity case, via truncated power iteration. Under certain assumptions, the unknown gains, phases, and the unknown signal can be recovered simultaneously. Numerical experiments show that power iteration algorithms work not only in the regime predicted by our main results, but also in regimes where theoretical analysis is limited. We also show that our power iteration algorithms for BGPC compare favorably with competing algorithms in adversarial conditions, e.g., with noisy measurement or with a bad initial estimate. |
| Tasks | Calibration |
| Published | 2017-11-30 |
| URL | http://arxiv.org/abs/1712.00111v1 |
| http://arxiv.org/pdf/1712.00111v1.pdf | |
| PWC | https://paperswithcode.com/paper/blind-gain-and-phase-calibration-via-sparse |
| Repo | |
| Framework | |
