Paper Group ANR 293
Learning Features from Co-occurrences: A Theoretical Analysis. Stochastic Feedback Control of Systems with Unknown Nonlinear Dynamics. Pseudo-labels for Supervised Learning on Dynamic Vision Sensor Data, Applied to Object Detection under Ego-motion. Meta Networks. Joint Blind Motion Deblurring and Depth Estimation of Light Field. Universal Denoisin …
Learning Features from Co-occurrences: A Theoretical Analysis
| Title | Learning Features from Co-occurrences: A Theoretical Analysis |
| Authors | Yanpeng Li |
| Abstract | Representing a word by its co-occurrences with other words in context is an effective way to capture the meaning of the word. However, the theory behind remains a challenge. In this work, taking the example of a word classification task, we give a theoretical analysis of the approaches that represent a word X by a function f(P(CX)), where C is a context feature, P(CX) is the conditional probability estimated from a text corpus, and the function f maps the co-occurrence measure to a prediction score. We investigate the impact of context feature C and the function f. We also explain the reasons why using the co-occurrences with multiple context features may be better than just using a single one. In addition, some of the results shed light on the theory of feature learning and machine learning in general. |
| Tasks | |
| Published | 2017-07-13 |
| URL | http://arxiv.org/abs/1707.04218v1 |
| http://arxiv.org/pdf/1707.04218v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-features-from-co-occurrences-a |
| Repo | |
| Framework | |
Stochastic Feedback Control of Systems with Unknown Nonlinear Dynamics
| Title | Stochastic Feedback Control of Systems with Unknown Nonlinear Dynamics |
| Authors | Dan Yu, Mohammadhussein Rafieisakhaei, Suman Chakravorty |
| Abstract | This paper studies the stochastic optimal control problem for systems with unknown dynamics. First, an open-loop deterministic trajectory optimization problem is solved without knowing the explicit form of the dynamical system. Next, a Linear Quadratic Gaussian (LQG) controller is designed for the nominal trajectory-dependent linearized system, such that under a small noise assumption, the actual states remain close to the optimal trajectory. The trajectory-dependent linearized system is identified using input-output experimental data consisting of the impulse responses of the nominal system. A computational example is given to illustrate the performance of the proposed approach. |
| Tasks | |
| Published | 2017-05-27 |
| URL | http://arxiv.org/abs/1705.09761v1 |
| http://arxiv.org/pdf/1705.09761v1.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-feedback-control-of-systems-with |
| Repo | |
| Framework | |
Pseudo-labels for Supervised Learning on Dynamic Vision Sensor Data, Applied to Object Detection under Ego-motion
| Title | Pseudo-labels for Supervised Learning on Dynamic Vision Sensor Data, Applied to Object Detection under Ego-motion |
| Authors | Nicholas F. Y. Chen |
| Abstract | In recent years, dynamic vision sensors (DVS), also known as event-based cameras or neuromorphic sensors, have seen increased use due to various advantages over conventional frame-based cameras. Using principles inspired by the retina, its high temporal resolution overcomes motion blurring, its high dynamic range overcomes extreme illumination conditions and its low power consumption makes it ideal for embedded systems on platforms such as drones and self-driving cars. However, event-based data sets are scarce and labels are even rarer for tasks such as object detection. We transferred discriminative knowledge from a state-of-the-art frame-based convolutional neural network (CNN) to the event-based modality via intermediate pseudo-labels, which are used as targets for supervised learning. We show, for the first time, event-based car detection under ego-motion in a real environment at 100 frames per second with a test average precision of 40.3% relative to our annotated ground truth. The event-based car detector handles motion blur and poor illumination conditions despite not explicitly trained to do so, and even complements frame-based CNN detectors, suggesting that it has learnt generalized visual representations. |
| Tasks | Object Detection, Self-Driving Cars |
| Published | 2017-09-27 |
| URL | http://arxiv.org/abs/1709.09323v3 |
| http://arxiv.org/pdf/1709.09323v3.pdf | |
| PWC | https://paperswithcode.com/paper/pseudo-labels-for-supervised-learning-on |
| Repo | |
| Framework | |
Meta Networks
| Title | Meta Networks |
| Authors | Tsendsuren Munkhdalai, Hong Yu |
| Abstract | Neural networks have been successfully applied in applications with a large amount of labeled data. However, the task of rapid generalization on new concepts with small training data while preserving performances on previously learned ones still presents a significant challenge to neural network models. In this work, we introduce a novel meta learning method, Meta Networks (MetaNet), that learns a meta-level knowledge across tasks and shifts its inductive biases via fast parameterization for rapid generalization. When evaluated on Omniglot and Mini-ImageNet benchmarks, our MetaNet models achieve a near human-level performance and outperform the baseline approaches by up to 6% accuracy. We demonstrate several appealing properties of MetaNet relating to generalization and continual learning. |
| Tasks | Continual Learning, Meta-Learning, Omniglot |
| Published | 2017-03-02 |
| URL | http://arxiv.org/abs/1703.00837v2 |
| http://arxiv.org/pdf/1703.00837v2.pdf | |
| PWC | https://paperswithcode.com/paper/meta-networks |
| Repo | |
| Framework | |
Joint Blind Motion Deblurring and Depth Estimation of Light Field
| Title | Joint Blind Motion Deblurring and Depth Estimation of Light Field |
| Authors | Dongwoo Lee, Haesol Park, In Kyu Park, Kyoung Mu Lee |
| Abstract | Removing camera motion blur from a single light field is a challenging task since it is highly ill-posed inverse problem. The problem becomes even worse when blur kernel varies spatially due to scene depth variation and high-order camera motion. In this paper, we propose a novel algorithm to estimate all blur model variables jointly, including latent sub-aperture image, camera motion, and scene depth from the blurred 4D light field. Exploiting multi-view nature of a light field relieves the inverse property of the optimization by utilizing strong depth cues and multi-view blur observation. The proposed joint estimation achieves high quality light field deblurring and depth estimation simultaneously under arbitrary 6-DOF camera motion and unconstrained scene depth. Intensive experiment on real and synthetic blurred light field confirms that the proposed algorithm outperforms the state-of-the-art light field deblurring and depth estimation methods. |
| Tasks | Deblurring, Depth Estimation |
| Published | 2017-11-29 |
| URL | http://arxiv.org/abs/1711.10918v2 |
| http://arxiv.org/pdf/1711.10918v2.pdf | |
| PWC | https://paperswithcode.com/paper/joint-blind-motion-deblurring-and-depth |
| Repo | |
| Framework | |
Universal Denoising Networks : A Novel CNN Architecture for Image Denoising
| Title | Universal Denoising Networks : A Novel CNN Architecture for Image Denoising |
| Authors | Stamatios Lefkimmiatis |
| Abstract | We design a novel network architecture for learning discriminative image models that are employed to efficiently tackle the problem of grayscale and color image denoising. Based on the proposed architecture, we introduce two different variants. The first network involves convolutional layers as a core component, while the second one relies instead on non-local filtering layers and thus it is able to exploit the inherent non-local self-similarity property of natural images. As opposed to most of the existing deep network approaches, which require the training of a specific model for each considered noise level, the proposed models are able to handle a wide range of noise levels using a single set of learned parameters, while they are very robust when the noise degrading the latent image does not match the statistics of the noise used during training. The latter argument is supported by results that we report on publicly available images corrupted by unknown noise and which we compare against solutions obtained by competing methods. At the same time the introduced networks achieve excellent results under additive white Gaussian noise (AWGN), which are comparable to those of the current state-of-the-art network, while they depend on a more shallow architecture with the number of trained parameters being one order of magnitude smaller. These properties make the proposed networks ideal candidates to serve as sub-solvers on restoration methods that deal with general inverse imaging problems such as deblurring, demosaicking, superresolution, etc. |
| Tasks | Deblurring, Demosaicking, Denoising, Image Denoising |
| Published | 2017-11-21 |
| URL | http://arxiv.org/abs/1711.07807v2 |
| http://arxiv.org/pdf/1711.07807v2.pdf | |
| PWC | https://paperswithcode.com/paper/universal-denoising-networks-a-novel-cnn |
| Repo | |
| Framework | |
A Savage-Like Axiomatization for Nonstandard Expected Utility
| Title | A Savage-Like Axiomatization for Nonstandard Expected Utility |
| Authors | Grant Molnar |
| Abstract | Since Leonard Savage’s epoch-making “Foundations of Statistics”, Subjective Expected Utility Theory has been the presumptive model for decision-making. Savage provided an act-based axiomatization of standard expected utility theory. In this article, we provide a Savage-like axiomatization of nonstandard expected utility theory. It corresponds to a weakening of Savage’s 6th axiom. |
| Tasks | Decision Making |
| Published | 2017-01-12 |
| URL | http://arxiv.org/abs/1701.03500v7 |
| http://arxiv.org/pdf/1701.03500v7.pdf | |
| PWC | https://paperswithcode.com/paper/a-savage-like-axiomatization-for-nonstandard |
| Repo | |
| Framework | |
Neural Networks for Predicting Algorithm Runtime Distributions
| Title | Neural Networks for Predicting Algorithm Runtime Distributions |
| Authors | Katharina Eggensperger, Marius Lindauer, Frank Hutter |
| Abstract | Many state-of-the-art algorithms for solving hard combinatorial problems in artificial intelligence (AI) include elements of stochasticity that lead to high variations in runtime, even for a fixed problem instance. Knowledge about the resulting runtime distributions (RTDs) of algorithms on given problem instances can be exploited in various meta-algorithmic procedures, such as algorithm selection, portfolios, and randomized restarts. Previous work has shown that machine learning can be used to individually predict mean, median and variance of RTDs. To establish a new state-of-the-art in predicting RTDs, we demonstrate that the parameters of an RTD should be learned jointly and that neural networks can do this well by directly optimizing the likelihood of an RTD given runtime observations. In an empirical study involving five algorithms for SAT solving and AI planning, we show that neural networks predict the true RTDs of unseen instances better than previous methods, and can even do so when only few runtime observations are available per training instance. |
| Tasks | |
| Published | 2017-09-22 |
| URL | http://arxiv.org/abs/1709.07615v3 |
| http://arxiv.org/pdf/1709.07615v3.pdf | |
| PWC | https://paperswithcode.com/paper/neural-networks-for-predicting-algorithm |
| Repo | |
| Framework | |
RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs
| Title | RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs |
| Authors | Yingying Fan, Emre Demirkaya, Gaorong Li, Jinchi Lv |
| Abstract | Power and reproducibility are key to enabling refined scientific discoveries in contemporary big data applications with general high-dimensional nonlinear models. In this paper, we provide theoretical foundations on the power and robustness for the model-free knockoffs procedure introduced recently in Cand`{e}s, Fan, Janson and Lv (2016) in high-dimensional setting when the covariate distribution is characterized by Gaussian graphical model. We establish that under mild regularity conditions, the power of the oracle knockoffs procedure with known covariate distribution in high-dimensional linear models is asymptotically one as sample size goes to infinity. When moving away from the ideal case, we suggest the modified model-free knockoffs method called graphical nonlinear knockoffs (RANK) to accommodate the unknown covariate distribution. We provide theoretical justifications on the robustness of our modified procedure by showing that the false discovery rate (FDR) is asymptotically controlled at the target level and the power is asymptotically one with the estimated covariate distribution. To the best of our knowledge, this is the first formal theoretical result on the power for the knockoffs procedure. Simulation results demonstrate that compared to existing approaches, our method performs competitively in both FDR control and power. A real data set is analyzed to further assess the performance of the suggested knockoffs procedure. |
| Tasks | |
| Published | 2017-08-31 |
| URL | http://arxiv.org/abs/1709.00092v1 |
| http://arxiv.org/pdf/1709.00092v1.pdf | |
| PWC | https://paperswithcode.com/paper/rank-large-scale-inference-with-graphical |
| Repo | |
| Framework | |
BreathRNNet: Breathing Based Authentication on Resource-Constrained IoT Devices using RNNs
| Title | BreathRNNet: Breathing Based Authentication on Resource-Constrained IoT Devices using RNNs |
| Authors | Jagmohan Chauhan, Suranga Seneviratne, Yining Hu, Archan Misra, Aruna Seneviratne, Youngki Lee |
| Abstract | Recurrent neural networks (RNNs) have shown promising results in audio and speech processing applications due to their strong capabilities in modelling sequential data. In many applications, RNNs tend to outperform conventional models based on GMM/UBMs and i-vectors. Increasing popularity of IoT devices makes a strong case for implementing RNN based inferences for applications such as acoustics based authentication, voice commands, and edge analytics for smart homes. Nonetheless, the feasibility and performance of RNN based inferences on resources-constrained IoT devices remain largely unexplored. In this paper, we investigate the feasibility of using RNNs for an end-to-end authentication system based on breathing acoustics. We evaluate the performance of RNN models on three types of devices; smartphone, smartwatch, and Raspberry Pi and show that unlike CNN models, RNN models can be easily ported onto resource-constrained devices without a significant loss in accuracy. |
| Tasks | |
| Published | 2017-09-22 |
| URL | http://arxiv.org/abs/1709.07626v1 |
| http://arxiv.org/pdf/1709.07626v1.pdf | |
| PWC | https://paperswithcode.com/paper/breathrnnet-breathing-based-authentication-on |
| Repo | |
| Framework | |
Group invariance principles for causal generative models
| Title | Group invariance principles for causal generative models |
| Authors | Michel Besserve, Naji Shajarisales, Bernhard Schölkopf, Dominik Janzing |
| Abstract | The postulate of independence of cause and mechanism (ICM) has recently led to several new causal discovery algorithms. The interpretation of independence and the way it is utilized, however, varies across these methods. Our aim in this paper is to propose a group theoretic framework for ICM to unify and generalize these approaches. In our setting, the cause-mechanism relationship is assessed by comparing it against a null hypothesis through the application of random generic group transformations. We show that the group theoretic view provides a very general tool to study the structure of data generating mechanisms with direct applications to machine learning. |
| Tasks | Causal Discovery |
| Published | 2017-05-05 |
| URL | http://arxiv.org/abs/1705.02212v1 |
| http://arxiv.org/pdf/1705.02212v1.pdf | |
| PWC | https://paperswithcode.com/paper/group-invariance-principles-for-causal |
| Repo | |
| Framework | |
OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
| Title | OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World |
| Authors | Tu-Hoa Pham, Giovanni De Magistris, Ryuki Tachibana |
| Abstract | While deep reinforcement learning techniques have recently produced considerable achievements on many decision-making problems, their use in robotics has largely been limited to simulated worlds or restricted motions, since unconstrained trial-and-error interactions in the real world can have undesirable consequences for the robot or its environment. To overcome such limitations, we propose a novel reinforcement learning architecture, OptLayer, that takes as inputs possibly unsafe actions predicted by a neural network and outputs the closest actions that satisfy chosen constraints. While learning control policies often requires carefully crafted rewards and penalties while exploring the range of possible actions, OptLayer ensures that only safe actions are actually executed and unsafe predictions are penalized during training. We demonstrate the effectiveness of our approach on robot reaching tasks, both simulated and in the real world. |
| Tasks | Decision Making |
| Published | 2017-09-22 |
| URL | http://arxiv.org/abs/1709.07643v2 |
| http://arxiv.org/pdf/1709.07643v2.pdf | |
| PWC | https://paperswithcode.com/paper/optlayer-practical-constrained-optimization |
| Repo | |
| Framework | |
Semantic Segmentation with Reverse Attention
| Title | Semantic Segmentation with Reverse Attention |
| Authors | Qin Huang, Chunyang Xia, Chihao Wu, Siyang Li, Ye Wang, Yuhang Song, C. -C. Jay Kuo |
| Abstract | Recent development in fully convolutional neural network enables efficient end-to-end learning of semantic segmentation. Traditionally, the convolutional classifiers are taught to learn the representative semantic features of labeled semantic objects. In this work, we propose a reverse attention network (RAN) architecture that trains the network to capture the opposite concept (i.e., what are not associated with a target class) as well. The RAN is a three-branch network that performs the direct, reverse and reverse-attention learning processes simultaneously. Extensive experiments are conducted to show the effectiveness of the RAN in semantic segmentation. Being built upon the DeepLabv2-LargeFOV, the RAN achieves the state-of-the-art mIoU score (48.1%) for the challenging PASCAL-Context dataset. Significant performance improvements are also observed for the PASCAL-VOC, Person-Part, NYUDv2 and ADE20K datasets. |
| Tasks | Semantic Segmentation |
| Published | 2017-07-20 |
| URL | http://arxiv.org/abs/1707.06426v1 |
| http://arxiv.org/pdf/1707.06426v1.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-segmentation-with-reverse-attention |
| Repo | |
| Framework | |
The CAPIO 2017 Conversational Speech Recognition System
| Title | The CAPIO 2017 Conversational Speech Recognition System |
| Authors | Kyu J. Han, Akshay Chandrashekaran, Jungsuk Kim, Ian Lane |
| Abstract | In this paper we show how we have achieved the state-of-the-art performance on the industry-standard NIST 2000 Hub5 English evaluation set. We explore densely connected LSTMs, inspired by the densely connected convolutional networks recently introduced for image classification tasks. We also propose an acoustic model adaptation scheme that simply averages the parameters of a seed neural network acoustic model and its adapted version. This method was applied with the CallHome training corpus and improved individual system performances by on average 6.1% (relative) against the CallHome portion of the evaluation set with no performance loss on the Switchboard portion. With RNN-LM rescoring and lattice combination on the 5 systems trained across three different phone sets, our 2017 speech recognition system has obtained 5.0% and 9.1% on Switchboard and CallHome, respectively, both of which are the best word error rates reported thus far. According to IBM in their latest work to compare human and machine transcriptions, our reported Switchboard word error rate can be considered to surpass the human parity (5.1%) of transcribing conversational telephone speech. |
| Tasks | Image Classification, Speech Recognition |
| Published | 2017-12-29 |
| URL | http://arxiv.org/abs/1801.00059v2 |
| http://arxiv.org/pdf/1801.00059v2.pdf | |
| PWC | https://paperswithcode.com/paper/the-capio-2017-conversational-speech |
| Repo | |
| Framework | |
Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
| Title | Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems |
| Authors | Alyson K. Fletcher, Mojtaba Sahraee-Ardakan, Philip Schniter, Sundeep Rangan |
| Abstract | The problem of estimating a random vector x from noisy linear measurements y = A x + w with unknown parameters on the distributions of x and w, which must also be learned, arises in a wide range of statistical learning and linear inverse problems. We show that a computationally simple iterative message-passing algorithm can provably obtain asymptotically consistent estimates in a certain high-dimensional large-system limit (LSL) under very general parameterizations. Previous message passing techniques have required i.i.d. sub-Gaussian A matrices and often fail when the matrix is ill-conditioned. The proposed algorithm, called adaptive vector approximate message passing (Adaptive VAMP) with auto-tuning, applies to all right-rotationally random A. Importantly, this class includes matrices with arbitrarily poor conditioning. We show that the parameter estimates and mean squared error (MSE) of x in each iteration converge to deterministic limits that can be precisely predicted by a simple set of state evolution (SE) equations. In addition, a simple testable condition is provided in which the MSE matches the Bayes-optimal value predicted by the replica method. The paper thus provides a computationally simple method with provable guarantees of optimality and consistency over a large class of linear inverse problems. |
| Tasks | |
| Published | 2017-06-19 |
| URL | http://arxiv.org/abs/1706.06054v1 |
| http://arxiv.org/pdf/1706.06054v1.pdf | |
| PWC | https://paperswithcode.com/paper/rigorous-dynamics-and-consistent-estimation |
| Repo | |
| Framework | |
