Paper Group ANR 51
Learning One-hidden-layer Neural Networks with Landscape Design. Fast Recurrent Fully Convolutional Networks for Direct Perception in Autonomous Driving. A Manifold Approach to Learning Mutually Orthogonal Subspaces. PCANet-II: When PCANet Meets the Second Order Pooling. Collision Selective Visual Neural Network Inspired by LGMD2 Neurons in Juvenil …
Learning One-hidden-layer Neural Networks with Landscape Design
| Title | Learning One-hidden-layer Neural Networks with Landscape Design |
| Authors | Rong Ge, Jason D. Lee, Tengyu Ma |
| Abstract | We consider the problem of learning a one-hidden-layer neural network: we assume the input $x\in \mathbb{R}^d$ is from Gaussian distribution and the label $y = a^\top \sigma(Bx) + \xi$, where $a$ is a nonnegative vector in $\mathbb{R}^m$ with $m\le d$, $B\in \mathbb{R}^{m\times d}$ is a full-rank weight matrix, and $\xi$ is a noise vector. We first give an analytic formula for the population risk of the standard squared loss and demonstrate that it implicitly attempts to decompose a sequence of low-rank tensors simultaneously. Inspired by the formula, we design a non-convex objective function $G(\cdot)$ whose landscape is guaranteed to have the following properties: 1. All local minima of $G$ are also global minima. 2. All global minima of $G$ correspond to the ground truth parameters. 3. The value and gradient of $G$ can be estimated using samples. With these properties, stochastic gradient descent on $G$ provably converges to the global minimum and learn the ground-truth parameters. We also prove finite sample complexity result and validate the results by simulations. |
| Tasks | |
| Published | 2017-11-01 |
| URL | http://arxiv.org/abs/1711.00501v2 |
| http://arxiv.org/pdf/1711.00501v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-one-hidden-layer-neural-networks-1 |
| Repo | |
| Framework | |
Fast Recurrent Fully Convolutional Networks for Direct Perception in Autonomous Driving
| Title | Fast Recurrent Fully Convolutional Networks for Direct Perception in Autonomous Driving |
| Authors | Yiqi Hou, Sascha Hornauer, Karl Zipser |
| Abstract | Deep convolutional neural networks (CNNs) have been shown to perform extremely well at a variety of tasks including subtasks of autonomous driving such as image segmentation and object classification. However, networks designed for these tasks typically require vast quantities of training data and long training periods to converge. We investigate the design rationale behind end-to-end driving network designs by proposing and comparing three small and computationally inexpensive deep end-to-end neural network models that generate driving control signals directly from input images. In contrast to prior work that segments the autonomous driving task, our models take on a novel approach to the autonomous driving problem by utilizing deep and thin Fully Convolutional Nets (FCNs) with recurrent neural nets and low parameter counts to tackle a complex end-to-end regression task predicting both steering and acceleration commands. In addition, we include layers optimized for classification to allow the networks to implicitly learn image semantics. We show that the resulting networks use 3x fewer parameters than the most recent comparable end-to-end driving network and 500x fewer parameters than the AlexNet variations and converge both faster and to lower losses while maintaining robustness against overfitting. |
| Tasks | Autonomous Driving, Object Classification, Semantic Segmentation |
| Published | 2017-11-17 |
| URL | http://arxiv.org/abs/1711.06459v2 |
| http://arxiv.org/pdf/1711.06459v2.pdf | |
| PWC | https://paperswithcode.com/paper/fast-recurrent-fully-convolutional-networks |
| Repo | |
| Framework | |
A Manifold Approach to Learning Mutually Orthogonal Subspaces
| Title | A Manifold Approach to Learning Mutually Orthogonal Subspaces |
| Authors | Stephen Giguere, Francisco Garcia, Sridhar Mahadevan |
| Abstract | Although many machine learning algorithms involve learning subspaces with particular characteristics, optimizing a parameter matrix that is constrained to represent a subspace can be challenging. One solution is to use Riemannian optimization methods that enforce such constraints implicitly, leveraging the fact that the feasible parameter values form a manifold. While Riemannian methods exist for some specific problems, such as learning a single subspace, there are more general subspace constraints that offer additional flexibility when setting up an optimization problem, but have not been formulated as a manifold. We propose the partitioned subspace (PS) manifold for optimizing matrices that are constrained to represent one or more subspaces. Each point on the manifold defines a partitioning of the input space into mutually orthogonal subspaces, where the number of partitions and their sizes are defined by the user. As a result, distinct groups of features can be learned by defining different objective functions for each partition. We illustrate the properties of the manifold through experiments on multiple dataset analysis and domain adaptation. |
| Tasks | Domain Adaptation |
| Published | 2017-03-08 |
| URL | http://arxiv.org/abs/1703.02992v1 |
| http://arxiv.org/pdf/1703.02992v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-manifold-approach-to-learning-mutually |
| Repo | |
| Framework | |
PCANet-II: When PCANet Meets the Second Order Pooling
| Title | PCANet-II: When PCANet Meets the Second Order Pooling |
| Authors | Lei Tian, Xiaopeng Hong, Guoying Zhao, Chunxiao Fan, Yue Ming, Matti Pietikäinen |
| Abstract | PCANet, as one noticeable shallow network, employs the histogram representation for feature pooling. However, there are three main problems about this kind of pooling method. First, the histogram-based pooling method binarizes the feature maps and leads to inevitable discriminative information loss. Second, it is difficult to effectively combine other visual cues into a compact representation, because the simple concatenation of various visual cues leads to feature representation inefficiency. Third, the dimensionality of histogram-based output grows exponentially with the number of feature maps used. In order to overcome these problems, we propose a novel shallow network model, named as PCANet-II. Compared with the histogram-based output, the second order pooling not only provides more discriminative information by preserving both the magnitude and sign of convolutional responses, but also dramatically reduces the size of output features. Thus we combine the second order statistical pooling method with the shallow network, i.e., PCANet. Moreover, it is easy to combine other discriminative and robust cues by using the second order pooling. So we introduce the binary feature difference encoding scheme into our PCANet-II to further improve robustness. Experiments demonstrate the effectiveness and robustness of our proposed PCANet-II method. |
| Tasks | |
| Published | 2017-09-30 |
| URL | http://arxiv.org/abs/1710.00166v1 |
| http://arxiv.org/pdf/1710.00166v1.pdf | |
| PWC | https://paperswithcode.com/paper/pcanet-ii-when-pcanet-meets-the-second-order |
| Repo | |
| Framework | |
Collision Selective Visual Neural Network Inspired by LGMD2 Neurons in Juvenile Locusts
| Title | Collision Selective Visual Neural Network Inspired by LGMD2 Neurons in Juvenile Locusts |
| Authors | Qinbing Fu, Cheng Hu, Shigang Yue |
| Abstract | For autonomous robots in dynamic environments mixed with human, it is vital to detect impending collision quickly and robustly. The biological visual systems evolved over millions of years may provide us efficient solutions for collision detection in complex environments. In the cockpit of locusts, two Lobula Giant Movement Detectors, i.e. LGMD1 and LGMD2, have been identified which respond to looming objects rigorously with high firing rates. Compared to LGMD1, LGMD2 matures early in the juvenile locusts with specific selectivity to dark moving objects against bright background in depth while not responding to light objects embedded in dark background - a similar situation which ground vehicles and robots are facing with. However, little work has been done on modeling LGMD2, let alone its potential in robotics and other vision-based applications. In this article, we propose a novel way of modeling LGMD2 neuron, with biased ON and OFF pathways splitting visual streams into parallel channels encoding brightness increments and decrements separately to fulfill its selectivity. Moreover, we apply a biophysical mechanism of spike frequency adaptation to shape the looming selectivity in such a collision-detecting neuron model. The proposed visual neural network has been tested with systematic experiments, challenged against synthetic and real physical stimuli, as well as image streams from the sensor of a miniature robot. The results demonstrated this framework is able to detect looming dark objects embedded in bright backgrounds selectively, which make it ideal for ground mobile platforms. The robotic experiments also showed its robustness in collision detection - it performed well for near range navigation in an arena with many obstacles. Its enhanced collision selectivity to dark approaching objects versus receding and translating ones has also been verified via systematic experiments. |
| Tasks | |
| Published | 2017-12-22 |
| URL | http://arxiv.org/abs/1801.06452v1 |
| http://arxiv.org/pdf/1801.06452v1.pdf | |
| PWC | https://paperswithcode.com/paper/collision-selective-visual-neural-network |
| Repo | |
| Framework | |
Nonnegative matrix factorization with side information for time series recovery and prediction
| Title | Nonnegative matrix factorization with side information for time series recovery and prediction |
| Authors | Jiali Mei, Yohann De Castro, Yannig Goude, Jean-Marc Azaïs, Georges Hébrail |
| Abstract | Motivated by the reconstruction and the prediction of electricity consumption, we extend Nonnegative Matrix Factorization~(NMF) to take into account side information (column or row features). We consider general linear measurement settings, and propose a framework which models non-linear relationships between features and the response variables. We extend previous theoretical results to obtain a sufficient condition on the identifiability of the NMF in this setting. Based the classical Hierarchical Alternating Least Squares~(HALS) algorithm, we propose a new algorithm (HALSX, or Hierarchical Alternating Least Squares with eXogeneous variables) which estimates the factorization model. The algorithm is validated on both simulated and real electricity consumption datasets as well as a recommendation dataset, to show its performance in matrix recovery and prediction for new rows and columns. |
| Tasks | Time Series |
| Published | 2017-09-19 |
| URL | http://arxiv.org/abs/1709.06320v1 |
| http://arxiv.org/pdf/1709.06320v1.pdf | |
| PWC | https://paperswithcode.com/paper/nonnegative-matrix-factorization-with-side |
| Repo | |
| Framework | |
A simple genome-wide association study algorithm
| Title | A simple genome-wide association study algorithm |
| Authors | Lev V. Utkin, Irina L. Utkina |
| Abstract | A computationally simple genome-wide association study (GWAS) algorithm for estimating the main and epistatic effects of markers or single nucleotide polymorphisms (SNPs) is proposed. It is based on the intuitive assumption that changes of alleles corresponding to important SNPs in a pair of individuals lead to large difference of phenotype values of these individuals. The algorithm is based on considering pairs of individuals instead of SNPs or pairs of SNPs. The main advantage of the algorithm is that it weakly depends on the number of SNPs in a genotype matrix. It mainly depends on the number of individuals, which is typically very small in comparison with the number of SNPs. Numerical experiments with real data sets illustrate the proposed algorithm. |
| Tasks | |
| Published | 2017-08-05 |
| URL | http://arxiv.org/abs/1708.01746v1 |
| http://arxiv.org/pdf/1708.01746v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-simple-genome-wide-association-study |
| Repo | |
| Framework | |
Role Playing Learning for Socially Concomitant Mobile Robot Navigation
| Title | Role Playing Learning for Socially Concomitant Mobile Robot Navigation |
| Authors | Mingming Li, Rui Jiang, Shuzhi Sam Ge, Tong Heng Lee |
| Abstract | In this paper, we present the Role Playing Learning (RPL) scheme for a mobile robot to navigate socially with its human companion in populated environments. Neural networks (NN) are constructed to parameterize a stochastic policy that directly maps sensory data collected by the robot to its velocity outputs, while respecting a set of social norms. An efficient simulative learning environment is built with maps and pedestrians trajectories collected from a number of real-world crowd data sets. In each learning iteration, a robot equipped with the NN policy is created virtually in the learning environment to play itself as a companied pedestrian and navigate towards a goal in a socially concomitant manner. Thus, we call this process Role Playing Learning, which is formulated under a reinforcement learning (RL) framework. The NN policy is optimized end-to-end using Trust Region Policy Optimization (TRPO), with consideration of the imperfectness of robot’s sensor measurements. Simulative and experimental results are provided to demonstrate the efficacy and superiority of our method. |
| Tasks | Robot Navigation |
| Published | 2017-05-29 |
| URL | http://arxiv.org/abs/1705.10092v1 |
| http://arxiv.org/pdf/1705.10092v1.pdf | |
| PWC | https://paperswithcode.com/paper/role-playing-learning-for-socially |
| Repo | |
| Framework | |
Breaking the curse of dimensionality in regression
| Title | Breaking the curse of dimensionality in regression |
| Authors | Yinchu Zhu, Jelena Bradic |
| Abstract | Models with many signals, high-dimensional models, often impose structures on the signal strengths. The common assumption is that only a few signals are strong and most of the signals are zero or close (collectively) to zero. However, such a requirement might not be valid in many real-life applications. In this article, we are interested in conducting large-scale inference in models that might have signals of mixed strengths. The key challenge is that the signals that are not under testing might be collectively non-negligible (although individually small) and cannot be accurately learned. This article develops a new class of tests that arise from a moment matching formulation. A virtue of these moment-matching statistics is their ability to borrow strength across features, adapt to the sparsity size and exert adjustment for testing growing number of hypothesis. GRoup-level Inference of Parameter, GRIP, test harvests effective sparsity structures with hypothesis formulation for an efficient multiple testing procedure. Simulated data showcase that GRIPs error control is far better than the alternative methods. We develop a minimax theory, demonstrating optimality of GRIP for a broad range of models, including those where the model is a mixture of a sparse and high-dimensional dense signals. |
| Tasks | |
| Published | 2017-08-01 |
| URL | http://arxiv.org/abs/1708.00430v1 |
| http://arxiv.org/pdf/1708.00430v1.pdf | |
| PWC | https://paperswithcode.com/paper/breaking-the-curse-of-dimensionality-in |
| Repo | |
| Framework | |
Harnessing Cognitive Features for Sarcasm Detection
| Title | Harnessing Cognitive Features for Sarcasm Detection |
| Authors | Abhijit Mishra, Diptesh Kanojia, Seema Nagar, Kuntal Dey, Pushpak Bhattacharyya |
| Abstract | In this paper, we propose a novel mechanism for enriching the feature vector, for the task of sarcasm detection, with cognitive features extracted from eye-movement patterns of human readers. Sarcasm detection has been a challenging research problem, and its importance for NLP applications such as review summarization, dialog systems and sentiment analysis is well recognized. Sarcasm can often be traced to incongruity that becomes apparent as the full sentence unfolds. This presence of incongruity- implicit or explicit- affects the way readers eyes move through the text. We observe the difference in the behaviour of the eye, while reading sarcastic and non sarcastic sentences. Motivated by his observation, we augment traditional linguistic and stylistic features for sarcasm detection with the cognitive features obtained from readers eye movement data. We perform statistical classification using the enhanced feature set so obtained. The augmented cognitive features improve sarcasm detection by 3.7% (in terms of F-score), over the performance of the best reported system. |
| Tasks | Sarcasm Detection, Sentiment Analysis |
| Published | 2017-01-19 |
| URL | http://arxiv.org/abs/1701.05574v1 |
| http://arxiv.org/pdf/1701.05574v1.pdf | |
| PWC | https://paperswithcode.com/paper/harnessing-cognitive-features-for-sarcasm |
| Repo | |
| Framework | |
WiFi based trajectory alignment, calibration and easy site survey using smart phones and foot-mounted IMUs
| Title | WiFi based trajectory alignment, calibration and easy site survey using smart phones and foot-mounted IMUs |
| Authors | Yang Gu, Caifa Zhou, Andreas Wieser, Zhimin Zhou |
| Abstract | Foot-mounted inertial positioning (FMIP) can face problems of inertial drifts and unknown initial states in real applications, which renders the estimated trajectories inaccurate and not obtained in a well defined coordinate system for matching trajectories of different users. In this paper, an approach adopting received signal strength (RSS) measurements for Wifi access points (APs) are proposed to align and calibrate the trajectories estimated from foot mounted inertial measurement units (IMUs). A crowd-sourced radio map (RM) can be built subsequently and can be used for fingerprinting based Wifi indoor positioning (FWIP). The foundation of the proposed approach is graph based simultaneously localization and mapping (SLAM). The nodes in the graph denote users poses and the edges denote the pairwise constrains between the nodes. The constrains are derived from: (1) inertial estimated trajectories; (2) vicinity in the RSS space. With these constrains, an error functions is defined. By minimizing the error function, the graph is optimized and the aligned/calibrated trajectories along with the RM are acquired. The experimental results have corroborated the effectiveness of the approach for trajectory alignment, calibration as well as RM construction. |
| Tasks | Calibration |
| Published | 2017-06-02 |
| URL | http://arxiv.org/abs/1706.00636v1 |
| http://arxiv.org/pdf/1706.00636v1.pdf | |
| PWC | https://paperswithcode.com/paper/wifi-based-trajectory-alignment-calibration |
| Repo | |
| Framework | |
ClickBAIT: Click-based Accelerated Incremental Training of Convolutional Neural Networks
| Title | ClickBAIT: Click-based Accelerated Incremental Training of Convolutional Neural Networks |
| Authors | Ervin Teng, João Diogo Falcão, Bob Iannucci |
| Abstract | Today’s general-purpose deep convolutional neural networks (CNN) for image classification and object detection are trained offline on large static datasets. Some applications, however, will require training in real-time on live video streams with a human-in-the-loop. We refer to this class of problem as Time-ordered Online Training (ToOT) - these problems will require a consideration of not only the quantity of incoming training data, but the human effort required to tag and use it. In this paper, we define training benefit as a metric to measure the effectiveness of a sequence in using each user interaction. We demonstrate and evaluate a system tailored to performing ToOT in the field, capable of training an image classifier on a live video stream through minimal input from a human operator. We show that by exploiting the time-ordered nature of the video stream through optical flow-based object tracking, we can increase the effectiveness of human actions by about 8 times. |
| Tasks | Image Classification, Object Detection, Object Tracking, Optical Flow Estimation |
| Published | 2017-09-15 |
| URL | http://arxiv.org/abs/1709.05021v1 |
| http://arxiv.org/pdf/1709.05021v1.pdf | |
| PWC | https://paperswithcode.com/paper/clickbait-click-based-accelerated-incremental |
| Repo | |
| Framework | |
Deconvolutional Latent-Variable Model for Text Sequence Matching
| Title | Deconvolutional Latent-Variable Model for Text Sequence Matching |
| Authors | Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, Lawrence Carin |
| Abstract | A latent-variable model is introduced for text matching, inferring sentence representations by jointly optimizing generative and discriminative objectives. To alleviate typical optimization challenges in latent-variable models for text, we employ deconvolutional networks as the sequence decoder (generator), providing learned latent codes with more semantic information and better generalization. Our model, trained in an unsupervised manner, yields stronger empirical predictive performance than a decoder based on Long Short-Term Memory (LSTM), with less parameters and considerably faster training. Further, we apply it to text sequence-matching problems. The proposed model significantly outperforms several strong sentence-encoding baselines, especially in the semi-supervised setting. |
| Tasks | Latent Variable Models, Text Matching |
| Published | 2017-09-21 |
| URL | http://arxiv.org/abs/1709.07109v3 |
| http://arxiv.org/pdf/1709.07109v3.pdf | |
| PWC | https://paperswithcode.com/paper/deconvolutional-latent-variable-model-for |
| Repo | |
| Framework | |
A method for the online construction of the set of states of a Markov Decision Process using Answer Set Programming
| Title | A method for the online construction of the set of states of a Markov Decision Process using Answer Set Programming |
| Authors | Leonardo A. Ferreira, Reinaldo A. C. Bianchi, Paulo E. Santos, Ramon Lopez de Mantaras |
| Abstract | Non-stationary domains, that change in unpredicted ways, are a challenge for agents searching for optimal policies in sequential decision-making problems. This paper presents a combination of Markov Decision Processes (MDP) with Answer Set Programming (ASP), named {\em Online ASP for MDP} (oASP(MDP)), which is a method capable of constructing the set of domain states while the agent interacts with a changing environment. oASP(MDP) updates previously obtained policies, learnt by means of Reinforcement Learning (RL), using rules that represent the domain changes observed by the agent. These rules represent a set of domain constraints that are processed as ASP programs reducing the search space. Results show that oASP(MDP) is capable of finding solutions for problems in non-stationary domains without interfering with the action-value function approximation process. |
| Tasks | Decision Making |
| Published | 2017-06-05 |
| URL | http://arxiv.org/abs/1706.01417v1 |
| http://arxiv.org/pdf/1706.01417v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-method-for-the-online-construction-of-the |
| Repo | |
| Framework | |
A Hybrid ACO Algorithm for the Next Release Problem
| Title | A Hybrid ACO Algorithm for the Next Release Problem |
| Authors | He Jiang, Jingyuan Zhang, Jifeng Xuan, Zhilei Ren, Yan Hu |
| Abstract | In this paper, we propose a Hybrid Ant Colony Optimization algorithm (HACO) for Next Release Problem (NRP). NRP, a NP-hard problem in requirement engineering, is to balance customer requests, resource constraints, and requirement dependencies by requirement selection. Inspired by the successes of Ant Colony Optimization algorithms (ACO) for solving NP-hard problems, we design our HACO to approximately solve NRP. Similar to traditional ACO algorithms, multiple artificial ants are employed to construct new solutions. During the solution construction phase, both pheromone trails and neighborhood information will be taken to determine the choices of every ant. In addition, a local search (first found hill climbing) is incorporated into HACO to improve the solution quality. Extensively wide experiments on typical NRP test instances show that HACO outperforms the existing algorithms (GRASP and simulated annealing) in terms of both solution uality and running time. |
| Tasks | |
| Published | 2017-04-16 |
| URL | http://arxiv.org/abs/1704.04777v1 |
| http://arxiv.org/pdf/1704.04777v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-hybrid-aco-algorithm-for-the-next-release |
| Repo | |
| Framework | |
