July 28, 2019

3097 words 15 mins read

Paper Group ANR 202

When Is the First Spurious Variable Selected by Sequential Regression Procedures?. Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data. Solving the Goddard problem by an influence diagram. Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks. An Investigation into the Pe …

When Is the First Spurious Variable Selected by Sequential Regression Procedures?


Title	When Is the First Spurious Variable Selected by Sequential Regression Procedures?
Authors	Weijie J. Su
Abstract	Applied statisticians use sequential regression procedures to produce a ranking of explanatory variables and, in settings of low correlations between variables and strong true effect sizes, expect that variables at the very top of this ranking are truly relevant to the response. In a regime of certain sparsity levels, however, three examples of sequential procedures–forward stepwise, the lasso, and least angle regression–are shown to include the first spurious variable unexpectedly early. We derive a rigorous, sharp prediction of the rank of the first spurious variable for these three procedures, demonstrating that the first spurious variable occurs earlier and earlier as the regression coefficients become denser. This counterintuitive phenomenon persists for statistically independent Gaussian random designs and an arbitrarily large magnitude of the true effects. We gain a better understanding of the phenomenon by identifying the underlying cause and then leverage the insights to introduce a simple visualization tool termed the double-ranking diagram to improve on sequential methods. As a byproduct of these findings, we obtain the first provable result certifying the exact equivalence between the lasso and least angle regression in the early stages of solution paths beyond orthogonal designs. This equivalence can seamlessly carry over many important model selection results concerning the lasso to least angle regression.
Tasks	Model Selection
Published	2017-08-10
URL	http://arxiv.org/abs/1708.03046v2
PDF	http://arxiv.org/pdf/1708.03046v2.pdf
PWC	https://paperswithcode.com/paper/when-is-the-first-spurious-variable-selected
Repo
Framework

Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data


Title	Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data
Authors	Steven Diamond, Vincent Sitzmann, Stephen Boyd, Gordon Wetzstein, Felix Heide
Abstract	Real-world sensors suffer from noise, blur, and other imperfections that make high-level computer vision tasks like scene segmentation, tracking, and scene understanding difficult. Making high-level computer vision networks robust is imperative for real-world applications like autonomous driving, robotics, and surveillance. We propose a novel end-to-end differentiable architecture for joint denoising, deblurring, and classification that makes classification robust to realistic noise and blur. The proposed architecture dramatically improves the accuracy of a classification network in low light and other challenging conditions, outperforming alternative approaches such as retraining the network on noisy and blurry images and preprocessing raw sensor inputs with conventional denoising and deblurring algorithms. The architecture learns denoising and deblurring pipelines optimized for classification whose outputs differ markedly from those of state-of-the-art denoising and deblurring methods, preserving fine detail at the cost of more noise and artifacts. Our results suggest that the best low-level image processing for computer vision is different from existing algorithms designed to produce visually pleasing images. The principles used to design the proposed architecture easily extend to other high-level computer vision tasks and image formation models, providing a general framework for integrating low-level and high-level image processing.
Tasks	Autonomous Driving, Deblurring, Denoising, Image Classification, Scene Segmentation, Scene Understanding
Published	2017-01-23
URL	http://arxiv.org/abs/1701.06487v1
PDF	http://arxiv.org/pdf/1701.06487v1.pdf
PWC	https://paperswithcode.com/paper/dirty-pixels-optimizing-image-classification
Repo
Framework

Solving the Goddard problem by an influence diagram


Title	Solving the Goddard problem by an influence diagram
Authors	Jiří Vomlel, Václav Kratochvíl
Abstract	Influence diagrams are a decision-theoretic extension of probabilistic graphical models. In this paper we show how they can be used to solve the Goddard problem. We present results of numerical experiments with this problem and compare the solutions provided by influence diagrams with the optimal solution.
Tasks
Published	2017-03-18
URL	http://arxiv.org/abs/1703.06321v2
PDF	http://arxiv.org/pdf/1703.06321v2.pdf
PWC	https://paperswithcode.com/paper/solving-the-goddard-problem-by-an-influence
Repo
Framework

Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks


Title	Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks
Authors	Pratik Chaudhari, Stefano Soatto
Abstract	Stochastic gradient descent (SGD) is widely believed to perform implicit regularization when used to train deep neural networks, but the precise manner in which this occurs has thus far been elusive. We prove that SGD minimizes an average potential over the posterior distribution of weights along with an entropic regularization term. This potential is however not the original loss function in general. So SGD does perform variational inference, but for a different loss than the one used to compute the gradients. Even more surprisingly, SGD does not even converge in the classical sense: we show that the most likely trajectories of SGD for deep networks do not behave like Brownian motion around critical points. Instead, they resemble closed loops with deterministic components. We prove that such “out-of-equilibrium” behavior is a consequence of highly non-isotropic gradient noise in SGD; the covariance matrix of mini-batch gradients for deep networks has a rank as small as 1% of its dimension. We provide extensive empirical validation of these claims, proven in the appendix.
Tasks
Published	2017-10-30
URL	http://arxiv.org/abs/1710.11029v2
PDF	http://arxiv.org/pdf/1710.11029v2.pdf
PWC	https://paperswithcode.com/paper/stochastic-gradient-descent-performs
Repo
Framework

An Investigation into the Pedagogical Features of Documents


Title	An Investigation into the Pedagogical Features of Documents
Authors	Emily Sheng, Prem Natarajan, Jonathan Gordon, Gully Burns
Abstract	Characterizing the content of a technical document in terms of its learning utility can be useful for applications related to education, such as generating reading lists from large collections of documents. We refer to this learning utility as the “pedagogical value” of the document to the learner. While pedagogical value is an important concept that has been studied extensively within the education domain, there has been little work exploring it from a computational, i.e., natural language processing (NLP), perspective. To allow a computational exploration of this concept, we introduce the notion of “pedagogical roles” of documents (e.g., Tutorial and Survey) as an intermediary component for the study of pedagogical value. Given the lack of available corpora for our exploration, we create the first annotated corpus of pedagogical roles and use it to test baseline techniques for automatic prediction of such roles.
Tasks
Published	2017-08-01
URL	http://arxiv.org/abs/1708.00179v1
PDF	http://arxiv.org/pdf/1708.00179v1.pdf
PWC	https://paperswithcode.com/paper/an-investigation-into-the-pedagogical
Repo
Framework

Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks


Title	Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks
Authors	Sayeh Sharify, Alberto Delmas Lascorz, Kevin Siu, Patrick Judd, Andreas Moshovos
Abstract	Loom (LM), a hardware inference accelerator for Convolutional Neural Networks (CNNs) is presented. In LM every bit of data precision that can be saved translates to proportional performance gains. Specifically, for convolutional layers LM’s execution time scales inversely proportionally with the precisions of both weights and activations. For fully-connected layers LM’s performance scales inversely proportionally with the precision of the weights. LM targets area- and bandwidth-constrained System-on-a-Chip designs such as those found on mobile devices that cannot afford the multi-megabyte buffers that would be needed to store each layer on-chip. Accordingly, given a data bandwidth budget, LM boosts energy efficiency and performance over an equivalent bit-parallel accelerator. For both weights and activations LM can exploit profile-derived perlayer precisions. However, at runtime LM further trims activation precisions at a much smaller than a layer granularity. Moreover, it can naturally exploit weight precision variability at a smaller granularity than a layer. On average, across several image classification CNNs and for a configuration that can perform the equivalent of 128 16b x 16b multiply-accumulate operations per cycle LM outperforms a state-of-the-art bit-parallel accelerator [1] by 4.38x without any loss in accuracy while being 3.54x more energy efficient. LM can trade-off accuracy for additional improvements in execution performance and energy efficiency and compares favorably to an accelerator that targeted only activation precisions. We also study 2- and 4-bit LM variants and find the the 2-bit per cycle variant is the most energy efficient.
Tasks	Image Classification
Published	2017-06-23
URL	http://arxiv.org/abs/1706.07853v2
PDF	http://arxiv.org/pdf/1706.07853v2.pdf
PWC	https://paperswithcode.com/paper/loom-exploiting-weight-and-activation
Repo
Framework

Accurate Optical Flow via Direct Cost Volume Processing


Title	Accurate Optical Flow via Direct Cost Volume Processing
Authors	Jia Xu, René Ranftl, Vladlen Koltun
Abstract	We present an optical flow estimation approach that operates on the full four-dimensional cost volume. This direct approach shares the structural benefits of leading stereo matching pipelines, which are known to yield high accuracy. To this day, such approaches have been considered impractical due to the size of the cost volume. We show that the full four-dimensional cost volume can be constructed in a fraction of a second due to its regularity. We then exploit this regularity further by adapting semi-global matching to the four-dimensional setting. This yields a pipeline that achieves significantly higher accuracy than state-of-the-art optical flow methods while being faster than most. Our approach outperforms all published general-purpose optical flow methods on both Sintel and KITTI 2015 benchmarks.
Tasks	Optical Flow Estimation, Stereo Matching, Stereo Matching Hand
Published	2017-04-24
URL	http://arxiv.org/abs/1704.07325v1
PDF	http://arxiv.org/pdf/1704.07325v1.pdf
PWC	https://paperswithcode.com/paper/accurate-optical-flow-via-direct-cost-volume
Repo
Framework

Leaf Identification Using a Deep Convolutional Neural Network


Title	Leaf Identification Using a Deep Convolutional Neural Network
Authors	Christoph Wick, Frank Puppe
Abstract	Convolutional neural networks (CNNs) have become popular especially in computer vision in the last few years because they achieved outstanding performance on different tasks, such as image classifications. We propose a nine-layer CNN for leaf identification using the famous Flavia and Foliage datasets. Usually the supervised learning of deep CNNs requires huge datasets for training. However, the used datasets contain only a few examples per plant species. Therefore, we apply data augmentation and transfer learning to prevent our network from overfitting. The trained CNNs achieve recognition rates above 99% on the Flavia and Foliage datasets, and slightly outperform current methods for leaf classification.
Tasks	Data Augmentation, Transfer Learning
Published	2017-12-04
URL	http://arxiv.org/abs/1712.00967v1
PDF	http://arxiv.org/pdf/1712.00967v1.pdf
PWC	https://paperswithcode.com/paper/leaf-identification-using-a-deep
Repo
Framework

Learning to Navigate by Growing Deep Networks


Title	Learning to Navigate by Growing Deep Networks
Authors	Thushan Ganegedara, Lionel Ott, Fabio Ramos
Abstract	Adaptability is central to autonomy. Intuitively, for high-dimensional learning problems such as navigating based on vision, internal models with higher complexity allow to accurately encode the information available. However, most learning methods rely on models with a fixed structure and complexity. In this paper, we present a self-supervised framework for robots to learn to navigate, without any prior knowledge of the environment, by incrementally building the structure of a deep network as new data becomes available. Our framework captures images from a monocular camera and self labels the images to continuously train and predict actions from a computationally efficient adaptive deep architecture based on Autoencoders (AE), in a self-supervised fashion. The deep architecture, named Reinforced Adaptive Denoising Autoencoders (RA-DAE), uses reinforcement learning to dynamically change the network structure by adding or removing neurons. Experiments were conducted in simulation and real-world indoor and outdoor environments to assess the potential of self-supervised navigation. RA-DAE demonstrates better performance than equivalent non-adaptive deep learning alternatives and can continue to expand its knowledge, trading-off past and present information.
Tasks	Denoising
Published	2017-12-14
URL	http://arxiv.org/abs/1712.05084v1
PDF	http://arxiv.org/pdf/1712.05084v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-navigate-by-growing-deep-networks
Repo
Framework

Evolutionary Image Composition Using Feature Covariance Matrices


Title	Evolutionary Image Composition Using Feature Covariance Matrices
Authors	Aneta Neumann, Zygmunt L. Szpak, Wojciech Chojnacki, Frank Neumann
Abstract	Evolutionary algorithms have recently been used to create a wide range of artistic work. In this paper, we propose a new approach for the composition of new images from existing ones, that retain some salient features of the original images. We introduce evolutionary algorithms that create new images based on a fitness function that incorporates feature covariance matrices associated with different parts of the images. This approach is very flexible in that it can work with a wide range of features and enables targeting specific regions in the images. For the creation of the new images, we propose a population-based evolutionary algorithm with mutation and crossover operators based on random walks. Our experimental results reveal a spectrum of aesthetically pleasing images that can be obtained with the aid of our evolutionary process.
Tasks
Published	2017-03-10
URL	http://arxiv.org/abs/1703.03773v1
PDF	http://arxiv.org/pdf/1703.03773v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-image-composition-using-feature
Repo
Framework

The Conference Paper Assignment Problem: Using Order Weighted Averages to Assign Indivisible Goods


Title	The Conference Paper Assignment Problem: Using Order Weighted Averages to Assign Indivisible Goods
Authors	Jing Wu Lian, Nicholas Mattei, Renee Noble, Toby Walsh
Abstract	Motivated by the common academic problem of allocating papers to referees for conference reviewing we propose a novel mechanism for solving the assignment problem when we have a two sided matching problem with preferences from one side (the agents/reviewers) over the other side (the objects/papers) and both sides have capacity constraints. The assignment problem is a fundamental problem in both computer science and economics with application in many areas including task and resource allocation. We draw inspiration from multi-criteria decision making and voting and use order weighted averages (OWAs) to propose a novel and flexible class of algorithms for the assignment problem. We show an algorithm for finding a $\Sigma$-OWA assignment in polynomial time, in contrast to the NP-hardness of finding an egalitarian assignment. Inspired by this setting we observe an interesting connection between our model and the classic proportional multi-winner election problem in social choice.
Tasks	Decision Making
Published	2017-05-19
URL	http://arxiv.org/abs/1705.06840v1
PDF	http://arxiv.org/pdf/1705.06840v1.pdf
PWC	https://paperswithcode.com/paper/the-conference-paper-assignment-problem-using
Repo
Framework

Face Translation between Images and Videos using Identity-aware CycleGAN


Title	Face Translation between Images and Videos using Identity-aware CycleGAN
Authors	Zhiwu Huang, Bernhard Kratzwald, Danda Pani Paudel, Jiqing Wu, Luc Van Gool
Abstract	This paper presents a new problem of unpaired face translation between images and videos, which can be applied to facial video prediction and enhancement. In this problem there exist two major technical challenges: 1) designing a robust translation model between static images and dynamic videos, and 2) preserving facial identity during image-video translation. To address such two problems, we generalize the state-of-the-art image-to-image translation network (Cycle-Consistent Adversarial Networks) to the image-to-video/video-to-image translation context by exploiting a image-video translation model and an identity preservation model. In particular, we apply the state-of-the-art Wasserstein GAN technique to the setting of image-video translation for better convergence, and we meanwhile introduce a face verificator to ensure the identity. Experiments on standard image/video face datasets demonstrate the effectiveness of the proposed model in both terms of qualitative and quantitative evaluations.
Tasks	Image-to-Image Translation, Video Prediction
Published	2017-12-04
URL	http://arxiv.org/abs/1712.00971v1
PDF	http://arxiv.org/pdf/1712.00971v1.pdf
PWC	https://paperswithcode.com/paper/face-translation-between-images-and-videos
Repo
Framework

Preparing for the Unknown: Learning a Universal Policy with Online System Identification


Title	Preparing for the Unknown: Learning a Universal Policy with Online System Identification
Authors	Wenhao Yu, Jie Tan, C. Karen Liu, Greg Turk
Abstract	We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots’ components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters mu. The value of mu from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.
Tasks
Published	2017-02-08
URL	http://arxiv.org/abs/1702.02453v3
PDF	http://arxiv.org/pdf/1702.02453v3.pdf
PWC	https://paperswithcode.com/paper/preparing-for-the-unknown-learning-a
Repo
Framework

Defining a Lingua Franca to Open the Black Box of a Naïve Bayes Recommender


Title	Defining a Lingua Franca to Open the Black Box of a Naïve Bayes Recommender
Authors	Kenneth L. Hess, Hugo D. Paz
Abstract	Many AI systems have a black box nature that makes it difficult to understand how they make their recommendations. This can be unsettling, as the designer cannot be certain how the system will respond to novelty. To penetrate our Na"ive Bayes recommender’s black box, we first asked, what do we want to know from our system, and how can it be obtained? The answers led us to recursively define a common lexicon with the AI, a lingua franca, using the very items that the system ranks to create meta-symbols recognized by the system, and enabling us to understand the system’s knowledge in plain terms and at different levels of abstraction. As one bonus, using its existing knowledge, the lingua franca can enable the system to extend recommendations to related, but entirely new areas, ameliorating the cold start problem. We also supplement the lingua franca with techniques for visualizing the system’s knowledge state, develop metrics for evaluating the meaningfulness of terms in the lingua franca, and generalize the requirements for developing a similar lingua franca in other applications.
Tasks
Published	2017-09-21
URL	http://arxiv.org/abs/1709.07528v1
PDF	http://arxiv.org/pdf/1709.07528v1.pdf
PWC	https://paperswithcode.com/paper/defining-a-lingua-franca-to-open-the-black
Repo
Framework

Application of a Shallow Neural Network to Short-Term Stock Trading


Title	Application of a Shallow Neural Network to Short-Term Stock Trading
Authors	Abhinav Madahar, Yuze Ma, Kunal Patel
Abstract	Machine learning is increasingly prevalent in stock market trading. Though neural networks have seen success in computer vision and natural language processing, they have not been as useful in stock market trading. To demonstrate the applicability of a neural network in stock trading, we made a single-layer neural network that recommends buying or selling shares of a stock by comparing the highest high of 10 consecutive days with that of the next 10 days, a process repeated for the stock’s year-long historical data. A chi-squared analysis found that the neural network can accurately and appropriately decide whether to buy or sell shares for a given stock, showing that a neural network can make simple decisions about the stock market.
Tasks
Published	2017-03-30
URL	http://arxiv.org/abs/1703.10458v1
PDF	http://arxiv.org/pdf/1703.10458v1.pdf
PWC	https://paperswithcode.com/paper/application-of-a-shallow-neural-network-to
Repo
Framework