January 29, 2020

3139 words 15 mins read

Paper Group ANR 743

Deep Learning for space-variant deconvolution in galaxy surveys. TF3P: Three-dimensional Force Fields Fingerprint Learned by Deep Capsular Network. PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices. Characterizing the invariances of learning algorithms using category theory. Apprenticeship Learning via Frank-Wolf …

Deep Learning for space-variant deconvolution in galaxy surveys


Title	Deep Learning for space-variant deconvolution in galaxy surveys
Authors	Florent Sureau, Alexis Lechat, Jean-Luc Starck
Abstract	Deconvolution of large survey images with millions of galaxies requires to develop a new generation of methods which can take into account a space variant Point Spread Function and have to be at the same time accurate and fast. We investigate in this paper how Deep Learning could be used to perform this task. We employ a U-NET Deep Neural Network architecture to learn in a supervised setting parameters adapted for galaxy image processing and study two strategies for deconvolution. The first approach is a post-processing of a mere Tikhonov deconvolution with closed form solution and the second one is an iterative deconvolution framework based on the Alternating Direction Method of Multipliers (ADMM). Our numerical results based on GREAT3 simulations with realistic galaxy images and PSFs show that our two approaches outperforms standard techniques based on convex optimization, whether assessed in galaxy image reconstruction or shape recovery. The approach based on Tikhonov deconvolution leads to the most accurate results except for ellipticity errors at high signal to noise ratio where the ADMM approach performs slightly better, is also more computation-time efficient to process a large number of galaxies, and is therefore recommended in this scenario.
Tasks	Image Reconstruction
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00443v1
PDF	https://arxiv.org/pdf/1911.00443v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-space-variant-deconvolution
Repo
Framework

TF3P: Three-dimensional Force Fields Fingerprint Learned by Deep Capsular Network


Title	TF3P: Three-dimensional Force Fields Fingerprint Learned by Deep Capsular Network
Authors	Yanxing Wang, Jianxing Hu, Junyong Lai, Yibo Li, Hongwei Jin, Lihe Zhang, Liangren Zhang, Zhenming Liu
Abstract	Molecular fingerprints are the workhorse in ligand-based drug discovery. In recent years, increasing number of research papers reported fascinating results on using deep neural networks to learn 2D molecular representations as fingerprints. One may anticipate that the integration of deep learning would also contribute to the prosperity of 3D fingerprints. Here, we presented a new 3D small molecule fingerprint, the three-dimensional force fields fingerprint (TF3P), learned by deep capsular network whose training is in no need of labeled dataset for specific predictive tasks. TF3P can encode the 3D force fields information of molecules and demonstrates its stronger ability to capture 3D structural changes, recognize molecules alike in 3D but not in 2D, and recognize similar targets inaccessible by other fingerprints, including the solely existing 3D fingerprint E3FP, based on only ligands similarity. Furthermore, TF3P is compatible with both statistical models (e.g. similarity ensemble approach) and machine learning models. Altogether, we report TF3P as a new 3D small molecule fingerprint with promising future in ligand-based drug discovery.
Tasks	Drug Discovery
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11430v2
PDF	https://arxiv.org/pdf/1912.11430v2.pdf
PWC	https://paperswithcode.com/paper/tf3p-three-dimensional-force-fields
Repo
Framework

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices


Title	PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices
Authors	Delia Velasco-Montero, Jorge Fernández-Berni, Ricardo Carmona-Galán, Ángel Rodríguez-Vázquez
Abstract	This paper presents PreVIous, a methodology to predict the performance of convolutional neural networks (CNNs) in terms of throughput and energy consumption on vision-enabled devices for the Internet of Things. CNNs typically constitute a massive computational load for such devices, which are characterized by scarce hardware resources to be shared among multiple concurrent tasks. Therefore, it is critical to select the optimal CNN architecture for a particular hardware platform according to prescribed application requirements. However, the zoo of CNN models is already vast and rapidly growing. To facilitate a suitable selection, we introduce a prediction framework that allows to evaluate the performance of CNNs prior to their actual implementation. The proposed methodology is based on PreVIousNet, a neural network specifically designed to build accurate per-layer performance predictive models. PreVIousNet incorporates the most usual parameters found in state-of-the-art network architectures. The resulting predictive models for inference time and energy have been tested against comprehensive characterizations of seven well-known CNN models running on two different software frameworks and two different embedded platforms. To the best of our knowledge, this is the most extensive study in the literature concerning CNN performance prediction on low-power low-cost devices. The average deviation between predictions and real measurements is remarkably low, ranging from 3% to 10%. This means state-of-the-art modeling accuracy. As an additional asset, the fine-grained a priori analysis provided by PreVIous could also be exploited by neural architecture search engines.
Tasks	Neural Architecture Search
Published	2019-12-13
URL	https://arxiv.org/abs/1912.06442v2
PDF	https://arxiv.org/pdf/1912.06442v2.pdf
PWC	https://paperswithcode.com/paper/previous-a-methodology-for-prediction-of
Repo
Framework

Characterizing the invariances of learning algorithms using category theory


Title	Characterizing the invariances of learning algorithms using category theory
Authors	Kenneth D. Harris
Abstract	Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are formalized by categories of predictor and target spaces, whose morphisms represent the algorithm’s invariances, and an index category whose morphisms represent permutations of the training examples. An invariant learning algorithm is a natural transformation between two functors from the product of these categories to the category of sets, representing training datasets and learned functions respectively. We illustrate the framework by characterizing and contrasting the invariances of linear regression and ridge regression.
Tasks
Published	2019-05-06
URL	https://arxiv.org/abs/1905.02072v1
PDF	https://arxiv.org/pdf/1905.02072v1.pdf
PWC	https://paperswithcode.com/paper/characterizing-the-invariances-of-learning
Repo
Framework

Apprenticeship Learning via Frank-Wolfe


Title	Apprenticeship Learning via Frank-Wolfe
Authors	Tom Zahavy, Alon Cohen, Haim Kaplan, Yishay Mansour
Abstract	We consider the applications of the Frank-Wolfe (FW) algorithm for Apprenticeship Learning (AL). In this setting, we are given a Markov Decision Process (MDP) without an explicit reward function. Instead, we observe an expert that acts according to some policy, and the goal is to find a policy whose feature expectations are closest to those of the expert policy. We formulate this problem as finding the projection of the feature expectations of the expert on the feature expectations polytope – the convex hull of the feature expectations of all the deterministic policies in the MDP. We show that this formulation is equivalent to the AL objective and that solving this problem using the FW algorithm is equivalent well-known Projection method of Abbeel and Ng (2004). This insight allows us to analyze AL with tools from convex optimization literature and derive tighter convergence bounds on AL. Specifically, we show that a variation of the FW method that is based on taking “away steps” achieves a linear rate of convergence when applied to AL and that a stochastic version of the FW algorithm can be used to avoid precise estimation of feature expectations. We also experimentally show that this version outperforms the FW baseline. To the best of our knowledge, this is the first work that shows linear convergence rates for AL.
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01679v2
PDF	https://arxiv.org/pdf/1911.01679v2.pdf
PWC	https://paperswithcode.com/paper/apprenticeship-learning-via-frank-wolfe
Repo
Framework

Follow the Leader: Documents on the Leading Edge of Semantic Change Get More Citations


Title	Follow the Leader: Documents on the Leading Edge of Semantic Change Get More Citations
Authors	Sandeep Soni, Kristina Lerman, Jacob Eisenstein
Abstract	Diachronic word embeddings offer remarkable insights into the evolution of language and provide a tool for quantifying socio-cultural change. However, while this method identifies words that have semantically shifted, it studies them in isolation; it does not facilitate the discovery of documents that lead or lag with respect to specific semantic innovations. In this paper, we propose a method to quantify the degree of semantic progressiveness in each usage. These usages can be aggregated to obtain scores for each document. We analyze two large collections of documents, representing legal opinions and scientific articles. Documents that are predicted to be semantically progressive receive a larger number of citations, indicating that they are especially influential. Our work thus provides a new technique for identifying lexical semantic leaders and demonstrates a new link between early adoption and influence in a citation network.
Tasks	Word Embeddings
Published	2019-09-09
URL	https://arxiv.org/abs/1909.04189v1
PDF	https://arxiv.org/pdf/1909.04189v1.pdf
PWC	https://paperswithcode.com/paper/follow-the-leader-documents-on-the-leading
Repo
Framework

Lenses and Learners


Title	Lenses and Learners
Authors	Brendan Fong, Michael Johnson
Abstract	Lenses are a well-established structure for modelling bidirectional transformations, such as the interactions between a database and a view of it. Lenses may be symmetric or asymmetric, and may be composed, forming the morphisms of a monoidal category. More recently, the notion of a learner has been proposed: these provide a compositional way of modelling supervised learning algorithms, and again form the morphisms of a monoidal category. In this paper, we show that the two concepts are tightly linked. We show both that there is a faithful, identity-on-objects symmetric monoidal functor embedding a category of asymmetric lenses into the category of learners, and furthermore there is such a functor embedding the category of learners into a category of symmetric lenses.
Tasks
Published	2019-03-05
URL	http://arxiv.org/abs/1903.03671v2
PDF	http://arxiv.org/pdf/1903.03671v2.pdf
PWC	https://paperswithcode.com/paper/lenses-and-learners
Repo
Framework

Learning Interpretable Disentangled Representations using Adversarial VAEs


Title	Learning Interpretable Disentangled Representations using Adversarial VAEs
Authors	Mhd Hasan Sarhan, Abouzar Eslami, Nassir Navab, Shadi Albarqouni
Abstract	Learning Interpretable representation in medical applications is becoming essential for adopting data-driven models into clinical practice. It has been recently shown that learning a disentangled feature representation is important for a more compact and explainable representation of the data. In this paper, we introduce a novel adversarial variational autoencoder with a total correlation constraint to enforce independence on the latent representation while preserving the reconstruction fidelity. Our proposed method is validated on a publicly available dataset showing that the learned disentangled representation is not only interpretable, but also superior to the state-of-the-art methods. We report a relative improvement of 81.50% in terms of disentanglement, 11.60% in clustering, and 2% in supervised classification with a few amounts of labeled data.
Tasks
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08491v1
PDF	http://arxiv.org/pdf/1904.08491v1.pdf
PWC	https://paperswithcode.com/paper/learning-interpretable-disentangled
Repo
Framework

Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models


Title	Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models
Authors	Victor Prokhorov, Mohammad Taher Pilehvar, Nigel Collier
Abstract	We present a novel method for mapping unrestricted text to knowledge graph entities by framing the task as a sequence-to-sequence problem. Specifically, given the encoded state of an input text, our decoder directly predicts paths in the knowledge graph, starting from the root and ending at the target node following hypernym-hyponym relationships. In this way, and in contrast to other text-to-entity mapping systems, our model outputs hierarchically structured predictions that are fully interpretable in the context of the underlying ontology, in an end-to-end manner. We present a proof-of-concept experiment with encouraging results, comparable to those of state-of-the-art systems.
Tasks
Published	2019-04-05
URL	http://arxiv.org/abs/1904.02996v1
PDF	http://arxiv.org/pdf/1904.02996v1.pdf
PWC	https://paperswithcode.com/paper/generating-knowledge-graph-paths-from-textual
Repo
Framework

Adaptive Sampling for Estimating Multiple Probability Distributions


Title	Adaptive Sampling for Estimating Multiple Probability Distributions
Authors	Shubhanshu Shekhar, Tara Javidi, Mohammad Ghavamzadeh
Abstract	We consider the problem of allocating samples to a finite set of discrete distributions in order to learn them uniformly well in terms of four common distance measures: $\ell_2^2$, $\ell_1$, $f$-divergence, and separation distance. To present a unified treatment of these distances, we first propose a general optimistic tracking algorithm and analyze its sample allocation performance w.r.t.~an oracle. We then instantiate this algorithm for the four distance measures and derive bounds on the regret of their resulting allocation schemes. We verify our theoretical findings through some experiments. Finally, we show that the techniques developed in the paper can be easily extended to the related setting of minimizing the average error (in terms of the four distances) in learning a set of distributions.
Tasks
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12406v2
PDF	https://arxiv.org/pdf/1910.12406v2.pdf
PWC	https://paperswithcode.com/paper/adaptive-sampling-for-estimating-multiple
Repo
Framework

VRLS: A Unified Reinforcement Learning Scheduler for Vehicle-to-Vehicle Communications


Title	VRLS: A Unified Reinforcement Learning Scheduler for Vehicle-to-Vehicle Communications
Authors	Taylan Şahin, Ramin Khalili, Mate Boban, Adam Wolisz
Abstract	Vehicle-to-vehicle (V2V) communications have distinct challenges that need to be taken into account when scheduling the radio resources. Although centralized schedulers (e.g., located on base stations) could be utilized to deliver high scheduling performance, they cannot be employed in case of coverage gaps. To address the issue of reliable scheduling of V2V transmissions out of coverage, we propose Vehicular Reinforcement Learning Scheduler (VRLS), a centralized scheduler that predictively assigns the resources for V2V communication while the vehicle is still in cellular network coverage. VRLS is a unified reinforcement learning (RL) solution, wherein the learning agent, the state representation, and the reward provided to the agent are applicable to different vehicular environments of interest (in terms of vehicular density, resource configuration, and wireless channel conditions). Such a unified solution eliminates the necessity of redesigning the RL components for a different environment, and facilitates transfer learning from one to another similar environment. We evaluate the performance of VRLS and show its ability to avoid collisions and half-duplex errors, and to reuse the resources better than the state of the art scheduling algorithms. We also show that pre-trained VRLS agent can adapt to different V2V environments with limited retraining, thus enabling real-world deployment in different scenarios.
Tasks	Transfer Learning
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09319v1
PDF	https://arxiv.org/pdf/1907.09319v1.pdf
PWC	https://paperswithcode.com/paper/vrls-a-unified-reinforcement-learning
Repo
Framework

A Generative Approach Towards Improved Robotic Detection of Marine Litter


Title	A Generative Approach Towards Improved Robotic Detection of Marine Litter
Authors	Jungseok Hong, Michael Fulton, Junaed Sattar
Abstract	This paper presents an approach to address data scarcity problems in underwater image datasets for visual detection of marine debris. The proposed approach relies on a two-stage variational autoencoder (VAE) and a binary classifier to evaluate the generated imagery for quality and realism. From the images generated by the two-stage VAE, the binary classifier selects “good quality” images and augments the given dataset with them. Lastly, a multi-class classifier is used to evaluate the impact of the augmentation process by measuring the accuracy of an object detector trained on combinations of real and generated trash images. Our results show that the classifier trained with the augmented data outperforms the one trained only with the real data. This approach will not only be valid for the underwater trash classification problem presented in this paper, but it will also be useful for any data-dependent task for which collecting more images is challenging or infeasible.
Tasks
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04754v1
PDF	https://arxiv.org/pdf/1910.04754v1.pdf
PWC	https://paperswithcode.com/paper/a-generative-approach-towards-improved
Repo
Framework

Lower Memory Oblivious (Tensor) Subspace Embeddings with Fewer Random Bits: Modewise Methods for Least Squares


Title	Lower Memory Oblivious (Tensor) Subspace Embeddings with Fewer Random Bits: Modewise Methods for Least Squares
Authors	M. A. Iwen, D. Needell, E. Rebrova, A. Zare
Abstract	In this paper new general modewise Johnson-Lindenstrauss (JL) subspace embeddings are proposed that are both considerably faster to generate and easier to store than traditional JL embeddings when working with extremely large vectors and/or tensors. Corresponding embedding results are then proven for two different types of low-dimensional (tensor) subspaces. The first of these new subspace embedding results produces improved space complexity bounds for embeddings of rank-$r$ tensors whose CP decompositions are contained in the span of a fixed (but unknown) set of $r$ rank-one basis tensors. In the traditional vector setting this first result yields new and very general near-optimal oblivious subspace embedding constructions that require fewer random bits to generate than standard JL embeddings when embedding subspaces of $\mathbb{C}^N$ spanned by basis vectors with special Kronecker structure. The second result proven herein provides new fast JL embeddings of arbitrary $r$-dimensional subspaces $\mathcal{S} \subset \mathbb{C}^N$ which also require fewer random bits (and so are easier to store - i.e., require less space) than standard fast JL embedding methods in order to achieve small $\epsilon$-distortions. These new oblivious subspace embedding results work by $(i)$ effectively folding any given vector in $\mathcal{S}$ into a (not necessarily low-rank) tensor, and then $(ii)$ embedding the resulting tensor into $\mathbb{C}^m$ for $m \leq C r \log^c(N) / \epsilon^2$. Applications related to compression and fast compressed least squares solution methods are also considered, including those used for fitting low-rank CP decompositions, and the proposed JL embedding results are shown to work well numerically in both settings.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08294v1
PDF	https://arxiv.org/pdf/1912.08294v1.pdf
PWC	https://paperswithcode.com/paper/lower-memory-oblivious-tensor-subspace
Repo
Framework

Truly Batch Apprenticeship Learning with Deep Successor Features


Title	Truly Batch Apprenticeship Learning with Deep Successor Features
Authors	Donghun Lee, Srivatsan Srinivasan, Finale Doshi-Velez
Abstract	We introduce a novel apprenticeship learning algorithm to learn an expert’s underlying reward structure in off-policy model-free \emph{batch} settings. Unlike existing methods that require a dynamics model or additional data acquisition for on-policy evaluation, our algorithm requires only the batch data of observed expert behavior. Such settings are common in real-world tasks—health care, finance or industrial processes —where accurate simulators do not exist or data acquisition is costly. To address challenges in batch settings, we introduce Deep Successor Feature Networks(DSFN) that estimate feature expectations in an off-policy setting and a transition-regularized imitation network that produces a near-expert initial policy and an efficient feature representation. Our algorithm achieves superior results in batch settings on both control benchmarks and a vital clinical task of sepsis management in the Intensive Care Unit.
Tasks
Published	2019-03-24
URL	http://arxiv.org/abs/1903.10077v1
PDF	http://arxiv.org/pdf/1903.10077v1.pdf
PWC	https://paperswithcode.com/paper/truly-batch-apprenticeship-learning-with-deep
Repo
Framework

Convex Relaxations for Consensus and Non-Minimal Problems in 3D Vision


Title	Convex Relaxations for Consensus and Non-Minimal Problems in 3D Vision
Authors	Thomas Probst, Danda Pani Paudel, Ajad Chhatkuli, Luc Van Gool
Abstract	In this paper, we formulate a generic non-minimal solver using the existing tools of Polynomials Optimization Problems (POP) from computational algebraic geometry. The proposed method exploits the well known Shor’s or Lasserre’s relaxations, whose theoretical aspects are also discussed. Notably, we further exploit the POP formulation of non-minimal solver also for the generic consensus maximization problems in 3D vision. Our framework is simple and straightforward to implement, which is also supported by three diverse applications in 3D vision, namely rigid body transformation estimation, Non-Rigid Structure-from-Motion (NRSfM), and camera autocalibration. In all three cases, both non-minimal and consensus maximization are tested, which are also compared against the state-of-the-art methods. Our results are competitive to the compared methods, and are also coherent with our theoretical analysis. The main contribution of this paper is the claim that a good approximate solution for many polynomial problems involved in 3D vision can be obtained using the existing theory of numerical computational algebra. This claim leads us to reason about why many relaxed methods in 3D vision behave so well? And also allows us to offer a generic relaxed solver in a rather straightforward way. We further show that the convex relaxation of these polynomials can easily be used for maximizing consensus in a deterministic manner. We support our claim using several experiments for aforementioned three diverse problems in 3D vision.
Tasks
Published	2019-09-26
URL	https://arxiv.org/abs/1909.12034v1
PDF	https://arxiv.org/pdf/1909.12034v1.pdf
PWC	https://paperswithcode.com/paper/convex-relaxations-for-consensus-and-non
Repo
Framework