October 17, 2019

2969 words 14 mins read

Paper Group ANR 688

Distilling with Performance Enhanced Students. On the Complexity of Value Iteration. Active Object Reconstruction Using a Guided View Planner. Superconducting Optoelectronic Neurons I: General Principles. An Improved Phrase-based Approach to Annotating and Summarizing Student Course Responses. Pathwise Derivatives for Multivariate Distributions. Ev …

Distilling with Performance Enhanced Students


Title	Distilling with Performance Enhanced Students
Authors	Jack Turner, Elliot J. Crowley, Valentin Radu, José Cano, Amos Storkey, Michael O’Boyle
Abstract	The task of accelerating large neural networks on general purpose hardware has, in recent years, prompted the use of channel pruning to reduce network size. However, the efficacy of pruning based approaches has since been called into question. In this paper, we turn to distillation for model compression—specifically, attention transfer—and develop a simple method for discovering performance enhanced student networks. We combine channel saliency metrics with empirical observations of runtime performance to design more accurate networks for a given latency budget. We apply our methodology to residual and densely-connected networks, and show that we are able to find resource-efficient student networks on different hardware platforms while maintaining very high accuracy. These performance-enhanced student networks achieve up to 10% boosts in top-1 ImageNet accuracy over their channel-pruned counterparts for the same inference time.
Tasks	Model Compression
Published	2018-10-24
URL	http://arxiv.org/abs/1810.10460v2
PDF	http://arxiv.org/pdf/1810.10460v2.pdf
PWC	https://paperswithcode.com/paper/distilling-with-performance-enhanced-students
Repo
Framework

On the Complexity of Value Iteration


Title	On the Complexity of Value Iteration
Authors	Nikhil Balaji, Stefan Kiefer, Petr Novotný, Guillermo A. Pérez, Mahsa Shirmohammadi
Abstract	Value iteration is a fundamental algorithm for solving Markov Decision Processes (MDPs). It computes the maximal $n$-step payoff by iterating $n$ times a recurrence equation which is naturally associated to the MDP. At the same time, value iteration provides a policy for the MDP that is optimal on a given finite horizon $n$. In this paper, we settle the computational complexity of value iteration. We show that, given a horizon $n$ in binary and an MDP, computing an optimal policy is EXP-complete, thus resolving an open problem that goes back to the seminal 1987 paper on the complexity of MDPs by Papadimitriou and Tsitsiklis. As a stepping stone, we show that it is EXP-complete to compute the $n$-fold iteration (with $n$ in binary) of a function given by a straight-line program over the integers with $\max$ and $+$ as operators.
Tasks
Published	2018-07-13
URL	http://arxiv.org/abs/1807.04920v3
PDF	http://arxiv.org/pdf/1807.04920v3.pdf
PWC	https://paperswithcode.com/paper/on-the-complexity-of-value-iteration
Repo
Framework

Active Object Reconstruction Using a Guided View Planner


Title	Active Object Reconstruction Using a Guided View Planner
Authors	Xin Yang, Yuanbo Wang, Yaru Wang, Baocai Yin, Qiang Zhang, Xiaopeng Wei, Hongbo Fu
Abstract	Inspired by the recent advance of image-based object reconstruction using deep learning, we present an active reconstruction model using a guided view planner. We aim to reconstruct a 3D model using images observed from a planned sequence of informative and discriminative views. But where are such informative and discriminative views around an object? To address this we propose a unified model for view planning and object reconstruction, which is utilized to learn a guided information acquisition model and to aggregate information from a sequence of images for reconstruction. Experiments show that our model (1) increases our reconstruction accuracy with an increasing number of views (2) and generally predicts a more informative sequence of views for object reconstruction compared to other alternative methods.
Tasks	Object Reconstruction
Published	2018-05-08
URL	http://arxiv.org/abs/1805.03081v1
PDF	http://arxiv.org/pdf/1805.03081v1.pdf
PWC	https://paperswithcode.com/paper/active-object-reconstruction-using-a-guided
Repo
Framework

Superconducting Optoelectronic Neurons I: General Principles


Title	Superconducting Optoelectronic Neurons I: General Principles
Authors	Jeffrey M. Shainline, Sonia M. Buckley, Adam N. McCaughan, Jeff Chiles, Richard P. Mirin, Sae Woo Nam
Abstract	The design of neural hardware is informed by the prominence of differentiated processing and information integration in cognitive systems. The central role of communication leads to the principal assumption of the hardware platform: signals between neurons should be optical to enable fanout and communication with minimal delay. The requirement of energy efficiency leads to the utilization of superconducting detectors to receive single-photon signals. We discuss the potential of superconducting optoelectronic hardware to achieve the spatial and temporal information integration advantageous for cognitive processing, and we consider physical scaling limits based on light-speed communication. We introduce the superconducting optoelectronic neurons and networks that are the subject of the subsequent papers in this series.
Tasks
Published	2018-05-04
URL	http://arxiv.org/abs/1805.01929v3
PDF	http://arxiv.org/pdf/1805.01929v3.pdf
PWC	https://paperswithcode.com/paper/superconducting-optoelectronic-neurons-i-1
Repo
Framework

An Improved Phrase-based Approach to Annotating and Summarizing Student Course Responses


Title	An Improved Phrase-based Approach to Annotating and Summarizing Student Course Responses
Authors	Wencan Luo, Fei Liu, Diane Litman
Abstract	Teaching large classes remains a great challenge, primarily because it is difficult to attend to all the student needs in a timely manner. Automatic text summarization systems can be leveraged to summarize the student feedback, submitted immediately after each lecture, but it is left to be discovered what makes a good summary for student responses. In this work we explore a new methodology that effectively extracts summary phrases from the student responses. Each phrase is tagged with the number of students who raise the issue. The phrases are evaluated along two dimensions: with respect to text content, they should be informative and well-formed, measured by the ROUGE metric; additionally, they shall attend to the most pressing student needs, measured by a newly proposed metric. This work is enabled by a phrase-based annotation and highlighting scheme, which is new to the summarization task. The phrase-based framework allows us to summarize the student responses into a set of bullet points and present to the instructor promptly.
Tasks	Text Summarization
Published	2018-05-25
URL	http://arxiv.org/abs/1805.10396v1
PDF	http://arxiv.org/pdf/1805.10396v1.pdf
PWC	https://paperswithcode.com/paper/an-improved-phrase-based-approach-to
Repo
Framework

Pathwise Derivatives for Multivariate Distributions


Title	Pathwise Derivatives for Multivariate Distributions
Authors	Martin Jankowiak, Theofanis Karaletsos
Abstract	We exploit the link between the transport equation and derivatives of expectations to construct efficient pathwise gradient estimators for multivariate distributions. We focus on two main threads. First, we use null solutions of the transport equation to construct adaptive control variates that can be used to construct gradient estimators with reduced variance. Second, we consider the case of multivariate mixture distributions. In particular we show how to compute pathwise derivatives for mixtures of multivariate Normal distributions with arbitrary means and diagonal covariances. We demonstrate in a variety of experiments in the context of variational inference that our gradient estimators can outperform other methods, especially in high dimensions.
Tasks
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01856v2
PDF	http://arxiv.org/pdf/1806.01856v2.pdf
PWC	https://paperswithcode.com/paper/pathwise-derivatives-for-multivariate
Repo
Framework

Evaluating Bayesian Deep Learning Methods for Semantic Segmentation


Title	Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
Authors	Jishnu Mukhoti, Yarin Gal
Abstract	Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes. This information is critical when using semantic segmentation for autonomous driving for example. Standard semantic segmentation systems have well-established evaluation metrics. However, with BDL’s rising popularity in computer vision we require new metrics to evaluate whether a BDL method produces better uncertainty estimates than another method. In this work we propose three such metrics to evaluate BDL models designed specifically for the task of semantic segmentation. We modify DeepLab-v3+, one of the state-of-the-art deep neural networks, and create its Bayesian counterpart using MC dropout and Concrete dropout as inference techniques. We then compare and test these two inference techniques on the well-known Cityscapes dataset using our suggested metrics. Our results provide new benchmarks for researchers to compare and evaluate their improved uncertainty quantification in pursuit of safer semantic segmentation.
Tasks	Autonomous Driving, Semantic Segmentation
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12709v2
PDF	http://arxiv.org/pdf/1811.12709v2.pdf
PWC	https://paperswithcode.com/paper/evaluating-bayesian-deep-learning-methods-for
Repo
Framework

Dense xUnit Networks


Title	Dense xUnit Networks
Authors	Idan Kligvasser, Tomer Michaeli
Abstract	Deep net architectures have constantly evolved over the past few years, leading to significant advancements in a wide array of computer vision tasks. However, besides high accuracy, many applications also require a low computational load and limited memory footprint. To date, efficiency has typically been achieved either by architectural choices at the macro level (e.g. using skip connections or pruning techniques) or modifications at the level of the individual layers (e.g. using depth-wise convolutions or channel shuffle operations). Interestingly, much less attention has been devoted to the role of the activation functions in constructing efficient nets. Recently, Kligvasser et al. showed that incorporating spatial connections within the activation functions, enables a significant boost in performance in image restoration tasks, at any given budget of parameters. However, the effectiveness of their xUnit module has only been tested on simple small models, which are not characteristic of those used in high-level vision tasks. In this paper, we adopt and improve the xUnit activation, show how it can be incorporated into the DenseNet architecture, and illustrate its high effectiveness for classification and image restoration tasks alike. While the DenseNet architecture is extremely efficient to begin with, our dense xUnit net (DxNet) can typically achieve the same performance with far fewer parameters. For example, on ImageNet, our DxNet outperforms a ReLU-based DenseNet having 30% more parameters and achieves state-of-the-art results for this budget of parameters. Furthermore, in denoising and super-resolution, DxNet significantly improves upon all existing lightweight solutions, including the xUnit-based nets of Kligvasser et al.
Tasks	Denoising, Image Restoration, Super-Resolution
Published	2018-11-27
URL	http://arxiv.org/abs/1811.11051v1
PDF	http://arxiv.org/pdf/1811.11051v1.pdf
PWC	https://paperswithcode.com/paper/dense-xunit-networks
Repo
Framework

Huge Automatically Extracted Training Sets for Multilingual Word Sense Disambiguation


Title	Huge Automatically Extracted Training Sets for Multilingual Word Sense Disambiguation
Authors	Tommaso Pasini, Francesco Maria Elia, Roberto Navigli
Abstract	We release to the community six large-scale sense-annotated datasets in multiple language to pave the way for supervised multilingual Word Sense Disambiguation. Our datasets cover all the nouns in the English WordNet and their translations in other languages for a total of millions of sense-tagged sentences. Experiments prove that these corpora can be effectively used as training sets for supervised WSD systems, surpassing the state of the art for low-resourced languages and providing competitive results for English, where manually annotated training sets are accessible. The data is available at trainomatic.org.
Tasks	Word Sense Disambiguation
Published	2018-05-12
URL	http://arxiv.org/abs/1805.04685v1
PDF	http://arxiv.org/pdf/1805.04685v1.pdf
PWC	https://paperswithcode.com/paper/huge-automatically-extracted-training-sets-1
Repo
Framework

Joint Learning of Interactive Spoken Content Retrieval and Trainable User Simulator


Title	Joint Learning of Interactive Spoken Content Retrieval and Trainable User Simulator
Authors	Pei-Hung Chung, Kuan Tung, Ching-Lun Tai, Hung-Yi Lee
Abstract	User-machine interaction is crucial for information retrieval, especially for spoken content retrieval, because spoken content is difficult to browse, and speech recognition has a high degree of uncertainty. In interactive retrieval, the machine takes different actions to interact with the user to obtain better retrieval results; here it is critical to select the most efficient action. In previous work, deep Q-learning techniques were proposed to train an interactive retrieval system but rely on a hand-crafted user simulator; building a reliable user simulator is difficult. In this paper, we further improve the interactive spoken content retrieval framework by proposing a learnable user simulator which is jointly trained with interactive retrieval system, making the hand-crafted user simulator unnecessary. The experimental results show that the learned simulated users not only achieve larger rewards than the hand-crafted ones but act more like real users.
Tasks	Information Retrieval, Q-Learning, Speech Recognition
Published	2018-04-01
URL	http://arxiv.org/abs/1804.00318v1
PDF	http://arxiv.org/pdf/1804.00318v1.pdf
PWC	https://paperswithcode.com/paper/joint-learning-of-interactive-spoken-content
Repo
Framework

Sample-to-Sample Correspondence for Unsupervised Domain Adaptation


Title	Sample-to-Sample Correspondence for Unsupervised Domain Adaptation
Authors	Debasmit Das, C. S. George Lee
Abstract	The assumption that training and testing samples are generated from the same distribution does not always hold for real-world machine-learning applications. The procedure of tackling this discrepancy between the training (source) and testing (target) domains is known as domain adaptation. We propose an unsupervised version of domain adaptation that considers the presence of only unlabelled data in the target domain. Our approach centers on finding correspondences between samples of each domain. The correspondences are obtained by treating the source and target samples as graphs and using a convex criterion to match them. The criteria used are first-order and second-order similarities between the graphs as well as a class-based regularization. We have also developed a computationally efficient routine for the convex optimization, thus allowing the proposed method to be used widely. To verify the effectiveness of the proposed method, computer simulations were conducted on synthetic, image classification and sentiment classification datasets. Results validated that the proposed local sample-to-sample matching method out-performs traditional moment-matching methods and is competitive with respect to current local domain-adaptation methods.
Tasks	Domain Adaptation, Image Classification, Sentiment Analysis, Unsupervised Domain Adaptation
Published	2018-05-01
URL	http://arxiv.org/abs/1805.00355v3
PDF	http://arxiv.org/pdf/1805.00355v3.pdf
PWC	https://paperswithcode.com/paper/sample-to-sample-correspondence-for
Repo
Framework

Improving On-policy Learning with Statistical Reward Accumulation


Title	Improving On-policy Learning with Statistical Reward Accumulation
Authors	Yubin Deng, Ke Yu, Dahua Lin, Xiaoou Tang, Chen Change Loy
Abstract	Deep reinforcement learning has obtained significant breakthroughs in recent years. Most methods in deep-RL achieve good results via the maximization of the reward signal provided by the environment, typically in the form of discounted cumulative returns. Such reward signals represent the immediate feedback of a particular action performed by an agent. However, tasks with sparse reward signals are still challenging to on-policy methods. In this paper, we introduce an effective characterization of past reward statistics (which can be seen as long-term feedback signals) to supplement this immediate reward feedback. In particular, value functions are learned with multi-critics supervision, enabling complex value functions to be more easily approximated in on-policy learning, even when the reward signals are sparse. We also introduce a novel exploration mechanism called “hot-wiring” that can give a boost to seemingly trapped agents. We demonstrate the effectiveness of our advantage actor multi-critic (A2MC) method across the discrete domains in Atari games as well as continuous domains in the MuJoCo environments. A video demo is provided at https://youtu.be/zBmpf3Yz8tc.
Tasks	Atari Games
Published	2018-09-07
URL	http://arxiv.org/abs/1809.02387v1
PDF	http://arxiv.org/pdf/1809.02387v1.pdf
PWC	https://paperswithcode.com/paper/improving-on-policy-learning-with-statistical
Repo
Framework

Visual Transfer between Atari Games using Competitive Reinforcement Learning


Title	Visual Transfer between Atari Games using Competitive Reinforcement Learning
Authors	Akshita Mittel, Sowmya Munukutla, Himanshi Yadav
Abstract	This paper explores the use of deep reinforcement learning agents to transfer knowledge from one environment to another. More specifically, the method takes advantage of asynchronous advantage actor critic (A3C) architecture to generalize a target game using an agent trained on a source game in Atari. Instead of fine-tuning a pre-trained model for the target game, we propose a learning approach to update the model using multiple agents trained in parallel with different representations of the target game. Visual mapping between video sequences of transfer pairs is used to derive new representations of the target game; training on these visual representations of the target game improves model updates in terms of performance, data efficiency and stability. In order to demonstrate the functionality of the architecture, Atari games Pong-v0 and Breakout-v0 are being used from the OpenAI gym environment; as the source and target environment.
Tasks	Atari Games
Published	2018-09-02
URL	http://arxiv.org/abs/1809.00397v1
PDF	http://arxiv.org/pdf/1809.00397v1.pdf
PWC	https://paperswithcode.com/paper/visual-transfer-between-atari-games-using
Repo
Framework


Title	A Data-driven Prior on Facet Orientation for Semantic Mesh Labeling
Authors	Andrea Romanoni, Matteo Matteucci
Abstract	Mesh labeling is the key problem of classifying the facets of a 3D mesh with a label among a set of possible ones. State-of-the-art methods model mesh labeling as a Markov Random Field over the facets. These algorithms map image segmentations to the mesh by minimizing an energy function that comprises a data term, a smoothness terms, and class-specific priors. The latter favor a labeling with respect to another depending on the orientation of the facet normals. In this paper we propose a novel energy term that acts as a prior, but does not require any prior knowledge about the scene nor scene-specific relationship among classes. It bootstraps from a coarse mapping of the 2D segmentations on the mesh, and it favors the facets to be labeled according to the statistics of the mesh normals in their neighborhood. We tested our approach against five different datasets and, even if we do not inject prior knowledge, our method adapts to the data and overcomes the state-of-the-art.
Tasks
Published	2018-07-26
URL	http://arxiv.org/abs/1807.09999v1
PDF	http://arxiv.org/pdf/1807.09999v1.pdf
PWC	https://paperswithcode.com/paper/a-data-driven-prior-on-facet-orientation-for
Repo
Framework

A Deep Learning Approach for Pose Estimation from Volumetric OCT Data


Title	A Deep Learning Approach for Pose Estimation from Volumetric OCT Data
Authors	Nils Gessert, Matthias Schlüter, Alexander Schlaefer
Abstract	Tracking the pose of instruments is a central problem in image-guided surgery. For microscopic scenarios, optical coherence tomography (OCT) is increasingly used as an imaging modality. OCT is suitable for accurate pose estimation due to its micrometer range resolution and volumetric field of view. However, OCT image processing is challenging due to speckle noise and reflection artifacts in addition to the images’ 3D nature. We address pose estimation from OCT volume data with a new deep learning-based tracking framework. For this purpose, we design a new 3D convolutional neural network (CNN) architecture to directly predict the 6D pose of a small marker geometry from OCT volumes. We use a hexapod robot to automatically acquire labeled data points which we use to train 3D CNN architectures for multi-output regression. We use this setup to provide an in-depth analysis on deep learning-based pose estimation from volumes. Specifically, we demonstrate that exploiting volume information for pose estimation yields higher accuracy than relying on 2D representations with depth information. Supporting this observation, we provide quantitative and qualitative results that 3D CNNs effectively exploit the depth structure of marker objects. Regarding the deep learning aspect, we present efficient design principles for 3D CNNs, making use of insights from the 2D deep learning community. In particular, we present Inception3D as a new architecture which performs best for our application. We show that our deep learning approach reaches errors at our ground-truth label’s resolution. We achieve a mean average error of $\SI{14.89 \pm 9.3}{\micro\metre}$ and $\SI{0.096 \pm 0.072}{\degree}$ for position and orientation learning, respectively.
Tasks	Pose Estimation
Published	2018-03-10
URL	http://arxiv.org/abs/1803.03852v1
PDF	http://arxiv.org/pdf/1803.03852v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-learning-approach-for-pose-estimation
Repo
Framework