January 29, 2020

3179 words 15 mins read

Paper Group ANR 563

Deep Neural Network Ensembles. Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks. Spaceland Embedding of Sparse Stochastic Graphs. Wearable-based Parkinson’s Disease Severity Monitoring using Deep Learning. Internal-transfer Weighting of Multi-task Learning for Lung Cancer Detection. Variational Quantum Circuits for Qua …

Deep Neural Network Ensembles


Title	Deep Neural Network Ensembles
Authors	Sean Tao
Abstract	Current deep neural networks suffer from two problems; first, they are hard to interpret, and second, they suffer from overfitting. There have been many attempts to define interpretability in neural networks, but they typically lack causality or generality. A myriad of regularization techniques have been developed to prevent overfitting, and this has driven deep learning to become the hot topic it is today; however, while most regularization techniques are justified empirically and even intuitively, there is not much underlying theory. This paper argues that to extract the features used in neural networks to make decisions, it’s important to look at the paths between clusters existing in the hidden spaces of neural networks. These features are of particular interest because they reflect the true decision making process of the neural network. This analysis is then furthered to present an ensemble algorithm for arbitrary neural networks which has guarantees for test accuracy. Finally, a discussion detailing the aforementioned guarantees is introduced and the implications to neural networks, including an intuitive explanation for all current regularization methods, are presented. The ensemble algorithm has generated state-of-the-art results for Wide-ResNets on CIFAR-10 (top 5 for all models) and has improved test accuracy for all models it has been applied to.
Tasks	Decision Making
Published	2019-04-11
URL	https://arxiv.org/abs/1904.05488v2
PDF	https://arxiv.org/pdf/1904.05488v2.pdf
PWC	https://paperswithcode.com/paper/deep-neural-network-ensembles
Repo
Framework

Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks


Title	Tool Substitution with Shape and Material Reasoning Using Dual Neural Networks
Authors	Nithin Shrivatsav, Lakshmi Nair, Sonia Chernova
Abstract	This paper explores the problem of tool substitution, namely, identifying substitute tools for performing a task from a given set of candidate tools. We introduce a novel approach to tool substitution, that unlike prior work in the area, combines both shape and material reasoning to effectively identify substitute tools. Our approach combines the use of visual and spectral reasoning using dual neural networks. It takes as input, the desired action to be performed, and outputs a ranking of the available candidate tools based on their suitability for performing the action. Our results on a test set of 30 real-world objects show that our approach is able to effectively match shape and material similarities, with improved tool substitution performance when combining both.
Tasks
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04521v1
PDF	https://arxiv.org/pdf/1911.04521v1.pdf
PWC	https://paperswithcode.com/paper/tool-substitution-with-shape-and-material
Repo
Framework

Spaceland Embedding of Sparse Stochastic Graphs


Title	Spaceland Embedding of Sparse Stochastic Graphs
Authors	Nikos Pitsianis, Alexandros-Stavros Iliopoulos, Dimitris Floros, Xiaobai Sun
Abstract	We introduce a nonlinear method for directly embedding large, sparse, stochastic graphs into low-dimensional spaces, without requiring vertex features to reside in, or be transformed into, a metric space. Graph data and models are prevalent in real-world applications. Direct graph embedding is fundamental to many graph analysis tasks, in addition to graph visualization. We name the novel approach SG-t-SNE, as it is inspired by and builds upon the core principle of t-SNE, a widely used method for nonlinear dimensionality reduction and data visualization. We also introduce t-SNE-$\Pi$, a high-performance software for 2D, 3D embedding of large sparse graphs on personal computers with superior efficiency. It empowers SG-t-SNE with modern computing techniques for exploiting in tandem both matrix structures and memory architectures. We present elucidating embedding results on one synthetic graph and four real-world networks.
Tasks	Dimensionality Reduction, Graph Embedding
Published	2019-06-13
URL	https://arxiv.org/abs/1906.05582v1
PDF	https://arxiv.org/pdf/1906.05582v1.pdf
PWC	https://paperswithcode.com/paper/spaceland-embedding-of-sparse-stochastic
Repo
Framework

Wearable-based Parkinson’s Disease Severity Monitoring using Deep Learning


Title	Wearable-based Parkinson’s Disease Severity Monitoring using Deep Learning
Authors	Jann Goschenhofer, Franz MJ Pfister, Kamer Ali Yuksel, Bernd Bischl, Urban Fietzek, Janek Thomas
Abstract	One major challenge in the medication of Parkinson’s disease is that the severity of the disease, reflected in the patients’ motor state, cannot be measured using accessible biomarkers. Therefore, we develop and examine a variety of statistical models to detect the motor state of such patients based on sensor data from a wearable device. We find that deep learning models consistently outperform a classical machine learning model applied on hand-crafted features in this time series classification task. Furthermore, our results suggest that treating this problem as a regression instead of an ordinal regression or a classification task is most appropriate. For consistent model evaluation and training, we adopt the leave-one-subject-out validation scheme to the training of deep learning models. We also employ a class-weighting scheme to successfully mitigate the problem of high multi-class imbalances in this domain. In addition, we propose a customized performance measure that reflects the requirements of the involved medical staff on the model. To solve the problem of limited availability of high quality training data, we propose a transfer learning technique which helps to improve model performance substantially. Our results suggest that deep learning techniques offer a high potential to autonomously detect motor states of patients with Parkinson’s disease.
Tasks	Time Series, Time Series Classification, Transfer Learning
Published	2019-04-24
URL	http://arxiv.org/abs/1904.10829v1
PDF	http://arxiv.org/pdf/1904.10829v1.pdf
PWC	https://paperswithcode.com/paper/wearable-based-parkinsons-disease-severity
Repo
Framework

Internal-transfer Weighting of Multi-task Learning for Lung Cancer Detection


Title	Internal-transfer Weighting of Multi-task Learning for Lung Cancer Detection
Authors	Yiyuan Yang, Riqiang Gao, Yucheng Tang, Sanja L. Antic, Steve Deppen, Yuankai Huo, Kim L. Sandler, Pierre P. Massion, Bennett A. Landman
Abstract	Recently, multi-task networks have shown to both offer additional estimation capabilities, and, perhaps more importantly, increased performance over single-task networks on a “main/primary” task. However, balancing the optimization criteria of multi-task networks across different tasks is an area of active exploration. Here, we extend a previously proposed 3D attention-based network with four additional multi-task subnetworks for the detection of lung cancer and four auxiliary tasks (diagnosis of asthma, chronic bronchitis, chronic obstructive pulmonary disease, and emphysema). We introduce and evaluate a learning policy, Periodic Focusing Learning Policy (PFLP), that alternates the dominance of tasks throughout the training. To improve performance on the primary task, we propose an Internal-Transfer Weighting (ITW) strategy to suppress the loss functions on auxiliary tasks for the final stages of training. To evaluate this approach, we examined 3386 patients (single scan per patient) from the National Lung Screening Trial (NLST) and de-identified data from the Vanderbilt Lung Screening Program, with a 2517/277/592 (scans) split for training, validation, and testing. Baseline networks include a single-task strategy and a multi-task strategy without adaptive weights (PFLP/ITW), while primary experiments are multi-task trials with either PFLP or ITW or both. On the test set for lung cancer prediction, the baseline single-task network achieved prediction AUC of 0.8080 and the multi-task baseline failed to converge (AUC 0.6720). However, applying PFLP helped multi-task network clarify and achieved test set lung cancer prediction AUC of 0.8402. Furthermore, our ITW technique boosted the PFLP enabled multi-task network and achieved an AUC of 0.8462 (McNemar test, p < 0.01).
Tasks	Multi-Task Learning
Published	2019-12-16
URL	https://arxiv.org/abs/1912.07167v1
PDF	https://arxiv.org/pdf/1912.07167v1.pdf
PWC	https://paperswithcode.com/paper/internal-transfer-weighting-of-multi-task
Repo
Framework

Variational Quantum Circuits for Quantum State Tomography


Title	Variational Quantum Circuits for Quantum State Tomography
Authors	Yong Liu, Dongyang Wang, Shichuan Xue, Anqi Huang, Xiang Fu, Xiaogang Qiang, Ping Xu, He-Liang Huang, Mingtang Deng, Chu Guo, Xuejun Yang, Junjie Wu
Abstract	We propose a hybrid quantum-classical algorithm for quantum state tomography. Given an unknown quantum state, a quantum machine learning algorithm is used to maximize the fidelity between the output of a variational quantum circuit and this state. The number of parameters of the variational quantum circuit grows linearly with the number of qubits and the circuit depth. After that, a subsequent classical algorithm is used to reconstruct the unknown quantum state. We demonstrate our method by performing numerical simulations to reconstruct the ground state of a one-dimensional quantum spin chain, using a variational quantum circuit simulator. Our method is suitable for near-term quantum computing platforms, and could be used for relatively large-scale quantum state tomography for experimentally relevant quantum states.
Tasks	Quantum Machine Learning, Quantum State Tomography
Published	2019-12-16
URL	https://arxiv.org/abs/1912.07286v1
PDF	https://arxiv.org/pdf/1912.07286v1.pdf
PWC	https://paperswithcode.com/paper/variational-quantum-circuits-for-quantum
Repo
Framework

Hierarchy Parsing for Image Captioning


Title	Hierarchy Parsing for Image Captioning
Authors	Ting Yao, Yingwei Pan, Yehao Li, Tao Mei
Abstract	It is always well believed that parsing an image into constituent visual patterns would be helpful for understanding and representing an image. Nevertheless, there has not been evidence in support of the idea on describing an image with a natural-language utterance. In this paper, we introduce a new design to model a hierarchy from instance level (segmentation), region level (detection) to the whole image to delve into a thorough image understanding for captioning. Specifically, we present a HIerarchy Parsing (HIP) architecture that novelly integrates hierarchical structure into image encoder. Technically, an image decomposes into a set of regions and some of the regions are resolved into finer ones. Each region then regresses to an instance, i.e., foreground of the region. Such process naturally builds a hierarchal tree. A tree-structured Long Short-Term Memory (Tree-LSTM) network is then employed to interpret the hierarchal structure and enhance all the instance-level, region-level and image-level features. Our HIP is appealing in view that it is pluggable to any neural captioning models. Extensive experiments on COCO image captioning dataset demonstrate the superiority of HIP. More remarkably, HIP plus a top-down attention-based LSTM decoder increases CIDEr-D performance from 120.1% to 127.2% on COCO Karpathy test split. When further endowing instance-level and region-level features from HIP with semantic relation learnt through Graph Convolutional Networks (GCN), CIDEr-D is boosted up to 130.6%.
Tasks	Image Captioning
Published	2019-09-09
URL	https://arxiv.org/abs/1909.03918v2
PDF	https://arxiv.org/pdf/1909.03918v2.pdf
PWC	https://paperswithcode.com/paper/hierarchy-parsing-for-image-captioning
Repo
Framework

Proactive Intention Recognition for Joint Human-Robot Search and Rescue Missions through Monte-Carlo Planning in POMDP Environments


Title	Proactive Intention Recognition for Joint Human-Robot Search and Rescue Missions through Monte-Carlo Planning in POMDP Environments
Authors	Dimitri Ognibene, Lorenzo Mirante, Letizia Marchegiani
Abstract	Proactively perceiving others’ intentions is a crucial skill to effectively interact in unstructured, dynamic and novel environments. This work proposes a first step towards embedding this skill in support robots for search and rescue missions. Predicting the responders’ intentions, indeed, will enable exploration approaches which will identify and prioritise areas that are more relevant for the responder and, thus, for the task, leading to the development of safer, more robust and efficient joint exploration strategies. More specifically, this paper presents an active intention recognition paradigm to perceive, even under sensory constraints, not only the target’s position but also the first responder’s movements, which can provide information on his/her intentions (e.g. reaching the position where he/she expects the target to be). This mechanism is implemented by employing an extension of Monte-Carlo-based planning techniques for partially observable environments, where the reward function is augmented with an entropy reduction bonus. We test in simulation several configurations of reward augmentation, both information theoretic and not, as well as belief state approximations and obtain substantial improvements over the basic approach.
Tasks	Intent Detection
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10125v1
PDF	https://arxiv.org/pdf/1908.10125v1.pdf
PWC	https://paperswithcode.com/paper/proactive-intention-recognition-for-joint
Repo
Framework


Title	Fast 3D Pose Refinement with RGB Images
Authors	Abhinav Jain, Frank Dellaert
Abstract	Pose estimation is a vital step in many robotics and perception tasks such as robotic manipulation, autonomous vehicle navigation, etc. Current state-of-the-art pose estimation methods rely on deep neural networks with complicated structures and long inference times. While highly robust, they require computing power often unavailable on mobile robots. We propose a CNN-based pose refinement system which takes a coarsely estimated 3D pose from a computationally cheaper algorithm along with a bounding box image of the object, and returns a highly refined pose. Our experiments on the YCB-Video dataset show that our system can refine 3D poses to an extremely high precision with minimal training data.
Tasks	Pose Estimation
Published	2019-11-17
URL	https://arxiv.org/abs/1911.07347v1
PDF	https://arxiv.org/pdf/1911.07347v1.pdf
PWC	https://paperswithcode.com/paper/fast-3d-pose-refinement-with-rgb-images
Repo
Framework

Noisy-As-Clean: Learning Unsupervised Denoising from the Corrupted Image


Title	Noisy-As-Clean: Learning Unsupervised Denoising from the Corrupted Image
Authors	Jun Xu, Yuan Huang, Li Liu, Fan Zhu, Xingsong Hou, Ling Shao
Abstract	In the past few years, supervised networks have achieved promising performance on image denoising. These methods learn image priors and synthetic noise statistics from plenty pairs of noisy and clean images. Recently, several unsupervised denoising networks are proposed only using external noisy images for training. However, the networks learned from external data inherently suffer from the domain gap dilemma, i.e., the image priors and noise statistics are very different between the training data and the corrupted test images. This dilemma becomes more clear when dealing with the signal dependent realistic noise in real photographs. In this work, we provide a statistically useful conclusion: it is possible to learn an unsupervised network only with the corrupted image, approximating the optimal parameters of a supervised network learned with pairs of noisy and clean images. This is achieved by proposing a “Noisy-As-Clean” strategy: taking the corrupted image as “clean” target and the simulated noisy images (based on the corrupted image) as inputs. Extensive experiments show that the unsupervised denoising networks learned with our “Noisy-As-Clean” strategy surprisingly outperforms previous supervised networks on removing several typical synthetic noise and realistic noise. The code will be publicly released.
Tasks	Denoising, Image Denoising
Published	2019-06-17
URL	https://arxiv.org/abs/1906.06878v3
PDF	https://arxiv.org/pdf/1906.06878v3.pdf
PWC	https://paperswithcode.com/paper/noisy-as-clean-learning-unsupervised
Repo
Framework

DashNet: A Hybrid Artificial and Spiking Neural Network for High-speed Object Tracking


Title	DashNet: A Hybrid Artificial and Spiking Neural Network for High-speed Object Tracking
Authors	Zheyu Yang, Yujie Wu, Guanrui Wang, Yukuan Yang, Guoqi Li, Lei Deng, Jun Zhu, Luping Shi
Abstract	Computer-science-oriented artificial neural networks (ANNs) have achieved tremendous success in a variety of scenarios via powerful feature extraction and high-precision data operations. It is well known, however, that ANNs usually suffer from expensive processing resources and costs. In contrast, neuroscience-oriented spiking neural networks (SNNs) are promising for energy-efficient information processing benefit from the event-driven spike activities, whereas, they are yet be evidenced to achieve impressive effectiveness on real complicated tasks. How to combine the advantage of these two model families is an open question of great interest. Two significant challenges need to be addressed: (1) lack of benchmark datasets including both ANN-oriented (frames) and SNN-oriented (spikes) signal resources; (2) the difficulty in jointly processing the synchronous activation from ANNs and event-driven spikes from SNNs. In this work, we proposed a hybrid paradigm, named as DashNet, to demonstrate the advantages of combining ANNs and SNNs in a single model. A simulator and benchmark dataset NFS-DAVIS is built, and a temporal complementary filter (TCF) and attention module are designed to address the two mentioned challenges, respectively. In this way, it is shown that DashNet achieves the record-breaking speed of 2083FPS on neuromorphic chips and the best tracking performance on NFS-DAVIS and PRED18 datasets. To the best of our knowledge, DashNet is the first framework that can integrate and process ANNs and SNNs in a hybrid paradigm, which provides a novel solution to achieve both effectiveness and efficiency for high-speed object tracking.
Tasks	Object Tracking
Published	2019-09-15
URL	https://arxiv.org/abs/1909.12942v1
PDF	https://arxiv.org/pdf/1909.12942v1.pdf
PWC	https://paperswithcode.com/paper/dashnet-a-hybrid-artificial-and-spiking
Repo
Framework

Artificial Neural Networks


Title	Artificial Neural Networks
Authors	B. Mehlig
Abstract	These are lecture notes for my course on Artificial Neural Networks that I have given at Chalmers and Gothenburg University. This course describes the use of neural networks in machine learning: deep learning, recurrent networks, and other supervised and unsupervised machine-learning algorithms.
Tasks
Published	2019-01-17
URL	http://arxiv.org/abs/1901.05639v2
PDF	http://arxiv.org/pdf/1901.05639v2.pdf
PWC	https://paperswithcode.com/paper/artificial-neural-networks
Repo
Framework

Growing Action Spaces


Title	Growing Action Spaces
Authors	Gregory Farquhar, Laura Gustafson, Zeming Lin, Shimon Whiteson, Nicolas Usunier, Gabriel Synnaeve
Abstract	In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.
Tasks	Starcraft
Published	2019-06-28
URL	https://arxiv.org/abs/1906.12266v1
PDF	https://arxiv.org/pdf/1906.12266v1.pdf
PWC	https://paperswithcode.com/paper/growing-action-spaces
Repo
Framework

Predicting US State-Level Agricultural Sentiment as a Measure of Food Security with Tweets from Farming Communities


Title	Predicting US State-Level Agricultural Sentiment as a Measure of Food Security with Tweets from Farming Communities
Authors	Jared Dunnmon, Swetava Ganguli, Darren Hau, Brooke Husic
Abstract	The ability to obtain accurate food security metrics in developing areas where relevant data can be sparse is critically important for policy makers tasked with implementing food aid programs. As a result, a great deal of work has been dedicated to predicting important food security metrics such as annual crop yields using a variety of methods including simulation, remote sensing, weather models, and human expert input. As a complement to existing techniques in crop yield prediction, this work develops neural network models for predicting the sentiment of Twitter feeds from farming communities. Specifically, we investigate the potential of both direct learning on a small dataset of agriculturally-relevant tweets and transfer learning from larger, well-labeled sentiment datasets from other domains (e.g.~politics) to accurately predict agricultural sentiment, which we hope would ultimately serve as a useful crop yield predictor. We find that direct learning from small, relevant datasets outperforms transfer learning from large, fully-labeled datasets, that convolutional neural networks broadly outperform recurrent neural networks on Twitter sentiment classification, and that these models perform substantially less well on ternary sentiment problems characteristic of practical settings than on binary problems often found in the literature.
Tasks	Sentiment Analysis, Transfer Learning
Published	2019-02-13
URL	http://arxiv.org/abs/1902.07087v2
PDF	http://arxiv.org/pdf/1902.07087v2.pdf
PWC	https://paperswithcode.com/paper/predicting-us-state-level-agricultural
Repo
Framework

On the Regularization Properties of Structured Dropout


Title	On the Regularization Properties of Structured Dropout
Authors	Ambar Pal, Connor Lane, René Vidal, Benjamin D. Haeffele
Abstract	Dropout and its extensions (eg. DropBlock and DropConnect) are popular heuristics for training neural networks, which have been shown to improve generalization performance in practice. However, a theoretical understanding of their optimization and regularization properties remains elusive. Recent work shows that in the case of single hidden-layer linear networks, Dropout is a stochastic gradient descent method for minimizing a regularized loss, and that the regularizer induces solutions that are low-rank and balanced. In this work we show that for single hidden-layer linear networks, DropBlock induces spectral k-support norm regularization, and promotes solutions that are low-rank and have factors with equal norm. We also show that the global minimizer for DropBlock can be computed in closed form, and that DropConnect is equivalent to Dropout. We then show that some of these results can be extended to a general class of Dropout-strategies, and, with some assumptions, to deep non-linear networks when Dropout is applied to the last layer. We verify our theoretical claims and assumptions experimentally with commonly used network architectures.
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1910.14186v1
PDF	https://arxiv.org/pdf/1910.14186v1.pdf
PWC	https://paperswithcode.com/paper/on-the-regularization-properties-of
Repo
Framework