October 17, 2019

2913 words 14 mins read

Paper Group ANR 703

Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning. k-RNN: Extending NN-heuristics for the TSP. Learning Under Distributed Features. A Peek Into the Hidden Layers of a Convolutional Neural Network Through a Factorization Lens. Higher Order of Motion Magnification for Vessel Localisation in Surgical Video. A Cha …

Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning


Title	Generating Diverse Programs with Instruction Conditioned Reinforced Adversarial Learning
Authors	Aishwarya Agrawal, Mateusz Malinowski, Felix Hill, Ali Eslami, Oriol Vinyals, Tejas Kulkarni
Abstract	Advances in Deep Reinforcement Learning have led to agents that perform well across a variety of sensory-motor domains. In this work, we study the setting in which an agent must learn to generate programs for diverse scenes conditioned on a given symbolic instruction. Final goals are specified to our agent via images of the scenes. A symbolic instruction consistent with the goal images is used as the conditioning input for our policies. Since a single instruction corresponds to a diverse set of different but still consistent end-goal images, the agent needs to learn to generate a distribution over programs given an instruction. We demonstrate that with simple changes to the reinforced adversarial learning objective, we can learn instruction conditioned policies to achieve the corresponding diverse set of goals. Most importantly, our agent’s stochastic policy is shown to more accurately capture the diversity in the goal distribution than a fixed pixel-based reward function baseline. We demonstrate the efficacy of our approach on two domains: (1) drawing MNIST digits with a paint software conditioned on instructions and (2) constructing scenes in a 3D editor that satisfies a certain instruction.
Tasks
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00898v1
PDF	http://arxiv.org/pdf/1812.00898v1.pdf
PWC	https://paperswithcode.com/paper/generating-diverse-programs-with-instruction
Repo
Framework

k-RNN: Extending NN-heuristics for the TSP


Title	k-RNN: Extending NN-heuristics for the TSP
Authors	Nikolas Klug, Alok Chauhan, Ramesh Ragala, V Vijayakumar
Abstract	In this paper we present an extension of existing Nearest-Neighbor heuristics to an algorithm called k-Repetitive-Nearest-Neighbor. The idea is to start with a tour of k nodes and then perform a Nearest-Neighbor search from there on. After doing this for all permutations of k nodes the result gets selected as the shortest tour found. Experimental results show that for 2-RNN the solutions quality remains relatively stable between about 10% to 40% above the optimum.
Tasks
Published	2018-10-17
URL	http://arxiv.org/abs/1810.08059v1
PDF	http://arxiv.org/pdf/1810.08059v1.pdf
PWC	https://paperswithcode.com/paper/k-rnn-extending-nn-heuristics-for-the-tsp
Repo
Framework

Learning Under Distributed Features


Title	Learning Under Distributed Features
Authors	Bicheng Ying, Kun Yuan, Ali H. Sayed
Abstract	This work studies the problem of learning under both large datasets and large-dimensional feature space scenarios. The feature information is assumed to be spread across agents in a network, where each agent observes some of the features. Through local cooperation, the agents are supposed to interact with each other to solve an inference problem and converge towards the global minimizer of an empirical risk. We study this problem exclusively in the primal domain, and propose new and effective distributed solutions with guaranteed convergence to the minimizer with linear rate under strong convexity. This is achieved by combining a dynamic diffusion construction, a pipeline strategy, and variance-reduced techniques. Simulation results illustrate the conclusions.
Tasks
Published	2018-05-29
URL	http://arxiv.org/abs/1805.11384v2
PDF	http://arxiv.org/pdf/1805.11384v2.pdf
PWC	https://paperswithcode.com/paper/learning-under-distributed-features
Repo
Framework

A Peek Into the Hidden Layers of a Convolutional Neural Network Through a Factorization Lens


Title	A Peek Into the Hidden Layers of a Convolutional Neural Network Through a Factorization Lens
Authors	Uday Singh Saini, Evangelos E. Papalexakis
Abstract	Despite their increasing popularity and success in a variety of supervised learning problems, deep neural networks are extremely hard to interpret and debug: Given and already trained Deep Neural Net, and a set of test inputs, how can we gain insight into how those inputs interact with different layers of the neural network? Furthermore, can we characterize a given deep neural network based on it’s observed behavior on different inputs? In this paper we propose a novel factorization based approach on understanding how different deep neural networks operate. In our preliminary results, we identify fascinating patterns that link the factorization rank (typically used as a measure of interestingness in unsupervised data analysis) with how well or poorly the deep network has been trained. Finally, our proposed approach can help provide visual insights on how high-level. interpretable patterns of the network’s input behave inside the hidden layers of the deep network.
Tasks
Published	2018-06-06
URL	http://arxiv.org/abs/1806.02012v1
PDF	http://arxiv.org/pdf/1806.02012v1.pdf
PWC	https://paperswithcode.com/paper/a-peek-into-the-hidden-layers-of-a
Repo
Framework

Higher Order of Motion Magnification for Vessel Localisation in Surgical Video


Title	Higher Order of Motion Magnification for Vessel Localisation in Surgical Video
Authors	Mirek Janatka, Ashwin Sridhar, John Kelly, Danail Stoyanov
Abstract	Locating vessels during surgery is critical for avoiding inadvertent damage, yet vasculature can be difficult to identify. Video motion magnification can potentially highlight vessels by exaggerating subtle motion embedded within the video to become perceivable to the surgeon. In this paper, we explore a physiological model of artery distension to extend motion magnification to incorporate higher orders of motion, leveraging the difference in acceleration over time (jerk) in pulsatile motion to highlight the vascular pulse wave. Our method is compared to first and second order motion based Eulerian video magnification algorithms. Using data from a surgical video retrieved during a robotic prostatectomy, we show that our method can accentuate cardio-physiological features and produce a more succinct and clearer video for motion magnification, with more similarities in areas without motion to the source video at large magnifications. We validate the approach with a Structure Similarity (SSIM) and Peak Signal to Noise Ratio (PSNR) assessment of three videos at an increasing working distance, using three different levels of optical magnification. Spatio-temporal cross sections are presented to show the effectiveness of our proposal and video samples are provided to demonstrates qualitatively our results.
Tasks
Published	2018-06-13
URL	http://arxiv.org/abs/1806.04955v1
PDF	http://arxiv.org/pdf/1806.04955v1.pdf
PWC	https://paperswithcode.com/paper/higher-order-of-motion-magnification-for
Repo
Framework

A Challenge Set for French –> English Machine Translation


Title	A Challenge Set for French –> English Machine Translation
Authors	Pierre Isabelle, Roland Kuhn
Abstract	We present a challenge set for French –> English machine translation based on the approach introduced in Isabelle, Cherry and Foster (EMNLP 2017). Such challenge sets are made up of sentences that are expected to be relatively difficult for machines to translate correctly because their most straightforward translations tend to be linguistically divergent. We present here a set of 506 manually constructed French sentences, 307 of which are targeted to the same kinds of structural divergences as in the paper mentioned above. The remaining 199 sentences are designed to test the ability of the systems to correctly translate difficult grammatical words such as prepositions. We report on the results of using this challenge set for testing two different systems, namely Google Translate and DEEPL, each on two different dates (October 2017 and January 2018). All the resulting data are made publicly available.
Tasks	Machine Translation
Published	2018-06-07
URL	http://arxiv.org/abs/1806.02725v2
PDF	http://arxiv.org/pdf/1806.02725v2.pdf
PWC	https://paperswithcode.com/paper/a-challenge-set-for-french-english-machine
Repo
Framework

The Quest for the Golden Activation Function


Title	The Quest for the Golden Activation Function
Authors	Mina Basirat, Peter M. Roth
Abstract	Deep Neural Networks have been shown to be beneficial for a variety of tasks, in particular allowing for end-to-end learning and reducing the requirement for manual design decisions. However, still many parameters have to be chosen in advance, also raising the need to optimize them. One important, but often ignored system parameter is the selection of a proper activation function. Thus, in this paper we target to demonstrate the importance of activation functions in general and show that for different tasks different activation functions might be meaningful. To avoid the manual design or selection of activation functions, we build on the idea of genetic algorithms to learn the best activation function for a given task. In addition, we introduce two new activation functions, ELiSH and HardELiSH, which can easily be incorporated in our framework. In this way, we demonstrate for three different image classification benchmarks that different activation functions are learned, also showing improved results compared to typically used baselines.
Tasks	Image Classification
Published	2018-08-02
URL	http://arxiv.org/abs/1808.00783v1
PDF	http://arxiv.org/pdf/1808.00783v1.pdf
PWC	https://paperswithcode.com/paper/the-quest-for-the-golden-activation-function
Repo
Framework

Conversational Group Detection With Deep Convolutional Networks


Title	Conversational Group Detection With Deep Convolutional Networks
Authors	Mason Swofford, John Peruzzi, Marynel Vázquez
Abstract	Detection of interacting and conversational groups from images has applications in video surveillance and social robotics. In this paper we build on prior attempts to find conversational groups by detection of social gathering spaces called o-spaces used to assign people to groups. As our contributions to the task, we are the first paper to incorporate features extracted from the room layout image, and the first to incorporate a deep network to generate an image representation of the proposed o-spaces. Specifically, this novel network builds on the PointNet architecture which allows unordered inputs of variable sizes. We present accuracies which demonstrate the ability to rival and sometimes outperform the best models, but due to a data imbalance issue we do not yet outperform existing models in our test results.
Tasks
Published	2018-10-07
URL	https://arxiv.org/abs/1810.04039v2
PDF	https://arxiv.org/pdf/1810.04039v2.pdf
PWC	https://paperswithcode.com/paper/conversational-group-detection-with-deep
Repo
Framework

Simultaneous Adversarial Training - Learn from Others Mistakes


Title	Simultaneous Adversarial Training - Learn from Others Mistakes
Authors	Zukang Liao
Abstract	Adversarial examples are maliciously tweaked images that can easily fool machine learning techniques, such as neural networks, but they are normally not visually distinguishable for human beings. One of the main approaches to solve this problem is to retrain the networks using those adversarial examples, namely adversarial training. However, standard adversarial training might not actually change the decision boundaries but cause the problem of gradient masking, resulting in a weaker ability to generate adversarial examples. Therefore, it cannot alleviate the problem of black-box attacks, where adversarial examples generated from other networks can transfer to the targeted one. In order to reduce the problem of black-box attacks, we propose a novel method that allows two networks to learn from each others’ adversarial examples and become resilient to black-box attacks. We also combine this method with a simple domain adaptation to further improve the performance.
Tasks	Domain Adaptation
Published	2018-07-21
URL	http://arxiv.org/abs/1807.08108v3
PDF	http://arxiv.org/pdf/1807.08108v3.pdf
PWC	https://paperswithcode.com/paper/simultaneous-adversarial-training-learn-from
Repo
Framework

Semantic Vector Spaces for Broadening Consideration of Consequences


Title	Semantic Vector Spaces for Broadening Consideration of Consequences
Authors	Douglas Summers Stay
Abstract	Reasoning systems with too simple a model of the world and human intent are unable to consider potential negative side effects of their actions and modify their plans to avoid them (e.g., avoiding potential errors). However, hand-encoding the enormous and subtle body of facts that constitutes common sense into a knowledge base has proved too difficult despite decades of work. Distributed semantic vector spaces learned from large text corpora, on the other hand, can learn representations that capture shades of meaning of common-sense concepts and perform analogical and associational reasoning in ways that knowledge bases are too rigid to perform, by encoding concepts and the relations between them as geometric structures. These have, however, the disadvantage of being unreliable, poorly understood, and biased in their view of the world by the source material. This chapter will discuss how these approaches may be combined in a way that combines the best properties of each for understanding the world and human intentions in a richer way.
Tasks	Common Sense Reasoning
Published	2018-02-23
URL	http://arxiv.org/abs/1802.08554v1
PDF	http://arxiv.org/pdf/1802.08554v1.pdf
PWC	https://paperswithcode.com/paper/semantic-vector-spaces-for-broadening
Repo
Framework

Towards Explainable Inference about Object Motion using Qualitative Reasoning


Title	Towards Explainable Inference about Object Motion using Qualitative Reasoning
Authors	Xiaoyu Ge, Jochen Renz, Hua Hua
Abstract	The capability of making explainable inferences regarding physical processes has long been desired. One fundamental physical process is object motion. Inferring what causes the motion of a group of objects can even be a challenging task for experts, e.g., in forensics science. Most of the work in the literature relies on physics simulation to draw such infer- ences. The simulation requires a precise model of the under- lying domain to work well and is essentially a black-box from which one can hardly obtain any useful explanation. By contrast, qualitative reasoning methods have the advan- tage in making transparent inferences with ambiguous infor- mation, which makes it suitable for this task. However, there has been no suitable qualitative theory proposed for object motion in three-dimensional space. In this paper, we take this challenge and develop a qualitative theory for the motion of rigid objects. Based on this theory, we develop a reasoning method to solve a very interesting problem: Assuming there are several objects that were initially at rest and now have started to move. We want to infer what action causes the movement of these objects.
Tasks
Published	2018-07-28
URL	http://arxiv.org/abs/1807.10935v1
PDF	http://arxiv.org/pdf/1807.10935v1.pdf
PWC	https://paperswithcode.com/paper/towards-explainable-inference-about-object
Repo
Framework

Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing


Title	Identifying Implementation Bugs in Machine Learning based Image Classifiers using Metamorphic Testing
Authors	Anurag Dwarakanath, Manish Ahuja, Samarth Sikand, Raghotham M. Rao, R. P. Jagadeesh Chandra Bose, Neville Dubash, Sanjay Podder
Abstract	We have recently witnessed tremendous success of Machine Learning (ML) in practical applications. Computer vision, speech recognition and language translation have all seen a near human level performance. We expect, in the near future, most business applications will have some form of ML. However, testing such applications is extremely challenging and would be very expensive if we follow today’s methodologies. In this work, we present an articulation of the challenges in testing ML based applications. We then present our solution approach, based on the concept of Metamorphic Testing, which aims to identify implementation bugs in ML based image classifiers. We have developed metamorphic relations for an application based on Support Vector Machine and a Deep Learning based application. Empirical validation showed that our approach was able to catch 71% of the implementation bugs in the ML applications.
Tasks	Speech Recognition
Published	2018-08-16
URL	http://arxiv.org/abs/1808.05353v1
PDF	http://arxiv.org/pdf/1808.05353v1.pdf
PWC	https://paperswithcode.com/paper/identifying-implementation-bugs-in-machine
Repo
Framework

Learning Time Dependent Choice


Title	Learning Time Dependent Choice
Authors	Zachary Chase, Siddharth Prasad
Abstract	We explore questions dealing with the learnability of models of choice over time. We present a large class of preference models defined by a structural criterion for which we are able to obtain an exponential improvement over previously known learning bounds for more general preference models. This in particular implies that the three most important discounted utility models of intertemporal choice – exponential, hyperbolic, and quasi-hyperbolic discounting – are learnable in the PAC setting with VC dimension that grows logarithmically in the number of time periods. We also examine these models in the framework of active learning. We find that the commonly studied stream-based setting is in general difficult to analyze for preference models, but we provide a redeeming situation in which the learner can indeed improve upon the guarantees provided by PAC learning. In contrast to the stream-based setting, we show that if the learner is given full power over the data he learns from – in the form of learning via membership queries – even very naive algorithms significantly outperform the guarantees provided by higher level active learning algorithms.
Tasks	Active Learning
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03154v1
PDF	http://arxiv.org/pdf/1809.03154v1.pdf
PWC	https://paperswithcode.com/paper/learning-time-dependent-choice
Repo
Framework

Unsupervised Ensemble Learning via Ising Model Approximation with Application to Phenotyping Prediction


Title	Unsupervised Ensemble Learning via Ising Model Approximation with Application to Phenotyping Prediction
Authors	Luwan Zhang, Tianrun Cai
Abstract	Unsupervised ensemble learning has long been an interesting yet challenging problem that comes to prominence in recent years with the increasing demand of crowdsourcing in various applications. In this paper, we propose a novel method– unsupervised ensemble learning via Ising model approximation (unElisa) that combines a pruning step with a predicting step. We focus on the binary case and use an Ising model to characterize interactions between the ensemble and the underlying true classifier. The presence of an edge between an observed classifier and the true classifier indicates a direct dependence whereas the absence indicates the corresponding one provides no additional information and shall be eliminated. This observation leads to the pruning step where the key is to recover the neighborhood of the true classifier. We show that it can be recovered successfully with exponentially decaying error in the high-dimensional setting by performing nodewise $\ell_1$-regularized logistic regression. The pruned ensemble allows us to get a consistent estimate of the Bayes classifier for predicting. We also propose an augmented version of majority voting by reversing all labels given by a subgroup of the pruned ensemble. We demonstrate the efficacy of our method through extensive numerical experiments and through the application to EHR-based phenotyping prediction on Rheumatoid Arthritis (RA) using data from Partners Healthcare System.
Tasks
Published	2018-10-15
URL	http://arxiv.org/abs/1810.06376v1
PDF	http://arxiv.org/pdf/1810.06376v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-ensemble-learning-via-ising
Repo
Framework

Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization


Title	Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
Authors	Yibo Yang, Nicholas Ruozzi, Vibhav Gogate
Abstract	We propose a simple and easy to implement neural network compression algorithm that achieves results competitive with more complicated state-of-the-art methods. The key idea is to modify the original optimization problem by adding K independent Gaussian priors (corresponding to the k-means objective) over the network parameters to achieve parameter quantization, as well as an L1 penalty to achieve pruning. Unlike many existing quantization-based methods, our method uses hard clustering assignments of network parameters, which adds minimal change or overhead to standard network training. We also demonstrate experimentally that tying neural network parameters provides less gain in generalization performance than changing network architecture and connectivity patterns entirely.
Tasks	Neural Network Compression, Quantization
Published	2018-06-14
URL	http://arxiv.org/abs/1806.05355v1
PDF	http://arxiv.org/pdf/1806.05355v1.pdf
PWC	https://paperswithcode.com/paper/scalable-neural-network-compression-and
Repo
Framework