February 1, 2020

3616 words 17 mins read

Paper Group AWR 137

CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning. Spatial Evolutionary Generative Adversarial Networks. Blending Diverse Physical Priors with Neural Networks. Synchronising audio and ultrasound by learning cross-modal embeddings. Machine learning algorithms to infer trait-matching and predict species interactions in ecological …

CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning


Title	CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning
Authors	Ziyu Yao, Jayavardhan Reddy Peddamail, Huan Sun
Abstract	To accelerate software development, much research has been performed to help people understand and reuse the huge amount of available code resources. Two important tasks have been widely studied: code retrieval, which aims to retrieve code snippets relevant to a given natural language query from a code base, and code annotation, where the goal is to annotate a code snippet with a natural language description. Despite their advancement in recent years, the two tasks are mostly explored separately. In this work, we investigate a novel perspective of Code annotation for Code retrieval (hence called `CoaCor’), where a code annotation model is trained to generate a natural language annotation that can represent the semantic meaning of a given code snippet and can be leveraged by a code retrieval model to better distinguish relevant code snippets from others. To this end, we propose an effective framework based on reinforcement learning, which explicitly encourages the code annotation model to generate annotations that can be used for the retrieval task. Through extensive experiments, we show that code annotations generated by our framework are much more detailed and more useful for code retrieval, and they can further improve the performance of existing code retrieval models significantly. \|
Tasks
Published	2019-03-13
URL	http://arxiv.org/abs/1904.00720v1
PDF	http://arxiv.org/pdf/1904.00720v1.pdf
PWC	https://paperswithcode.com/paper/coacor-code-annotation-for-code-retrieval
Repo	https://github.com/LittleYUYU/CoaCor
Framework	pytorch

Spatial Evolutionary Generative Adversarial Networks


Title	Spatial Evolutionary Generative Adversarial Networks
Authors	Jamal Toutouh, Erik Hemberg, Una-May O’Reilly
Abstract	Generative adversary networks (GANs) suffer from training pathologies such as instability and mode collapse. These pathologies mainly arise from a lack of diversity in their adversarial interactions. Evolutionary generative adversarial networks apply the principles of evolutionary computation to mitigate these problems. We hybridize two of these approaches that promote training diversity. One, E-GAN, at each batch, injects mutation diversity by training the (replicated) generator with three independent objective functions then selecting the resulting best performing generator for the next batch. The other, Lipizzaner, injects population diversity by training a two-dimensional grid of GANs with a distributed evolutionary algorithm that includes neighbor exchanges of additional training adversaries, performance based selection and population-based hyper-parameter tuning. We propose to combine mutation and population approaches to diversity improvement. We contribute a superior evolutionary GANs training method, Mustangs, that eliminates the single loss function used across Lipizzaner’s grid. Instead, each training round, a loss function is selected with equal probability, from among the three E-GAN uses. Experimental analyses on standard benchmarks, MNIST and CelebA, demonstrate that Mustangs provides a statistically faster training method resulting in more accurate networks.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12702v1
PDF	https://arxiv.org/pdf/1905.12702v1.pdf
PWC	https://paperswithcode.com/paper/spatial-evolutionary-generative-adversarial
Repo	https://github.com/mustang-gan/mustang
Framework	pytorch

Blending Diverse Physical Priors with Neural Networks


Title	Blending Diverse Physical Priors with Neural Networks
Authors	Yunhao Ba, Guangyuan Zhao, Achuta Kadambi
Abstract	Machine learning in context of physical systems merits a re-examination of the learning strategy. In addition to data, one can leverage a vast library of physical prior models (e.g. kinematics, fluid flow, etc) to perform more robust inference. The nascent sub-field of \emph{physics-based learning} (PBL) studies the blending of neural networks with physical priors. While previous PBL algorithms have been applied successfully to specific tasks, it is hard to generalize existing PBL methods to a wide range of physics-based problems. Such generalization would require an architecture that can adapt to variations in the correctness of the physics, or in the quality of training data. No such architecture exists. In this paper, we aim to generalize PBL, by making a first attempt to bring neural architecture search (NAS) to the realm of PBL. We introduce a new method known as physics-based neural architecture search (PhysicsNAS) that is a top-performer across a diverse range of quality in the physical model and the dataset.
Tasks	Neural Architecture Search
Published	2019-10-01
URL	https://arxiv.org/abs/1910.00201v1
PDF	https://arxiv.org/pdf/1910.00201v1.pdf
PWC	https://paperswithcode.com/paper/blending-diverse-physical-priors-with-neural
Repo	https://github.com/PhysicsNAS/PhysicsNAS
Framework	pytorch


Title	Synchronising audio and ultrasound by learning cross-modal embeddings
Authors	Aciel Eshky, Manuel Sam Ribeiro, Korin Richmond, Steve Renals
Abstract	Audiovisual synchronisation is the task of determining the time offset between speech audio and a video recording of the articulators. In child speech therapy, audio and ultrasound videos of the tongue are captured using instruments which rely on hardware to synchronise the two modalities at recording time. Hardware synchronisation can fail in practice, and no mechanism exists to synchronise the signals post hoc. To address this problem, we employ a two-stream neural network which exploits the correlation between the two modalities to find the offset. We train our model on recordings from 69 speakers, and show that it correctly synchronises 82.9% of test utterances from unseen therapy sessions and unseen speakers, thus considerably reducing the number of utterances to be manually synchronised. An analysis of model performance on the test utterances shows that directed phone articulations are more difficult to automatically synchronise compared to utterances containing natural variation in speech such as words, sentences, or conversations.
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00758v2
PDF	https://arxiv.org/pdf/1907.00758v2.pdf
PWC	https://paperswithcode.com/paper/synchronising-audio-and-ultrasound-by
Repo	https://github.com/aeshky/ultrasync
Framework	none

Machine learning algorithms to infer trait-matching and predict species interactions in ecological networks


Title	Machine learning algorithms to infer trait-matching and predict species interactions in ecological networks
Authors	Maximilian Pichler, Virginie Boreux, Alexandra-Maria Klein, Matthias Schleuning, Florian Hartig
Abstract	Ecologists have long suspected that species are more likely to interact if their traits match in a particular way. For example, a pollination interaction may be more likely if the proportions of a bee’s tongue fit a plant’s flower shape. Empirical estimates of the importance of trait-matching for determining species interactions, however, vary significantly among different types of ecological networks. Here, we show that ambiguity among empirical trait-matching studies may have arisen at least in parts from using overly simple statistical models. Using simulated and real data, we contrast conventional generalized linear models (GLM) with more flexible Machine Learning (ML) models (Random Forest, Boosted Regression Trees, Deep Neural Networks, Convolutional Neural Networks, Support Vector Machines, naive Bayes, and k-Nearest-Neighbor), testing their ability to predict species interactions based on traits, and infer trait combinations causally responsible for species interactions. We find that the best ML models can successfully predict species interactions in plant-pollinator networks, outperforming GLMs by a substantial margin. Our results also demonstrate that ML models can better identify the causally responsible trait-matching combinations than GLMs. In two case studies, the best ML models successfully predicted species interactions in a global plant-pollinator database and inferred ecologically plausible trait-matching rules for a plant-hummingbird network, without any prior assumptions. We conclude that flexible ML models offer many advantages over traditional regression models for understanding interaction networks. We anticipate that these results extrapolate to other ecological network types. More generally, our results highlight the potential of machine learning and artificial intelligence for inference in ecology, beyond standard tasks such as image or pattern recognition.
Tasks
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09853v2
PDF	https://arxiv.org/pdf/1908.09853v2.pdf
PWC	https://paperswithcode.com/paper/machine-learning-algorithms-to-infer-trait
Repo	https://github.com/TheoreticalEcology/Pichler-et-al-2019
Framework	none

Mean field theory for deep dropout networks: digging up gradient backpropagation deeply


Title	Mean field theory for deep dropout networks: digging up gradient backpropagation deeply
Authors	Wei Huang, Richard Yi Da Xu, Weitao Du, Yutian Zeng, Yunce Zhao
Abstract	In recent years, the mean field theory has been applied to the study of neural networks and has achieved a great deal of success. The theory has been applied to various neural network structures, including CNNs, RNNs, Residual networks, and Batch normalization. Inevitably, recent work has also covered the use of dropout. The mean field theory shows that the existence of depth scales that limit the maximum depth of signal propagation and gradient backpropagation. However, the gradient backpropagation is derived under the gradient independence assumption that weights used during feed forward are drawn independently from the ones used in backpropagation. This is not how neural networks are trained in a real setting. Instead, the same weights used in a feed-forward step needs to be carried over to its corresponding backpropagation. Using this realistic condition, we perform theoretical computation on linear dropout networks and a series of experiments on dropout networks. Our empirical results show an interesting phenomenon that the length gradients can backpropagate for a single input and a pair of inputs are governed by the same depth scale. Besides, we study the relationship between variance and mean of statistical metrics of the gradient and shown an emergence of universality. Finally, we investigate the maximum trainable length for deep dropout networks through a series of experiments using MNIST and CIFAR10 and provide a more precise empirical formula that describes the trainable length than original work.
Tasks
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09132v2
PDF	https://arxiv.org/pdf/1912.09132v2.pdf
PWC	https://paperswithcode.com/paper/mean-field-theory-for-deep-dropout-networks
Repo	https://github.com/WeiHuang05/Mean-field-theory-for-deep-dropout-networks
Framework	tf

Compositional Questions Do Not Necessitate Multi-hop Reasoning


Title	Compositional Questions Do Not Necessitate Multi-hop Reasoning
Authors	Sewon Min, Eric Wallace, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi, Luke Zettlemoyer
Abstract	Multi-hop reading comprehension (RC) questions are challenging because they require reading and reasoning over multiple paragraphs. We argue that it can be difficult to construct large multi-hop RC datasets. For example, even highly compositional questions can be answered with a single hop if they target specific entity types, or the facts needed to answer them are redundant. Our analysis is centered on HotpotQA, where we show that single-hop reasoning can solve much more of the dataset than previously thought. We introduce a single-hop BERT-based RC model that achieves 67 F1—comparable to state-of-the-art multi-hop models. We also design an evaluation setting where humans are not shown all of the necessary paragraphs for the intended multi-hop reasoning but can still answer over 80% of questions. Together with detailed error analysis, these results suggest there should be an increasing focus on the role of evidence in multi-hop reasoning and possibly even a shift towards information retrieval style evaluations with large and diverse evidence collections.
Tasks	Information Retrieval, Multi-Hop Reading Comprehension, Reading Comprehension
Published	2019-06-07
URL	https://arxiv.org/abs/1906.02900v1
PDF	https://arxiv.org/pdf/1906.02900v1.pdf
PWC	https://paperswithcode.com/paper/compositional-questions-do-not-necessitate
Repo	https://github.com/shmsw25/single-hop-rc
Framework	pytorch

A Cluster Ranking Model for Full Anaphora Resolution


Title	A Cluster Ranking Model for Full Anaphora Resolution
Authors	Juntao Yu, Alexandra Uma, Massimo Poesio
Abstract	Anaphora resolution (coreference) systems designed for the CONLL 2012 dataset typically cannot handle key aspects of the full anaphora resolution task such as the identification of singletons and of certain types of non-referring expressions (e.g., expletives), as these aspects are not annotated in that corpus. However, the recently released dataset for the CRAC 2018 Shared Task can now be used for that purpose. In this paper, we introduce an architecture to simultaneously identify non-referring expressions (including expletives, predicative {\NP}s, and other types) and build coreference chains, including singletons. Our cluster-ranking system uses an attention mechanism to determine the relative importance of the mentions in the same cluster. Additional classifiers are used to identify singletons and non-referring markables. Our contributions are as follows. First all, we report the first result on the CRAC data using system mentions; our result is 5.8% better than the shared task baseline system, which used gold mentions. Second, we demonstrate that the availability of singleton clusters and non-referring expressions can lead to substantially improved performance on non-singleton clusters as well. Third, we show that despite our model not being designed specifically for the CONLL data, it achieves a score equivalent to that of the state-of-the-art system by Kantor and Globerson (2019) on that dataset.
Tasks
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09532v1
PDF	https://arxiv.org/pdf/1911.09532v1.pdf
PWC	https://paperswithcode.com/paper/a-cluster-ranking-model-for-full-anaphora
Repo	https://github.com/juntaoy/dali-full-anaphora
Framework	tf

Decision Explanation and Feature Importance for Invertible Networks


Title	Decision Explanation and Feature Importance for Invertible Networks
Authors	Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Junlin Yang, James S. Duncan
Abstract	Deep neural networks are vulnerable to adversarial attacks and hard to interpret because of their black-box nature. The recently proposed invertible network is able to accurately reconstruct the inputs to a layer from its outputs, thus has the potential to unravel the black-box model. An invertible network classifier can be viewed as a two-stage model: (1) invertible transformation from input space to the feature space; (2) a linear classifier in the feature space. We can determine the decision boundary of a linear classifier in the feature space; since the transform is invertible, we can invert the decision boundary from the feature space to the input space. Furthermore, we propose to determine the projection of a data point onto the decision boundary, and define explanation as the difference between data and its projection. Finally, we propose to locally approximate a neural network with its first-order Taylor expansion, and define feature importance using a local linear model. We provide the implementation of our method: \url{https://github.com/juntang-zhuang/explain_invertible}.
Tasks	Feature Importance
Published	2019-09-30
URL	https://arxiv.org/abs/1910.00406v2
PDF	https://arxiv.org/pdf/1910.00406v2.pdf
PWC	https://paperswithcode.com/paper/decision-explanation-and-feature-importance
Repo	https://github.com/juntang-zhuang/explain_invertible
Framework	pytorch

From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network


Title	From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
Authors	Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, Hongsheng Li
Abstract	3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-$A^2$ net). The whole framework consists of the part-aware stage and the part-aggregation stage. Firstly, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-$A^2$ net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. Code is available at https://github.com/sshaoshuai/PointCloudDet3D.
Tasks	3D Object Detection, Object Detection, Scene Understanding
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03670v3
PDF	https://arxiv.org/pdf/1907.03670v3.pdf
PWC	https://paperswithcode.com/paper/part-a2-net-3d-part-aware-and-aggregation
Repo	https://github.com/sshaoshuai/PointCloudDet3D
Framework	pytorch

Learnable Tree Filter for Structure-preserving Feature Transform


Title	Learnable Tree Filter for Structure-preserving Feature Transform
Authors	Lin Song, Yanwei Li, Zeming Li, Gang Yu, Hongbin Sun, Jian Sun, Nanning Zheng
Abstract	Learning discriminative global features plays a vital role in semantic segmentation. And most of the existing methods adopt stacks of local convolutions or non-local blocks to capture long-range context. However, due to the absence of spatial structure preservation, these operators ignore the object details when enlarging receptive fields. In this paper, we propose the learnable tree filter to form a generic tree filtering module that leverages the structural property of minimal spanning tree to model long-range dependencies while preserving the details. Furthermore, we propose a highly efficient linear-time algorithm to reduce resource consumption. Thus, the designed modules can be plugged into existing deep neural networks conveniently. To this end, tree filtering modules are embedded to formulate a unified framework for semantic segmentation. We conduct extensive ablation studies to elaborate on the effectiveness and efficiency of the proposed method. Specifically, it attains better performance with much less overhead compared with the classic PSP block and Non-local operation under the same backbone. Our approach is proved to achieve consistent improvements on several benchmarks without bells-and-whistles. Code and models are available at https://github.com/StevenGrove/TreeFilter-Torch.
Tasks	Semantic Segmentation
Published	2019-09-27
URL	https://arxiv.org/abs/1909.12513v1
PDF	https://arxiv.org/pdf/1909.12513v1.pdf
PWC	https://paperswithcode.com/paper/learnable-tree-filter-for-structure
Repo	https://github.com/StevenGrove/TreeFilter-Torch
Framework	pytorch

Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments


Title	Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments
Authors	Zhizheng Zhang, Jiale Chen, Zhibo Chen, Weiping Li
Abstract	Deep Deterministic Policy Gradient (DDPG) has been proved to be a successful reinforcement learning (RL) algorithm for continuous control tasks. However, DDPG still suffers from data insufficiency and training inefficiency, especially in computationally complex environments. In this paper, we propose Asynchronous Episodic DDPG (AE-DDPG), as an expansion of DDPG, which can achieve more effective learning with less training time required. First, we design a modified scheme for data collection in an asynchronous fashion. Generally, for asynchronous RL algorithms, sample efficiency or/and training stability diminish as the degree of parallelism increases. We consider this problem from the perspectives of both data generation and data utilization. In detail, we re-design experience replay by introducing the idea of episodic control so that the agent can latch on good trajectories rapidly. In addition, we also inject a new type of noise in action space to enrich the exploration behaviors. Experiments demonstrate that our AE-DDPG achieves higher rewards and requires less time consuming than most popular RL algorithms in Learning to Run task which has a computationally complex environment. Not limited to the control tasks in computationally complex environments, AE-DDPG also achieves higher rewards and 2- to 4-fold improvement in sample efficiency on average compared to other variants of DDPG in MuJoCo environments. Furthermore, we verify the effectiveness of each proposed technique component through abundant ablation study.
Tasks	Continuous Control
Published	2019-03-03
URL	http://arxiv.org/abs/1903.00827v1
PDF	http://arxiv.org/pdf/1903.00827v1.pdf
PWC	https://paperswithcode.com/paper/asynchronous-episodic-deep-deterministic
Repo	https://github.com/anita-hu/TF2-RL
Framework	tf

Counterexample-Driven Synthesis for Probabilistic Program Sketches


Title	Counterexample-Driven Synthesis for Probabilistic Program Sketches
Authors	Milan Češka, Christian Hensel, Sebastian Junges, Joost-Pieter Katoen
Abstract	Probabilistic programs are key to deal with uncertainty in e.g. controller synthesis. They are typically small but intricate. Their development is complex and error prone requiring quantitative reasoning over a myriad of alternative designs. To mitigate this complexity, we adopt counterexample-guided inductive synthesis (CEGIS) to automatically synthesise finite-state probabilistic programs. Our approach leverages efficient model checking, modern SMT solving, and counterexample generation at program level. Experiments on practically relevant case studies show that design spaces with millions of candidate designs can be fully explored using a few thousand verification queries.
Tasks
Published	2019-04-28
URL	http://arxiv.org/abs/1904.12371v1
PDF	http://arxiv.org/pdf/1904.12371v1.pdf
PWC	https://paperswithcode.com/paper/counterexample-driven-synthesis-for
Repo	https://github.com/moves-rwth/sketching
Framework	none

Skin Lesion Classification Using CNNs with Patch-Based Attention and Diagnosis-Guided Loss Weighting


Title	Skin Lesion Classification Using CNNs with Patch-Based Attention and Diagnosis-Guided Loss Weighting
Authors	Nils Gessert, Thilo Sentker, Frederic Madesta, Rüdiger Schmitz, Helge Kniep, Ivo Baltruschat, René Werner, Alexander Schlaefer
Abstract	Objective: This work addresses two key problems of skin lesion classification. The first problem is the effective use of high-resolution images with pretrained standard architectures for image classification. The second problem is the high class imbalance encountered in real-world multi-class datasets. Methods: To use high-resolution images, we propose a novel patch-based attention architecture that provides global context between small, high-resolution patches. We modify three pretrained architectures and study the performance of patch-based attention. To counter class imbalance problems, we compare oversampling, balanced batch sampling, and class-specific loss weighting. Additionally, we propose a novel diagnosis-guided loss weighting method which takes the method used for ground-truth annotation into account. Results: Our patch-based attention mechanism outperforms previous methods and improves the mean sensitivity by 7%. Class balancing significantly improves the mean sensitivity and we show that our diagnosis-guided loss weighting method improves the mean sensitivity by 3% over normal loss balancing. Conclusion: The novel patch-based attention mechanism can be integrated into pretrained architectures and provides global context between local patches while outperforming other patch-based methods. Hence, pretrained architectures can be readily used with high-resolution images without downsampling. The new diagnosis-guided loss weighting method outperforms other methods and allows for effective training when facing class imbalance. Significance: The proposed methods improve automatic skin lesion classification. They can be extended to other clinical applications where high-resolution image data and class imbalance are relevant.
Tasks	Image Classification, Skin Lesion Classification
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02793v2
PDF	https://arxiv.org/pdf/1905.02793v2.pdf
PWC	https://paperswithcode.com/paper/skin-lesion-classification-using-cnns-with
Repo	https://github.com/ngessert/patch-lesion
Framework	pytorch

Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data


Title	Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data
Authors	Nils Gessert, Maximilian Nielsen, Mohsin Shaikh, René Werner, Alexander Schlaefer
Abstract	In this paper, we describe our method for the ISIC 2019 Skin Lesion Classification Challenge. The challenge comes with two tasks. For task 1, skin lesions have to be classified based on dermoscopic images. For task 2, dermoscopic images and additional patient meta data have to be used. A diverse dataset of 25000 images was provided for training, containing images from eight classes. The final test set contains an additional, unknown class. We address this challenging problem with a simple, data driven approach by including external data with skin lesions types that are not present in the training set. Furthermore, multi-class skin lesion classification comes with the problem of severe class imbalance. We try to overcome this problem by using loss balancing. Also, the dataset contains images with very different resolutions. We take care of this property by considering different model input resolutions and different cropping strategies. To incorporate meta data such as age, anatomical site, and sex, we use an additional dense neural network and fuse its features with the CNN. We aggregate all our models with an ensembling strategy where we search for the optimal subset of models. Our best ensemble achieves a balanced accuracy of 74.2% using five-fold cross-validation. On the official test set our method is ranked first for both tasks with a balanced accuracy of 63.6% for task 1 and 63.4% for task 2.
Tasks	Skin Lesion Classification
Published	2019-10-09
URL	https://arxiv.org/abs/1910.03910v1
PDF	https://arxiv.org/pdf/1910.03910v1.pdf
PWC	https://paperswithcode.com/paper/skin-lesion-classification-using-ensembles-of
Repo	https://github.com/ngessert/isic2019
Framework	pytorch