January 31, 2020

3196 words 16 mins read

Paper Group AWR 388

On Single Source Robustness in Deep Fusion Models. Predicting materials properties without crystal structure: Deep representation learning from stoichiometry. Predicting Research Trends with Semantic and Neural Networks with an application in Quantum Physics. Understanding and Controlling Memory in Recurrent Neural Networks. Calculating Optimistic …

On Single Source Robustness in Deep Fusion Models


Title	On Single Source Robustness in Deep Fusion Models
Authors	Taewan Kim, Joydeep Ghosh
Abstract	Algorithms that fuse multiple input sources benefit from both complementary and shared information. Shared information may provide robustness against faulty or noisy inputs, which is indispensable for safety-critical applications like self-driving cars. We investigate learning fusion algorithms that are robust against noise added to a single source. We first demonstrate that robustness against single source noise is not guaranteed in a linear fusion model. Motivated by this discovery, two possible approaches are proposed to increase robustness: a carefully designed loss with corresponding training algorithms for deep fusion models, and a simple convolutional fusion layer that has a structural advantage in dealing with noise. Experimental results show that both training algorithms and our fusion layer make a deep fusion-based 3D object detector robust against noise applied to a single source, while preserving the original performance on clean data.
Tasks	Self-Driving Cars
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04691v2
PDF	https://arxiv.org/pdf/1906.04691v2.pdf
PWC	https://paperswithcode.com/paper/on-single-source-robustness-in-deep-fusion
Repo	https://github.com/twankim/avod_ssn
Framework	tf

Predicting materials properties without crystal structure: Deep representation learning from stoichiometry


Title	Predicting materials properties without crystal structure: Deep representation learning from stoichiometry
Authors	Rhys E. A. Goodall, Alpha A. Lee
Abstract	Machine learning can accelerate materials discovery by accurately predicting materials properties with low computational cost. However, the model inputs remain a key stumbling block: current methods typically use hand-engineered descriptors constructed from knowledge of either the full crystal structure – applicable only to materials with experimentally measured structures as crystal structure prediction is computationally expensive – or the stoichiometry. We develop a machine learning approach that takes only the stoichiometry as input and automatically learns appropriate and systematically improvable descriptors from data. Our key insight is to treat the stoichiometric formula as a dense weighted graph between elements. Compared to the state of the art, our approach achieves lower error on a plethora of challenging material properties. Moreover, our model can estimate its own uncertainty as well as transfer its learnt representation, extracting useful information from a cognate data-abundant task to deploy on a data-poor task.
Tasks	Representation Learning
Published	2019-10-01
URL	https://arxiv.org/abs/1910.00617v2
PDF	https://arxiv.org/pdf/1910.00617v2.pdf
PWC	https://paperswithcode.com/paper/predicting-materials-properties-without
Repo	https://github.com/CompRhys/roost
Framework	pytorch

Predicting Research Trends with Semantic and Neural Networks with an application in Quantum Physics


Title	Predicting Research Trends with Semantic and Neural Networks with an application in Quantum Physics
Authors	Mario Krenn, Anton Zeilinger
Abstract	The vast and growing number of publications in all disciplines of science cannot be comprehended by a single human researcher. As a consequence, researchers have to specialize in narrow sub-disciplines, which makes it challenging to uncover scientific connections beyond the own field of research. Thus access to structured knowledge from a large corpus of publications could help pushing the frontiers of science. Here we demonstrate a method to build a semantic network from published scientific literature, which we call SemNet. We use SemNet to predict future trends in research and to inspire new, personalized and surprising seeds of ideas in science. We apply it in the discipline of quantum physics, which has seen an unprecedented growth of activity in recent years. In SemNet, scientific knowledge is represented as an evolving network using the content of 750,000 scientific papers published since 1919. The nodes of the network correspond to physical concepts, and links between two nodes are drawn when two physical concepts are concurrently studied in research articles. We identify influential and prize-winning research topics from the past inside SemNet thus confirm that it stores useful semantic knowledge. We train a deep neural network using states of SemNet of the past, to predict future developments in quantum physics research, and confirm high quality predictions using historic data. With the neural network and theoretical network tools we are able to suggest new, personalized, out-of-the-box ideas, by identifying pairs of concepts which have unique and extremal semantic network properties. Finally, we consider possible future developments and implications of our findings.
Tasks
Published	2019-06-17
URL	https://arxiv.org/abs/1906.06843v2
PDF	https://arxiv.org/pdf/1906.06843v2.pdf
PWC	https://paperswithcode.com/paper/predicting-research-trends-with-semantic-and
Repo	https://github.com/MarioKrenn6240/SEMNET
Framework	none

Understanding and Controlling Memory in Recurrent Neural Networks


Title	Understanding and Controlling Memory in Recurrent Neural Networks
Authors	Doron Haviv, Alexander Rivkind, Omri Barak
Abstract	To be effective in sequential data processing, Recurrent Neural Networks (RNNs) are required to keep track of past events by creating memories. While the relation between memories and the network’s hidden state dynamics was established over the last decade, previous works in this direction were of a predominantly descriptive nature focusing mainly on locating the dynamical objects of interest. In particular, it remained unclear how dynamical observables affect the performance, how they form and whether they can be manipulated. Here, we utilize different training protocols, datasets and architectures to obtain a range of networks solving a delayed classification task with similar performance, alongside substantial differences in their ability to extrapolate for longer delays. We analyze the dynamics of the network’s hidden state, and uncover the reasons for this difference. Each memory is found to be associated with a nearly steady state of the dynamics which we refer to as a ‘slow point’. Slow point speeds predict extrapolation performance across all datasets, protocols and architectures tested. Furthermore, by tracking the formation of the slow points we are able to understand the origin of differences between training protocols. Finally, we propose a novel regularization technique that is based on the relation between hidden state speeds and memory longevity. Our technique manipulates these speeds, thereby leading to a dramatic improvement in memory robustness over time, and could pave the way for a new class of regularization methods.
Tasks
Published	2019-02-19
URL	https://arxiv.org/abs/1902.07275v3
PDF	https://arxiv.org/pdf/1902.07275v3.pdf
PWC	https://paperswithcode.com/paper/understanding-and-controlling-memory-in
Repo	https://github.com/DoronHaviv/MemoryRNN
Framework	tf

Calculating Optimistic Likelihoods Using (Geodesically) Convex Optimization


Title	Calculating Optimistic Likelihoods Using (Geodesically) Convex Optimization
Authors	Viet Anh Nguyen, Soroosh Shafieezadeh-Abadeh, Man-Chung Yue, Daniel Kuhn, Wolfram Wiesemann
Abstract	A fundamental problem arising in many areas of machine learning is the evaluation of the likelihood of a given observation under different nominal distributions. Frequently, these nominal distributions are themselves estimated from data, which makes them susceptible to estimation errors. We thus propose to replace each nominal distribution with an ambiguity set containing all distributions in its vicinity and to evaluate an \emph{optimistic likelihood}, that is, the maximum of the likelihood over all distributions in the ambiguity set. When the proximity of distributions is quantified by the Fisher-Rao distance or the Kullback-Leibler divergence, the emerging optimistic likelihoods can be computed efficiently using either geodesic or standard convex optimization techniques. We showcase the advantages of working with optimistic likelihoods on a classification problem using synthetic as well as empirical data.
Tasks
Published	2019-10-17
URL	https://arxiv.org/abs/1910.07817v1
PDF	https://arxiv.org/pdf/1910.07817v1.pdf
PWC	https://paperswithcode.com/paper/calculating-optimistic-likelihoods-using
Repo	https://github.com/sorooshafiee/Optimistic_Likelihoods
Framework	none

Efficient Smoothing of Dilated Convolutions for Image Segmentation


Title	Efficient Smoothing of Dilated Convolutions for Image Segmentation
Authors	Thomas Ziegler, Manuel Fritsche, Lorenz Kuhn, Konstantin Donhauser
Abstract	Dilated Convolutions have been shown to be highly useful for the task of image segmentation. By introducing gaps into convolutional filters, they enable the use of larger receptive fields without increasing the original kernel size. Even though this allows for the inexpensive capturing of features at different scales, the structure of the dilated convolutional filter leads to a loss of information. We hypothesise that inexpensive modifications to Dilated Convolutional Neural Networks, such as additional averaging layers, could overcome this limitation. In this project we test this hypothesis by evaluating the effect of these modifications for a state-of-the art image segmentation system and compare them to existing approaches with the same objective. Our experiments show that our proposed methods improve the performance of dilated convolutions for image segmentation. Crucially, our modifications achieve these results at a much lower computational cost than previous smoothing approaches.
Tasks	Semantic Segmentation
Published	2019-03-19
URL	http://arxiv.org/abs/1903.07992v1
PDF	http://arxiv.org/pdf/1903.07992v1.pdf
PWC	https://paperswithcode.com/paper/efficient-smoothing-of-dilated-convolutions
Repo	https://github.com/ThomasZiegler/Efficient-Smoothing-of-Dilated-Convolutions
Framework	tf

Neural Text Generation with Unlikelihood Training


Title	Neural Text Generation with Unlikelihood Training
Authors	Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, Jason Weston
Abstract	Neural text generation is a key tool in natural language applications, but it is well known there are major problems at its core. In particular, standard likelihood training and decoding leads to dull and repetitive outputs. While some post-hoc fixes have been proposed, in particular top-$k$ and nucleus sampling, they do not address the fact that the token-level probabilities predicted by the model are poor. In this paper we show that the likelihood objective itself is at fault, resulting in a model that assigns too much probability to sequences containing repeats and frequent words, unlike those from the human training distribution. We propose a new objective, unlikelihood training, which forces unlikely generations to be assigned lower probability by the model. We show that both token and sequence level unlikelihood training give less repetitive, less dull text while maintaining perplexity, giving superior generations using standard greedy or beam search. According to human evaluations, our approach with standard beam search also outperforms the currently popular decoding methods of nucleus sampling or beam blocking, thus providing a strong alternative to existing techniques.
Tasks	Text Generation
Published	2019-08-12
URL	https://arxiv.org/abs/1908.04319v2
PDF	https://arxiv.org/pdf/1908.04319v2.pdf
PWC	https://paperswithcode.com/paper/neural-text-generation-with-unlikelihood
Repo	https://github.com/facebookresearch/unlikelihood_training
Framework	pytorch

ICLabel: An automated electroencephalographic independent component classifier, dataset, and website


Title	ICLabel: An automated electroencephalographic independent component classifier, dataset, and website
Authors	Luca Pion-Tonachini, Ken Kreutz-Delgado, Scott Makeig
Abstract	The electroencephalogram (EEG) provides a non-invasive, minimally restrictive, and relatively low cost measure of mesoscale brain dynamics with high temporal resolution. Although signals recorded in parallel by multiple, near-adjacent EEG scalp electrode channels are highly-correlated and combine signals from many different sources, biological and non-biological, independent component analysis (ICA) has been shown to isolate the various source generator processes underlying those recordings. Independent components (IC) found by ICA decomposition can be manually inspected, selected, and interpreted, but doing so requires both time and practice as ICs have no particular order or intrinsic interpretations and therefore require further study of their properties. Alternatively, sufficiently-accurate automated IC classifiers can be used to classify ICs into broad source categories, speeding the analysis of EEG studies with many subjects and enabling the use of ICA decomposition in near-real-time applications. While many such classifiers have been proposed recently, this work presents the ICLabel project comprised of (1) an IC dataset containing spatiotemporal measures for over 200,000 ICs from more than 6,000 EEG recordings, (2) a website for collecting crowdsourced IC labels and educating EEG researchers and practitioners about IC interpretation, and (3) the automated ICLabel classifier. The classifier improves upon existing methods in two ways: by improving the accuracy of the computed label estimates and by enhancing its computational efficiency. The ICLabel classifier outperforms or performs comparably to the previous best publicly available method for all measured IC categories while computing those labels ten times faster than that classifier as shown in a rigorous comparison against all other publicly available EEG IC classifiers.
Tasks	EEG
Published	2019-01-22
URL	http://arxiv.org/abs/1901.07915v2
PDF	http://arxiv.org/pdf/1901.07915v2.pdf
PWC	https://paperswithcode.com/paper/iclabel-an-automated-electroencephalographic
Repo	https://github.com/lucapton/ICLabel-Train
Framework	tf

Optimisation of Overparametrized Sum-Product Networks


Title	Optimisation of Overparametrized Sum-Product Networks
Authors	Martin Trapp, Robert Peharz, Franz Pernkopf
Abstract	It seems to be a pearl of conventional wisdom that parameter learning in deep sum-product networks is surprisingly fast compared to shallow mixture models. This paper examines the effects of overparameterization in sum-product networks on the speed of parameter optimisation. Using theoretical analysis and empirical experiments, we show that deep sum-product networks exhibit an implicit acceleration compared to their shallow counterpart. In fact, gradient-based optimisation in deep tree-structured sum-product networks is equal to gradient ascend with adaptive and time-varying learning rates and additional momentum terms.
Tasks
Published	2019-05-20
URL	https://arxiv.org/abs/1905.08196v2
PDF	https://arxiv.org/pdf/1905.08196v2.pdf
PWC	https://paperswithcode.com/paper/optimisation-of-overparametrized-sum-product
Repo	https://github.com/trappmartin/TPM2019
Framework	none

Learning to Interpret Satellite Images in Global Scale Using Wikipedia


Title	Learning to Interpret Satellite Images in Global Scale Using Wikipedia
Authors	Burak Uzkent, Evan Sheehan, Chenlin Meng, Zhongyi Tang, Marshall Burke, David Lobell, Stefano Ermon
Abstract	Despite recent progress in computer vision, finegrained interpretation of satellite images remains challenging because of a lack of labeled training data. To overcome this limitation, we construct a novel dataset called WikiSatNet by pairing georeferenced Wikipedia articles with satellite imagery of their corresponding locations. We then propose two strategies to learn representations of satellite images by predicting properties of the corresponding articles from the images. Leveraging this new multi-modal dataset, we can drastically reduce the quantity of human-annotated labels and time required for downstream tasks. On the recently released fMoW dataset, our pre-training strategies can boost the performance of a model pre-trained on ImageNet by up to 4:5% in F1 score.
Tasks
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02506v3
PDF	https://arxiv.org/pdf/1905.02506v3.pdf
PWC	https://paperswithcode.com/paper/learning-to-interpret-satellite-images-in
Repo	https://github.com/ermongroup/PretrainingWikiSatNet
Framework	pytorch

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis


Title	Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
Authors	Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, Hongsheng Li
Abstract	Semantic image synthesis aims at generating photorealistic images from semantic layouts. Previous approaches with conditional generative adversarial networks (GAN) show state-of-the-art performance on this task, which either feed the semantic label maps as inputs to the generator, or use them to modulate the activations in normalization layers via affine transformations. We argue that convolutional kernels in the generator should be aware of the distinct semantic labels at different locations when generating images. In order to better exploit the semantic layout for the image generator, we propose to predict convolutional kernels conditioned on the semantic label map to generate the intermediate feature maps from the noise maps and eventually generate the images. Moreover, we propose a feature pyramid semantics-embedding discriminator, which is more effective in enhancing fine details and semantic alignments between the generated images and the input semantic layouts than previous multi-scale discriminators. We achieve state-of-the-art results on both quantitative metrics and subjective evaluation on various semantic segmentation datasets, demonstrating the effectiveness of our approach.
Tasks	Image Generation, Image-to-Image Translation, Semantic Segmentation
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06809v3
PDF	https://arxiv.org/pdf/1910.06809v3.pdf
PWC	https://paperswithcode.com/paper/learning-to-predict-layout-to-image
Repo	https://github.com/xh-liu/CC-FPSE
Framework	pytorch

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment


Title	What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment
Authors	Paritosh Parmar, Brendan Tran Morris
Abstract	Can performance on the task of action quality assessment (AQA) be improved by exploiting a description of the action and its quality? Current AQA and skills assessment approaches propose to learn features that serve only one task - estimating the final score. In this paper, we propose to learn spatio-temporal features that explain three related tasks - fine-grained action recognition, commentary generation, and estimating the AQA score. A new multitask-AQA dataset, the largest to date, comprising of 1412 diving samples was collected to evaluate our approach (https://github.com/ParitoshParmar/MTL-AQA). We show that our MTL approach outperforms STL approach using two different kinds of architectures: C3D-AVG and MSCADC. The C3D-AVG-MTL approach achieves the new state-of-the-art performance with a rank correlation of 90.44%. Detailed experiments were performed to show that MTL offers better generalization than STL, and representations from action recognition models are not sufficient for the AQA task and instead should be learned.
Tasks	Temporal Action Localization
Published	2019-04-08
URL	https://arxiv.org/abs/1904.04346v2
PDF	https://arxiv.org/pdf/1904.04346v2.pdf
PWC	https://paperswithcode.com/paper/what-and-how-well-you-performed-a-multitask
Repo	https://github.com/ParitoshParmar/MTL-AQA
Framework	none

Interactive Machine Comprehension with Information Seeking Agents


Title	Interactive Machine Comprehension with Information Seeking Agents
Authors	Xingdi Yuan, Jie Fu, Marc-Alexandre Cote, Yi Tay, Christopher Pal, Adam Trischler
Abstract	Existing machine reading comprehension (MRC) models do not scale effectively to real-world applications like web-level information retrieval and question answering (QA). We argue that this stems from the nature of MRC datasets: most of these are static environments wherein the supporting documents and all necessary information are fully observed. In this paper, we propose a simple method that reframes existing MRC datasets as interactive, partially observable environments. Specifically, we “occlude” the majority of a document’s text and add context-sensitive commands that reveal “glimpses” of the hidden text to a model. We repurpose SQuAD and NewsQA as an initial case study, and then show how the interactive corpora can be used to train a model that seeks relevant information through sequential decision making. We believe that this setting can contribute in scaling models to web-level QA scenarios.
Tasks	Decision Making, Information Retrieval, Machine Reading Comprehension, Question Answering, Reading Comprehension
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10449v2
PDF	https://arxiv.org/pdf/1908.10449v2.pdf
PWC	https://paperswithcode.com/paper/interactive-machine-comprehension-with
Repo	https://github.com/xingdi-eric-yuan/imrc_public
Framework	pytorch

GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension


Title	GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension
Authors	Yu Chen, Lingfei Wu, Mohammed J. Zaki
Abstract	Conversational machine reading comprehension (MRC) has proven significantly more challenging compared to traditional MRC since it requires better utilization of conversation history. However, most existing approaches do not effectively capture conversation history and thus have trouble handling questions involving coreference or ellipsis. We propose a novel graph neural network (GNN) based model, namely GraphFlow, which captures conversational flow in the dialog. Specifically, we first propose a new approach to dynamically construct a question-aware context graph from passage text at each turn. We then present a novel flow mechanism to model the temporal dependencies in the sequence of context graphs. The proposed GraphFlow model shows superior performance compared to existing state-of-the-art methods. For instance, GraphFlow outperforms two recently proposed models on the CoQA benchmark dataset: FlowQA by 2.3% and SDNet by 0.7% on F1 score, respectively. In addition, visualization experiments show that our proposed model can better mimic the human reasoning process for conversational MRC compared to existing models.
Tasks	Machine Reading Comprehension, Reading Comprehension
Published	2019-07-31
URL	https://arxiv.org/abs/1908.00059v1
PDF	https://arxiv.org/pdf/1908.00059v1.pdf
PWC	https://paperswithcode.com/paper/graphflow-exploiting-conversation-flow-with
Repo	https://github.com/hugochan/GraphFlow
Framework	pytorch

Smoothed Inference for Adversarially-Trained Models


Title	Smoothed Inference for Adversarially-Trained Models
Authors	Yaniv Nemcovsky, Evgenii Zheltonozhskii, Chaim Baskin, Brian Chmiel, Maxim Fishman, Alex M. Bronstein, Avi Mendelson
Abstract	Deep neural networks are known to be vulnerable to adversarial attacks. Current methods of defense from such attacks are based on either implicit or explicit regularization, e.g., adversarial training. Randomized smoothing, the averaging of the classifier outputs over a random distribution centered in the sample, has been shown to guarantee the performance of a classifier subject to bounded perturbations of the input. In this work, we study the application of randomized smoothing as a way to improve performance on unperturbed data as well as to increase robustness to adversarial attacks. The proposed technique can be applied on top of any existing adversarial defense, but works particularly well with the randomized approaches. We examine its performance on common white-box (PGD) and black-box (transfer and NAttack) attacks on CIFAR-10 and CIFAR-100, substantially outperforming previous art for most scenarios and comparable on others. For example, we achieve 60.4% accuracy under a PGD attack on CIFAR-10 using ResNet-20, outperforming previous art by 11.7%. Since our method is based on sampling, it lends itself well for trading-off between the model inference complexity and its performance. A reference implementation of the proposed techniques is provided at https://github.com/yanemcovsky/SIAM
Tasks	Adversarial Defense
Published	2019-11-17
URL	https://arxiv.org/abs/1911.07198v2
PDF	https://arxiv.org/pdf/1911.07198v2.pdf
PWC	https://paperswithcode.com/paper/smoothed-inference-for-adversarially-trained
Repo	https://github.com/yanemcovsky/SIAM
Framework	pytorch