January 29, 2020

3068 words 15 mins read

Paper Group ANR 577

Deciphering Dynamical Nonlinearities in Short Time Series Using Recurrent Neural Networks. Learning Functions over Sets via Permutation Adversarial Networks. Rethinking Full Connectivity in Recurrent Neural Networks. REBEC: Robust Evolutionary-based Calibration Approach for the Numerical Wind Wave Model. Semi-Supervised Medical Image Segmentation v …

Deciphering Dynamical Nonlinearities in Short Time Series Using Recurrent Neural Networks


Title	Deciphering Dynamical Nonlinearities in Short Time Series Using Recurrent Neural Networks
Authors	Radhakrishnan Nagarajan
Abstract	Surrogate testing techniques have been used widely to investigate the presence of dynamical nonlinearities, an essential ingredient of deterministic chaotic processes. Traditional surrogate testing subscribes to statistical hypothesis testing and investigates potential differences in discriminant statistics between the given empirical sample and its surrogate counterparts. The choice and estimation of the discriminant statistics can be challenging across short time series. Also, conclusion based on a single empirical sample is an inherent limitation. The present study proposes a recurrent neural network classification framework that uses the raw time series obviating the need for discriminant statistic while accommodating multiple time series realizations for enhanced generalizability of the findings. The results are demonstrated on short time series with lengths (L = 32, 64, 128) from continuous and discrete dynamical systems in chaotic regimes, nonlinear transform of linearly correlated noise and experimental data. Accuracy of the classifier is shown to be markedly higher than » 50% for the processes in chaotic regimes whereas those of nonlinearly correlated noise were around ~50% similar to that of random guess from a one-sample binomial test. These results are promising and elucidate the usefulness of the proposed framework in identifying potential dynamical nonlinearities from short experimental time series.
Tasks	Time Series
Published	2019-07-15
URL	https://arxiv.org/abs/1907.07181v1
PDF	https://arxiv.org/pdf/1907.07181v1.pdf
PWC	https://paperswithcode.com/paper/deciphering-dynamical-nonlinearities-in-short
Repo
Framework

Learning Functions over Sets via Permutation Adversarial Networks


Title	Learning Functions over Sets via Permutation Adversarial Networks
Authors	Chirag Pabbaraju, Prateek Jain
Abstract	In this paper, we consider the problem of learning functions over sets, i.e., functions that are invariant to permutations of input set items. Recent approaches of pooling individual element embeddings can necessitate extremely large embedding sizes for challenging functions. We address this challenge by allowing standard neural networks like LSTMs to succinctly capture the function over the set. However, to ensure invariance with respect to permutations of set elements, we propose a novel architecture called SPAN that simultaneously learns the function as well as adversarial or worst-case permutations for each input set. The learning problem reduces to a min-max optimization problem that is solved via a simple alternating block coordinate descent technique. We conduct extensive experiments on a variety of set-learning tasks and demonstrate that SPAN learns nearly permutation-invariant functions while still ensuring accuracy on test data. On a variety of tasks sampled from the domains of statistics, graph functions and linear algebra, we show that our method can significantly outperform state-of-the-art methods such as DeepSets and Janossy Pooling. Finally, we present a case study of how learning set-functions can help extract powerful features for recommendation systems, and show that such a method can be as much as 2% more accurate than carefully hand-tuned features on a real-world recommendation system.
Tasks	Recommendation Systems
Published	2019-07-12
URL	https://arxiv.org/abs/1907.05638v2
PDF	https://arxiv.org/pdf/1907.05638v2.pdf
PWC	https://paperswithcode.com/paper/learning-functions-over-sets-via-permutation
Repo
Framework

Rethinking Full Connectivity in Recurrent Neural Networks


Title	Rethinking Full Connectivity in Recurrent Neural Networks
Authors	Matthijs Van Keirsbilck, Alexander Keller, Xiaodong Yang
Abstract	Recurrent neural networks (RNNs) are omnipresent in sequence modeling tasks. Practical models usually consist of several layers of hundreds or thousands of neurons which are fully connected. This places a heavy computational and memory burden on hardware, restricting adoption in practical low-cost and low-power devices. Compared to fully convolutional models, the costly sequential operation of RNNs severely hinders performance on parallel hardware. This paper challenges the convention of full connectivity in RNNs. We study structurally sparse RNNs, showing that they are well suited for acceleration on parallel hardware, with a greatly reduced cost of the recurrent operations as well as orders of magnitude less recurrent weights. Extensive experiments on challenging tasks ranging from language modeling and speech recognition to video action recognition reveal that structurally sparse RNNs achieve competitive performance as compared to fully-connected networks. This allows for using large sparse RNNs for a wide range of real-world tasks that previously were too costly with fully connected networks.
Tasks	Language Modelling, Speech Recognition, Temporal Action Localization
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12340v1
PDF	https://arxiv.org/pdf/1905.12340v1.pdf
PWC	https://paperswithcode.com/paper/rethinking-full-connectivity-in-recurrent
Repo
Framework

REBEC: Robust Evolutionary-based Calibration Approach for the Numerical Wind Wave Model


Title	REBEC: Robust Evolutionary-based Calibration Approach for the Numerical Wind Wave Model
Authors	Pavel Vychuzhanin, Nikolay O. Nikitin, Anna V. Kalyuzhnaya
Abstract	The adaptation of numerical wind wave models to the local time-spatial conditions is a problem that can be solved by using various calibration techniques. However, the obtained sets of physical parameters become over-tuned to specific events if there is a lack of observations. In this paper, we propose a robust evolutionary calibration approach that allows to build the stochastic ensemble of perturbed models and use it to achieve the trade-off between quality and robustness of the target model. The implemented robust ensemble-based evolutionary calibration (REBEC) approach was compared to the baseline SPEA2 algorithm in a set of experiments with the SWAN wind wave model configuration for the Kara Sea domain. Provided metrics for the set of scenarios confirm the effectiveness of the REBEC approach for the majority of calibration scenarios.
Tasks	Calibration
Published	2019-03-19
URL	https://arxiv.org/abs/1906.08587v1
PDF	https://arxiv.org/pdf/1906.08587v1.pdf
PWC	https://paperswithcode.com/paper/rebec-robust-evolutionary-based-calibration
Repo
Framework

Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations


Title	Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations
Authors	Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, Marleen de Bruijne
Abstract	The scarcity of labeled data often limits the application of supervised deep learning techniques for medical image segmentation. This has motivated the development of semi-supervised techniques that learn from a mixture of labeled and unlabeled images. In this paper, we propose a novel semi-supervised method that, in addition to supervised learning on labeled training images, learns to predict segmentations consistent under a given class of transformations on both labeled and unlabeled images. More specifically, in this work we explore learning equivariance to elastic deformations. We implement this through: 1) a Siamese architecture with two identical branches, each of which receives a differently transformed image, and 2) a composite loss function with a supervised segmentation loss term and an unsupervised term that encourages segmentation consistency between the predictions of the two branches. We evaluate the method on a public dataset of chest radiographs with segmentations of anatomical structures using 5-fold cross-validation. The proposed method reaches significantly higher segmentation accuracy compared to supervised learning. This is due to learning transformation consistency on both labeled and unlabeled images, with the latter contributing the most. We achieve the performance comparable to state-of-the-art chest X-ray segmentation methods while using substantially fewer labeled images.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01218v1
PDF	https://arxiv.org/pdf/1911.01218v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-medical-image-segmentation
Repo
Framework

Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction


Title	Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction
Authors	Yufang Hou, Charles Jochim, Martin Gleize, Francesca Bonin, Debasis Ganguly
Abstract	While the fast-paced inception of novel tasks and new datasets helps foster active research in a community towards interesting directions, keeping track of the abundance of research activity in different areas on different datasets is likely to become increasingly difficult. The community could greatly benefit from an automatic system able to summarize scientific results, e.g., in the form of a leaderboard. In this paper we build two datasets and develop a framework (TDMS-IE) aimed at automatically extracting task, dataset, metric and score from NLP papers, towards the automatic construction of leaderboards. Experiments show that our model outperforms several baselines by a large margin. Our model is a first step towards automatic leaderboard construction, e.g., in the NLP domain.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.09317v1
PDF	https://arxiv.org/pdf/1906.09317v1.pdf
PWC	https://paperswithcode.com/paper/identification-of-tasks-datasets-evaluation
Repo
Framework

Extracting evidence of supplement-drug interactions from literature


Title	Extracting evidence of supplement-drug interactions from literature
Authors	Lucy Lu Wang, Oyvind Tafjord, Sarthak Jain, Arman Cohan, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Waleed Ammar
Abstract	To improve discovery of dietary supplement safety information, we demonstrate an automated method for extracting evidence of supplement-drug interactions (SDIs) from scientific text. To address the lack of labeled data in this domain, we use labels of the closely related task of identifying drug-drug interactions (DDIs) for supervision. We fine-tune the contextualized word representations of BERT-large using labeled data from the PDDI corpus. We process 22M abstracts from PubMed using this model, and extract evidence for 55946 unique interactions between 1923 supplements and 2727 drugs (precision: 0.74, accuracy: 0.83), demonstrating that learning the task of DDI classification transfers successfully to the related problem of SDI classification. We implement a freely-available public interface supp.ai to browse and search evidence sentences extracted by our model.
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.08135v2
PDF	https://arxiv.org/pdf/1909.08135v2.pdf
PWC	https://paperswithcode.com/paper/extracting-evidence-of-supplement-drug
Repo
Framework

Improved Differentially Private Decentralized Source Separation for fMRI Data


Title	Improved Differentially Private Decentralized Source Separation for fMRI Data
Authors	Hafiz Imtiaz, Jafar Mohammadi, Rogers Silva, Bradley Baker, Sergey M. Plis, Anand D. Sarwate, Vince Calhoun
Abstract	Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. In order to leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, precluding centralized analyses that pool the data at a single site. A recently proposed algorithm uses message-passing between sites and a central aggregator to perform a decentralized joint ICA (djICA) without sharing the data. However, this method does not satisfy formal privacy guarantees. We propose a differentially private algorithm for performing ICA in a decentralized data setting. Differential privacy provides a formal and mathematically rigorous privacy guarantee by introducing noise into the messages. Conventional approaches to decentralized differentially private algorithms may require too much noise due to the typically small sample sizes at each site. We leverage a recently proposed correlated noise protocol to remedy the excessive noise problem of the conventional schemes. We investigate the performance of the proposed algorithm on synthetic and real fMRI datasets to show that our algorithm outperforms existing approaches and can sometimes reach the same level of utility as the corresponding non-private algorithm. This indicates that it is possible to have meaningful utility while preserving privacy.
Tasks
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12913v1
PDF	https://arxiv.org/pdf/1910.12913v1.pdf
PWC	https://paperswithcode.com/paper/improved-differentially-private-decentralized
Repo
Framework

Analog forecasting of extreme-causing weather patterns using deep learning


Title	Analog forecasting of extreme-causing weather patterns using deep learning
Authors	Ashesh Chattopadhyay, Ebrahim Nabizadeh, Pedram Hassanzadeh
Abstract	Numerical weather prediction (NWP) models require ever-growing computing time/resources, but still, have difficulties with predicting weather extremes. Here we introduce a data-driven framework that is based on analog forecasting (prediction using past similar patterns) and employs a novel deep learning pattern-recognition technique (capsule neural networks, CapsNets) and impact-based auto-labeling strategy. CapsNets are trained on mid-tropospheric large-scale circulation patterns (Z500) labeled $0-4$ depending on the existence and geographical region of surface temperature extremes over North America several days ahead. The trained networks predict the occurrence/region of cold or heat waves, only using Z500, with accuracies (recalls) of $69%-45%$ $(77%-48%)$ or $62%-41%$ $(73%-47%)$ $1-5$ days ahead. CapsNets outperform simpler techniques such as convolutional neural networks and logistic regression. Using both temperature and Z500, accuracies (recalls) with CapsNets increase to $\sim 80%$ $(88%)$, showing the promises of multi-modal data-driven frameworks for accurate/fast extreme weather predictions, which can augment NWP efforts in providing early warnings.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11617v2
PDF	https://arxiv.org/pdf/1907.11617v2.pdf
PWC	https://paperswithcode.com/paper/analog-forecasting-of-extreme-causing-weather
Repo
Framework


Title	ASR is all you need: cross-modal distillation for lip reading
Authors	Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman
Abstract	The goal of this work is to train strong models for visual speech recognition without requiring human annotated ground truth data. We achieve this by distilling from an Automatic Speech Recognition (ASR) model that has been trained on a large-scale audio-only corpus. We use a cross-modal distillation method that combines Connectionist Temporal Classification (CTC) with a frame-wise cross-entropy loss. Our contributions are fourfold: (i) we show that ground truth transcriptions are not necessary to train a lip reading system; (ii) we show how arbitrary amounts of unlabelled video data can be leveraged to improve performance; (iii) we demonstrate that distillation significantly speeds up training; and, (iv) we obtain state-of-the-art results on the challenging LRS2 and LRS3 datasets for training only on publicly available data.
Tasks	Speech Recognition, Visual Speech Recognition
Published	2019-11-28
URL	https://arxiv.org/abs/1911.12747v2
PDF	https://arxiv.org/pdf/1911.12747v2.pdf
PWC	https://paperswithcode.com/paper/asr-is-all-you-need-cross-modal-distillation
Repo
Framework

Improved Conditional VRNNs for Video Prediction


Title	Improved Conditional VRNNs for Video Prediction
Authors	Lluis Castrejon, Nicolas Ballas, Aaron Courville
Abstract	Predicting future frames for a video sequence is a challenging generative modeling task. Promising approaches include probabilistic latent variable models such as the Variational Auto-Encoder. While VAEs can handle uncertainty and model multiple possible future outcomes, they have a tendency to produce blurry predictions. In this work we argue that this is a sign of underfitting. To address this issue, we propose to increase the expressiveness of the latent distributions and to use higher capacity likelihood models. Our approach relies on a hierarchy of latent variables, which defines a family of flexible prior and posterior distributions in order to better model the probability of future sequences. We validate our proposal through a series of ablation experiments and compare our approach to current state-of-the-art latent variable models. Our method performs favorably under several metrics in three different datasets.
Tasks	Latent Variable Models, Video Prediction
Published	2019-04-27
URL	http://arxiv.org/abs/1904.12165v1
PDF	http://arxiv.org/pdf/1904.12165v1.pdf
PWC	https://paperswithcode.com/paper/improved-conditional-vrnns-for-video
Repo
Framework

Learning interaction kernels in heterogeneous systems of agents from multiple trajectories


Title	Learning interaction kernels in heterogeneous systems of agents from multiple trajectories
Authors	Fei Lu, Mauro Maggioni, Sui Tang
Abstract	Systems of interacting particles or agents have wide applications in many disciplines such as Physics, Chemistry, Biology and Economics. These systems are governed by interaction laws, which are often unknown: estimating them from observation data is a fundamental task that can provide meaningful insights and accurate predictions of the behaviour of the agents. In this paper, we consider the inverse problem of learning interaction laws given data from multiple trajectories, in a nonparametric fashion, when the interaction kernels depend on pairwise distances. We establish a condition for learnability of interaction kernels, and construct estimators that are guaranteed to converge in a suitable $L^2$ space, at the optimal min-max rate for 1-dimensional nonparametric regression. We propose an efficient learning algorithm based on least squares, which can be implemented in parallel for multiple trajectories and is therefore well-suited for the high dimensional, big data regime. Numerical simulations on a variety examples, including opinion dynamics, predator-swarm dynamics and heterogeneous particle dynamics, suggest that the learnability condition is satisfied in models used in practice, and the rate of convergence of our estimator is consistent with the theory. These simulations also suggest that our estimators are robust to noise in the observations, and produce accurate predictions of dynamics in relative large time intervals, even when they are learned from data collected in short time intervals.
Tasks
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04832v2
PDF	https://arxiv.org/pdf/1910.04832v2.pdf
PWC	https://paperswithcode.com/paper/learning-interaction-kernels-in-heterogeneous
Repo
Framework

A multi-series framework for demand forecasts in E-commerce


Title	A multi-series framework for demand forecasts in E-commerce
Authors	Rémy Garnier, Arnaud Belletoile
Abstract	Sales forecasts are crucial for the E-commerce business. State-of-the-art techniques typically apply only univariate methods to make prediction for each series independently. However, due to the short nature of sales times series in E-commerce, univariate methods don’t apply well. In this article, we propose a global model which outperforms state-of-the-art models on real dataset. It is achieved by using Tree Boosting Methods that exploit non-linearity and cross-series information. We also proposed a preprocessing framework to overcome the inherent difficulties in the E-commerce data. In particular, we use different schemes to limit the impact of the volatility of the data.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13614v1
PDF	https://arxiv.org/pdf/1905.13614v1.pdf
PWC	https://paperswithcode.com/paper/a-multi-series-framework-for-demand-forecasts
Repo
Framework

Unified Multi-scale Feature Abstraction for Medical Image Segmentation


Title	Unified Multi-scale Feature Abstraction for Medical Image Segmentation
Authors	Xi Fang, Bo Du, Sheng Xu, Bradford J. Wood, Pingkun Yan
Abstract	Automatic medical image segmentation, an essential component of medical image analysis, plays an importantrole in computer-aided diagnosis. For example, locating and segmenting the liver can be very helpful in livercancer diagnosis and treatment. The state-of-the-art models in medical image segmentation are variants ofthe encoder-decoder architecture such as fully convolutional network (FCN) and U-Net.1A major focus ofthe FCN based segmentation methods has been on network structure engineering by incorporating the latestCNN structures such as ResNet2and DenseNet.3In addition to exploring new network structures for efficientlyabstracting high level features, incorporating structures for multi-scale image feature extraction in FCN hashelped to improve performance in segmentation tasks. In this paper, we design a new multi-scale networkarchitecture, which takes multi-scale inputs with dedicated convolutional paths to efficiently combine featuresfrom different scales to better utilize the hierarchical information.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2019-10-24
URL	https://arxiv.org/abs/1910.11456v1
PDF	https://arxiv.org/pdf/1910.11456v1.pdf
PWC	https://paperswithcode.com/paper/unified-multi-scale-feature-abstraction-for
Repo
Framework

CROWN: Conversational Passage Ranking by Reasoning over Word Networks


Title	CROWN: Conversational Passage Ranking by Reasoning over Word Networks
Authors	Magdalena Kaiser, Rishiraj Saha Roy, Gerhard Weikum
Abstract	Information needs around a topic cannot be satisfied in a single turn; users typically ask follow-up questions referring to the same theme and a system must be capable of understanding the conversational context of a request to retrieve correct answers. In this paper, we present our submission to the TREC Conversational Assistance Track 2019, in which such a conversational setting is explored. We propose a simple unsupervised method for conversational passage ranking by formulating the passage score for a query as a combination of similarity and coherence. To be specific, passages are preferred that contain words semantically similar to the words used in the question, and where such words appear close by. We built a word-proximity network (WPN) from a large corpus, where words are nodes and there is an edge between two nodes if they co-occur in the same passages in a statistically significant way, within a context window. Our approach, named CROWN, improved nDCG scores over a provided Indri baseline on the CAsT training data. On the evaluation data for CAsT, our best run submission achieved above-average performance with respect to AP@5 and nDCG@1000.
Tasks
Published	2019-11-07
URL	https://arxiv.org/abs/1911.02850v3
PDF	https://arxiv.org/pdf/1911.02850v3.pdf
PWC	https://paperswithcode.com/paper/crown-conversational-passage-ranking-by
Repo
Framework