January 29, 2020

3667 words 18 mins read

Paper Group ANR 740

One-dimensional Deep Image Prior for Time Series Inverse Problems. A Semi-Supervised Machine Learning Approach to Detecting Recurrent Metastatic Breast Cancer Cases Using Linked Cancer Registry and Electronic Medical Record Data. Detecting Machine-Translated Paragraphs by Matching Similar Words. Subspace Robust Wasserstein Distances. Training recur …

One-dimensional Deep Image Prior for Time Series Inverse Problems


Title	One-dimensional Deep Image Prior for Time Series Inverse Problems
Authors	Sriram Ravula, Alexandros G. Dimakis
Abstract	We extend the Deep Image Prior (DIP) framework to one-dimensional signals. DIP is using a randomly initialized convolutional neural network (CNN) to solve linear inverse problems by optimizing over weights to fit the observed measurements. Our main finding is that properly tuned one-dimensional convolutional architectures provide an excellent Deep Image Prior for various types of temporal signals including audio, biological signals, and sensor measurements. We show that our network can be used in a variety of recovery tasks including missing value imputation, blind denoising, and compressed sensing from random Gaussian projections. The key challenge is how to avoid overfitting by carefully tuning early stopping, total variation, and weight decay regularization. Our method requires up to 4 times fewer measurements than Lasso and outperforms NLM-VAMP for random Gaussian measurements on audio signals, has similar imputation performance to a Kalman state-space model on a variety of data, and outperforms wavelet filtering in removing additive noise from air-quality sensor readings.
Tasks	Denoising, Imputation, Time Series
Published	2019-04-18
URL	http://arxiv.org/abs/1904.08594v1
PDF	http://arxiv.org/pdf/1904.08594v1.pdf
PWC	https://paperswithcode.com/paper/one-dimensional-deep-image-prior-for-time
Repo
Framework

A Semi-Supervised Machine Learning Approach to Detecting Recurrent Metastatic Breast Cancer Cases Using Linked Cancer Registry and Electronic Medical Record Data


Title	A Semi-Supervised Machine Learning Approach to Detecting Recurrent Metastatic Breast Cancer Cases Using Linked Cancer Registry and Electronic Medical Record Data
Authors	Albee Y. Ling, Allison W. Kurian, Jennifer L. Caswell-Jin, George W. Sledge Jr., Nigam H. Shah, Suzanne R. Tamang
Abstract	Objectives: Most cancer data sources lack information on metastatic recurrence. Electronic medical records (EMRs) and population-based cancer registries contain complementary information on cancer treatment and outcomes, yet are rarely used synergistically. To enable detection of metastatic breast cancer (MBC), we applied a semi-supervised machine learning framework to linked EMR-California Cancer Registry (CCR) data. Materials and Methods: We studied 11,459 female patients treated at Stanford Health Care who received an incident breast cancer diagnosis from 2000-2014. The dataset consisted of structured data and unstructured free-text clinical notes from EMR, linked to CCR, a component of the Surveillance, Epidemiology and End Results (SEER) database. We extracted information on metastatic disease from patient notes to infer a class label and then trained a regularized logistic regression model for MBC classification. We evaluated model performance on a gold standard set of set of 146 patients. Results: There are 495 patients with de novo stage IV MBC, 1,374 patients initially diagnosed with Stage 0-III disease had recurrent MBC, and 9,590 had no evidence of metastatis. The median follow-up time is 96.3 months (mean 97.8, standard deviation 46.7). The best-performing model incorporated both EMR and CCR features. The area under the receiver-operating characteristic curve=0.925 [95% confidence interval: 0.880-0.969], sensitivity=0.861, specificity=0.878 and overall accuracy=0.870. Discussion and Conclusion: A framework for MBC case detection combining EMR and CCR data achieved good sensitivity, specificity and discrimination without requiring expert-labeled examples. This approach enables population-based research on how patients die from cancer and may identify novel predictors of cancer recurrence.
Tasks	Epidemiology
Published	2019-01-17
URL	http://arxiv.org/abs/1901.05958v1
PDF	http://arxiv.org/pdf/1901.05958v1.pdf
PWC	https://paperswithcode.com/paper/a-semi-supervised-machine-learning-approach
Repo
Framework

Detecting Machine-Translated Paragraphs by Matching Similar Words


Title	Detecting Machine-Translated Paragraphs by Matching Similar Words
Authors	Hoang-Quoc Nguyen-Son, Tran Phuong Thao, Seira Hidano, Shinsaku Kiyomoto
Abstract	Machine-translated text plays an important role in modern life by smoothing communication from various communities using different languages. However, unnatural translation may lead to misunderstanding, a detector is thus needed to avoid the unfortunate mistakes. While a previous method measured the naturalness of continuous words using a N-gram language model, another method matched noncontinuous words across sentences but this method ignores such words in an individual sentence. We have developed a method matching similar words throughout the paragraph and estimating the paragraph-level coherence, that can identify machine-translated text. Experiment evaluates on 2000 English human-generated and 2000 English machine-translated paragraphs from German showing that the coherence-based method achieves high performance (accuracy = 87.0%; equal error rate = 13.0%). It is efficiently better than previous methods (best accuracy = 72.4%; equal error rate = 29.7%). Similar experiments on Dutch and Japanese obtain 89.2% and 97.9% accuracy, respectively. The results demonstrate the persistence of the proposed method in various languages with different resource levels.
Tasks	Language Modelling
Published	2019-04-24
URL	http://arxiv.org/abs/1904.10641v1
PDF	http://arxiv.org/pdf/1904.10641v1.pdf
PWC	https://paperswithcode.com/paper/detecting-machine-translated-paragraphs-by
Repo
Framework

Subspace Robust Wasserstein Distances


Title	Subspace Robust Wasserstein Distances
Authors	François-Pierre Paty, Marco Cuturi
Abstract	Making sense of Wasserstein distances between discrete measures in high-dimensional settings remains a challenge. Recent work has advocated a two-step approach to improve robustness and facilitate the computation of optimal transport, using for instance projections on random real lines, or a preliminary quantization of the measures to reduce the size of their support. We propose in this work a “max-min” robust variant of the Wasserstein distance by considering the maximal possible distance that can be realized between two measures, assuming they can be projected orthogonally on a lower $k$-dimensional subspace. Alternatively, we show that the corresponding “min-max” OT problem has a tight convex relaxation which can be cast as that of finding an optimal transport plan with a low transportation cost, where the cost is alternatively defined as the sum of the $k$ largest eigenvalues of the second order moment matrix of the displacements (or matchings) corresponding to that plan (the usual OT definition only considers the trace of that matrix). We show that both quantities inherit several favorable properties from the OT geometry. We propose two algorithms to compute the latter formulation using entropic regularization, and illustrate the interest of this approach empirically.
Tasks	Quantization
Published	2019-01-25
URL	https://arxiv.org/abs/1901.08949v5
PDF	https://arxiv.org/pdf/1901.08949v5.pdf
PWC	https://paperswithcode.com/paper/subspace-robust-wasserstein-distances
Repo
Framework

Training recurrent neural networks robust to incomplete data: application to Alzheimer’s disease progression modeling


Title	Training recurrent neural networks robust to incomplete data: application to Alzheimer’s disease progression modeling
Authors	Mostafa Mehdipour Ghazi, Mads Nielsen, Akshay Pai, M. Jorge Cardoso, Marc Modat, Sebastien Ourselin, Lauge Sørensen
Abstract	Disease progression modeling (DPM) using longitudinal data is a challenging machine learning task. Existing DPM algorithms neglect temporal dependencies among measurements, make parametric assumptions about biomarker trajectories, do not model multiple biomarkers jointly, and need an alignment of subjects’ trajectories. In this paper, recurrent neural networks (RNNs) are utilized to address these issues. However, in many cases, longitudinal cohorts contain incomplete data, which hinders the application of standard RNNs and requires a pre-processing step such as imputation of the missing values. Instead, we propose a generalized training rule for the most widely used RNN architecture, long short-term memory (LSTM) networks, that can handle both missing predictor and target values. The proposed LSTM algorithm is applied to model the progression of Alzheimer’s disease (AD) using six volumetric magnetic resonance imaging (MRI) biomarkers, i.e., volumes of ventricles, hippocampus, whole brain, fusiform, middle temporal gyrus, and entorhinal cortex, and it is compared to standard LSTM networks with data imputation and a parametric, regression-based DPM method. The results show that the proposed algorithm achieves a significantly lower mean absolute error (MAE) than the alternatives with p < 0.05 using Wilcoxon signed rank test in predicting values of almost all of the MRI biomarkers. Moreover, a linear discriminant analysis (LDA) classifier applied to the predicted biomarker values produces a significantly larger AUC of 0.90 vs. at most 0.84 with p < 0.001 using McNemar’s test for clinical diagnosis of AD. Inspection of MAE curves as a function of the amount of missing data reveals that the proposed LSTM algorithm achieves the best performance up until more than 74% missing values. Finally, it is illustrated how the method can successfully be applied to data with varying time intervals.
Tasks	Imputation
Published	2019-03-17
URL	http://arxiv.org/abs/1903.07173v1
PDF	http://arxiv.org/pdf/1903.07173v1.pdf
PWC	https://paperswithcode.com/paper/training-recurrent-neural-networks-robust-to
Repo
Framework

Focus Is All You Need: Loss Functions For Event-based Vision


Title	Focus Is All You Need: Loss Functions For Event-based Vision
Authors	Guillermo Gallego, Mathias Gehrig, Davide Scaramuzza
Abstract	Event cameras are novel vision sensors that output pixel-level brightness changes (“events”) instead of traditional video frames. These asynchronous sensors offer several advantages over traditional cameras, such as, high temporal resolution, very high dynamic range, and no motion blur. To unlock the potential of such sensors, motion compensation methods have been recently proposed. We present a collection and taxonomy of twenty two objective functions to analyze event alignment in motion compensation approaches (Fig. 1). We call them Focus Loss Functions since they have strong connections with functions used in traditional shape-from-focus applications. The proposed loss functions allow bringing mature computer vision tools to the realm of event cameras. We compare the accuracy and runtime performance of all loss functions on a publicly available dataset, and conclude that the variance, the gradient and the Laplacian magnitudes are among the best loss functions. The applicability of the loss functions is shown on multiple tasks: rotational motion, depth and optical flow estimation. The proposed focus loss functions allow to unlock the outstanding properties of event cameras.
Tasks	Event-based vision, Motion Compensation, Optical Flow Estimation
Published	2019-04-15
URL	http://arxiv.org/abs/1904.07235v1
PDF	http://arxiv.org/pdf/1904.07235v1.pdf
PWC	https://paperswithcode.com/paper/focus-is-all-you-need-loss-functions-for
Repo
Framework

Multi-Kernel Capsule Network for Schizophrenia Identification


Title	Multi-Kernel Capsule Network for Schizophrenia Identification
Authors	Tian Wang, Anastasios Bezerianos, Andrzej Cichocki, Junhua Li
Abstract	Objective: Schizophrenia seriously affects the quality of life. To date, both simple (linear discriminant analysis) and complex (deep neural network) machine learning methods have been utilized to identify schizophrenia based on functional connectivity features. The existing simple methods need two separate steps (i.e., feature extraction and classification) to achieve the identification, which disables simultaneous tuning for the best feature extraction and classifier training. The complex methods integrate two steps and can be simultaneously tuned to achieve optimal performance, but these methods require a much larger amount of data for model training. Methods: To overcome the aforementioned drawbacks, we proposed a multi-kernel capsule network (MKCapsnet), which was developed by considering the brain anatomical structure. Kernels were set to match with partition sizes of brain anatomical structure in order to capture interregional connectivities at the varying scales. With the inspiration of widely-used dropout strategy in deep learning, we developed vector dropout in the capsule layer to prevent overfitting of the model. Results: The comparison results showed that the proposed method outperformed the state-of-the-art methods. Besides, we compared performances using different parameters and illustrated the routing process to reveal characteristics of the proposed method. Conclusion: MKCapsnet is promising for schizophrenia identification. Significance: Our study not only proposed a multi-kernel capsule network but also provided useful information in the parameter setting, which is informative for further studies using a capsule network for neurophysiological signal classification.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1907.12827v1
PDF	https://arxiv.org/pdf/1907.12827v1.pdf
PWC	https://paperswithcode.com/paper/multi-kernel-capsule-network-for
Repo
Framework

Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction


Title	Logician: A Unified End-to-End Neural Approach for Open-Domain Information Extraction
Authors	Mingming Sun, Xu Li, Xin Wang, Miao Fan, Yue Feng, Ping Li
Abstract	In this paper, we consider the problem of open information extraction (OIE) for extracting entity and relation level intermediate structures from sentences in open-domain. We focus on four types of valuable intermediate structures (Relation, Attribute, Description, and Concept), and propose a unified knowledge expression form, SAOKE, to express them. We publicly release a data set which contains more than forty thousand sentences and the corresponding facts in the SAOKE format labeled by crowd-sourcing. To our knowledge, this is the largest publicly available human labeled data set for open information extraction tasks. Using this labeled SAOKE data set, we train an end-to-end neural model using the sequenceto-sequence paradigm, called Logician, to transform sentences into facts. For each sentence, different to existing algorithms which generally focus on extracting each single fact without concerning other possible facts, Logician performs a global optimization over all possible involved facts, in which facts not only compete with each other to attract the attention of words, but also cooperate to share words. An experimental study on various types of open domain relation extraction tasks reveals the consistent superiority of Logician to other states-of-the-art algorithms. The experiments verify the reasonableness of SAOKE format, the valuableness of SAOKE data set, the effectiveness of the proposed Logician model, and the feasibility of the methodology to apply end-to-end learning paradigm on supervised data sets for the challenging tasks of open information extraction.
Tasks	Open Information Extraction, Relation Extraction
Published	2019-04-29
URL	http://arxiv.org/abs/1904.12535v1
PDF	http://arxiv.org/pdf/1904.12535v1.pdf
PWC	https://paperswithcode.com/paper/logician-a-unified-end-to-end-neural-approach
Repo
Framework

Domain-Aware Dynamic Networks


Title	Domain-Aware Dynamic Networks
Authors	Tianyuan Zhang, Bichen Wu, Xin Wang, Joseph Gonzalez, Kurt Keutzer
Abstract	Deep neural networks with more parameters and FLOPs have higher capacity and generalize better to diverse domains. But to be deployed on edge devices, the model’s complexity has to be constrained due to limited compute resource. In this work, we propose a method to improve the model capacity without increasing inference-time complexity. Our method is based on an assumption of data locality: for an edge device, within a short period of time, the input data to the device are sampled from a single domain with relatively low diversity. Therefore, it is possible to utilize a specialized, low-complexity model to achieve good performance in that input domain. To leverage this, we propose Domain-aware Dynamic Network (DDN), which is a high-capacity dynamic network in which each layer contains multiple weights. During inference, based on the input domain, DDN dynamically combines those weights into one single weight that specializes in the given domain. This way, DDN can keep the inference-time complexity low but still maintain a high capacity. Experiments show that without increasing the parameters, FLOPs, and actual latency, DDN achieves up to 2.6% higher AP50 than a static network on the BDD100K object-detection benchmark.
Tasks	Object Detection
Published	2019-11-26
URL	https://arxiv.org/abs/1911.13237v1
PDF	https://arxiv.org/pdf/1911.13237v1.pdf
PWC	https://paperswithcode.com/paper/domain-aware-dynamic-networks
Repo
Framework

SANNS: Scaling Up Secure Approximate k-Nearest Neighbors Search


Title	SANNS: Scaling Up Secure Approximate k-Nearest Neighbors Search
Authors	Hao Chen, Ilaria Chillotti, Yihe Dong, Oxana Poburinnaya, Ilya Razenshteyn, M. Sadegh Riazi
Abstract	The $k$-Nearest Neighbor Search ($k$-NNS) is the backbone of several cloud-based services such as recommender systems, face recognition, and database search on text and images. In these services, the client sends the query to the cloud server and receives the response in which case the query and response are revealed to the service provider. Such data disclosures are unacceptable in several scenarios due to the sensitivity of data and/or privacy laws. In this paper, we introduce SANNS, a system for secure $k$-NNS that keeps client’s query and the search result confidential. SANNS comprises two protocols: an optimized linear scan and a protocol based on a novel sublinear time clustering-based algorithm. We prove the security of both protocols in the standard semi-honest model. The protocols are built upon several state-of-the-art cryptographic primitives such as lattice-based additively homomorphic encryption, distributed oblivious RAM, and garbled circuits. We provide several contributions to each of these primitives which are applicable to other secure computation tasks. Both of our protocols rely on a new circuit for the approximate top-$k$ selection from $n$ numbers that is built from $O(n + k^2)$ comparators. We have implemented our proposed system and performed extensive experimental results on four datasets in two different computation environments, demonstrating more than $18-31\times$ faster response time compared to optimally implemented protocols from the prior work. Moreover, SANNS is the first work that scales to the database of 10 million entries, pushing the limit by more than two orders of magnitude.
Tasks	Face Recognition, Recommendation Systems
Published	2019-04-03
URL	https://arxiv.org/abs/1904.02033v5
PDF	https://arxiv.org/pdf/1904.02033v5.pdf
PWC	https://paperswithcode.com/paper/sanns-scaling-up-secure-approximate-k-nearest
Repo
Framework

Improved spectral convergence rates for graph Laplacians on epsilon-graphs and k-NN graphs


Title	Improved spectral convergence rates for graph Laplacians on epsilon-graphs and k-NN graphs
Authors	Jeff Calder, Nicolas Garcia Trillos
Abstract	In this paper we improve the spectral convergence rates for graph-based approximations of Laplace-Beltrami operators constructed from random data. We utilize regularity of the continuum eigenfunctions and strong pointwise consistency results to prove that spectral convergence rates are the same as the pointwise consistency rates for graph Laplacians. In particular, for an optimal choice of the graph connectivity $\varepsilon$, our results show that the eigenvalues and eigenvectors of the graph Laplacian converge to those of the Laplace-Beltrami operator at a rate of $O(n^{-1/(m+4)})$, up to log factors, where $m$ is the manifold dimension and $n$ is the number of vertices in the graph. Our approach is general and allows us to analyze a large variety of graph constructions that include $\varepsilon$-graphs and $k$-NN graphs.
Tasks
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13476v1
PDF	https://arxiv.org/pdf/1910.13476v1.pdf
PWC	https://paperswithcode.com/paper/improved-spectral-convergence-rates-for-graph
Repo
Framework

Accelerated Alternating Minimization, Accelerated Sinkhorn’s Algorithm and Accelerated Iterative Bregman Projections


Title	Accelerated Alternating Minimization, Accelerated Sinkhorn’s Algorithm and Accelerated Iterative Bregman Projections
Authors	Sergey Guminov, Pavel Dvurechensky, Nazarii Tupitsa, Alexander Gasnikov
Abstract	Motivated by the alternating minimization nature of the Sinkhorn’s algorithm and the theoretically faster convergence of accelerated gradient method, in this paper we propose a way to combine alternating minimization and Nesterov-type momentum acceleration. We propose a generic accelerated alternating minimization method and its primal-dual modification for problems with linear constraints enjoying a $1/k^2$ convergence rate, where $k$ is the iteration counter. Moreover, our algorithm converges faster than gradient-type methods in practice as it is free of the choice of the step-size and is adaptive to the local smoothness of the problem. We show how this generic method can be applied to the Optimal Transport problem, we introduce an accelerated Sinkhorn algorithm and estimate its theoretical complexity for the OT problem. We also demonstrate how one can apply the same generic method to the Wasserstein Barycenter problem. As we demonstrate by numerical experiments, the new method is more stable and has faster convergence in practice than the Sinkhorn’s algorithm, especially in the regime of high accuracy.
Tasks
Published	2019-06-09
URL	https://arxiv.org/abs/1906.03622v3
PDF	https://arxiv.org/pdf/1906.03622v3.pdf
PWC	https://paperswithcode.com/paper/accelerated-alternating-minimization
Repo
Framework

Overcoming Multi-Model Forgetting


Title	Overcoming Multi-Model Forgetting
Authors	Yassine Benyahia, Kaicheng Yu, Kamil Bennani-Smires, Martin Jaggi, Anthony Davison, Mathieu Salzmann, Claudiu Musat
Abstract	We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, due to the overwriting of shared parameters. To overcome this, we introduce a statistically-justified weight plasticity loss that regularizes the learning of a model’s shared parameters according to their importance for the previous models, and demonstrate its effectiveness when training two models sequentially and for neural architecture search. Adding weight plasticity in neural architecture search preserves the best models to the end of the search and yields improved results in both natural language processing and computer vision tasks.
Tasks	Neural Architecture Search
Published	2019-02-21
URL	http://arxiv.org/abs/1902.08232v2
PDF	http://arxiv.org/pdf/1902.08232v2.pdf
PWC	https://paperswithcode.com/paper/overcoming-multi-model-forgetting
Repo
Framework

Hue-Net: Intensity-based Image-to-Image Translation with Differentiable Histogram Loss Functions


Title	Hue-Net: Intensity-based Image-to-Image Translation with Differentiable Histogram Loss Functions
Authors	Mor Avi-Aharon, Assaf Arbelle, Tammy Riklin Raviv
Abstract	We present the Hue-Net - a novel Deep Learning framework for Intensity-based Image-to-Image Translation. The key idea is a new technique termed network augmentation which allows a differentiable construction of intensity histograms from images. We further introduce differentiable representations of (1D) cyclic and joint (2D) histograms and use them for defining loss functions based on cyclic Earth Mover’s Distance (EMD) and Mutual Information (MI). While the Hue-Net can be applied to several image-to-image translation tasks, we choose to demonstrate its strength on color transfer problems, where the aim is to paint a source image with the colors of a different target image. Note that the desired output image does not exist and therefore cannot be used for supervised pixel-to-pixel learning. This is accomplished by using the HSV color-space and defining an intensity-based loss that is built on the EMD between the cyclic hue histograms of the output and the target images. To enforce color-free similarity between the source and the output images, we define a semantic-based loss by a differentiable approximation of the MI of these images. The incorporation of histogram loss functions in addition to an adversarial loss enables the construction of semantically meaningful and realistic images. Promising results are presented for different datasets.
Tasks	Image-to-Image Translation
Published	2019-12-12
URL	https://arxiv.org/abs/1912.06044v1
PDF	https://arxiv.org/pdf/1912.06044v1.pdf
PWC	https://paperswithcode.com/paper/hue-net-intensity-based-image-to-image
Repo
Framework

Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images


Title	Modified U-Net (mU-Net) with Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images
Authors	Hyunseok Seo, Charles Huang, Maxime Bassenne, Ruoxiu Xiao, Lei Xing
Abstract	Segmentation of livers and liver tumors is one of the most important steps in radiation therapy of hepatocellular carcinoma. The segmentation task is often done manually, making it tedious, labor intensive, and subject to intra-/inter- operator variations. While various algorithms for delineating organ-at-risks (OARs) and tumor targets have been proposed, automatic segmentation of livers and liver tumors remains intractable due to their low tissue contrast with respect to the surrounding organs and their deformable shape in CT images. The U-Net has gained increasing popularity recently for image analysis tasks and has shown promising results. Conventional U-Net architectures, however, suffer from three major drawbacks. To cope with these problems, we added a residual path with deconvolution and activation operations to the skip connection of the U-Net to avoid duplication of low resolution information of features. In the case of small object inputs, features in the skip connection are not incorporated with features in the residual path. Furthermore, the proposed architecture has additional convolution layers in the skip connection in order to extract high level global features of small object inputs as well as high level features of high resolution edge information of large object inputs. Efficacy of the modified U-Net (mU-Net) was demonstrated using the public dataset of Liver tumor segmentation (LiTS) challenge 2017. The proposed mU-Net outperformed existing state-of-art networks.
Tasks
Published	2019-10-31
URL	https://arxiv.org/abs/1911.00140v1
PDF	https://arxiv.org/pdf/1911.00140v1.pdf
PWC	https://paperswithcode.com/paper/modified-u-net-mu-net-with-incorporation-of
Repo
Framework