January 26, 2020

3031 words 15 mins read

Paper Group ANR 1557

Controllable Data Synthesis Method for Grammatical Error Correction. Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans. Interpreting deep learning prediction of the Parkinson’s disease diagnosis from SPECT imaging. Obfuscation via Information Density Estimation. Analyzing the Limitations of Cross-lingual Word Embedding Mappings. Multisc …

Controllable Data Synthesis Method for Grammatical Error Correction


Title	Controllable Data Synthesis Method for Grammatical Error Correction
Authors	Chencheng Wang, Liner Yang, Yun Chen, Yongping Du, Erhong Yang
Abstract	Due to the lack of parallel data in current Grammatical Error Correction (GEC) task, models based on Sequence to Sequence framework cannot be adequately trained to obtain higher performance. We propose two data synthesis methods which can control the error rate and the ratio of error types on synthetic data. The first approach is to corrupt each word in the monolingual corpus with a fixed probability, including replacement, insertion and deletion. Another approach is to train error generation models and further filtering the decoding results of the models. The experiments on different synthetic data show that the error rate is 40% and the ratio of error types is the same can improve the model performance better. Finally, we synthesize about 100 million data and achieve comparable performance as the state of the art, which uses twice as much data as we use.
Tasks	Grammatical Error Correction
Published	2019-09-29
URL	https://arxiv.org/abs/1909.13302v3
PDF	https://arxiv.org/pdf/1909.13302v3.pdf
PWC	https://paperswithcode.com/paper/controllable-data-synthesis-method-for
Repo
Framework

Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans


Title	Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans
Authors	Mohammad Hamghalam, Baiying Lei, Tianfu Wang
Abstract	The magnetic resonance (MR) analysis of brain tumors is widely used for diagnosis and examination of tumor subregions. The overlapping area among the intensity distribution of healthy, enhancing, non-enhancing, and edema region makes the automatic segmentation a challenging task. Here, we show that a convolutional neural network trained on high-contrast images can transform intensity distribution of brain lesion in its internal subregions. Specifically, generative adversarial network (GAN) is extended to synthesize high-contrast images. A comparison of these synthetic images and real images of brain tumor tissue in MR scans showed significant segmentation improvement and decreased the number of real channels for segmentation. The synthetic images are used as a substitute for real channels and can bypass real modalities in the multimodal brain tumor segmentation framework. Segmentation results on BraTS 2019 dataset demonstrate that our proposed approach can efficiently segment the tumor areas.
Tasks	Brain Tumor Segmentation
Published	2019-09-27
URL	https://arxiv.org/abs/1909.13640v1
PDF	https://arxiv.org/pdf/1909.13640v1.pdf
PWC	https://paperswithcode.com/paper/brain-tumor-synthetic-segmentation-in-3d
Repo
Framework

Interpreting deep learning prediction of the Parkinson’s disease diagnosis from SPECT imaging


Title	Interpreting deep learning prediction of the Parkinson’s disease diagnosis from SPECT imaging
Authors	Theerasarn Pianpanit, Sermkiat Lolak, Phattarapong Sawangjai, Apiwat Ditthapron, Sanparith Marukatat, Ekapol Chuangsuwanich, Theerawit Wilaiprasitporn
Abstract	Parkinson’s disease (PD) diagnosis mainly relies on the visual and semi-quantitative analysis of medical imaging using single-photon emission computed tomography (SPECT) with 123I-Ioflupane (DaTSCAN). Deep learning approach has the benefits over other machine learning methods because the model does not rely on feature extraction. However, the complexity of the deep learning model usually results in difficulty of interpretation of the model when uses in clinical. Several interpretation methods were created for this approach to show the attention map which reveals important features of the input data, giving the model interpretability. However, it is still unclear whether these methods can be applied to explain PD diagnosis or not. In this paper, four different models of the deep learning approach based on the 3-dimensional convolution neural network (3D-CNN) of well-established architectures have been trained. All the models give high classification performance of PD diagnosis with accuracy up to 95-96%. These four models have been used to evaluate the interpretation performance of six well-known interpretation methods. In general, radiologists interpret SPECT images for a healthy subject by confirming a homogeneous symmetrical comma type shape of the I123-Ioflupane uptake in the striatal nuclei. Any other shape is interpreted as abnormal. To evaluate the interpretation performance, the segmented striatal nuclei of the SPECT images are chosen to be the ground truth. {\Blue Guided backpropagation which is one of the interpretation methods shows the best performance among all other methods. Guided backpropagation has the best performance to generate the attention map that focuses on the location of striatal nuclei. By using the result from guided backpropagation, 3D CNN architecture that has the highest classification and interpretation performance can be chosen for SPECT diagnosis.
Tasks
Published	2019-08-23
URL	https://arxiv.org/abs/1908.11199v2
PDF	https://arxiv.org/pdf/1908.11199v2.pdf
PWC	https://paperswithcode.com/paper/a-comparative-study-for-interpreting-deep
Repo
Framework

Obfuscation via Information Density Estimation


Title	Obfuscation via Information Density Estimation
Authors	Hsiang Hsu, Shahab Asoodeh, Flavio du Pin Calmon
Abstract	Identifying features that leak information about sensitive attributes is a key challenge in the design of information obfuscation mechanisms. In this paper, we propose a framework to identify information-leaking features via information density estimation. Here, features whose information densities exceed a pre-defined threshold are deemed information-leaking features. Once these features are identified, we sequentially pass them through a targeted obfuscation mechanism with a provable leakage guarantee in terms of $\mathsf{E}_\gamma$-divergence. The core of this mechanism relies on a data-driven estimate of the trimmed information density for which we propose a novel estimator, named the trimmed information density estimator (TIDE). We then use TIDE to implement our mechanism on three real-world datasets. Our approach can be used as a data-driven pipeline for designing obfuscation mechanisms targeting specific features.
Tasks	Density Estimation
Published	2019-10-17
URL	https://arxiv.org/abs/1910.08109v1
PDF	https://arxiv.org/pdf/1910.08109v1.pdf
PWC	https://paperswithcode.com/paper/obfuscation-via-information-density
Repo
Framework

Analyzing the Limitations of Cross-lingual Word Embedding Mappings


Title	Analyzing the Limitations of Cross-lingual Word Embedding Mappings
Authors	Aitor Ormazabal, Mikel Artetxe, Gorka Labaka, Aitor Soroa, Eneko Agirre
Abstract	Recent research in cross-lingual word embeddings has almost exclusively focused on offline methods, which independently train word embeddings in different languages and map them to a shared space through linear transformations. While several authors have questioned the underlying isomorphism assumption, which states that word embeddings in different languages have approximately the same structure, it is not clear whether this is an inherent limitation of mapping approaches or a more general issue when learning cross-lingual embeddings. So as to answer this question, we experiment with parallel corpora, which allows us to compare offline mapping to an extension of skip-gram that jointly learns both embedding spaces. We observe that, under these ideal conditions, joint learning yields to more isomorphic embeddings, is less sensitive to hubness, and obtains stronger results in bilingual lexicon induction. We thus conclude that current mapping methods do have strong limitations, calling for further research to jointly learn cross-lingual embeddings with a weaker cross-lingual signal.
Tasks	Word Embeddings
Published	2019-06-12
URL	https://arxiv.org/abs/1906.05407v1
PDF	https://arxiv.org/pdf/1906.05407v1.pdf
PWC	https://paperswithcode.com/paper/analyzing-the-limitations-of-cross-lingual
Repo
Framework

Multiscale CNN based Deep Metric Learning for Bioacoustic Classification: Overcoming Training Data Scarcity Using Dynamic Triplet Loss


Title	Multiscale CNN based Deep Metric Learning for Bioacoustic Classification: Overcoming Training Data Scarcity Using Dynamic Triplet Loss
Authors	Anshul Thakur, Daksh Thapar, Padmanabhan Rajan, Aditya Nigam
Abstract	This paper proposes multiscale convolutional neural network (CNN)-based deep metric learning for bioacoustic classification, under low training data conditions. The proposed CNN is characterized by the utilization of four different filter sizes at each level to analyze input feature maps. This multiscale nature helps in describing different bioacoustic events effectively: smaller filters help in learning the finer details of bioacoustic events, whereas, larger filters help in analyzing a larger context leading to global details. A dynamic triplet loss is employed in the proposed CNN architecture to learn a transformation from the input space to the embedding space, where classification is performed. The triplet loss helps in learning this transformation by analyzing three examples, referred to as triplets, at a time where intra-class distance is minimized while maximizing the inter-class separation by a dynamically increasing margin. The number of possible triplets increases cubically with the dataset size, making triplet loss more suitable than the softmax cross-entropy loss in low training data conditions. Experiments on three different publicly available datasets show that the proposed framework performs better than existing bioacoustic classification frameworks. Experimental results also confirm the superiority of the triplet loss over the cross-entropy loss in low training data conditions
Tasks	Metric Learning
Published	2019-03-26
URL	http://arxiv.org/abs/1903.10713v2
PDF	http://arxiv.org/pdf/1903.10713v2.pdf
PWC	https://paperswithcode.com/paper/multiscale-cnn-based-deep-metric-learning-for
Repo
Framework

FIS-GAN: GAN with Flow-based Importance Sampling


Title	FIS-GAN: GAN with Flow-based Importance Sampling
Authors	Shiyu Yi, Donglin Zhan, Zhengyang Geng, Wenqing Zhang, Chang Xu
Abstract	Generative Adversarial Networks (GAN) training process, in most cases, apply uniform and Gaussian sampling methods in latent space, which probably spends most of the computation on examples that can be properly handled and easy to generate. Theoretically, importance sampling speeds up stochastic gradient algorithms for supervised learning by prioritizing training examples. In this paper, we explore the possibility for adapting importance sampling into adversarial learning. We use importance sampling to replace uniform and Gaussian sampling methods in latent space and combine normalizing flow with importance sampling to approximate latent space posterior distribution by density estimation. Empirically, results on MNIST and Fashion-MNIST demonstrate that our method significantly accelerates the convergence of generative process while retaining visual fidelity in generated samples.
Tasks	Density Estimation
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02519v1
PDF	https://arxiv.org/pdf/1910.02519v1.pdf
PWC	https://paperswithcode.com/paper/fis-gan-gan-with-flow-based-importance
Repo
Framework

Probing What Different NLP Tasks Teach Machines about Function Word Comprehension


Title	Probing What Different NLP Tasks Teach Machines about Function Word Comprehension
Authors	Najoung Kim, Roma Patel, Adam Poliak, Alex Wang, Patrick Xia, R. Thomas McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, Ellie Pavlick
Abstract	We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to target the comprehension of specific types of function words (e.g., prepositions, wh-words). Using these probing tasks, we explore the effects of various pretraining objectives for sentence encoders (e.g., language modeling, CCG supertagging and natural language inference (NLI)) on the learned representations. Our results show that pretraining on language modeling performs the best on average across our probing tasks, supporting its widespread use for pretraining state-of-the-art NLP models, and CCG supertagging and NLI pretraining perform comparably. Overall, no pretraining objective dominates across the board, and our function word probing tasks highlight several intuitive differences between pretraining objectives, e.g., that NLI helps the comprehension of negation.
Tasks	CCG Supertagging, Language Modelling, Natural Language Inference
Published	2019-04-25
URL	https://arxiv.org/abs/1904.11544v2
PDF	https://arxiv.org/pdf/1904.11544v2.pdf
PWC	https://paperswithcode.com/paper/probing-what-different-nlp-tasks-teach
Repo
Framework

Dream Distillation: A Data-Independent Model Compression Framework


Title	Dream Distillation: A Data-Independent Model Compression Framework
Authors	Kartikeya Bhardwaj, Naveen Suda, Radu Marculescu
Abstract	Model compression is eminently suited for deploying deep learning on IoT-devices. However, existing model compression techniques rely on access to the original or some alternate dataset. In this paper, we address the model compression problem when no real data is available, e.g., when data is private. To this end, we propose Dream Distillation, a data-independent model compression framework. Our experiments show that Dream Distillation can achieve 88.5% accuracy on the CIFAR-10 test set without actually training on the original data!
Tasks	Model Compression
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07072v1
PDF	https://arxiv.org/pdf/1905.07072v1.pdf
PWC	https://paperswithcode.com/paper/dream-distillation-a-data-independent-model
Repo
Framework

Optimization with soft Dice can lead to a volumetric bias


Title	Optimization with soft Dice can lead to a volumetric bias
Authors	Jeroen Bertels, David Robben, Dirk Vandermeulen, Paul Suetens
Abstract	Segmentation is a fundamental task in medical image analysis. The clinical interest is often to measure the volume of a structure. To evaluate and compare segmentation methods, the similarity between a segmentation and a predefined ground truth is measured using metrics such as the Dice score. Recent segmentation methods based on convolutional neural networks use a differentiable surrogate of the Dice score, such as soft Dice, explicitly as the loss function during the learning phase. Even though this approach leads to improved Dice scores, we find that, both theoretically and empirically on four medical tasks, it can introduce a volumetric bias for tasks with high inherent uncertainty. As such, this may limit the method’s clinical applicability.
Tasks
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02278v1
PDF	https://arxiv.org/pdf/1911.02278v1.pdf
PWC	https://paperswithcode.com/paper/optimization-with-soft-dice-can-lead-to-a
Repo
Framework

Bridging Theory and Algorithm for Domain Adaptation


Title	Bridging Theory and Algorithm for Domain Adaptation
Authors	Yuchen Zhang, Tianle Liu, Mingsheng Long, Michael I. Jordan
Abstract	This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.
Tasks	Domain Adaptation
Published	2019-04-11
URL	https://arxiv.org/abs/1904.05801v2
PDF	https://arxiv.org/pdf/1904.05801v2.pdf
PWC	https://paperswithcode.com/paper/bridging-theory-and-algorithm-for-domain
Repo
Framework

Detecting Human-Object Interactions via Functional Generalization


Title	Detecting Human-Object Interactions via Functional Generalization
Authors	Ankan Bansal, Sai Saketh Rambhatla, Abhinav Shrivastava, Rama Chellappa
Abstract	We present an approach for detecting human-object interactions (HOIs) in images, based on the idea that humans interact with functionally similar objects in a similar manner. The proposed model is simple and efficiently uses the data, visual features of the human, relative spatial orientation of the human and the object, and the knowledge that functionally similar objects take part in similar interactions with humans. We provide extensive experimental validation for our approach and demonstrate state-of-the-art results for HOI detection. On the HICO-Det dataset our method achieves a gain of over 2.5% absolute points in mean average precision (mAP) over state-of-the-art. We also show that our approach leads to significant performance gains for zero-shot HOI detection in the seen object setting. We further demonstrate that using a generic object detector, our model can generalize to interactions involving previously unseen objects.
Tasks	Human-Object Interaction Detection
Published	2019-04-05
URL	https://arxiv.org/abs/1904.03181v2
PDF	https://arxiv.org/pdf/1904.03181v2.pdf
PWC	https://paperswithcode.com/paper/detecting-human-object-interactions-via
Repo
Framework

Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness


Title	Towards A Unified Min-Max Framework for Adversarial Exploration and Robustness
Authors	Jingkang Wang, Tianyun Zhang, Sijia Liu, Pin-Yu Chen, Jiacen Xu, Makan Fardad, Bo Li
Abstract	The worst-case training principle that minimizes the maximal adversarial loss, also known as adversarial training (AT), has shown to be a state-of-the-art approach for enhancing adversarial robustness against norm-ball bounded input perturbations. Nonetheless, min-max optimization beyond the purpose of AT has not been rigorously explored in the research of adversarial attack and defense. In particular, given a set of risk sources (domains), minimizing the maximal loss induced from the domain set can be reformulated as a general min-max problem that is different from AT. Examples of this general formulation include attacking model ensembles, devising universal perturbation under multiple inputs or data transformations, and generalized AT over different types of attack models. We show that these problems can be solved under a unified and theoretically principled min-max optimization framework. We also show that the self-adjusted domain weights learned from our method provides a means to explain the difficulty level of attack and defense over multiple domains. Extensive experiments show that our approach leads to substantial performance improvement over the conventional averaging strategy.
Tasks	Adversarial Attack
Published	2019-06-09
URL	https://arxiv.org/abs/1906.03563v2
PDF	https://arxiv.org/pdf/1906.03563v2.pdf
PWC	https://paperswithcode.com/paper/beyond-adversarial-training-min-max
Repo
Framework

Online Semi-Supervised Concept Drift Detection with Density Estimation


Title	Online Semi-Supervised Concept Drift Detection with Density Estimation
Authors	Chang How Tan, Vincent CS Lee, Mahsa Salehi
Abstract	Concept drift is formally defined as the change in joint distribution of a set of input variables X and a target variable y. The two types of drift that are extensively studied are real drift and virtual drift where the former is the change in posterior probabilities p(yX) while the latter is the change in distribution of X without affecting the posterior probabilities. Many approaches on concept drift detection either assume full availability of data labels, y or handle only the virtual drift. In a streaming environment, the assumption of full availability of data labels, y is questioned. On the other hand, approaches that deal with virtual drift failed to address real drift. Rather than improving the state-of-the-art methods, this paper presents a semi-supervised framework to deal with the challenges above. The objective of the proposed framework is to learn from streaming environment with limited data labels, y and detect real drift concurrently. This paper proposes a novel concept drift detection method utilizing the densities of posterior probabilities in partially labeled streaming environments. Experimental results on both synthetic and realworld datasets show that our proposed semi-supervised framework enables the detection of concept drift in such environment while achieving comparable prediction performance to the state-of-the-art methods.
Tasks	Density Estimation
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11251v2
PDF	https://arxiv.org/pdf/1909.11251v2.pdf
PWC	https://paperswithcode.com/paper/online-semi-supervised-concept-drift
Repo
Framework

Efficient Black-box Assessment of Autonomous Vehicle Safety


Title	Efficient Black-box Assessment of Autonomous Vehicle Safety
Authors	Justin Norden, Matthew O’Kelly, Aman Sinha
Abstract	While autonomous vehicle (AV) technology has shown substantial progress, we still lack tools for rigorous and scalable testing. Real-world testing, the $\textit{de-facto}$ evaluation method, is dangerous to the public. Moreover, due to the rare nature of failures, billions of miles of driving are needed to statistically validate performance claims. Thus, the industry has largely turned to simulation to evaluate AV systems. However, having a simulation stack alone is not a solution. A simulation testing framework needs to prioritize which scenarios to run, learn how the chosen scenarios provide coverage of failure modes, and rank failure scenarios in order of importance. We implement a simulation testing framework that evaluates an entire modern AV system as a black box. This framework estimates the probability of accidents under a base distribution governing standard traffic behavior. In order to accelerate rare-event probability evaluation, we efficiently learn to identify and rank failure scenarios via adaptive importance-sampling methods. Using this framework, we conduct the first independent evaluation of a full-stack commercial AV system, Comma AI’s OpenPilot.
Tasks
Published	2019-12-08
URL	https://arxiv.org/abs/1912.03618v1
PDF	https://arxiv.org/pdf/1912.03618v1.pdf
PWC	https://paperswithcode.com/paper/efficient-black-box-assessment-of-autonomous
Repo
Framework