January 28, 2020

2902 words 14 mins read

Paper Group ANR 946

Defending Against Universal Attacks Through Selective Feature Regeneration. TRADI: Tracking deep neural network weight distributions. Why Should we Combine Training and Post-Training Methods for Out-of-Distribution Detection?. Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting. Improving Interpretability of Word Embe …

Defending Against Universal Attacks Through Selective Feature Regeneration


Title	Defending Against Universal Attacks Through Selective Feature Regeneration
Authors	Tejas Borkar, Felix Heide, Lina Karam
Abstract	Deep neural network (DNN) predictions have been shown to be vulnerable to carefully crafted adversarial perturbations. Specifically, image-agnostic (universal adversarial) perturbations added to any image can fool a target network into making erroneous predictions. Departing from existing defense strategies that work mostly in the image domain, we present a novel defense which operates in the DNN feature domain and effectively defends against such universal perturbations. Our approach identifies pre-trained convolutional features that are most vulnerable to adversarial noise and deploys trainable feature regeneration units which transform these DNN filter activations into resilient features that are robust to universal perturbations. Regenerating only the top 50% adversarially susceptible activations in at most 6 DNN layers and leaving all remaining DNN activations unchanged, we outperform existing defense strategies across different network architectures by more than 10% in restored accuracy. We show that without any additional modification, our defense trained on ImageNet with one type of universal attack examples effectively defends against other types of unseen universal attacks.
Tasks	Adversarial Defense
Published	2019-06-08
URL	https://arxiv.org/abs/1906.03444v3
PDF	https://arxiv.org/pdf/1906.03444v3.pdf
PWC	https://paperswithcode.com/paper/defending-against-adversarial-attacks-through
Repo
Framework

TRADI: Tracking deep neural network weight distributions


Title	TRADI: Tracking deep neural network weight distributions
Authors	Gianni Franchi, Andrei Bursuc, Emanuel Aldea, Severine Dubuisson, Isabelle Bloch
Abstract	During training, the weights of a Deep Neural Network (DNN) are optimized from a random initialization towards a nearly optimum value minimizing a loss function. Only this final state of the weights is typically kept for testing, while the wealth of information on the geometry of the weight space, accumulated over the descent towards the minimum is discarded. In this work we propose to make use of this knowledge and leverage it for computing the distributions of the weights of the DNN. This can be further used for estimating the epistemic uncertainty of the DNN by sampling an ensemble of networks from these distributions. To this end we introduce a method for tracking the trajectory of the weights during optimization, that does not require any changes in the architecture nor on the training procedure. We evaluate our method on standard classification and regression benchmarks, and on out-of-distribution detection for classification and semantic segmentation. We achieve competitive results, while preserving computational efficiency in comparison to other popular approaches.
Tasks	Out-of-Distribution Detection, Semantic Segmentation
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11316v2
PDF	https://arxiv.org/pdf/1912.11316v2.pdf
PWC	https://paperswithcode.com/paper/tradi-tracking-deep-neural-network-weight
Repo
Framework

Why Should we Combine Training and Post-Training Methods for Out-of-Distribution Detection?


Title	Why Should we Combine Training and Post-Training Methods for Out-of-Distribution Detection?
Authors	Aristotelis-Angelos Papadopoulos, Nazim Shaikh, Mohammad Reza Rajati
Abstract	Deep neural networks are known to achieve superior results in classification tasks. However, it has been recently shown that they are incapable to detect examples that are generated by a distribution which is different than the one they have been trained on since they are making overconfident prediction for Out-Of-Distribution (OOD) examples. OOD detection has attracted a lot of attention recently. In this paper, we review some of the most seminal recent algorithms in the OOD detection field, we divide those methods into training and post-training and we experimentally show how the combination of the former with the latter can achieve state-of-the-art results in the OOD detection task.
Tasks	Out-of-Distribution Detection
Published	2019-12-05
URL	https://arxiv.org/abs/1912.03133v1
PDF	https://arxiv.org/pdf/1912.03133v1.pdf
PWC	https://paperswithcode.com/paper/why-should-we-combine-training-and-post
Repo
Framework

Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting


Title	Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting
Authors	Aakash Srinivasan, Harshavardhan Kamarthi, Devi Ganesan, Sutanu Chakraborti
Abstract	Neural network based word embeddings, such as Word2Vec and GloVe, are purely data driven in that they capture the distributional information about words from the training corpus. Past works have attempted to improve these embeddings by incorporating semantic knowledge from lexical resources like WordNet. Some techniques like retrofitting modify word embeddings in the post-processing stage while some others use a joint learning approach by modifying the objective function of neural networks. In this paper, we discuss two novel approaches for incorporating semantic knowledge into word embeddings. In the first approach, we take advantage of Levy et al’s work which showed that using SVD based methods on co-occurrence matrix provide similar performance to neural network based embeddings. We propose a ‘sprinkling’ technique to add semantic relations to the co-occurrence matrix directly before factorization. In the second approach, WordNet similarity scores are used to improve the retrofitting method. We evaluate the proposed methods in both intrinsic and extrinsic tasks and observe significant improvements over the baselines in many of the datasets.
Tasks	Word Embeddings
Published	2019-12-14
URL	https://arxiv.org/abs/1912.06889v2
PDF	https://arxiv.org/pdf/1912.06889v2.pdf
PWC	https://paperswithcode.com/paper/integrating-lexical-knowledge-in-word
Repo
Framework

Improving Interpretability of Word Embeddings by Generating Definition and Usage


Title	Improving Interpretability of Word Embeddings by Generating Definition and Usage
Authors	Haitong Zhang, Yongping Du, Jiaxin Sun, Qingxiao Li
Abstract	Word Embeddings, which encode semantic and syntactic features, have achieved success in many natural language processing tasks recently. However, the lexical semantics captured by these embeddings are difficult to interpret due to the dense vector representations. In order to improve the interpretability of word vectors, we explore definition modeling task and propose a novel framework (Semantics-Generator) to generate more reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and study whether it is possible to utilize distributed representations to generate example sentences of words. These ways of semantics generation are a more direct and explicit expression of embedding’s semantics. Two multi-task learning methods are used to combine usage modeling and definition modeling. To verify our approach, we construct Oxford-2019 dataset, where each entry contains word, context, example sentence and corresponding definition. Experimental results show that Semantics-Generator achieves the state-of-the-art result in definition modeling and the multi-task learning methods are helpful for two tasks to improve the performance.
Tasks	Multi-Task Learning, Word Embeddings
Published	2019-12-12
URL	https://arxiv.org/abs/1912.05898v1
PDF	https://arxiv.org/pdf/1912.05898v1.pdf
PWC	https://paperswithcode.com/paper/improving-interpretability-of-word-embeddings
Repo
Framework

Evolutionary Multitasking for Semantic Web Service Composition


Title	Evolutionary Multitasking for Semantic Web Service Composition
Authors	Chen Wang, Hui Ma, Gang Chen, Sven Hartmann
Abstract	Web services are basic functions of a software system to support the concept of service-oriented architecture. They are often composed together to provide added values, known as web service composition. Researchers often employ Evolutionary Computation techniques to efficiently construct composite services with near-optimized functional quality (i.e., Quality of Semantic Matchmaking) or non-functional quality (i.e., Quality of Service) or both due to the complexity of this problem. With a significant increase in service composition requests, many composition requests have similar input and output requirements but may vary due to different preferences from different user segments. This problem is often treated as a multi-objective service composition so as to cope with different preferences from different user segments simultaneously. Without taking a multi-objective approach that gives rise to a solution selection challenge, we perceive multiple similar service composition requests as jointly forming an evolutionary multi-tasking problem in this work. We propose an effective permutation-based evolutionary multi-tasking approach that can simultaneously generate a set of solutions, with one for each service request. We also introduce a neighborhood structure over multiple tasks to allow newly evolved solutions to be evaluated on related tasks. Our proposed method can perform better at the cost of only a fraction of time, compared to one state-of-art single-tasking EC-based method. We also found that the use of the proper neighborhood structure can enhance the effectiveness of our approach.
Tasks
Published	2019-02-18
URL	http://arxiv.org/abs/1902.06370v1
PDF	http://arxiv.org/pdf/1902.06370v1.pdf
PWC	https://paperswithcode.com/paper/evolutionary-multitasking-for-semantic-web
Repo
Framework

The Probabilistic Object Detection Challenge


Title	The Probabilistic Object Detection Challenge
Authors	John Skinner, David Hall, Haoyang Zhang, Feras Dayoub, Niko Sünderhauf
Abstract	We introduce a new challenge for computer and robotic vision, the first ACRV Robotic Vision Challenge, Probabilistic Object Detection. Probabilistic object detection is a new variation on traditional object detection tasks, requiring estimates of spatial and semantic uncertainty. We extend the traditional bounding box format of object detection to express spatial uncertainty using gaussian distributions for the box corners. The challenge introduces a new test dataset of video sequences, which are designed to more closely resemble the kind of data available to a robotic system. We evaluate probabilistic detections using a new probability-based detection quality (PDQ) measure. The goal in creating this challenge is to draw the computer and robotic vision communities together, toward applying object detection solutions for practical robotics applications.
Tasks	Object Detection
Published	2019-03-19
URL	http://arxiv.org/abs/1903.07840v2
PDF	http://arxiv.org/pdf/1903.07840v2.pdf
PWC	https://paperswithcode.com/paper/the-probabilistic-object-detection-challenge
Repo
Framework

AntNet: Deep Answer Understanding Network for Natural Reverse QA


Title	AntNet: Deep Answer Understanding Network for Natural Reverse QA
Authors	Lei Yang, Qing Yin, Linlin Hou, Jie Gui, Ou Wu, James Kwok
Abstract	This study refers to a reverse question answering(reverse QA) procedure, in which machines proactively raise questions and humans supply answers. This procedure exists in many real human-machine interaction applications. A crucial problem in human-machine interaction is answer understanding. Existing solutions rely on mandatory option term selection to avoid automatic answer understanding. However, these solutions lead to unnatural human-computer interaction and harm user experience. To this end, this study proposed a novel deep answer understanding network, called AntNet, for reverse QA. The network consists of three new modules, namely, skeleton extraction for questions, relevance-aware representation of answers, and multi-hop based fusion. As answer understanding for reverse QA has not been explored, a new data corpus is compiled in this study. Experimental results indicate that our proposed network is significantly better than existing methods and those modified from classical natural language processing (NLP) deep models. The effectiveness of the three new modules is also verified.
Tasks	Question Answering
Published	2019-12-01
URL	https://arxiv.org/abs/1912.00398v1
PDF	https://arxiv.org/pdf/1912.00398v1.pdf
PWC	https://paperswithcode.com/paper/antnet-deep-answer-understanding-network-for
Repo
Framework

SUMNet: Fully Convolutional Model for Fast Segmentation of Anatomical Structures in Ultrasound Volumes


Title	SUMNet: Fully Convolutional Model for Fast Segmentation of Anatomical Structures in Ultrasound Volumes
Authors	Sumanth Nandamuri, Debarghya China, Pabitra Mitra, Debdoot Sheet
Abstract	Ultrasound imaging is generally employed for real-time investigation of internal anatomy of the human body for disease identification. Delineation of the anatomical boundary of organs and pathological lesions is quite challenging due to the stochastic nature of speckle intensity in the images, which also introduces visual fatigue for the observer. This paper introduces a fully convolutional neural network based method to segment organ and pathologies in ultrasound volume by learning the spatial-relationship between closely related classes in the presence of stochastically varying speckle intensity. We propose a convolutional encoder-decoder like framework with (i) feature concatenation across matched layers in encoder and decoder and (ii) index passing based unpooling at the decoder for semantic segmentation of ultrasound volumes. We have experimentally evaluated the performance on publicly available datasets consisting of $10$ intravascular ultrasound pullback acquired at $20$ MHz and $16$ freehand thyroid ultrasound volumes acquired $11 - 16$ MHz. We have obtained a dice score of $0.93 \pm 0.08$ and $0.92 \pm 0.06$ respectively, following a $10$-fold cross-validation experiment while processing frame of $256 \times 384$ pixel in $0.035$s and a volume of $256 \times 384 \times 384$ voxel in $13.44$s.
Tasks	Semantic Segmentation
Published	2019-01-21
URL	http://arxiv.org/abs/1901.06920v1
PDF	http://arxiv.org/pdf/1901.06920v1.pdf
PWC	https://paperswithcode.com/paper/sumnet-fully-convolutional-model-for-fast
Repo
Framework

A Distance Map Regularized CNN for Cardiac Cine MR Image Segmentation


Title	A Distance Map Regularized CNN for Cardiac Cine MR Image Segmentation
Authors	Shusil Dangi, Cristian Linte, Ziv Yaniv
Abstract	Cardiac image segmentation is a critical process for generating personalized models of the heart and for quantifying cardiac performance parameters. Several convolutional neural network (CNN) architectures have been proposed to segment the heart chambers from cardiac cine MR images. Here we propose a multi-task learning (MTL)-based regularization framework for cardiac MR image segmentation. The network is trained to perform the main task of semantic segmentation, along with a simultaneous, auxiliary task of pixel-wise distance map regression. The proposed distance map regularizer is a decoder network added to the bottleneck layer of an existing CNN architecture, facilitating the network to learn robust global features. The regularizer block is removed after training, so that the original number of network parameters does not change. We show that the proposed regularization method improves both binary and multi-class segmentation performance over the corresponding state-of-the-art CNN architectures on two publicly available cardiac cine MRI datasets, obtaining average dice coefficient of 0.84$\pm$0.03 and 0.91$\pm$0.04, respectively. Furthermore, we also demonstrate improved generalization performance of the distance map regularized network on cross-dataset segmentation, showing as much as 42% improvement in myocardium Dice coefficient from 0.56$\pm$0.28 to 0.80$\pm$0.14.
Tasks	Multi-Task Learning, Semantic Segmentation
Published	2019-01-04
URL	http://arxiv.org/abs/1901.01238v2
PDF	http://arxiv.org/pdf/1901.01238v2.pdf
PWC	https://paperswithcode.com/paper/a-distance-map-regularized-cnn-for-cardiac
Repo
Framework

Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey


Title	Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey
Authors	Sicheng Zhao, Shangfei Wang, Mohammad Soleymani, Dhiraj Joshi, Qiang Ji
Abstract	The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.
Tasks
Published	2019-10-03
URL	https://arxiv.org/abs/1911.05609v1
PDF	https://arxiv.org/pdf/1911.05609v1.pdf
PWC	https://paperswithcode.com/paper/affective-computing-for-large-scale
Repo
Framework

Compositional generalization in a deep seq2seq model by separating syntax and semantics


Title	Compositional generalization in a deep seq2seq model by separating syntax and semantics
Authors	Jake Russin, Jason Jo, Randall C. O’Reilly, Yoshua Bengio
Abstract	Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution. However, human learners readily generalize in this way, e.g. by applying known grammatical rules to novel words. Inspired by work in neuroscience suggesting separate brain systems for syntactic and semantic processing, we implement a modification to standard approaches in neural machine translation, imposing an analogous separation. The novel model, which we call Syntactic Attention, substantially outperforms standard methods in deep learning on the SCAN dataset, a compositional generalization task, without any hand-engineered features or additional supervision. Our work suggests that separating syntactic from semantic learning may be a useful heuristic for capturing compositional structure.
Tasks	Machine Translation
Published	2019-04-22
URL	https://arxiv.org/abs/1904.09708v3
PDF	https://arxiv.org/pdf/1904.09708v3.pdf
PWC	https://paperswithcode.com/paper/compositional-generalization-in-a-deep
Repo
Framework

Smoothing Policies and Safe Policy Gradients


Title	Smoothing Policies and Safe Policy Gradients
Authors	Matteo Papini, Matteo Pirotta, Marcello Restelli
Abstract	Policy gradient algorithms are among the best candidates for the much anticipated application of reinforcement learning to real-world control tasks, such as the ones arising in robotics. However, the trial-and-error nature of these methods introduces safety issues whenever the learning phase itself must be performed on a physical system. In this paper, we address a specific safety formulation, where danger is encoded in the reward signal and the learning agent is constrained to never worsen its performance. By studying actor-only policy gradient from a stochastic optimization perspective, we establish improvement guarantees for a wide class of parametric policies, generalizing existing results on Gaussian policies. This, together with novel upper bounds on the variance of policy gradient estimators, allows to identify those meta-parameter schedules that guarantee monotonic improvement with high probability. The two key meta-parameters are the step size of the parameter updates and the batch size of the gradient estimators. By a joint, adaptive selection of these meta-parameters, we obtain a safe policy gradient algorithm.
Tasks	Stochastic Optimization
Published	2019-05-08
URL	https://arxiv.org/abs/1905.03231v1
PDF	https://arxiv.org/pdf/1905.03231v1.pdf
PWC	https://paperswithcode.com/paper/smoothing-policies-and-safe-policy-gradients
Repo
Framework

A Critique of the Smooth Inverse Frequency Sentence Embeddings


Title	A Critique of the Smooth Inverse Frequency Sentence Embeddings
Authors	Aidana Karipbayeva, Alena Sorokina, Zhenisbek Assylbekov
Abstract	We critically review the smooth inverse frequency sentence embedding method of Arora, Liang, and Ma (2017), and show inconsistencies in its setup, derivation, and evaluation.
Tasks	Sentence Embedding, Sentence Embeddings
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13494v1
PDF	https://arxiv.org/pdf/1909.13494v1.pdf
PWC	https://paperswithcode.com/paper/a-critique-of-the-smooth-inverse-frequency
Repo
Framework

Gaussians on Riemannian Manifolds: Applications for Robot Learning and Adaptive Control


Title	Gaussians on Riemannian Manifolds: Applications for Robot Learning and Adaptive Control
Authors	Sylvain Calinon
Abstract	This article presents an overview of robot learning and adaptive control applications that can benefit from a joint use of Riemannian geometry and probabilistic representations. The roles of Riemannian manifolds, geodesics and parallel transport in robotics are first discussed. Several forms of manifolds already employed in robotics are then presented, by also listing manifolds that have been underexploited but that have potentials in future robot learning applications. A varied range of techniques employing Gaussian distributions on Riemannian manifolds is then introduced, including clustering, regression, information fusion, planning and control problems. Two examples of applications are presented, involving the control of a prosthetic hand from surface electromyography (sEMG) data, and the teleoperation of a bimanual underwater robot. Further perspectives are finally discussed, with suggestions of promising research directions.
Tasks
Published	2019-09-12
URL	https://arxiv.org/abs/1909.05946v4
PDF	https://arxiv.org/pdf/1909.05946v4.pdf
PWC	https://paperswithcode.com/paper/gaussians-on-riemannian-manifolds-for-robot
Repo
Framework