October 18, 2019

3265 words 16 mins read

Paper Group ANR 511

A Dynamic-Adversarial Mining Approach to the Security of Machine Learning. Weighted total variation based convex clustering. Question Relevance in Visual Question Answering. Image-based deep learning for classification of noise transients in gravitational wave detectors. Minimizing Area and Energy of Deep Learning Hardware Design Using Collective L …

A Dynamic-Adversarial Mining Approach to the Security of Machine Learning


Title	A Dynamic-Adversarial Mining Approach to the Security of Machine Learning
Authors	Tegjyot Singh Sethi, Mehmed Kantardzic, Lingyu Lyua, Jiashun Chen
Abstract	Operating in a dynamic real world environment requires a forward thinking and adversarial aware design for classifiers, beyond fitting the model to the training data. In such scenarios, it is necessary to make classifiers - a) harder to evade, b) easier to detect changes in the data distribution over time, and c) be able to retrain and recover from model degradation. While most works in the security of machine learning has concentrated on the evasion resistance (a) problem, there is little work in the areas of reacting to attacks (b and c). Additionally, while streaming data research concentrates on the ability to react to changes to the data distribution, they often take an adversarial agnostic view of the security problem. This makes them vulnerable to adversarial activity, which is aimed towards evading the concept drift detection mechanism itself. In this paper, we analyze the security of machine learning, from a dynamic and adversarial aware perspective. The existing techniques of Restrictive one class classifier models, Complex learning models and Randomization based ensembles, are shown to be myopic as they approach security as a static task. These methodologies are ill suited for a dynamic environment, as they leak excessive information to an adversary, who can subsequently launch attacks which are indistinguishable from the benign data. Based on empirical vulnerability analysis against a sophisticated adversary, a novel feature importance hiding approach for classifier design, is proposed. The proposed design ensures that future attacks on classifiers can be detected and recovered from. The proposed work presents motivation, by serving as a blueprint, for future work in the area of Dynamic-Adversarial mining, which combines lessons learned from Streaming data mining, Adversarial learning and Cybersecurity.
Tasks	Feature Importance, One-class classifier
Published	2018-03-24
URL	http://arxiv.org/abs/1803.09162v1
PDF	http://arxiv.org/pdf/1803.09162v1.pdf
PWC	https://paperswithcode.com/paper/a-dynamic-adversarial-mining-approach-to-the
Repo
Framework

Weighted total variation based convex clustering


Title	Weighted total variation based convex clustering
Authors	Guodong Xu, Yu Xia, Hui Ji
Abstract	Data clustering is a fundamental problem with a wide range of applications. Standard methods, eg the $k$-means method, usually require solving a non-convex optimization problem. Recently, total variation based convex relaxation to the $k$-means model has emerged as an attractive alternative for data clustering. However, the existing results on its exact clustering property, ie, the condition imposed on data so that the method can provably give correct identification of all cluster memberships, is only applicable to very specific data and is also much more restrictive than that of some other methods. This paper aims at the revisit of total variation based convex clustering, by proposing a weighted sum-of-$\ell_1$-norm relating convex model. Its exact clustering property established in this paper, in both deterministic and probabilistic context, is applicable to general data and is much sharper than the existing results. These results provided good insights to advance the research on convex clustering. Moreover, the experiments also demonstrated that the proposed convex model has better empirical performance when be compared to standard clustering methods, and thus it can see its potential in practice.
Tasks
Published	2018-08-28
URL	http://arxiv.org/abs/1808.09144v1
PDF	http://arxiv.org/pdf/1808.09144v1.pdf
PWC	https://paperswithcode.com/paper/weighted-total-variation-based-convex
Repo
Framework

Question Relevance in Visual Question Answering


Title	Question Relevance in Visual Question Answering
Authors	Prakruthi Prabhakar, Nitish Kulkarni, Linghao Zhang
Abstract	Free-form and open-ended Visual Question Answering systems solve the problem of providing an accurate natural language answer to a question pertaining to an image. Current VQA systems do not evaluate if the posed question is relevant to the input image and hence provide nonsensical answers when posed with irrelevant questions to an image. In this paper, we solve the problem of identifying the relevance of the posed question to an image. We address the problem as two sub-problems. We first identify if the question is visual or not. If the question is visual, we then determine if it’s relevant to the image or not. For the second problem, we generate a large dataset from existing visual question answering datasets in order to enable the training of complex architectures and model the relevance of a visual question to an image. We also compare the results of our Long Short-Term Memory Recurrent Neural Network based models to Logistic Regression, XGBoost and multi-layer perceptron based approaches to the problem.
Tasks	Question Answering, Visual Question Answering
Published	2018-07-23
URL	http://arxiv.org/abs/1807.08435v1
PDF	http://arxiv.org/pdf/1807.08435v1.pdf
PWC	https://paperswithcode.com/paper/question-relevance-in-visual-question
Repo
Framework

Image-based deep learning for classification of noise transients in gravitational wave detectors


Title	Image-based deep learning for classification of noise transients in gravitational wave detectors
Authors	Massimiliano Razzano, Elena Cuoco
Abstract	The detection of gravitational waves has inaugurated the era of gravitational astronomy and opened new avenues for the multimessenger study of cosmic sources. Thanks to their sensitivity, the Advanced LIGO and Advanced Virgo interferometers will probe a much larger volume of space and expand the capability of discovering new gravitational wave emitters. The characterization of these detectors is a primary task in order to recognize the main sources of noise and optimize the sensitivity of interferometers. Glitches are transient noise events that can impact the data quality of the interferometers and their classification is an important task for detector characterization. Deep learning techniques are a promising tool for the recognition and classification of glitches. We present a classification pipeline that exploits convolutional neural networks to classify glitches starting from their time-frequency evolution represented as images. We evaluated the classification accuracy on simulated glitches, showing that the proposed algorithm can automatically classify glitches on very fast timescales and with high accuracy, thus providing a promising tool for online detector characterization.
Tasks
Published	2018-03-27
URL	http://arxiv.org/abs/1803.09933v1
PDF	http://arxiv.org/pdf/1803.09933v1.pdf
PWC	https://paperswithcode.com/paper/image-based-deep-learning-for-classification
Repo
Framework

Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression


Title	Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression
Authors	Shihui Yin, Gaurav Srivastava, Shreyas K. Venkataramanaiah, Chaitali Chakrabarti, Visar Berisha, Jae-sun Seo
Abstract	Deep learning algorithms have shown tremendous success in many recognition tasks; however, these algorithms typically include a deep neural network (DNN) structure and a large number of parameters, which makes it challenging to implement them on power/area-constrained embedded platforms. To reduce the network size, several studies investigated compression by introducing element-wise or row-/column-/block-wise sparsity via pruning and regularization. In addition, many recent works have focused on reducing precision of activations and weights with some reducing down to a single bit. However, combining various sparsity structures with binarized or very-low-precision (2-3 bit) neural networks have not been comprehensively explored. In this work, we present design techniques for minimum-area/-energy DNN hardware with minimal degradation in accuracy. During training, both binarization/low-precision and structured sparsity are applied as constraints to find the smallest memory footprint for a given deep learning algorithm. The DNN model for CIFAR-10 dataset with weight memory reduction of 50X exhibits accuracy comparable to that of the floating-point counterpart. Area, performance and energy results of DNN hardware in 40nm CMOS are reported for the MNIST dataset. The optimized DNN that combines 8X structured compression and 3-bit weight precision showed 98.4% accuracy at 20nJ per classification.
Tasks
Published	2018-04-19
URL	http://arxiv.org/abs/1804.07370v1
PDF	http://arxiv.org/pdf/1804.07370v1.pdf
PWC	https://paperswithcode.com/paper/minimizing-area-and-energy-of-deep-learning
Repo
Framework

Adaptive Variational Particle Filtering in Non-stationary Environments


Title	Adaptive Variational Particle Filtering in Non-stationary Environments
Authors	Mahdi Azarafrooz
Abstract	Online convex optimization is a sequential prediction framework with the goal to track and adapt to the environment through evaluating proper convex loss functions. We study efficient particle filtering methods from the perspective of such a framework. We formulate an efficient particle filtering methods for the non-stationary environment by making connections with the online mirror descent algorithm which is known to be a universal online convex optimization algorithm. As a result of this connection, our proposed particle filtering algorithm proves to achieve optimal particle efficiency.
Tasks
Published	2018-07-19
URL	http://arxiv.org/abs/1807.07612v1
PDF	http://arxiv.org/pdf/1807.07612v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-variational-particle-filtering-in
Repo
Framework

Inducing Grammars with and for Neural Machine Translation


Title	Inducing Grammars with and for Neural Machine Translation
Authors	Ke Tran, Yonatan Bisk
Abstract	Machine translation systems require semantic knowledge and grammatical understanding. Neural machine translation (NMT) systems often assume this information is captured by an attention mechanism and a decoder that ensures fluency. Recent work has shown that incorporating explicit syntax alleviates the burden of modeling both types of knowledge. However, requiring parses is expensive and does not explore the question of what syntax a model needs during translation. To address both of these issues we introduce a model that simultaneously translates while inducing dependency trees. In this way, we leverage the benefits of structure while investigating what syntax NMT must induce to maximize performance. We show that our dependency trees are 1. language pair dependent and 2. improve translation quality.
Tasks	Machine Translation
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10850v1
PDF	http://arxiv.org/pdf/1805.10850v1.pdf
PWC	https://paperswithcode.com/paper/inducing-grammars-with-and-for-neural-machine
Repo
Framework

A Hybrid Q-Learning Sine-Cosine-based Strategy for Addressing the Combinatorial Test Suite Minimization Problem


Title	A Hybrid Q-Learning Sine-Cosine-based Strategy for Addressing the Combinatorial Test Suite Minimization Problem
Authors	Kamal Z. Zamli, Fakhrud Din, Bestoun S. Ahmed, Miroslav Bures
Abstract	The sine-cosine algorithm (SCA) is a new population-based meta-heuristic algorithm. In addition to exploiting sine and cosine functions to perform local and global searches (hence the name sine-cosine), the SCA introduces several random and adaptive parameters to facilitate the search process. Although it shows promising results, the search process of the SCA is vulnerable to local minima/maxima due to the adoption of a fixed switch probability and the bounded magnitude of the sine and cosine functions (from -1 to 1). In this paper, we propose a new hybrid Q-learning sine-cosine- based strategy, called the Q-learning sine-cosine algorithm (QLSCA). Within the QLSCA, we eliminate the switching probability. Instead, we rely on the Q-learning algorithm (based on the penalty and reward mechanism) to dynamically identify the best operation during runtime. Additionally, we integrate two new operations (L'evy flight motion and crossover) into the QLSCA to facilitate jumping out of local minima/maxima and enhance the solution diversity. To assess its performance, we adopt the QLSCA for the combinatorial test suite minimization problem. Experimental results reveal that the QLSCA is statistically superior with regard to test suite size reduction compared to recent state-of-the-art strategies, including the original SCA, the particle swarm test generator (PSTG), adaptive particle swarm optimization (APSO) and the cuckoo search strategy (CS) at the 95% confidence level. However, concerning the comparison with discrete particle swarm optimization (DPSO), there is no significant difference in performance at the 95% confidence level. On a positive note, the QLSCA statistically outperforms the DPSO in certain configurations at the 90% confidence level.
Tasks	Q-Learning
Published	2018-04-27
URL	http://arxiv.org/abs/1805.00873v1
PDF	http://arxiv.org/pdf/1805.00873v1.pdf
PWC	https://paperswithcode.com/paper/a-hybrid-q-learning-sine-cosine-based
Repo
Framework

A case for deep learning in semantics


Title	A case for deep learning in semantics
Authors	Christopher Potts
Abstract	Pater’s target article builds a persuasive case for establishing stronger ties between theoretical linguistics and connectionism (deep learning). This commentary extends his arguments to semantics, focusing in particular on issues of learning, compositionality, and lexical meaning.
Tasks
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03068v1
PDF	http://arxiv.org/pdf/1809.03068v1.pdf
PWC	https://paperswithcode.com/paper/a-case-for-deep-learning-in-semantics
Repo
Framework

CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps


Title	CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps
Authors	Paul Hongsuck Seo, Tobias Weyand, Jack Sim, Bohyung Han
Abstract	Image geolocalization is the task of identifying the location depicted in a photo based only on its visual information. This task is inherently challenging since many photos have only few, possibly ambiguous cues to their geolocation. Recent work has cast this task as a classification problem by partitioning the earth into a set of discrete cells that correspond to geographic regions. The granularity of this partitioning presents a critical trade-off; using fewer but larger cells results in lower location accuracy while using more but smaller cells reduces the number of training examples per class and increases model size, making the model prone to overfitting. To tackle this issue, we propose a simple but effective algorithm, combinatorial partitioning, which generates a large number of fine-grained output classes by intersecting multiple coarse-grained partitionings of the earth. Each classifier votes for the fine-grained classes that overlap with their respective coarse-grained ones. This technique allows us to predict locations at a fine scale while maintaining sufficient training examples per class. Our algorithm achieves the state-of-the-art performance in location recognition on multiple benchmark datasets.
Tasks
Published	2018-08-06
URL	http://arxiv.org/abs/1808.02130v1
PDF	http://arxiv.org/pdf/1808.02130v1.pdf
PWC	https://paperswithcode.com/paper/cplanet-enhancing-image-geolocalization-by
Repo
Framework

Collaborative Deep Learning Across Multiple Data Centers


Title	Collaborative Deep Learning Across Multiple Data Centers
Authors	Kele Xu, Haibo Mi, Dawei Feng, Huaimin Wang, Chuan Chen, Zibin Zheng, Xu Lan
Abstract	Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data to a centralized data center due to not only bandwidth limitation but also the constraints of privacy regulations. Model averaging is a conventional choice for data parallelized training, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model averaging can be effective in the decentralized environment by using two strategies, namely, the cyclical learning rate and the increased number of epochs for local model training. With the two strategies, we show that model averaging can provide competitive performance in the decentralized mode compared to the data-centralized one. In a practical environment with multiple data centers, we conduct extensive experiments using state-of-the-art deep network architectures on different types of data. Results demonstrate the effectiveness and robustness of the proposed method.
Tasks
Published	2018-10-16
URL	http://arxiv.org/abs/1810.06877v1
PDF	http://arxiv.org/pdf/1810.06877v1.pdf
PWC	https://paperswithcode.com/paper/collaborative-deep-learning-across-multiple
Repo
Framework

No-reference Image Denoising Quality Assessment


Title	No-reference Image Denoising Quality Assessment
Authors	Si Lu
Abstract	A wide variety of image denoising methods are available now. However, the performance of a denoising algorithm often depends on individual input noisy images as well as its parameter setting. In this paper, we present a no-reference image denoising quality assessment method that can be used to select for an input noisy image the right denoising algorithm with the optimal parameter setting. This is a challenging task as no ground truth is available. This paper presents a data-driven approach to learn to predict image denoising quality. Our method is based on the observation that while individual existing quality metrics and denoising models alone cannot robustly rank denoising results, they often complement each other. We accordingly design denoising quality features based on these existing metrics and models and then use Random Forests Regression to aggregate them into a more powerful unified metric. Our experiments on images with various types and levels of noise show that our no-reference denoising quality assessment method significantly outperforms the state-of-the-art quality metrics. This paper also provides a method that leverages our quality assessment method to automatically tune the parameter settings of a denoising algorithm for an input noisy image to produce an optimal denoising result.
Tasks	Denoising, Image Denoising
Published	2018-10-13
URL	http://arxiv.org/abs/1810.05919v1
PDF	http://arxiv.org/pdf/1810.05919v1.pdf
PWC	https://paperswithcode.com/paper/no-reference-image-denoising-quality
Repo
Framework

Multi-region segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks


Title	Multi-region segmentation of bladder cancer structures in MRI with progressive dilated convolutional networks
Authors	Jose Dolz, Xiaopan Xu, Jerome Rony, Jing Yuan, Yang Liu, Eric Granger, Christian Desrosiers, Xi Zhang, Ismail Ben Ayed, Hongbing Lu
Abstract	Precise segmentation of bladder walls and tumor regions is an essential step towards non-invasive identification of tumor stage and grade, which is critical for treatment decision and prognosis of patients with bladder cancer (BC). However, the automatic delineation of bladder walls and tumor in magnetic resonance images (MRI) is a challenging task, due to important bladder shape variations, strong intensity inhomogeneity in urine and very high variability across population, particularly on tumors appearance. To tackle these issues, we propose to use a deep fully convolutional neural network. The proposed network includes dilated convolutions to increase the receptive field without incurring extra cost nor degrading its performance. Furthermore, we introduce progressive dilations in each convolutional block, thereby enabling extensive receptive fields without the need for large dilation rates. The proposed network is evaluated on 3.0T T2-weighted MRI scans from 60 pathologically confirmed patients with BC. Experiments shows the proposed model to achieve high accuracy, with a mean Dice similarity coefficient of 0.98, 0.84 and 0.69 for inner wall, outer wall and tumor region, respectively. These results represent a very good agreement with reference contours and an increase in performance compared to existing methods. In addition, inference times are less than a second for a whole 3D volume, which is between 2-3 orders of magnitude faster than related state-of-the-art methods for this application. We showed that a CNN can yield precise segmentation of bladder walls and tumors in bladder cancer patients on MRI. The whole segmentation process is fully-automatic and yields results in very good agreement with the reference standard, demonstrating the viability of deep learning models for the automatic multi-region segmentation of bladder cancer MRI images.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10720v4
PDF	http://arxiv.org/pdf/1805.10720v4.pdf
PWC	https://paperswithcode.com/paper/multi-region-segmentation-of-bladder-cancer
Repo
Framework

Inferring Parameters Through Inverse Multiobjective Optimization


Title	Inferring Parameters Through Inverse Multiobjective Optimization
Authors	Chaosheng Dong, Bo Zeng
Abstract	Given a set of human’s decisions that are observed, inverse optimization has been developed and utilized to infer the underlying decision making problem. The majority of existing studies assumes that the decision making problem is with a single objective function, and attributes data divergence to noises, errors or bounded rationality, which, however, could lead to a corrupted inference when decisions are tradeoffs among multiple criteria. In this paper, we take a data-driven approach and design a more sophisticated inverse optimization formulation to explicitly infer parameters of a multiobjective decision making problem from noisy observations. This framework, together with our mathematical analyses and advanced algorithm developments, demonstrates a strong capacity in estimating critical parameters, decoupling “interpretable” components from noises or errors, deriving the denoised \emph{optimal} decisions, and ensuring statistical significance. In particular, for the whole decision maker population, if suitable conditions hold, we will be able to understand the overall diversity and the distribution of their preferences over multiple criteria, which is important when a precise inference on every single decision maker is practically unnecessary or infeasible. Numerical results on a large number of experiments are reported to confirm the effectiveness of our unique inverse optimization model and the computational efficacy of the developed algorithms.
Tasks	Decision Making, Multiobjective Optimization
Published	2018-08-02
URL	http://arxiv.org/abs/1808.00935v1
PDF	http://arxiv.org/pdf/1808.00935v1.pdf
PWC	https://paperswithcode.com/paper/inferring-parameters-through-inverse
Repo
Framework

A machine learning framework for data driven acceleration of computations of differential equations


Title	A machine learning framework for data driven acceleration of computations of differential equations
Authors	Siddhartha Mishra
Abstract	We propose a machine learning framework to accelerate numerical computations of time-dependent ODEs and PDEs. Our method is based on recasting (generalizations of) existing numerical methods as artificial neural networks, with a set of trainable parameters. These parameters are determined in an offline training process by (approximately) minimizing suitable (possibly non-convex) loss functions by (stochastic) gradient descent methods. The proposed algorithm is designed to be always consistent with the underlying differential equation. Numerical experiments involving both linear and non-linear ODE and PDE model problems demonstrate a significant gain in computational efficiency over standard numerical methods.
Tasks
Published	2018-07-25
URL	http://arxiv.org/abs/1807.09519v1
PDF	http://arxiv.org/pdf/1807.09519v1.pdf
PWC	https://paperswithcode.com/paper/a-machine-learning-framework-for-data-driven
Repo
Framework