January 28, 2020

3015 words 15 mins read

Paper Group ANR 836

Boosting Image Recognition with Non-differentiable Constraints. Dynamic Past and Future for Neural Machine Translation. Liver Steatosis Segmentation with Deep Learning Methods. Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples. Spatial Filtering Pipeline Evaluation of Cortically Coupled Compute …

Boosting Image Recognition with Non-differentiable Constraints


Title	Boosting Image Recognition with Non-differentiable Constraints
Authors	Xuan Li, Yuchen Lu, Peng Xu, Jizong Peng, Christian Desrosiers, Xue Liu
Abstract	In this paper, we study the problem of image recognition with non-differentiable constraints. A lot of real-life recognition applications require a rich output structure with deterministic constraints that are discrete or modeled by a non-differentiable function. A prime example is recognizing digit sequences, which are restricted by such rules (e.g., \textit{container code detection}, \textit{social insurance number recognition}, etc.). We investigate the usefulness of adding non-differentiable constraints in learning for the task of digit sequence recognition. Toward this goal, we synthesize six different datasets from MNIST and Cropped SVHN, with three discrete rules inspired by real-life protocols. To deal with the non-differentiability of these rules, we propose a reinforcement learning approach based on the policy gradient method. We find that incorporating this rule-based reinforcement can effectively increase the accuracy for all datasets and provide a good inductive bias which improves the model even with limited data. On one of the datasets, MNIST_Rule2, models trained with rule-based reinforcement increase the accuracy by 4.7% for 2000 samples and 23.6% for 500 samples. We further test our model against synthesized adversarial examples, e.g., blocking out digits, and observe that adding our rule-based reinforcement increases the model robustness with a relatively smaller performance drop.
Tasks
Published	2019-10-02
URL	https://arxiv.org/abs/1910.00736v1
PDF	https://arxiv.org/pdf/1910.00736v1.pdf
PWC	https://paperswithcode.com/paper/boosting-image-recognition-with-non
Repo
Framework

Dynamic Past and Future for Neural Machine Translation


Title	Dynamic Past and Future for Neural Machine Translation
Authors	Zaixiang Zheng, Shujian Huang, Zhaopeng Tu, Xin-Yu Dai, Jiajun Chen
Abstract	Previous studies have shown that neural machine translation (NMT) models can benefit from explicitly modeling translated (Past) and untranslated (Future) to groups of translated and untranslated contents through parts-to-wholes assignment. The assignment is learned through a novel variant of routing-by-agreement mechanism (Sabour et al., 2017), namely {\em Guided Dynamic Routing}, where the translating status at each decoding step {\em guides} the routing process to assign each source word to its associated group (i.e., translated or untranslated content) represented by a capsule, enabling translation to be made from holistic context. Experiments show that our approach achieves substantial improvements over both RNMT and Transformer by producing more adequate translations. Extensive analysis demonstrates that our method is highly interpretable, which is able to recognize the translated and untranslated contents as expected.
Tasks	Machine Translation
Published	2019-04-21
URL	https://arxiv.org/abs/1904.09646v2
PDF	https://arxiv.org/pdf/1904.09646v2.pdf
PWC	https://paperswithcode.com/paper/dynamic-past-and-future-for-neural-machine
Repo
Framework

Liver Steatosis Segmentation with Deep Learning Methods


Title	Liver Steatosis Segmentation with Deep Learning Methods
Authors	Xiaoyuan Guo, Fusheng Wang, George Teodorou, Alton B. Farris, Jun Kong
Abstract	Liver steatosis is known as the abnormal accumulation of lipids within cells. An accurate quantification of steatosis area within the liver histopathological microscopy images plays an important role in liver disease diagnosis and trans-plantation assessment. Such a quantification analysis often requires a precise steatosis segmentation that is challenging due to abundant presence of highly overlapped steatosis droplets. In this paper, a deep learning model Mask-RCNN is used to segment the steatosis droplets in clumps. Extended from Faster R-CNN, Mask-RCNN can predict object masks in addition to bounding box detection. With transfer learning, the resulting model is able to segment overlapped steatosis regions at 75.87% by Average Precision, 60.66% by Recall,65.88% by F1-score, and 76.97% by Jaccard index, promising to support liver disease diagnosis and allograft rejection prediction in future clinical practice.
Tasks	Transfer Learning
Published	2019-11-16
URL	https://arxiv.org/abs/1911.07088v1
PDF	https://arxiv.org/pdf/1911.07088v1.pdf
PWC	https://paperswithcode.com/paper/liver-steatosis-segmentation-with-deep
Repo
Framework

Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples


Title	Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples
Authors	Vikash Sehwag, Arjun Nitin Bhagoji, Liwei Song, Chawin Sitawarin, Daniel Cullina, Mung Chiang, Prateek Mittal
Abstract	A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follow a pre-specified distribution. However, real-world implementations of deep learning applications, such as autonomous driving and content classification are likely to operate in the open-world environment. In this paper, we demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out-of-distribution inputs (OOD adversarial examples). In our study, we use 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity. We first demonstrate that state-of-the-art detectors for out-of-distribution data are not robust against OOD adversarial examples. We then consider 5 known defenses for adversarial examples, including state-of-the-art robust training methods, and show that against these defenses, OOD adversarial examples can achieve up to 4$\times$ higher target success rates compared to adversarial examples generated from in-distribution data. We also take a quantitative look at how open-world evasion attacks may affect real-world systems. Finally, we present the first steps towards a robust open-world machine learning system.
Tasks	Autonomous Driving
Published	2019-05-05
URL	https://arxiv.org/abs/1905.01726v1
PDF	https://arxiv.org/pdf/1905.01726v1.pdf
PWC	https://paperswithcode.com/paper/better-the-devil-you-know-an-analysis-of
Repo
Framework

Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation


Title	Spatial Filtering Pipeline Evaluation of Cortically Coupled Computer Vision System for Rapid Serial Visual Presentation
Authors	Zhengwei Wang, Graham Healy, Alan F. Smeaton, Tomas E. Ward
Abstract	Rapid Serial Visual Presentation (RSVP) is a paradigm that supports the application of cortically coupled computer vision to rapid image search. In RSVP, images are presented to participants in a rapid serial sequence which can evoke Event-related Potentials (ERPs) detectable in their Electroencephalogram (EEG). The contemporary approach to this problem involves supervised spatial filtering techniques which are applied for the purposes of enhancing the discriminative information in the EEG data. In this paper we make two primary contributions to that field: 1) We propose a novel spatial filtering method which we call the Multiple Time Window LDA Beamformer (MTWLB) method; 2) we provide a comprehensive comparison of nine spatial filtering pipelines using three spatial filtering schemes namely, MTWLB, xDAWN, Common Spatial Pattern (CSP) and three linear classification methods Linear Discriminant Analysis (LDA), Bayesian Linear Regression (BLR) and Logistic Regression (LR). Three pipelines without spatial filtering are used as baseline comparison. The Area Under Curve (AUC) is used as an evaluation metric in this paper. The results reveal that MTWLB and xDAWN spatial filtering techniques enhance the classification performance of the pipeline but CSP does not. The results also support the conclusion that LR can be effective for RSVP based BCI if discriminative features are available.
Tasks	EEG, Image Retrieval
Published	2019-01-15
URL	http://arxiv.org/abs/1901.04618v1
PDF	http://arxiv.org/pdf/1901.04618v1.pdf
PWC	https://paperswithcode.com/paper/spatial-filtering-pipeline-evaluation-of
Repo
Framework

Propose-and-Attend Single Shot Detector


Title	Propose-and-Attend Single Shot Detector
Authors	Ho-Deok Jang, Sanghyun Woo, Philipp Benz, Jinsun Park, In So Kweon
Abstract	We present a simple yet effective prediction module for a one-stage detector. The main process is conducted in a coarse-to-fine manner. First, the module roughly adjusts the default boxes to well capture the extent of target objects in an image. Second, given the adjusted boxes, the module aligns the receptive field of the convolution filters accordingly, not requiring any embedding layers. Both steps build a propose-and-attend mechanism, mimicking two-stage detectors in a highly efficient manner. To verify its effectiveness, we apply the proposed module to a basic one-stage detector SSD. Our final model achieves an accuracy comparable to that of state-of-the-art detectors while using a fraction of their model parameters and computational overheads. Moreover, we found that the proposed module has two strong applications. 1) The module can be successfully integrated into a lightweight backbone, further pushing the efficiency of the one-stage detector. 2) The module also allows train-from-scratch without relying on any sophisticated base networks as previous methods do.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1907.12736v1
PDF	https://arxiv.org/pdf/1907.12736v1.pdf
PWC	https://paperswithcode.com/paper/propose-and-attend-single-shot-detector
Repo
Framework

Learning in Competitive Network with Haeusslers Equation adapted using FIREFLY algorithm


Title	Learning in Competitive Network with Haeusslers Equation adapted using FIREFLY algorithm
Authors	N. Joshi
Abstract	Many of the competitive neural network consists of spatially arranged neurons. The weigh matrix that connects cells represents local excitation and long-range inhibition. They are known as soft-winner-take-all networks and shown to exhibit desirable information-processing. The local excitatory connections are many times predefined hand-wired based depending on spatial arrangement which is chosen using the previous knowledge of data. Here we present learning in recurrent network through Haeusslers equation and modified wiring scheme based on biologically based Firefly algorithm. Following results show learning in such network from input patterns without hand-wiring with fixed topology.
Tasks
Published	2019-07-09
URL	https://arxiv.org/abs/1907.04160v1
PDF	https://arxiv.org/pdf/1907.04160v1.pdf
PWC	https://paperswithcode.com/paper/learning-in-competitive-network-with
Repo
Framework

Anti-Alignments – Measuring The Precision of Process Models and Event Logs


Title	Anti-Alignments – Measuring The Precision of Process Models and Event Logs
Authors	Thomas Chatain, Mathilde Boltenhagen, Josep Carmona
Abstract	Processes are a crucial artefact in organizations, since they coordinate the execution of activities so that products and services are provided. The use of models to analyse the underlying processes is a well-known practice. However, due to the complexity and continuous evolution of their processes, organizations need an effective way of analysing the relation between processes and models. Conformance checking techniques asses the suitability of a process model in representing an underlying process, observed through a collection of real executions. One important metric in conformance checking is to asses the precision of the model with respect to the observed executions, i.e., characterize the ability of the model to produce behavior unrelated to the one observed. In this paper we present the notion of anti-alignment as a concept to help unveiling runs in the model that may deviate significantly from the observed behavior. Using anti-alignments, a new metric for precision is proposed. In contrast to existing metrics, anti-alignment based precision metrics satisfy most of the required axioms highlighted in a recent publication. Moreover, a complexity analysis of the problem of computing anti-alignments is provided, which sheds light into the practicability of using anti-alignment to estimate precision. Experiments are provided that witness the validity of the concepts introduced in this paper.
Tasks
Published	2019-11-28
URL	https://arxiv.org/abs/1912.05907v1
PDF	https://arxiv.org/pdf/1912.05907v1.pdf
PWC	https://paperswithcode.com/paper/anti-alignments-measuring-the-precision-of
Repo
Framework

Enhancing Out-Of-Domain Utterance Detection with Data Augmentation Based on Word Embeddings


Title	Enhancing Out-Of-Domain Utterance Detection with Data Augmentation Based on Word Embeddings
Authors	Yueqi Feng, Jiali Lin
Abstract	For most intelligent assistant systems, it is essential to have a mechanism that detects out-of-domain (OOD) utterances automatically to handle noisy input properly. One typical approach would be introducing a separate class that contains OOD utterance examples combined with in-domain text samples into the classifier. However, since OOD utterances are usually unseen to the training datasets, the detection performance largely depends on the quality of the attached OOD text data with restricted sizes of samples due to computing limits. In this paper, we study how augmented OOD data based on sampling impact OOD utterance detection with a small sample size. We hypothesize that OOD utterance samples chosen randomly can increase the coverage of unknown OOD utterance space and enhance detection accuracy if they are more dispersed. Experiments show that given the same dataset with the same OOD sample size, the OOD utterance detection performance improves when OOD samples are more spread-out.
Tasks	Data Augmentation, Word Embeddings
Published	2019-11-24
URL	https://arxiv.org/abs/1911.10439v3
PDF	https://arxiv.org/pdf/1911.10439v3.pdf
PWC	https://paperswithcode.com/paper/enhancing-out-of-domain-utterance-detection
Repo
Framework

HAD-GAN: A Human-perception Auxiliary Defense GAN to Defend Adversarial Examples


Title	HAD-GAN: A Human-perception Auxiliary Defense GAN to Defend Adversarial Examples
Authors	Wanting Yu, Hongyi Yu, Lingyun Jiang, Mengli Zhang, Kai Qiao, Linyuan Wang, Bin Yan
Abstract	Adversarial examples reveal the vulnerability and unexplained nature of neural networks. Studying the defense of adversarial examples is of considerable practical importance. Most adversarial examples that misclassify networks are often undetectable by humans. In this paper, we propose a defense model to train the classifier into a human-perception classification model with shape preference. The proposed model comprising a texture transfer network (TTN) and an auxiliary defense generative adversarial networks (GAN) is called Human-perception Auxiliary Defense GAN (HAD-GAN). The TTN is used to extend the texture samples of a clean image and helps classifiers focus on its shape. GAN is utilized to form a training framework for the model and generate the necessary images. A series of experiments conducted on MNIST, Fashion-MNIST and CIFAR10 show that the proposed model outperforms the state-of-the-art defense methods for network robustness. The model also demonstrates a significant improvement on defense capability of adversarial examples.
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.07558v2
PDF	https://arxiv.org/pdf/1909.07558v2.pdf
PWC	https://paperswithcode.com/paper/had-gan-a-human-perception-auxiliary-defense
Repo
Framework

Predicting Citywide Crowd Flows in Irregular Regions Using Multi-View Graph Convolutional Networks


Title	Predicting Citywide Crowd Flows in Irregular Regions Using Multi-View Graph Convolutional Networks
Authors	Junkai Sun, Junbo Zhang, Qiaofei Li, Xiuwen Yi, Yu Zheng
Abstract	Being able to predict the crowd flows in each and every part of a city, especially in irregular regions, is strategically important for traffic control, risk assessment, and public safety. However, it is very challenging because of interactions and spatial correlations between different regions. In addition, it is affected by many factors: i) multiple temporal correlations among different time intervals: closeness, period, trend; ii) complex external influential factors: weather, events; iii) meta features: time of the day, day of the week, and so on. In this paper, we formulate crowd flow forecasting in irregular regions as a spatio-temporal graph (STG) prediction problem in which each node represents a region with time-varying flows. By extending graph convolution to handle the spatial information, we propose using spatial graph convolution to build a multi-view graph convolutional network (MVGCN) for the crowd flow forecasting problem, where different views can capture different factors as mentioned above. We evaluate MVGCN using four real-world datasets (taxicabs and bikes) and extensive experimental results show that our approach outperforms the adaptations of state-of-the-art methods. And we have developed a crowd flow forecasting system for irregular regions that can now be used internally.
Tasks
Published	2019-03-19
URL	http://arxiv.org/abs/1903.07789v1
PDF	http://arxiv.org/pdf/1903.07789v1.pdf
PWC	https://paperswithcode.com/paper/predicting-citywide-crowd-flows-in-irregular
Repo
Framework

Improving Deep Image Clustering With Spatial Transformer Layers


Title	Improving Deep Image Clustering With Spatial Transformer Layers
Authors	Thiago V. M. Souza, Cleber Zanchettin
Abstract	Image clustering is an important but challenging task in machine learning. As in most image processing areas, the latest improvements came from models based on the deep learning approach. However, classical deep learning methods have problems to deal with spatial image transformations like scale and rotation. In this paper, we propose the use of visual attention techniques to reduce this problem in image clustering methods. We evaluate the combination of a deep image clustering model called Deep Adaptive Clustering (DAC) with the Spatial Transformer Networks (STN). The proposed model is evaluated in the datasets MNIST and FashionMNIST and outperformed the baseline model.
Tasks	Image Clustering
Published	2019-02-09
URL	https://arxiv.org/abs/1902.05401v2
PDF	https://arxiv.org/pdf/1902.05401v2.pdf
PWC	https://paperswithcode.com/paper/improving-deep-image-clustering-with-spatial
Repo
Framework

Dealing with Topological Information within a Fully Convolutional Neural Network


Title	Dealing with Topological Information within a Fully Convolutional Neural Network
Authors	Etienne Decencière, Santiago Velasco-Forero, Fu Min, Juanjuan Chen, Hélène Burdin, Gervais Gauthier, Bruno Laÿ, Thomas Bornschloegl, Thérèse Baldeweck
Abstract	A fully convolutional neural network has a receptive field of limited size and therefore cannot exploit global information, such as topological information. A solution is proposed in this paper to solve this problem, based on pre-processing with a geodesic operator. It is applied to the segmentation of histological images of pigmented reconstructed epidermis acquired via Whole Slide Imaging.
Tasks
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11600v1
PDF	https://arxiv.org/pdf/1906.11600v1.pdf
PWC	https://paperswithcode.com/paper/dealing-with-topological-information-within-a
Repo
Framework

Convex optimization for the densest subgraph and densest submatrix problems


Title	Convex optimization for the densest subgraph and densest submatrix problems
Authors	Polina Bombina, Brendan Ames
Abstract	We consider the densest $k$-subgraph problem, which seeks to identify the $k$-node subgraph of a given input graph with maximum number of edges. This problem is well-known to be NP-hard, by reduction to the maximum clique problem. We propose a new convex relaxation for the densest $k$-subgraph problem, based on a nuclear norm relaxation of a low-rank plus sparse decomposition of the adjacency matrices of $k$-node subgraphs to partially address this intractability. We establish that the densest $k$-subgraph can be recovered with high probability from the optimal solution of this convex relaxation if the input graph is randomly sampled from a distribution of random graphs constructed to contain an especially dense $k$-node subgraph with high probability. Specifically, the relaxation is exact when the edges of the input graph are added independently at random, with edges within a particular $k$-node subgraph added with higher probability than other edges in the graph. We provide a sufficient condition on the size of this subgraph $k$ and the expected density under which the optimal solution of the proposed relaxation recovers this $k$-node subgraph with high probability. Further, we propose a first-order method for solving this relaxation based on the alternating direction method of multipliers, and empirically confirm our predicted recovery thresholds using simulations involving randomly generated graphs, as well as graphs drawn from social and collaborative networks.
Tasks
Published	2019-04-05
URL	http://arxiv.org/abs/1904.03272v1
PDF	http://arxiv.org/pdf/1904.03272v1.pdf
PWC	https://paperswithcode.com/paper/convex-optimization-for-the-densest-subgraph
Repo
Framework

A Non-Negative Factorization approach to node pooling in Graph Convolutional Neural Networks


Title	A Non-Negative Factorization approach to node pooling in Graph Convolutional Neural Networks
Authors	Davide Bacciu, Luigi Di Sotto
Abstract	The paper discusses a pooling mechanism to induce subsampling in graph structured data and introduces it as a component of a graph convolutional neural network. The pooling mechanism builds on the Non-Negative Matrix Factorization (NMF) of a matrix representing node adjacency and node similarity as adaptively obtained through the vertices embedding learned by the model. Such mechanism is applied to obtain an incrementally coarser graph where nodes are adaptively pooled into communities based on the outcomes of the non-negative factorization. The empirical analysis on graph classification benchmarks shows how such coarsening process yields significant improvements in the predictive performance of the model with respect to its non-pooled counterpart.
Tasks	Graph Classification
Published	2019-09-07
URL	https://arxiv.org/abs/1909.03287v1
PDF	https://arxiv.org/pdf/1909.03287v1.pdf
PWC	https://paperswithcode.com/paper/a-non-negative-factorization-approach-to-node
Repo
Framework