July 29, 2019

2604 words 13 mins read

Paper Group AWR 175

Synthesizing Robust Adversarial Examples. Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks. Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition. Working hard to know your neighbor’s margins: Local descriptor learning loss. Deep MIMO Detection. Light-weight place recognition and loop de …

Synthesizing Robust Adversarial Examples


Title	Synthesizing Robust Adversarial Examples
Authors	Anish Athalye, Logan Engstrom, Andrew Ilyas, Kevin Kwok
Abstract	Standard methods for generating adversarial examples for neural networks do not consistently fool neural network classifiers in the physical world due to a combination of viewpoint shifts, camera noise, and other natural transformations, limiting their relevance to real-world systems. We demonstrate the existence of robust 3D adversarial objects, and we present the first algorithm for synthesizing examples that are adversarial over a chosen distribution of transformations. We synthesize two-dimensional adversarial images that are robust to noise, distortion, and affine transformation. We apply our algorithm to complex three-dimensional objects, using 3D-printing to manufacture the first physical adversarial objects. Our results demonstrate the existence of 3D adversarial objects in the physical world.
Tasks
Published	2017-07-24
URL	http://arxiv.org/abs/1707.07397v3
PDF	http://arxiv.org/pdf/1707.07397v3.pdf
PWC	https://paperswithcode.com/paper/synthesizing-robust-adversarial-examples
Repo	https://github.com/prabhant/synthesizing-robust-adversarial-examples
Framework	tf

Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks


Title	Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
Authors	Ying Zhang, Mohammad Pezeshki, Philemon Brakel, Saizheng Zhang, Cesar Laurent Yoshua Bengio, Aaron Courville
Abstract	Convolutional Neural Networks (CNNs) are effective models for reducing spectral variations and modeling spectral correlations in acoustic features for automatic speech recognition (ASR). Hybrid speech recognition systems incorporating CNNs with Hidden Markov Models/Gaussian Mixture Models (HMMs/GMMs) have achieved the state-of-the-art in various benchmarks. Meanwhile, Connectionist Temporal Classification (CTC) with Recurrent Neural Networks (RNNs), which is proposed for labeling unsegmented sequences, makes it feasible to train an end-to-end speech recognition system instead of hybrid settings. However, RNNs are computationally expensive and sometimes difficult to train. In this paper, inspired by the advantages of both CNNs and the CTC approach, we propose an end-to-end speech framework for sequence labeling, by combining hierarchical CNNs with CTC directly without recurrent connections. By evaluating the approach on the TIMIT phoneme recognition task, we show that the proposed model is not only computationally efficient, but also competitive with the existing baseline systems. Moreover, we argue that CNNs have the capability to model temporal correlations with appropriate context information.
Tasks	End-To-End Speech Recognition, Speech Recognition
Published	2017-01-10
URL	http://arxiv.org/abs/1701.02720v1
PDF	http://arxiv.org/pdf/1701.02720v1.pdf
PWC	https://paperswithcode.com/paper/towards-end-to-end-speech-recognition-with
Repo	https://github.com/sdrobert/more-or-let
Framework	tf

Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition


Title	Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition
Authors	Earnest E. Hansley, Mauricio Pamplona Segundo, Sudeep Sarkar
Abstract	We present an unconstrained ear recognition framework that outperforms state-of-the-art systems in different publicly available image databases. To this end, we developed CNN-based solutions for ear normalization and description, we used well-known handcrafted descriptors, and we fused learned and handcrafted features to improve recognition. We designed a two-stage landmark detector that successfully worked under untrained scenarios. We used the results generated to perform a geometric image normalization that boosted the performance of all evaluated descriptors. Our CNN descriptor outperformed other CNN-based works in the literature, specially in more difficult scenarios. The fusion of learned and handcrafted matchers appears to be complementary as it achieved the best performance in all experiments. The obtained results outperformed all other reported results for the UERC challenge, which contains the most difficult database nowadays.
Tasks
Published	2017-10-20
URL	http://arxiv.org/abs/1710.07662v1
PDF	http://arxiv.org/pdf/1710.07662v1.pdf
PWC	https://paperswithcode.com/paper/employing-fusion-of-learned-and-handcrafted
Repo	https://github.com/maups/ear-recognition
Framework	none

Working hard to know your neighbor’s margins: Local descriptor learning loss


Title	Working hard to know your neighbor’s margins: Local descriptor learning loss
Authors	Anastasiya Mishchuk, Dmytro Mishkin, Filip Radenovic, Jiri Matas
Abstract	We introduce a novel loss for learning local feature descriptors which is inspired by the Lowe’s matching criterion for SIFT. We show that the proposed loss that maximizes the distance between the closest positive and closest negative patch in the batch is better than complex regularization methods; it works well for both shallow and deep convolution network architectures. Applying the novel loss to the L2Net CNN architecture results in a compact descriptor – it has the same dimensionality as SIFT (128) that shows state-of-art performance in wide baseline stereo, patch verification and instance retrieval benchmarks. It is fast, computing a descriptor takes about 1 millisecond on a low-end GPU.
Tasks
Published	2017-05-30
URL	http://arxiv.org/abs/1705.10872v4
PDF	http://arxiv.org/pdf/1705.10872v4.pdf
PWC	https://paperswithcode.com/paper/working-hard-to-know-your-neighbors-margins
Repo	https://github.com/DagnyT/hardnet
Framework	pytorch

Deep MIMO Detection


Title	Deep MIMO Detection
Authors	Neev Samuel, Tzvi Diskin, Ami Wiesel
Abstract	In this paper, we consider the use of deep neural networks in the context of Multiple-Input-Multiple-Output (MIMO) detection. We give a brief introduction to deep learning and propose a modern neural network architecture suitable for this detection task. First, we consider the case in which the MIMO channel is constant, and we learn a detector for a specific system. Next, we consider the harder case in which the parameters are known yet changing and a single detector must be learned for all multiple varying channels. We demonstrate the performance of our deep MIMO detector using numerical simulations in comparison to competing methods including approximate message passing and semidefinite relaxation. The results show that deep networks can achieve state of the art accuracy with significantly lower complexity while providing robustness against ill conditioned channels and mis-specified noise variance.
Tasks
Published	2017-06-04
URL	http://arxiv.org/abs/1706.01151v1
PDF	http://arxiv.org/pdf/1706.01151v1.pdf
PWC	https://paperswithcode.com/paper/deep-mimo-detection
Repo	https://github.com/ZeyuRuan/DetNet
Framework	tf

Light-weight place recognition and loop detection using road markings


Title	Light-weight place recognition and loop detection using road markings
Authors	Oleksandr Bailo, Francois Rameau, In So Kweon
Abstract	In this paper, we propose an efficient algorithm for robust place recognition and loop detection using camera information only. Our pipeline purely relies on spatial localization and semantic information of road markings. The creation of the database of road markings sequences is performed online, which makes the method applicable for real-time loop closure for visual SLAM techniques. Furthermore, our algorithm is robust to various weather conditions, occlusions from vehicles, and shadows. We have performed an extensive number of experiments which highlight the effectiveness and scalability of the proposed method.
Tasks
Published	2017-10-20
URL	http://arxiv.org/abs/1710.07434v1
PDF	http://arxiv.org/pdf/1710.07434v1.pdf
PWC	https://paperswithcode.com/paper/light-weight-place-recognition-and-loop
Repo	https://github.com/BAILOOL/PlaceRecognition-LoopDetection
Framework	none

Real-Time Multiple Object Tracking - A Study on the Importance of Speed


Title	Real-Time Multiple Object Tracking - A Study on the Importance of Speed
Authors	Samuel Murray
Abstract	In this project, we implement a multiple object tracker, following the tracking-by-detection paradigm, as an extension of an existing method. It works by modelling the movement of objects by solving the filtering problem, and associating detections with predicted new locations in new frames using the Hungarian algorithm. Three different similarity measures are used, which use the location and shape of the bounding boxes. Compared to other trackers on the MOTChallenge leaderboard, our method, referred to as C++SORT, is the fastest non-anonymous submission, while also achieving decent score on other metrics. By running our model on the Okutama-Action dataset, sampled at different frame-rates, we show that the performance is greatly reduced when running the model - including detecting objects - in real-time. In most metrics, the score is reduced by 50%, but in certain cases as much as 90%. We argue that this indicates that other, slower methods could not be used for tracking in real-time, but that more research is required specifically on this.
Tasks	Multiple Object Tracking, Object Tracking
Published	2017-09-11
URL	http://arxiv.org/abs/1709.03572v2
PDF	http://arxiv.org/pdf/1709.03572v2.pdf
PWC	https://paperswithcode.com/paper/real-time-multiple-object-tracking-a-study-on
Repo	https://github.com/samuelmurray/tracking-by-detection
Framework	none

MAGAN: Margin Adaptation for Generative Adversarial Networks


Title	MAGAN: Margin Adaptation for Generative Adversarial Networks
Authors	Ruohan Wang, Antoine Cully, Hyung Jin Chang, Yiannis Demiris
Abstract	We propose the Margin Adaptation for Generative Adversarial Networks (MAGANs) algorithm, a novel training procedure for GANs to improve stability and performance by using an adaptive hinge loss function. We estimate the appropriate hinge loss margin with the expected energy of the target distribution, and derive principled criteria for when to update the margin. We prove that our method converges to its global optimum under certain assumptions. Evaluated on the task of unsupervised image generation, the proposed training procedure is simple yet robust on a diverse set of data, and achieves qualitative and quantitative improvements compared to the state-of-the-art.
Tasks	Image Generation
Published	2017-04-12
URL	http://arxiv.org/abs/1704.03817v3
PDF	http://arxiv.org/pdf/1704.03817v3.pdf
PWC	https://paperswithcode.com/paper/magan-margin-adaptation-for-generative
Repo	https://github.com/RuohanW/magan
Framework	tf

OpenML: An R Package to Connect to the Machine Learning Platform OpenML


Title	OpenML: An R Package to Connect to the Machine Learning Platform OpenML
Authors	Giuseppe Casalicchio, Jakob Bossek, Michel Lang, Dominik Kirchhoff, Pascal Kerschke, Benjamin Hofner, Heidi Seibold, Joaquin Vanschoren, Bernd Bischl
Abstract	OpenML is an online machine learning platform where researchers can easily share data, machine learning tasks and experiments as well as organize them online to work and collaborate more efficiently. In this paper, we present an R package to interface with the OpenML platform and illustrate its usage in combination with the machine learning R package mlr. We show how the OpenML package allows R users to easily search, download and upload data sets and machine learning tasks. Furthermore, we also show how to upload results of experiments, share them with others and download results from other users. Beyond ensuring reproducibility of results, the OpenML platform automates much of the drudge work, speeds up research, facilitates collaboration and increases the users’ visibility online.
Tasks
Published	2017-01-05
URL	http://arxiv.org/abs/1701.01293v2
PDF	http://arxiv.org/pdf/1701.01293v2.pdf
PWC	https://paperswithcode.com/paper/openml-an-r-package-to-connect-to-the-machine
Repo	https://github.com/mlr-org/mlr
Framework	none

One Network to Solve Them All — Solving Linear Inverse Problems using Deep Projection Models


Title	One Network to Solve Them All — Solving Linear Inverse Problems using Deep Projection Models
Authors	J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan
Abstract	While deep learning methods have achieved state-of-the-art performance in many challenging inverse problems like image inpainting and super-resolution, they invariably involve problem-specific training of the networks. Under this approach, different problems require different networks. In scenarios where we need to solve a wide variety of problems, e.g., on a mobile camera, it is inefficient and costly to use these specially-trained networks. On the other hand, traditional methods using signal priors can be used in all linear inverse problems but often have worse performance on challenging tasks. In this work, we provide a middle ground between the two kinds of methods — we propose a general framework to train a single deep neural network that solves arbitrary linear inverse problems. The proposed network acts as a proximal operator for an optimization algorithm and projects non-image signals onto the set of natural images defined by the decision boundary of a classifier. In our experiments, the proposed framework demonstrates superior performance over traditional methods using a wavelet sparsity prior and achieves comparable performance of specially-trained networks on tasks including compressive sensing and pixel-wise inpainting.
Tasks	Compressive Sensing, Image Inpainting, Super-Resolution
Published	2017-03-29
URL	http://arxiv.org/abs/1703.09912v1
PDF	http://arxiv.org/pdf/1703.09912v1.pdf
PWC	https://paperswithcode.com/paper/one-network-to-solve-them-all-solving-linear
Repo	https://github.com/image-science-lab/OneNet
Framework	tf

Adversarial Transformation Networks: Learning to Generate Adversarial Examples


Title	Adversarial Transformation Networks: Learning to Generate Adversarial Examples
Authors	Shumeet Baluja, Ian Fischer
Abstract	Multiple different approaches of generating adversarial examples have been proposed to attack deep neural networks. These approaches involve either directly computing gradients with respect to the image pixels, or directly solving an optimization on the image pixels. In this work, we present a fundamentally new method for generating adversarial examples that is fast to execute and provides exceptional diversity of output. We efficiently train feed-forward neural networks in a self-supervised manner to generate adversarial examples against a target network or set of networks. We call such a network an Adversarial Transformation Network (ATN). ATNs are trained to generate adversarial examples that minimally modify the classifier’s outputs given the original input, while constraining the new classification to match an adversarial target class. We present methods to train ATNs and analyze their effectiveness targeting a variety of MNIST classifiers as well as the latest state-of-the-art ImageNet classifier Inception ResNet v2.
Tasks
Published	2017-03-28
URL	http://arxiv.org/abs/1703.09387v1
PDF	http://arxiv.org/pdf/1703.09387v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-transformation-networks-learning
Repo	https://github.com/cs-giung/course-dl-TP
Framework	pytorch

Reducing Reparameterization Gradient Variance


Title	Reducing Reparameterization Gradient Variance
Authors	Andrew C. Miller, Nicholas J. Foti, Alexander D’Amour, Ryan P. Adams
Abstract	Optimization with noisy gradients has become ubiquitous in statistics and machine learning. Reparameterization gradients, or gradient estimates computed via the “reparameterization trick,” represent a class of noisy gradients often used in Monte Carlo variational inference (MCVI). However, when these gradient estimators are too noisy, the optimization procedure can be slow or fail to converge. One way to reduce noise is to use more samples for the gradient estimate, but this can be computationally expensive. Instead, we view the noisy gradient as a random variable, and form an inexpensive approximation of the generating procedure for the gradient sample. This approximation has high correlation with the noisy gradient by construction, making it a useful control variate for variance reduction. We demonstrate our approach on non-conjugate multi-level hierarchical models and a Bayesian neural net where we observed gradient variance reductions of multiple orders of magnitude (20-2,000x).
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.07880v1
PDF	http://arxiv.org/pdf/1705.07880v1.pdf
PWC	https://paperswithcode.com/paper/reducing-reparameterization-gradient-variance
Repo	https://github.com/andymiller/ReducedVarianceReparamGradients
Framework	none

Streaming Sparse Gaussian Process Approximations


Title	Streaming Sparse Gaussian Process Approximations
Authors	Thang D. Bui, Cuong V. Nguyen, Richard E. Turner
Abstract	Sparse pseudo-point approximations for Gaussian process (GP) models provide a suite of methods that support deployment of GPs in the large data regime and enable analytic intractabilities to be sidestepped. However, the field lacks a principled method to handle streaming data in which both the posterior distribution over function values and the hyperparameter estimates are updated in an online fashion. The small number of existing approaches either use suboptimal hand-crafted heuristics for hyperparameter learning, or suffer from catastrophic forgetting or slow updating when new data arrive. This paper develops a new principled framework for deploying Gaussian process probabilistic models in the streaming setting, providing methods for learning hyperparameters and optimising pseudo-input locations. The proposed framework is assessed using synthetic and real-world datasets.
Tasks
Published	2017-05-19
URL	http://arxiv.org/abs/1705.07131v2
PDF	http://arxiv.org/pdf/1705.07131v2.pdf
PWC	https://paperswithcode.com/paper/streaming-sparse-gaussian-process
Repo	https://github.com/thangbui/streaming_sparse_gp
Framework	tf

Generative Adversarial Nets for Multiple Text Corpora


Title	Generative Adversarial Nets for Multiple Text Corpora
Authors	Baiyang Wang, Diego Klabjan
Abstract	Generative adversarial nets (GANs) have been successfully applied to the artificial generation of image data. In terms of text data, much has been done on the artificial generation of natural language from a single corpus. We consider multiple text corpora as the input data, for which there can be two applications of GANs: (1) the creation of consistent cross-corpus word embeddings given different word embeddings per corpus; (2) the generation of robust bag-of-words document embeddings for each corpora. We demonstrate our GAN models on real-world text data sets from different corpora, and show that embeddings from both models lead to improvements in supervised learning problems.
Tasks	Word Embeddings
Published	2017-12-25
URL	http://arxiv.org/abs/1712.09127v1
PDF	http://arxiv.org/pdf/1712.09127v1.pdf
PWC	https://paperswithcode.com/paper/generative-adversarial-nets-for-multiple-text
Repo	https://github.com/baiyangwang/emgan
Framework	none

The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings


Title	The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings
Authors	Krzysztof Choromanski, Mark Rowland, Adrian Weller
Abstract	We examine a class of embeddings based on structured random matrices with orthogonal rows which can be applied in many machine learning applications including dimensionality reduction and kernel approximation. For both the Johnson-Lindenstrauss transform and the angular kernel, we show that we can select matrices yielding guaranteed improved performance in accuracy and/or speed compared to earlier methods. We introduce matrices with complex entries which give significant further accuracy improvement. We provide geometric and Markov chain-based perspectives to help understand the benefits, and empirical results which suggest that the approach is helpful in a wider range of applications.
Tasks	Dimensionality Reduction
Published	2017-03-02
URL	http://arxiv.org/abs/1703.00864v5
PDF	http://arxiv.org/pdf/1703.00864v5.pdf
PWC	https://paperswithcode.com/paper/the-unreasonable-effectiveness-of-structured
Repo	https://github.com/dnbaker/frp
Framework	none