July 30, 2019

3122 words 15 mins read

Paper Group AWR 28

Neural Sketch Learning for Conditional Program Generation. ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models. W-Net: A Deep Model for Fully Unsupervised Image Segmentation. Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation. Wembedder: Wikidata entity embedding …

Neural Sketch Learning for Conditional Program Generation


Title	Neural Sketch Learning for Conditional Program Generation
Authors	Vijayaraghavan Murali, Letao Qi, Swarat Chaudhuri, Chris Jermaine
Abstract	We study the problem of generating source code in a strongly typed, Java-like programming language, given a label (for example a set of API calls or types) carrying a small amount of information about the code that is desired. The generated programs are expected to respect a “realistic” relationship between programs and labels, as exemplified by a corpus of labeled programs available during training. Two challenges in such conditional program generation are that the generated programs must satisfy a rich set of syntactic and semantic constraints, and that source code contains many low-level features that impede learning. We address these problems by training a neural generator not on code but on program sketches, or models of program syntax that abstract out names and operations that do not generalize across programs. During generation, we infer a posterior distribution over sketches, then concretize samples from this distribution into type-safe programs using combinatorial techniques. We implement our ideas in a system for generating API-heavy Java code, and show that it can often predict the entire body of a method given just a few API calls or data types that appear in the method.
Tasks
Published	2017-03-16
URL	http://arxiv.org/abs/1703.05698v5
PDF	http://arxiv.org/pdf/1703.05698v5.pdf
PWC	https://paperswithcode.com/paper/neural-sketch-learning-for-conditional
Repo	https://github.com/capergroup/bayou
Framework	tf

ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models


Title	ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models
Authors	Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh
Abstract	Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern on the robustness to adversarial examples, especially for security-critical tasks such as traffic sign identification for autonomous driving. Studies have unveiled the vulnerability of a well-trained DNN by demonstrating the ability of generating barely noticeable (to both human and machines) adversarial images that lead to misclassification. Furthermore, researchers have shown that these adversarial images are highly transferable by simply training and attacking a substitute model built upon the target model, known as a black-box attack to DNNs. Similar to the setting of training substitute models, in this paper we propose an effective black-box attack that also only has access to the input (images) and the output (confidence scores) of a targeted DNN. However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples. We use zeroth order stochastic coordinate descent along with dimension reduction, hierarchical attack and importance sampling techniques to efficiently attack black-box models. By exploiting zeroth order optimization, improved attacks to the targeted DNN can be accomplished, sparing the need for training substitute models and avoiding the loss in attack transferability. Experimental results on MNIST, CIFAR10 and ImageNet show that the proposed ZOO attack is as effective as the state-of-the-art white-box attack and significantly outperforms existing black-box attacks via substitute models.
Tasks	Adversarial Attack, Adversarial Defense, Autonomous Driving, Dimensionality Reduction, Image Classification
Published	2017-08-14
URL	http://arxiv.org/abs/1708.03999v2
PDF	http://arxiv.org/pdf/1708.03999v2.pdf
PWC	https://paperswithcode.com/paper/zoo-zeroth-order-optimization-based-black-box
Repo	https://github.com/huanzhang12/ZOO-Attack
Framework	tf

W-Net: A Deep Model for Fully Unsupervised Image Segmentation


Title	W-Net: A Deep Model for Fully Unsupervised Image Segmentation
Authors	Xide Xia, Brian Kulis
Abstract	While significant attention has been recently focused on designing supervised deep semantic segmentation algorithms for vision tasks, there are many domains in which sufficient supervised pixel-level labels are difficult to obtain. In this paper, we revisit the problem of purely unsupervised image segmentation and propose a novel deep architecture for this problem. We borrow recent ideas from supervised semantic segmentation methods, in particular by concatenating two fully convolutional networks together into an autoencoder–one for encoding and one for decoding. The encoding layer produces a k-way pixelwise prediction, and both the reconstruction error of the autoencoder as well as the normalized cut produced by the encoder are jointly minimized during training. When combined with suitable postprocessing involving conditional random field smoothing and hierarchical segmentation, our resulting algorithm achieves impressive results on the benchmark Berkeley Segmentation Data Set, outperforming a number of competing methods.
Tasks	Semantic Segmentation
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08506v1
PDF	http://arxiv.org/pdf/1711.08506v1.pdf
PWC	https://paperswithcode.com/paper/w-net-a-deep-model-for-fully-unsupervised
Repo	https://github.com/wau/Unsupervised-SIS
Framework	none

Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation


Title	Efficient Yet Deep Convolutional Neural Networks for Semantic Segmentation
Authors	Sharif Amit Kamran, Ali Shihab Sabbir
Abstract	Semantic Segmentation using deep convolutional neural network pose more complex challenge for any GPU intensive task. As it has to compute million of parameters, it results to huge memory consumption. Moreover, extracting finer features and conducting supervised training tends to increase the complexity. With the introduction of Fully Convolutional Neural Network, which uses finer strides and utilizes deconvolutional layers for upsampling, it has been a go to for any image segmentation task. In this paper, we propose two segmentation architecture which not only needs one-third the parameters to compute but also gives better accuracy than the similar architectures. The model weights were transferred from the popular neural net like VGG19 and VGG16 which were trained on Imagenet classification data-set. Then we transform all the fully connected layers to convolutional layers and use dilated convolution for decreasing the parameters. Lastly, we add finer strides and attach four skip architectures which are element-wise summed with the deconvolutional layers in steps. We train and test on different sparse and fine data-sets like Pascal VOC2012, Pascal-Context and NYUDv2 and show how better our model performs in this tasks. On the other hand our model has a faster inference time and consumes less memory for training and testing on NVIDIA Pascal GPUs, making it more efficient and less memory consuming architecture for pixel-wise segmentation.
Tasks	Scene Segmentation, Semantic Segmentation
Published	2017-07-26
URL	http://arxiv.org/abs/1707.08254v3
PDF	http://arxiv.org/pdf/1707.08254v3.pdf
PWC	https://paperswithcode.com/paper/efficient-yet-deep-convolutional-neural
Repo	https://github.com/SharifAmit/DilatedFCNSegmentation
Framework	caffe2

Wembedder: Wikidata entity embedding web service


Title	Wembedder: Wikidata entity embedding web service
Authors	Finn Årup Nielsen
Abstract	I present a web service for querying an embedding of entities in the Wikidata knowledge graph. The embedding is trained on the Wikidata dump using Gensim’s Word2Vec implementation and a simple graph walk. A REST API is implemented. Together with the Wikidata API the web service exposes a multilingual resource for over 600’000 Wikidata items and properties.
Tasks
Published	2017-10-11
URL	http://arxiv.org/abs/1710.04099v1
PDF	http://arxiv.org/pdf/1710.04099v1.pdf
PWC	https://paperswithcode.com/paper/wembedder-wikidata-entity-embedding-web
Repo	https://github.com/fnielsen/wembedder
Framework	none

Channel Pruning for Accelerating Very Deep Neural Networks


Title	Channel Pruning for Accelerating Very Deep Neural Networks
Authors	Yihui He, Xiangyu Zhang, Jian Sun
Abstract	In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks.Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We further generalize this algorithm to multi-layer and multi-branch cases. Our method reduces the accumulated error and enhance the compatibility with various architectures. Our pruned VGG-16 achieves the state-of-the-art results by 5x speed-up along with only 0.3% increase of error. More importantly, our method is able to accelerate modern networks like ResNet, Xception and suffers only 1.4%, 1.0% accuracy loss under 2x speed-up respectively, which is significant. Code has been made publicly available.
Tasks
Published	2017-07-19
URL	http://arxiv.org/abs/1707.06168v2
PDF	http://arxiv.org/pdf/1707.06168v2.pdf
PWC	https://paperswithcode.com/paper/channel-pruning-for-accelerating-very-deep
Repo	https://github.com/yihui-he/channel-pruning
Framework	none

Deep Photo Style Transfer


Title	Deep Photo Style Transfer
Authors	Fujun Luan, Sylvain Paris, Eli Shechtman, Kavita Bala
Abstract	This paper introduces a deep-learning approach to photographic style transfer that handles a large variety of image content while faithfully transferring the reference style. Our approach builds upon the recent work on painterly transfer that separates style from the content of an image by considering different layers of a neural network. However, as is, this approach is not suitable for photorealistic style transfer. Even when both the input and reference images are photographs, the output still exhibits distortions reminiscent of a painting. Our contribution is to constrain the transformation from the input to the output to be locally affine in colorspace, and to express this constraint as a custom fully differentiable energy term. We show that this approach successfully suppresses distortion and yields satisfying photorealistic style transfers in a broad variety of scenarios, including transfer of the time of day, weather, season, and artistic edits.
Tasks	Style Transfer
Published	2017-03-22
URL	http://arxiv.org/abs/1703.07511v3
PDF	http://arxiv.org/pdf/1703.07511v3.pdf
PWC	https://paperswithcode.com/paper/deep-photo-style-transfer
Repo	https://github.com/brstream/links
Framework	none

Deep Unsupervised Clustering Using Mixture of Autoencoders


Title	Deep Unsupervised Clustering Using Mixture of Autoencoders
Authors	Dejiao Zhang, Yifan Sun, Brian Eriksson, Laura Balzano
Abstract	Unsupervised clustering is one of the most fundamental challenges in machine learning. A popular hypothesis is that data are generated from a union of low-dimensional nonlinear manifolds; thus an approach to clustering is identifying and separating these manifolds. In this paper, we present a novel approach to solve this problem by using a mixture of autoencoders. Our model consists of two parts: 1) a collection of autoencoders where each autoencoder learns the underlying manifold of a group of similar objects, and 2) a mixture assignment neural network, which takes the concatenated latent vectors from the autoencoders as input and infers the distribution over clusters. By jointly optimizing the two parts, we simultaneously assign data to clusters and learn the underlying manifolds of each cluster.
Tasks
Published	2017-12-21
URL	http://arxiv.org/abs/1712.07788v2
PDF	http://arxiv.org/pdf/1712.07788v2.pdf
PWC	https://paperswithcode.com/paper/deep-unsupervised-clustering-using-mixture-of
Repo	https://github.com/icannos/mixture-autoencoder
Framework	tf

A Read-Write Memory Network for Movie Story Understanding


Title	A Read-Write Memory Network for Movie Story Understanding
Authors	Seil Na, Sangho Lee, Jisung Kim, Gunhee Kim
Abstract	We propose a novel memory network model named Read-Write Memory Network (RWMN) to perform question and answering tasks for large-scale, multimodal movie story understanding. The key focus of our RWMN model is to design the read network and the write network that consist of multiple convolutional layers, which enable memory read and write operations to have high capacity and flexibility. While existing memory-augmented network models treat each memory slot as an independent block, our use of multi-layered CNNs allows the model to read and write sequential memory cells as chunks, which is more reasonable to represent a sequential story because adjacent memory blocks often have strong correlations. For evaluation, we apply our model to all the six tasks of the MovieQA benchmark, and achieve the best accuracies on several tasks, especially on the visual QA task. Our model shows a potential to better understand not only the content in the story, but also more abstract information, such as relationships between characters and the reasons for their actions.
Tasks	Video Story QA
Published	2017-09-27
URL	http://arxiv.org/abs/1709.09345v4
PDF	http://arxiv.org/pdf/1709.09345v4.pdf
PWC	https://paperswithcode.com/paper/a-read-write-memory-network-for-movie-story
Repo	https://github.com/seilna/RWMN
Framework	tf

Neural Style Transfer: A Review


Title	Neural Style Transfer: A Review
Authors	Yongcheng Jing, Yezhou Yang, Zunlei Feng, Jingwen Ye, Yizhou Yu, Mingli Song
Abstract	The seminal work of Gatys et al. demonstrated the power of Convolutional Neural Networks (CNNs) in creating artistic imagery by separating and recombining image content and style. This process of using CNNs to render a content image in different styles is referred to as Neural Style Transfer (NST). Since then, NST has become a trending topic both in academic literature and industrial applications. It is receiving increasing attention and a variety of approaches are proposed to either improve or extend the original NST algorithm. In this paper, we aim to provide a comprehensive overview of the current progress towards NST. We first propose a taxonomy of current algorithms in the field of NST. Then, we present several evaluation methods and compare different NST algorithms both qualitatively and quantitatively. The review concludes with a discussion of various applications of NST and open problems for future research. A list of papers discussed in this review, corresponding codes, pre-trained models and more comparison results are publicly available at https://github.com/ycjing/Neural-Style-Transfer-Papers.
Tasks	Style Transfer
Published	2017-05-11
URL	http://arxiv.org/abs/1705.04058v7
PDF	http://arxiv.org/pdf/1705.04058v7.pdf
PWC	https://paperswithcode.com/paper/neural-style-transfer-a-review
Repo	https://github.com/andy-yangz/writing_style_transfer
Framework	none

Deep Neural Networks as Gaussian Processes


Title	Deep Neural Networks as Gaussian Processes
Authors	Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl-Dickstein
Abstract	It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of evaluating the corresponding GP. Recently, kernel functions which mimic multi-layer random neural networks have been developed, but only outside of a Bayesian framework. As such, previous work has not identified that these kernels can be used as covariance functions for GPs and allow fully Bayesian prediction with a deep neural network. In this work, we derive the exact equivalence between infinitely wide deep networks and GPs. We further develop a computationally efficient pipeline to compute the covariance function for these GPs. We then use the resulting GPs to perform Bayesian inference for wide deep neural networks on MNIST and CIFAR-10. We observe that trained neural network accuracy approaches that of the corresponding GP with increasing layer width, and that the GP uncertainty is strongly correlated with trained network prediction error. We further find that test performance increases as finite-width trained networks are made wider and more similar to a GP, and thus that GP predictions typically outperform those of finite-width networks. Finally we connect the performance of these GPs to the recent theory of signal propagation in random neural networks.
Tasks	Bayesian Inference, Gaussian Processes
Published	2017-11-01
URL	http://arxiv.org/abs/1711.00165v3
PDF	http://arxiv.org/pdf/1711.00165v3.pdf
PWC	https://paperswithcode.com/paper/deep-neural-networks-as-gaussian-processes
Repo	https://github.com/yumaloop/dnn_hsic
Framework	tf

Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models


Title	Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models
Authors	Roman Klokov, Victor Lempitsky
Abstract	We present a new deep learning architecture (called Kd-network) that is designed for 3D model recognition tasks and works with unstructured point clouds. The new architecture performs multiplicative transformations and share parameters of these transformations according to the subdivisions of the point clouds imposed onto them by Kd-trees. Unlike the currently dominant convolutional architectures that usually require rasterization on uniform two-dimensional or three-dimensional grids, Kd-networks do not rely on such grids in any way and therefore avoid poor scaling behaviour. In a series of experiments with popular shape recognition benchmarks, Kd-networks demonstrate competitive performance in a number of shape recognition tasks such as shape classification, shape retrieval and shape part segmentation.
Tasks	3D Part Segmentation
Published	2017-04-04
URL	http://arxiv.org/abs/1704.01222v2
PDF	http://arxiv.org/pdf/1704.01222v2.pdf
PWC	https://paperswithcode.com/paper/escape-from-cells-deep-kd-networks-for-the
Repo	https://github.com/fxia22/kdnet.pytorch
Framework	pytorch

Like What You Like: Knowledge Distill via Neuron Selectivity Transfer


Title	Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
Authors	Zehao Huang, Naiyan Wang
Abstract	Despite deep neural networks have demonstrated extraordinary power in various applications, their superior performances are at expense of high storage and computational costs. Consequently, the acceleration and compression of neural networks have attracted much attention recently. Knowledge Transfer (KT), which aims at training a smaller student network by transferring knowledge from a larger teacher model, is one of the popular solutions. In this paper, we propose a novel knowledge transfer method by treating it as a distribution matching problem. Particularly, we match the distributions of neuron selectivity patterns between teacher and student networks. To achieve this goal, we devise a new KT loss function by minimizing the Maximum Mean Discrepancy (MMD) metric between these distributions. Combined with the original loss function, our method can significantly improve the performance of student networks. We validate the effectiveness of our method across several datasets, and further combine it with other KT methods to explore the best possible results. Last but not least, we fine-tune the model to other tasks such as object detection. The results are also encouraging, which confirm the transferability of the learned features.
Tasks	Object Detection, Transfer Learning
Published	2017-07-05
URL	http://arxiv.org/abs/1707.01219v2
PDF	http://arxiv.org/pdf/1707.01219v2.pdf
PWC	https://paperswithcode.com/paper/like-what-you-like-knowledge-distill-via
Repo	https://github.com/TuSimple/neuron-selectivity-transfer
Framework	tf

Learning model-based planning from scratch


Title	Learning model-based planning from scratch
Authors	Razvan Pascanu, Yujia Li, Oriol Vinyals, Nicolas Heess, Lars Buesing, Sebastien Racanière, David Reichert, Théophane Weber, Daan Wierstra, Peter Battaglia
Abstract	Conventional wisdom holds that model-based planning is a powerful approach to sequential decision-making. It is often very challenging in practice, however, because while a model can be used to evaluate a plan, it does not prescribe how to construct a plan. Here we introduce the “Imagination-based Planner”, the first model-based, sequential decision-making agent that can learn to construct, evaluate, and execute plans. Before any action, it can perform a variable number of imagination steps, which involve proposing an imagined action and evaluating it with its model-based imagination. All imagined actions and outcomes are aggregated, iteratively, into a “plan context” which conditions future real and imagined actions. The agent can even decide how to imagine: testing out alternative imagined actions, chaining sequences of actions together, or building a more complex “imagination tree” by navigating flexibly among the previously imagined states using a learned policy. And our agent can learn to plan economically, jointly optimizing for external rewards and computational costs associated with using its imagination. We show that our architecture can learn to solve a challenging continuous control problem, and also learn elaborate planning strategies in a discrete maze-solving task. Our work opens a new direction toward learning the components of a model-based planning system and how to use them.
Tasks	Continuous Control, Decision Making
Published	2017-07-19
URL	http://arxiv.org/abs/1707.06170v1
PDF	http://arxiv.org/pdf/1707.06170v1.pdf
PWC	https://paperswithcode.com/paper/learning-model-based-planning-from-scratch
Repo	https://github.com/Driesssens/ibp-pytorch
Framework	pytorch

Towards Generalization and Simplicity in Continuous Control


Title	Towards Generalization and Simplicity in Continuous Control
Authors	Aravind Rajeswaran, Kendall Lowrey, Emanuel Todorov, Sham Kakade
Abstract	This work shows that policies with simple linear and RBF parameterizations can be trained to solve a variety of continuous control tasks, including the OpenAI gym benchmarks. The performance of these trained policies are competitive with state of the art results, obtained with more elaborate parameterizations such as fully connected neural networks. Furthermore, existing training and testing scenarios are shown to be very limited and prone to over-fitting, thus giving rise to only trajectory-centric policies. Training with a diverse initial state distribution is shown to produce more global policies with better generalization. This allows for interactive control scenarios where the system recovers from large on-line perturbations; as shown in the supplementary video.
Tasks	Continuous Control
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02660v2
PDF	http://arxiv.org/pdf/1703.02660v2.pdf
PWC	https://paperswithcode.com/paper/towards-generalization-and-simplicity-in
Repo	https://github.com/cvoelcker/reinforcement-learning-material-ws-2018
Framework	none