October 18, 2019

3100 words 15 mins read

Paper Group ANR 552

ClosNets: a Priori Sparse Topologies for Faster DNN Training. Superconducting Optoelectronic Neurons II: Receiver Circuits. The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion. A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration. Cross-dataset Person Re-Identification Using Similarity Preserved Generative …

ClosNets: a Priori Sparse Topologies for Faster DNN Training


Title	ClosNets: a Priori Sparse Topologies for Faster DNN Training
Authors	Mihailo Isakov, Michel A. Kinsy
Abstract	Fully-connected layers in deep neural networks (DNN) are often the throughput and power bottleneck during training. This is due to their large size and low data reuse. Pruning dense layers can significantly reduce the size of these networks, but this approach can only be applied after training. In this work we propose a novel fully-connected layer that reduces the memory requirements of DNNs without sacrificing accuracy. We replace a dense matrix with products of sparse matrices whose topologies we pick in advance. This allows us to: (1) train significantly smaller networks without a loss in accuracy, and (2) store the network weights without having to store connection indices. We therefore achieve significant training speedups due to the smaller network size, and a reduced amount of computation per epoch. We tested several sparse layer topologies and found that Clos networks perform well due to their high path diversity, shallowness, and high model accuracy. With the ClosNets, we are able to reduce dense layer sizes by as much as an order of magnitude without hurting model accuracy.
Tasks
Published	2018-02-12
URL	http://arxiv.org/abs/1802.03885v1
PDF	http://arxiv.org/pdf/1802.03885v1.pdf
PWC	https://paperswithcode.com/paper/closnets-a-priori-sparse-topologies-for
Repo
Framework

Superconducting Optoelectronic Neurons II: Receiver Circuits


Title	Superconducting Optoelectronic Neurons II: Receiver Circuits
Authors	Jeffrey M. Shainline, Sonia M. Buckley, Adam N. McCaughan, Manuel Castellanos-Beltran, Christine A. Donnelly, Michael L. Schneider, Richard P. Mirin, Sae Woo Nam
Abstract	Circuits using superconducting single-photon detectors and Josephson junctions to perform signal reception, synaptic weighting, and integration are investigated. The circuits convert photon-detection events into flux quanta, the number of which is determined by the synaptic weight. The current from many synaptic connections is inductively coupled to a superconducting loop that implements the neuronal threshold operation. Designs are presented for synapses and neurons that perform integration as well as detect coincidence events for temporal coding. Both excitatory and inhibitory connections are demonstrated. It is shown that a neuron with a single integration loop can receive input from 1000 such synaptic connections, and neurons of similar design could employ many loops for dendritic processing.
Tasks
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02599v3
PDF	http://arxiv.org/pdf/1805.02599v3.pdf
PWC	https://paperswithcode.com/paper/superconducting-optoelectronic-neurons-ii
Repo
Framework

The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion


Title	The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion
Authors	Michael Schmitt, Lloyd Haydn Hughes, Xiao Xiang Zhu
Abstract	While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing. In particular, this holds for applications involving data from multiple sensors with heterogeneous characteristics. One example for that is the fusion of synthetic aperture radar (SAR) data and optical imagery. With this paper, we publish the SEN1-2 dataset to foster deep learning research in SAR-optical data fusion. SEN1-2 comprises 282,384 pairs of corresponding image patches, collected from across the globe and throughout all meteorological seasons. Besides a detailed description of the dataset, we show exemplary results for several possible applications, such as SAR image colorization, SAR-optical image matching, and creation of artificial optical images from SAR input data. Since SEN1-2 is the first large open dataset of this kind, we believe it will support further developments in the field of deep learning for remote sensing as well as multi-sensor data fusion.
Tasks	Colorization
Published	2018-07-04
URL	http://arxiv.org/abs/1807.01569v1
PDF	http://arxiv.org/pdf/1807.01569v1.pdf
PWC	https://paperswithcode.com/paper/the-sen1-2-dataset-for-deep-learning-in-sar
Repo
Framework

A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration


Title	A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration
Authors	Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Hessam Sokooti, Marius Staring, Ivana Isgum
Abstract	Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for \textit{unsupervised} affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.
Tasks	Image Registration
Published	2018-09-17
URL	http://arxiv.org/abs/1809.06130v2
PDF	http://arxiv.org/pdf/1809.06130v2.pdf
PWC	https://paperswithcode.com/paper/a-deep-learning-framework-for-unsupervised
Repo
Framework

Cross-dataset Person Re-Identification Using Similarity Preserved Generative Adversarial Networks


Title	Cross-dataset Person Re-Identification Using Similarity Preserved Generative Adversarial Networks
Authors	Jianming Lv, Xintong Wang
Abstract	Person re-identification (Re-ID) aims to match the image frames which contain the same person in the surveillance videos. Most of the Re-ID algorithms conduct supervised training in some small labeled datasets, so directly deploying these trained models to the real-world large camera networks may lead to a poor performance due to underfitting. The significant difference between the source training dataset and the target testing dataset makes it challenging to incrementally optimize the model. To address this challenge, we propose a novel solution by transforming the unlabeled images in the target domain to fit the original classifier by using our proposed similarity preserved generative adversarial networks model, SimPGAN. Specifically, SimPGAN adopts the generative adversarial networks with the cycle consistency constraint to transform the unlabeled images in the target domain to the style of the source domain. Meanwhile, SimPGAN uses the similarity consistency loss, which is measured by a siamese deep convolutional neural network, to preserve the similarity of the transformed images of the same person. Comprehensive experiments based on multiple real surveillance datasets are conducted, and the results show that our algorithm is better than the state-of-the-art cross-dataset unsupervised person Re-ID algorithms.
Tasks	Person Re-Identification
Published	2018-06-11
URL	http://arxiv.org/abs/1806.04533v2
PDF	http://arxiv.org/pdf/1806.04533v2.pdf
PWC	https://paperswithcode.com/paper/cross-dataset-person-re-identification-using
Repo
Framework

Adversarial Domain Adaptation for Stable Brain-Machine Interfaces


Title	Adversarial Domain Adaptation for Stable Brain-Machine Interfaces
Authors	Ali Farshchian, Juan A. Gallego, Joseph P. Cohen, Yoshua Bengio, Lee E. Miller, Sara A. Solla
Abstract	Brain-Machine Interfaces (BMIs) have recently emerged as a clinically viable option to restore voluntary movements after paralysis. These devices are based on the ability to extract information about movement intent from neural signals recorded using multi-electrode arrays chronically implanted in the motor cortices of the brain. However, the inherent loss and turnover of recorded neurons requires repeated recalibrations of the interface, which can potentially alter the day-to-day user experience. The resulting need for continued user adaptation interferes with the natural, subconscious use of the BMI. Here, we introduce a new computational approach that decodes movement intent from a low-dimensional latent representation of the neural data. We implement various domain adaptation methods to stabilize the interface over significantly long times. This includes Canonical Correlation Analysis used to align the latent variables across days; this method requires prior point-to-point correspondence of the time series across domains. Alternatively, we match the empirical probability distributions of the latent variables across days through the minimization of their Kullback-Leibler divergence. These two methods provide a significant and comparable improvement in the performance of the interface. However, implementation of an Adversarial Domain Adaptation Network trained to match the empirical probability distribution of the residuals of the reconstructed neural signals outperforms the two methods based on latent variables, while requiring remarkably few data points to solve the domain adaptation problem.
Tasks	Domain Adaptation, Time Series
Published	2018-09-28
URL	http://arxiv.org/abs/1810.00045v2
PDF	http://arxiv.org/pdf/1810.00045v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-domain-adaptation-for-stable
Repo
Framework

Fast Weight Long Short-Term Memory


Title	Fast Weight Long Short-Term Memory
Authors	T. Anderson Keller, Sharath Nittur Sridhar, Xin Wang
Abstract	Associative memory using fast weights is a short-term memory mechanism that substantially improves the memory capacity and time scale of recurrent neural networks (RNNs). As recent studies introduced fast weights only to regular RNNs, it is unknown whether fast weight memory is beneficial to gated RNNs. In this work, we report a significant synergy between long short-term memory (LSTM) networks and fast weight associative memories. We show that this combination, in learning associative retrieval tasks, results in much faster training and lower test error, a performance boost most prominent at high memory task difficulties.
Tasks
Published	2018-04-18
URL	http://arxiv.org/abs/1804.06511v1
PDF	http://arxiv.org/pdf/1804.06511v1.pdf
PWC	https://paperswithcode.com/paper/fast-weight-long-short-term-memory
Repo
Framework

DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing


Title	DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
Authors	Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Hongxu Chen, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, Simon See
Abstract	In company with the data explosion over the past decade, deep neural network (DNN) based software has experienced unprecedented leap and is becoming the key driving force of many novel industrial applications, including many safety-critical scenarios such as autonomous driving. Despite great success achieved in various human intelligence tasks, similar to traditional software, DNNs could also exhibit incorrect behaviors caused by hidden defects causing severe accidents and losses. In this paper, we propose DeepHunter, an automated fuzz testing framework for hunting potential defects of general-purpose DNNs. DeepHunter performs metamorphic mutation to generate new semantically preserved tests, and leverages multiple plugable coverage criteria as feedback to guide the test generation from different perspectives. To be scalable towards practical-sized DNNs, DeepHunter maintains multiple tests in a batch, and prioritizes the tests selection based on active feedback. The effectiveness of DeepHunter is extensively investigated on 3 popular datasets (MNIST, CIFAR-10, ImageNet) and 7 DNNs with diverse complexities, under a large set of 6 coverage criteria as feedback. The large-scale experiments demonstrate that DeepHunter can (1) significantly boost the coverage with guidance; (2) generate useful tests to detect erroneous behaviors and facilitate the DNN model quality evaluation; (3) accurately capture potential defects during DNN quantization for platform migration.
Tasks	Autonomous Driving, Quantization
Published	2018-09-04
URL	http://arxiv.org/abs/1809.01266v3
PDF	http://arxiv.org/pdf/1809.01266v3.pdf
PWC	https://paperswithcode.com/paper/deephunter-hunting-deep-neural-network
Repo
Framework

Evaluating neural network explanation methods using hybrid documents and morphological agreement


Title	Evaluating neural network explanation methods using hybrid documents and morphological agreement
Authors	Nina Poerner, Benjamin Roth, Hinrich Schütze
Abstract	The behavior of deep neural networks (DNNs) is hard to understand. This makes it necessary to explore post hoc explanation methods. We conduct the first comprehensive evaluation of explanation methods for NLP. To this end, we design two novel evaluation paradigms that cover two important classes of NLP problems: small context and large context problems. Both paradigms require no manual annotation and are therefore broadly applicable. We also introduce LIMSSE, an explanation method inspired by LIME that is designed for NLP. We show empirically that LIMSSE, LRP and DeepLIFT are the most effective explanation methods and recommend them for explaining DNNs in NLP.
Tasks
Published	2018-01-19
URL	https://arxiv.org/abs/1801.06422v3
PDF	https://arxiv.org/pdf/1801.06422v3.pdf
PWC	https://paperswithcode.com/paper/evaluating-neural-network-explanation-methods
Repo
Framework

How To Solve Moral Conundrums with Computability Theory


Title	How To Solve Moral Conundrums with Computability Theory
Authors	Jongmin Jerome Baek
Abstract	Various moral conundrums plague population ethics: The Non-Identity Problem, The Procreation Asymmetry, The Repugnant Conclusion, and more. I argue that the aforementioned moral conundrums have a structure neatly accounted for, and solved by, some ideas in computability theory. I introduce a mathematical model based on computability theory and show how previous arguments pertaining to these conundrums fit into the model. This paper proceeds as follows. First, I do a very brief survey of the history of computability theory in moral philosophy. Second, I follow various papers, and show how their arguments fit into, or don’t fit into, our model. Third, I discuss the implications of our model to the question why the human race should or should not continue to exist. Finally, I show that our model ineluctably leads us to a Confucian moral principle.
Tasks
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08347v1
PDF	http://arxiv.org/pdf/1805.08347v1.pdf
PWC	https://paperswithcode.com/paper/how-to-solve-moral-conundrums-with
Repo
Framework

Cross Domain Image Generation through Latent Space Exploration with Adversarial Loss


Title	Cross Domain Image Generation through Latent Space Exploration with Adversarial Loss
Authors	Yingjing Lu
Abstract	Conditional domain generation is a good way to interactively control sample generation process of deep generative models. However, once a conditional generative model has been created, it is often expensive to allow it to adapt to new conditional controls, especially the network structure is relatively deep. We propose a conditioned latent domain transfer framework across latent spaces of unconditional variational autoencoders(VAE). With this framework, we can allow unconditionally trained VAEs to generate images in its domain with conditionals provided by a latent representation of another domain. This framework does not assume commonalities between two domains. We demonstrate effectiveness and robustness of our model under widely used image datasets.
Tasks	Image Generation
Published	2018-05-24
URL	http://arxiv.org/abs/1805.10130v1
PDF	http://arxiv.org/pdf/1805.10130v1.pdf
PWC	https://paperswithcode.com/paper/cross-domain-image-generation-through-latent
Repo
Framework

Modular Verification of Vehicle Platooning with Respect to Decisions, Space and Time


Title	Modular Verification of Vehicle Platooning with Respect to Decisions, Space and Time
Authors	Maryam Kamali, Sven Linker, Michael Fisher
Abstract	The spread of autonomous systems into safety-critical areas has increased the demand for their formal verification, not only due to stronger certification requirements but also to public uncertainty over these new technologies. However, the complex nature of such systems, for example, the intricate combination of discrete and continuous aspects, ensures that whole system verification is often infeasible. This motivates the need for novel analysis approaches that modularise the problem, allowing us to restrict our analysis to one particular aspect of the system while abstracting away from others. For instance, while verifying the real-time properties of an autonomous system we might hide the details of the internal decision-making components. In this paper we describe verification of a range of properties across distinct dimesnions on a practical hybrid agent architecture. This allows us to verify the autonomous decision-making, real-time aspects, and spatial aspects of an autonomous vehicle platooning system. This modular approach also illustrates how both algorithmic and deductive verification techniques can be applied for the analysis of different system subcomponents.
Tasks	Decision Making
Published	2018-04-18
URL	http://arxiv.org/abs/1804.06647v1
PDF	http://arxiv.org/pdf/1804.06647v1.pdf
PWC	https://paperswithcode.com/paper/modular-verification-of-vehicle-platooning
Repo
Framework

A Simple Convolutional Generative Network for Next Item Recommendation


Title	A Simple Convolutional Generative Network for Next Item Recommendation
Authors	Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, Xiangnan He
Abstract	Convolutional Neural Networks (CNNs) have been recently introduced in the domain of session-based next item recommendation. An ordered collection of past items the user has interacted with in a session (or sequence) are embedded into a 2-dimensional latent matrix, and treated as an image. The convolution and pooling operations are then applied to the mapped item embeddings. In this paper, we first examine the typical session-based CNN recommender and show that both the generative model and network architecture are suboptimal when modeling long-range dependencies in the item sequence. To address the issues, we introduce a simple, but very effective generative model that is capable of learning high-level representation from both short- and long-range item dependencies. The network architecture of the proposed model is formed of a stack of \emph{holed} convolutional layers, which can efficiently increase the receptive fields without relying on the pooling operation. Another contribution is the effective use of residual block structure in recommender systems, which can ease the optimization for much deeper networks. The proposed generative model attains state-of-the-art accuracy with less training time in the next item recommendation task. It accordingly can be used as a powerful recommendation baseline to beat in future, especially when there are long sequences of user feedback.
Tasks	Recommendation Systems
Published	2018-08-15
URL	http://arxiv.org/abs/1808.05163v4
PDF	http://arxiv.org/pdf/1808.05163v4.pdf
PWC	https://paperswithcode.com/paper/a-simple-convolutional-generative-network-for
Repo
Framework

Image Generation and Translation with Disentangled Representations


Title	Image Generation and Translation with Disentangled Representations
Authors	Tobias Hinz, Stefan Wermter
Abstract	Generative models have made significant progress in the tasks of modeling complex data distributions such as natural images. The introduction of Generative Adversarial Networks (GANs) and auto-encoders lead to the possibility of training on big data sets in an unsupervised manner. However, for many generative models it is not possible to specify what kind of image should be generated and it is not possible to translate existing images into new images of similar domains. Furthermore, models that can perform image-to-image translation often need distinct models for each domain, making it hard to scale these systems to multiple domain image-to-image translation. We introduce a model that can do both, controllable image generation and image-to-image translation between multiple domains. We split our image representation into two parts encoding unstructured and structured information respectively. The latter is designed in a disentangled manner, so that different parts encode different image characteristics. We train an encoder to encode images into these representations and use a small amount of labeled data to specify what kind of information should be encoded in the disentangled part. A generator is trained to generate images from these representations using the characteristics provided by the disentangled part of the representation. Through this we can control what kind of images the generator generates, translate images between different domains, and even learn unknown data-generating factors while only using one single model.
Tasks	Conditional Image Generation, Face Generation, Image Generation, Image-to-Image Translation, Representation Learning
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10567v1
PDF	http://arxiv.org/pdf/1803.10567v1.pdf
PWC	https://paperswithcode.com/paper/image-generation-and-translation-with
Repo
Framework

Deep Embedding using Bayesian Risk Minimization with Application to Sketch Recognition


Title	Deep Embedding using Bayesian Risk Minimization with Application to Sketch Recognition
Authors	Anand Mishra, Ajeet Kumar Singh
Abstract	In this paper, we address the problem of hand-drawn sketch recognition. Inspired by the Bayesian decision theory, we present a deep metric learning loss with the objective to minimize the Bayesian risk of misclassification. We estimate this risk for every mini-batch during training, and learn robust deep embeddings by backpropagating it to a deep neural network in an end-to-end trainable paradigm. Our learnt embeddings are discriminative and robust despite of intra-class variations and inter-class similarities naturally present in hand-drawn sketch images. Outperforming the state of the art on sketch recognition, our method achieves 82.2% and 88.7% on TU-Berlin-250 and TU-Berlin-160 benchmarks respectively.
Tasks	Metric Learning, Sketch Recognition
Published	2018-12-06
URL	http://arxiv.org/abs/1812.02466v1
PDF	http://arxiv.org/pdf/1812.02466v1.pdf
PWC	https://paperswithcode.com/paper/deep-embedding-using-bayesian-risk
Repo
Framework