October 18, 2019

3008 words 15 mins read

Paper Group ANR 594

Accelerated physical emulation of Bayesian inference in spiking neural networks. Are Generative Classifiers More Robust to Adversarial Attacks?. Neural Latent Extractive Document Summarization. Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks. Convolutional neural network-based regression for depth prediction in digi …

Accelerated physical emulation of Bayesian inference in spiking neural networks


Title	Accelerated physical emulation of Bayesian inference in spiking neural networks
Authors	Akos F. Kungl, Sebastian Schmitt, Johann Klähn, Paul Müller, Andreas Baumbach, Dominik Dold, Alexander Kugele, Nico Gürtler, Luziwei Leng, Eric Müller, Christoph Koke, Mitja Kleider, Christian Mauch, Oliver Breitwieser, Maurice Güttler, Dan Husmann, Kai Husmann, Joscha Ilmberger, Andreas Hartel, Vitali Karasenko, Andreas Grübl, Johannes Schemmel, Karlheinz Meier, Mihai A. Petrovici
Abstract	The massively parallel nature of biological information processing plays an important role for its superiority to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.
Tasks	Bayesian Inference
Published	2018-07-06
URL	https://arxiv.org/abs/1807.02389v4
PDF	https://arxiv.org/pdf/1807.02389v4.pdf
PWC	https://paperswithcode.com/paper/generative-models-on-accelerated-neuromorphic
Repo
Framework

Are Generative Classifiers More Robust to Adversarial Attacks?


Title	Are Generative Classifiers More Robust to Adversarial Attacks?
Authors	Yingzhen Li, John Bradshaw, Yash Sharma
Abstract	There is a rising interest in studying the robustness of deep neural network classifiers against adversaries, with both advanced attack and defence techniques being actively developed. However, most recent work focuses on discriminative classifiers, which only model the conditional distribution of the labels given the inputs. In this paper, we propose and investigate the deep Bayes classifier, which improves classical naive Bayes with conditional deep generative models. We further develop detection methods for adversarial examples, which reject inputs with low likelihood under the generative model. Experimental results suggest that deep Bayes classifiers are more robust than deep discriminative classifiers, and that the proposed detection methods are effective against many recently proposed attacks.
Tasks
Published	2018-02-19
URL	https://arxiv.org/abs/1802.06552v3
PDF	https://arxiv.org/pdf/1802.06552v3.pdf
PWC	https://paperswithcode.com/paper/are-generative-classifiers-more-robust-to
Repo
Framework

Neural Latent Extractive Document Summarization


Title	Neural Latent Extractive Document Summarization
Authors	Xingxing Zhang, Mirella Lapata, Furu Wei, Ming Zhou
Abstract	Extractive summarization models require sentence-level labels, which are usually created heuristically (e.g., with rule-based methods) given that most summarization datasets only have document-summary pairs. Since these labels might be suboptimal, we propose a latent variable extractive model where sentences are viewed as latent variables and sentences with activated variables are used to infer gold summaries. During training the loss comes \emph{directly} from gold summaries. Experiments on the CNN/Dailymail dataset show that our model improves over a strong extractive baseline trained on heuristically approximated labels and also performs competitively to several recent models.
Tasks	Document Summarization, Extractive Document Summarization
Published	2018-08-22
URL	http://arxiv.org/abs/1808.07187v2
PDF	http://arxiv.org/pdf/1808.07187v2.pdf
PWC	https://paperswithcode.com/paper/neural-latent-extractive-document
Repo
Framework

Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks


Title	Freeform Diffractive Metagrating Design Based on Generative Adversarial Networks
Authors	Jiaqi Jiang, David Sell, Stephan Hoyer, Jason Hickey, Jianji Yang, Jonathan A. Fan
Abstract	A key challenge in metasurface design is the development of algorithms that can effectively and efficiently produce high performance devices. Design methods based on iterative optimization can push the performance limits of metasurfaces, but they require extensive computational resources that limit their implementation to small numbers of microscale devices. We show that generative neural networks can train from images of periodic, topology-optimized metagratings to produce high-efficiency, topologically complex devices operating over a broad range of deflection angles and wavelengths. Further iterative optimization of these designs yields devices with enhanced robustness and efficiencies, and these devices can be utilized as additional training data for network refinement. In this manner, generative networks can be trained, with a onetime computation cost, and used as a design tool to facilitate the production of near-optimal, topologically-complex device designs. We envision that such data-driven design methodologies can apply to other physical sciences domains that require the design of functional elements operating across a wide parameter space.
Tasks
Published	2018-11-29
URL	https://arxiv.org/abs/1811.12436v2
PDF	https://arxiv.org/pdf/1811.12436v2.pdf
PWC	https://paperswithcode.com/paper/data-driven-metasurface-discovery
Repo
Framework

Convolutional neural network-based regression for depth prediction in digital holography


Title	Convolutional neural network-based regression for depth prediction in digital holography
Authors	Tomoyoshi Shimobaba, Takashi Kakue, Tomoyoshi Ito
Abstract	Digital holography enables us to reconstruct objects in three-dimensional space from holograms captured by an imaging device. For the reconstruction, we need to know the depth position of the recoded object in advance. In this study, we propose depth prediction using convolutional neural network (CNN)-based regression. In the previous researches, the depth of an object was estimated through reconstructed images at different depth positions from a hologram using a certain metric that indicates the most focused depth position; however, such a depth search is time-consuming. The CNN of the proposed method can directly predict the depth position with millimeter precision from holograms.
Tasks	Depth Estimation
Published	2018-02-02
URL	http://arxiv.org/abs/1802.00664v1
PDF	http://arxiv.org/pdf/1802.00664v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-neural-network-based-regression
Repo
Framework

Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization


Title	Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization
Authors	Gang Wang, Georgios B. Giannakis, Jie Chen
Abstract	Neural networks with REctified Linear Unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being e.g. Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can \emph{provably} train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory.
Tasks
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04685v2
PDF	http://arxiv.org/pdf/1808.04685v2.pdf
PWC	https://paperswithcode.com/paper/learning-relu-networks-on-linearly-separable
Repo
Framework

Generalized Zero-Shot Recognition based on Visually Semantic Embedding


Title	Generalized Zero-Shot Recognition based on Visually Semantic Embedding
Authors	Pengkai Zhu, Hanxiao Wang, Venkatesh Saligrama
Abstract	We propose a novel Generalized Zero-Shot learning (GZSL) method that is agnostic to both unseen images and unseen semantic vectors during training. Prior works in this context propose to map high-dimensional visual features to the semantic domain, we believe contributes to the semantic gap. To bridge the gap, we propose a novel low-dimensional embedding of visual instances that is “visually semantic.” Analogous to semantic data that quantifies the existence of an attribute in the presented instance, components of our visual embedding quantifies existence of a prototypical part-type in the presented instance. In parallel, as a thought experiment, we quantify the impact of noisy semantic data by utilizing a novel visual oracle to visually supervise a learner. These factors, namely semantic noise, visual-semantic gap and label noise lead us to propose a new graphical model for inference with pairwise interactions between label, semantic data, and inputs. We tabulate results on a number of benchmark datasets demonstrating significant improvement in accuracy over state-of-the-art under both semantic and visual supervision.
Tasks	Zero-Shot Learning
Published	2018-11-19
URL	http://arxiv.org/abs/1811.07993v2
PDF	http://arxiv.org/pdf/1811.07993v2.pdf
PWC	https://paperswithcode.com/paper/generalized-zero-shot-recognition-based-on
Repo
Framework

Realizing quantum linear regression with auxiliary qumodes


Title	Realizing quantum linear regression with auxiliary qumodes
Authors	Dan-Bo Zhang, Zheng-Yuan Xue, Shi-Liang Zhu, Z. D. Wang
Abstract	In order to exploit quantum advantages, quantum algorithms are indispensable for operating machine learning with quantum computers. We here propose an intriguing hybrid approach of quantum information processing for quantum linear regression, which utilizes both discrete and continuous quantum variables, in contrast to existing wisdoms based solely upon discrete qubits. In our framework, data information is encoded in a qubit system, while information processing is tackled using auxiliary continuous qumodes via qubit-qumode interactions. Moreover, it is also elaborated that finite squeezing is quite helpful for efficiently running the quantum algorithms in realistic setup. Comparing with an all-qubit approach, the present hybrid approach is more efficient and feasible for implementing quantum algorithms, still retaining exponential quantum speed-up.
Tasks
Published	2018-08-27
URL	http://arxiv.org/abs/1808.08888v2
PDF	http://arxiv.org/pdf/1808.08888v2.pdf
PWC	https://paperswithcode.com/paper/realizing-quantum-linear-regression-with
Repo
Framework

Real-time image-based instrument classification for laparoscopic surgery


Title	Real-time image-based instrument classification for laparoscopic surgery
Authors	Sebastian Bodenstedt, Antonia Ohnemus, Darko Katic, Anna-Laura Wekerle, Martin Wagner, Hannes Kenngott, Beat Müller-Stich, Rüdiger Dillmann, Stefanie Speidel
Abstract	During laparoscopic surgery, context-aware assistance systems aim to alleviate some of the difficulties the surgeon faces. To ensure that the right information is provided at the right time, the current phase of the intervention has to be known. Real-time locating and classification the surgical tools currently in use are key components of both an activity-based phase recognition and assistance generation. In this paper, we present an image-based approach that detects and classifies tools during laparoscopic interventions in real-time. First, potential instrument bounding boxes are detected using a pixel-wise random forest segmentation. Each of these bounding boxes is then classified using a cascade of random forest. For this, multiple features, such as histograms over hue and saturation, gradients and SURF feature, are extracted from each detected bounding box. We evaluated our approach on five different videos from two different types of procedures. We distinguished between the four most common classes of instruments (LigaSure, atraumatic grasper, aspirator, clip applier) and background. Our method succesfully located up to 86% of all instruments respectively. On manually provided bounding boxes, we achieve a instrument type recognition rate of up to 58% and on automatically detected bounding boxes up to 49%. To our knowledge, this is the first approach that allows an image-based classification of surgical tools in a laparoscopic setting in real-time.
Tasks
Published	2018-08-01
URL	http://arxiv.org/abs/1808.00178v1
PDF	http://arxiv.org/pdf/1808.00178v1.pdf
PWC	https://paperswithcode.com/paper/real-time-image-based-instrument
Repo
Framework

A Novel Variational Autoencoder with Applications to Generative Modelling, Classification, and Ordinal Regression


Title	A Novel Variational Autoencoder with Applications to Generative Modelling, Classification, and Ordinal Regression
Authors	Joel Jaskari, Jyri J. Kivinen
Abstract	We develop a novel probabilistic generative model based on the variational autoencoder approach. Notable aspects of our architecture are: a novel way of specifying the latent variables prior, and the introduction of an ordinality enforcing unit. We describe how to do supervised, unsupervised and semi-supervised learning, and nominal and ordinal classification, with the model. We analyze generative properties of the approach, and the classification effectiveness under nominal and ordinal classification, using two benchmark datasets. Our results show that our model can achieve comparable results with relevant baselines in both of the classification tasks.
Tasks
Published	2018-12-18
URL	http://arxiv.org/abs/1812.07352v2
PDF	http://arxiv.org/pdf/1812.07352v2.pdf
PWC	https://paperswithcode.com/paper/a-novel-variational-autoencoder-with
Repo
Framework

Decoupled Learning for Conditional Adversarial Networks


Title	Decoupled Learning for Conditional Adversarial Networks
Authors	Zhifei Zhang, Yang Song, Hairong Qi
Abstract	Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks. We observe that the state-of-the-art achievements were obtained by carefully balancing the reconstruction loss and adversarial loss, and such balance shifts with different network structures, datasets, and training strategies. Empirical studies have demonstrated that an inappropriate weight between the two losses may cause instability, and it is tricky to search for the optimal setting, especially when lacking prior knowledge on the data and network. This paper gives the first attempt to relax the need of manual balancing by proposing the concept of \textit{decoupled learning}, where a novel network structure is designed that explicitly disentangles the backpropagation paths of the two losses. Experimental results demonstrate the effectiveness, robustness, and generality of the proposed method. The other contribution of the paper is the design of a new evaluation metric to measure the image quality of generative models. We propose the so-called \textit{normalized relative discriminative score} (NRDS), which introduces the idea of relative comparison, rather than providing absolute estimates like existing metrics.
Tasks	Image Generation
Published	2018-01-21
URL	http://arxiv.org/abs/1801.06790v1
PDF	http://arxiv.org/pdf/1801.06790v1.pdf
PWC	https://paperswithcode.com/paper/decoupled-learning-for-conditional
Repo
Framework

Tensor N-tubal rank and its convex relaxation for low-rank tensor recovery


Title	Tensor N-tubal rank and its convex relaxation for low-rank tensor recovery
Authors	Yu-Bang Zheng, Ting-Zhu Huang, Xi-Le Zhao, Tai-Xiang Jiang, Teng-Yu Ji, Tian-Hui Ma
Abstract	As low-rank modeling has achieved great success in tensor recovery, many research efforts devote to defining the tensor rank. Among them, the recent popular tensor tubal rank, defined based on the tensor singular value decomposition (t-SVD), obtains promising results. However, the framework of the t-SVD and the tensor tubal rank are applicable only to three-way tensors and lack of flexibility to handle different correlations along different modes. To tackle these two issues, we define a new tensor unfolding operator, named mode-$k_1k_2$ tensor unfolding, as the process of lexicographically stacking the mode-$k_1k_2$ slices of an $N$-way tensor into a three-way tensor, which is a three-way extension of the well-known mode-$k$ tensor matricization. Based on it, we define a novel tensor rank, the tensor $N$-tubal rank, as a vector whose elements contain the tubal rank of all mode-$k_1k_2$ unfolding tensors, to depict the correlations along different modes. To efficiently minimize the proposed $N$-tubal rank, we establish its convex relaxation: the weighted sum of tensor nuclear norm (WSTNN). Then, we apply WSTNN to low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA). The corresponding WSTNN-based LRTC and TRPCA models are proposed, and two efficient alternating direction method of multipliers (ADMM)-based algorithms are developed to solve the proposed models. Numerical experiments demonstrate that the proposed models significantly outperform the compared ones.
Tasks
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00688v1
PDF	http://arxiv.org/pdf/1812.00688v1.pdf
PWC	https://paperswithcode.com/paper/tensor-n-tubal-rank-and-its-convex-relaxation
Repo
Framework

prDeep: Robust Phase Retrieval with a Flexible Deep Network


Title	prDeep: Robust Phase Retrieval with a Flexible Deep Network
Authors	Christopher A. Metzler, Philip Schniter, Ashok Veeraraghavan, Richard G. Baraniuk
Abstract	Phase retrieval algorithms have become an important component in many modern computational imaging systems. For instance, in the context of ptychography and speckle correlation imaging, they enable imaging past the diffraction limit and through scattering media, respectively. Unfortunately, traditional phase retrieval algorithms struggle in the presence of noise. Progress has been made recently on more robust algorithms using signal priors, but at the expense of limiting the range of supported measurement models (e.g., to Gaussian or coded diffraction patterns). In this work we leverage the regularization-by-denoising framework and a convolutional neural network denoiser to create prDeep, a new phase retrieval algorithm that is both robust and broadly applicable. We test and validate prDeep in simulation to demonstrate that it is robust to noise and can handle a variety of system models. A MatConvNet implementation of prDeep is available at https://github.com/ricedsp/prDeep.
Tasks	Denoising
Published	2018-03-01
URL	http://arxiv.org/abs/1803.00212v2
PDF	http://arxiv.org/pdf/1803.00212v2.pdf
PWC	https://paperswithcode.com/paper/prdeep-robust-phase-retrieval-with-a-flexible
Repo
Framework

Comparative Analysis of Neural QA models on SQuAD


Title	Comparative Analysis of Neural QA models on SQuAD
Authors	Soumya Wadhwa, Khyathi Raghavi Chandu, Eric Nyberg
Abstract	The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language. Large datasets for Machine Reading have led to the development of neural models that cater to deeper language understanding compared to information retrieval tasks. Different components in these neural architectures are intended to tackle different challenges. As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering Dataset (SQuAD) by performing quantitative as well as qualitative analysis of the results attained by each of them. We observed that prediction errors reflect certain model-specific biases, which we further discuss in this paper.
Tasks	Information Retrieval, Question Answering, Reading Comprehension
Published	2018-06-18
URL	http://arxiv.org/abs/1806.06972v1
PDF	http://arxiv.org/pdf/1806.06972v1.pdf
PWC	https://paperswithcode.com/paper/comparative-analysis-of-neural-qa-models-on
Repo
Framework

SafeRNet: Safe Transportation Routing in the era of Internet of Vehicles and Mobile Crowd Sensing


Title	SafeRNet: Safe Transportation Routing in the era of Internet of Vehicles and Mobile Crowd Sensing
Authors	Qun Liu, Suman Kumar, Vijay Mago
Abstract	World wide road traffic fatality and accident rates are high, and this is true even in technologically advanced countries like the USA. Despite the advances in Intelligent Transportation Systems, safe transportation routing i.e., finding safest routes is largely an overlooked paradigm. In recent years, large amount of traffic data has been produced by people, Internet of Vehicles and Internet of Things (IoT). Also, thanks to advances in cloud computing and proliferation of mobile communication technologies, it is now possible to perform analysis on vast amount of generated data (crowd sourced) and deliver the result back to users in real time. This paper proposes SafeRNet, a safe route computation framework which takes advantage of these technologies to analyze streaming traffic data and historical data to effectively infer safe routes and deliver them back to users in real time. SafeRNet utilizes Bayesian network to formulate safe route model. Furthermore, a case study is presented to demonstrate the effectiveness of our approach using real traffic data. SafeRNet intends to improve drivers safety in a modern technology rich transportation system.
Tasks
Published	2018-05-03
URL	http://arxiv.org/abs/1805.01162v1
PDF	http://arxiv.org/pdf/1805.01162v1.pdf
PWC	https://paperswithcode.com/paper/safernet-safe-transportation-routing-in-the
Repo
Framework