April 1, 2020

3087 words 15 mins read

Paper Group ANR 496

Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization. Efficient Bitwidth Search for Practical Mixed Precision Neural Network. Trajectory Poisson multi-Bernoulli filters. Obliviousness Makes Poisoning Adversaries Weaker. Conversations with Documents. An Exploration of Document-Centered Assistance. Language …

Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization


Title	Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization
Authors	Meng Li, Yilei Li, Pierce Chuang, Liangzhen Lai, Vikas Chandra
Abstract	Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric. The data-path energy, including the computation energy and the data movement energy among the arithmetic units, claims a significant part of the total accelerator energy. By revisiting the basic physics of the arithmetic logic circuits, we show that the data-path energy is highly correlated with the bit flips when streaming the input operands into the arithmetic units, defined as the hamming distance of the input operand matrices. Based on the insight, we propose a post-training optimization algorithm and a hamming-distance-aware training algorithm to co-design and co-optimize the accelerator and the network synergistically. The experimental results based on post-layout simulation with MobileNetV2 demonstrate on average 2.85X data-path energy reduction and up to 8.51X data-path energy reduction for certain layers.
Tasks
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05293v1
PDF	https://arxiv.org/pdf/2002.05293v1.pdf
PWC	https://paperswithcode.com/paper/improving-efficiency-in-neural-network
Repo
Framework

Efficient Bitwidth Search for Practical Mixed Precision Neural Network


Title	Efficient Bitwidth Search for Practical Mixed Precision Neural Network
Authors	Yuhang Li, Wei Wang, Haoli Bai, Ruihao Gong, Xin Dong, Fengwei Yu
Abstract	Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activations from different layers with different precision to improve the overall performance. However, it is challenging to find the optimal bitwidth (i.e., precision) for weights and activations of each layer efficiently. Meanwhile, it is yet unclear how to perform convolution for weights and activations of different precision efficiently on generic hardware platforms. To resolve these two issues, in this paper, we first propose an Efficient Bitwidth Search (EBS) algorithm, which reuses the meta weights for different quantization bitwidth and thus the strength for each candidate precision can be optimized directly w.r.t the objective without superfluous copies, reducing both the memory and computational cost significantly. Second, we propose a binary decomposition algorithm that converts weights and activations of different precision into binary matrices to make the mixed precision convolution efficient and practical. Experiment results on CIFAR10 and ImageNet datasets demonstrate our mixed precision QNN outperforms the handcrafted uniform bitwidth counterparts and other mixed precision techniques.
Tasks	Quantization
Published	2020-03-17
URL	https://arxiv.org/abs/2003.07577v1
PDF	https://arxiv.org/pdf/2003.07577v1.pdf
PWC	https://paperswithcode.com/paper/efficient-bitwidth-search-for-practical-mixed
Repo
Framework

Trajectory Poisson multi-Bernoulli filters


Title	Trajectory Poisson multi-Bernoulli filters
Authors	Ángel F. García-Fernández, Lennart Svensson, Jason L. Williams, Yuxuan Xia, Karl Granström
Abstract	This paper presents two trajectory Poisson multi-Bernoulli (TPMB) filters for multi-target tracking: one to estimate the set of alive trajectories at each time step and another to estimate the set of all trajectories, which includes alive and dead trajectories, at each time step. The filters are based on propagating a Poisson multi-Bernoulli (PMB) density on the corresponding set of trajectories through the filtering recursion. After the update step, the posterior is a PMB mixture (PMBM) so, in order to obtain a PMB density, a Kullback-Leibler divergence minimisation on an augmented space is performed. The developed filters are computationally lighter alternatives to the trajectory PMBM filters, which provide the closed-form recursion for sets of trajectories with Poisson birth model, and are shown to outperform previous multi-target tracking algorithms.
Tasks
Published	2020-03-28
URL	https://arxiv.org/abs/2003.12767v1
PDF	https://arxiv.org/pdf/2003.12767v1.pdf
PWC	https://paperswithcode.com/paper/trajectory-poisson-multi-bernoulli-filters
Repo
Framework

Obliviousness Makes Poisoning Adversaries Weaker


Title	Obliviousness Makes Poisoning Adversaries Weaker
Authors	Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Abhradeep Thakurta
Abstract	Poisoning attacks have emerged as a significant security threat to machine learning (ML) algorithms. It has been demonstrated that adversaries who make small changes to the training set, such as adding specially crafted data points, can hurt the performance of the output model. Most of these attacks require the full knowledge of training data or the underlying data distribution. In this paper we study the power of oblivious adversaries who do not have any information about the training set. We show a separation between oblivious and full-information poisoning adversaries. Specifically, we construct a sparse linear regression problem for which LASSO estimator is robust against oblivious adversaries whose goal is to add a non-relevant features to the model with certain poisoning budget. On the other hand, non-oblivious adversaries, with the same budget, can craft poisoning examples based on the rest of the training data and successfully add non-relevant features to the model.
Tasks
Published	2020-03-26
URL	https://arxiv.org/abs/2003.12020v1
PDF	https://arxiv.org/pdf/2003.12020v1.pdf
PWC	https://paperswithcode.com/paper/obliviousness-makes-poisoning-adversaries
Repo
Framework

Conversations with Documents. An Exploration of Document-Centered Assistance


Title	Conversations with Documents. An Exploration of Document-Centered Assistance
Authors	Maartje ter Hoeve, Robert Sim, Elnaz Nouri, Adam Fourney, Maarten de Rijke, Ryen W. White
Abstract	The role of conversational assistants has become more prevalent in helping people increase their productivity. Document-centered assistance, for example to help an individual quickly review a document, has seen less significant progress, even though it has the potential to tremendously increase a user’s productivity. This type of document-centered assistance is the focus of this paper. Our contributions are three-fold: (1) We first present a survey to understand the space of document-centered assistance and the capabilities people expect in this scenario. (2) We investigate the types of queries that users will pose while seeking assistance with documents, and show that document-centered questions form the majority of these queries. (3) We present a set of initial machine learned models that show that (a) we can accurately detect document-centered questions, and (b) we can build reasonably accurate models for answering such questions. These positive results are encouraging, and suggest that even greater results may be attained with continued study of this interesting and novel problem space. Our findings have implications for the design of intelligent systems to support task completion via natural interactions with documents.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2002.00747v1
PDF	https://arxiv.org/pdf/2002.00747v1.pdf
PWC	https://paperswithcode.com/paper/conversations-with-documents-an-exploration
Repo
Framework

Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration


Title	Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
Authors	Cédric Colas, Tristan Karch, Nicolas Lair, Jean-Michel Dussoux, Clément Moulin-Frier, Peter Ford Dominey, Pierre-Yves Oudeyer
Abstract	Autonomous reinforcement learning agents must be intrinsically motivated to explore their environment, discover potential goals, represent them and learn how to achieve them. As children do the same, they benefit from exposure to language, using it to formulate goals and imagine new ones as they learn their meaning. In our proposed learning architecture (IMAGINE), the agent freely explores its environment and turns natural language descriptions of interesting interactions from a social partner into potential goals. IMAGINE learns to represent goals by jointly learning a language model and a goal-conditioned reward function. Just like humans, our agent uses language compositionality to generate new goals by composing known ones. Leveraging modular model architectures based on Deep Sets and gated-attention mechanisms, IMAGINE autonomously builds a repertoire of behaviors and shows good zero-shot generalization properties for various types of generalization. When imagining its own goals, the agent leverages zero-shot generalization of the reward function to further train on imagined goals and refine its behavior. We present experiments in a simulated domain where the agent interacts with procedurally generated scenes containing objects of various types and colors, discovers goals, imagines others and learns to achieve them.
Tasks	Language Modelling
Published	2020-02-21
URL	https://arxiv.org/abs/2002.09253v1
PDF	https://arxiv.org/pdf/2002.09253v1.pdf
PWC	https://paperswithcode.com/paper/language-as-a-cognitive-tool-to-imagine-goals
Repo
Framework

Binary Classification from Positive Data with Skewed Confidence


Title	Binary Classification from Positive Data with Skewed Confidence
Authors	Kazuhiko Shinoda, Hirotaka Kaji, Masashi Sugiyama
Abstract	Positive-confidence (Pconf) classification [Ishida et al., 2018] is a promising weakly-supervised learning method which trains a binary classifier only from positive data equipped with confidence. However, in practice, the confidence may be skewed by bias arising in an annotation process. The Pconf classifier cannot be properly learned with skewed confidence, and consequently, the classification performance might be deteriorated. In this paper, we introduce the parameterized model of the skewed confidence, and propose the method for selecting the hyperparameter which cancels out the negative impact of skewed confidence under the assumption that we have the misclassification rate of positive samples as a prior knowledge. We demonstrate the effectiveness of the proposed method through a synthetic experiment with simple linear models and benchmark problems with neural network models. We also apply our method to drivers’ drowsiness prediction to show that it works well with a real-world problem where confidence is obtained based on manual annotation.
Tasks
Published	2020-01-29
URL	https://arxiv.org/abs/2001.10642v1
PDF	https://arxiv.org/pdf/2001.10642v1.pdf
PWC	https://paperswithcode.com/paper/binary-classification-from-positive-data-with
Repo
Framework

Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems


Title	Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems
Authors	Filip Hanzely, Dmitry Kovalev, Peter Richtarik
Abstract	We propose an accelerated version of stochastic variance reduced coordinate descent – ASVRCD. As other variance reduced coordinate descent methods such as SEGA or SVRCD, our method can deal with problems that include a non-separable and non-smooth regularizer, while accessing a random block of partial derivatives in each iteration only. However, ASVRCD incorporates Nesterov’s momentum, which offers favorable iteration complexity guarantees over both SEGA and SVRCD. As a by-product of our theory, we show that a variant of Allen-Zhu (2017) is a specific case of ASVRCD, recovering the optimal oracle complexity for the finite sum objective.
Tasks
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04670v1
PDF	https://arxiv.org/pdf/2002.04670v1.pdf
PWC	https://paperswithcode.com/paper/variance-reduced-coordinate-descent-with
Repo
Framework

Single-shot autofocusing of microscopy images using deep learning


Title	Single-shot autofocusing of microscopy images using deep learning
Authors	Yilin Luo, Luzhe Huang, Yair Rivenson, Aydogan Ozcan
Abstract	We demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard online algorithmic autofocusing methods. This deep learning-based blind autofocusing framework opens up new opportunities for rapid microscopic imaging of large sample areas, also reducing the photon dose on the sample.
Tasks
Published	2020-03-21
URL	https://arxiv.org/abs/2003.09585v1
PDF	https://arxiv.org/pdf/2003.09585v1.pdf
PWC	https://paperswithcode.com/paper/single-shot-autofocusing-of-microscopy-images
Repo
Framework

Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients


Title	Improving Sampling Accuracy of Stochastic Gradient MCMC Methods via Non-uniform Subsampling of Gradients
Authors	Ruilin Li, Xin Wang, Hongyuan Zha, Molei Tao
Abstract	Common Stochastic Gradient MCMC methods approximate gradients by stochastic ones via uniformly subsampled data points. We propose that a non-uniform subsampling can reduce the variance introduced by the stochastic approximation, hence making the sampling of a target distribution more accurate. An exponentially weighted stochastic gradient approach (EWSG) is developed for this objective by matching the transition kernels of SG-MCMC methods respectively based on stochastic and batch gradients. A demonstration of EWSG combined with second-order Langevin equation for sampling purposes is provided. In our method, non-uniform subsampling is done efficiently via a Metropolis-Hasting chain on the data index, which is coupled to the sampling algorithm. The fact that our method has reduced local variance with high probability is theoretically analyzed. A non-asymptotic global error analysis is also presented. Numerical experiments based on both synthetic and real world data sets are also provided to demonstrate the efficacy of the proposed approaches. While statistical accuracy has improved, the speed of convergence was empirically observed to be at least comparable to the uniform version.
Tasks
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08949v1
PDF	https://arxiv.org/pdf/2002.08949v1.pdf
PWC	https://paperswithcode.com/paper/improving-sampling-accuracy-of-stochastic
Repo
Framework

Stable Neural Flows


Title	Stable Neural Flows
Authors	Stefano Massaroli, Michael Poli, Michelangelo Bin, Jinkyoo Park, Atsushi Yamashita, Hajime Asama
Abstract	We introduce a provably stable variant of neural ordinary differential equations (neural ODEs) whose trajectories evolve on an energy functional parametrised by a neural network. Stable neural flows provide an implicit guarantee on asymptotic stability of the depth-flows, leading to robustness against input perturbations and low computational burden for the numerical solver. The learning procedure is cast as an optimal control problem, and an approximate solution is proposed based on adjoint sensivity analysis. We further introduce novel regularizers designed to ease the optimization process and speed up convergence. The proposed model class is evaluated on non-linear classification and function approximation tasks.
Tasks
Published	2020-03-18
URL	https://arxiv.org/abs/2003.08063v1
PDF	https://arxiv.org/pdf/2003.08063v1.pdf
PWC	https://paperswithcode.com/paper/stable-neural-flows
Repo
Framework

Learning Speaker Embedding with Momentum Contrast


Title	Learning Speaker Embedding with Momentum Contrast
Authors	Ke Ding, Xuanji He, Guanglu Wan
Abstract	Speaker verification can be formulated as a representation learning task, where speaker-discriminative embeddings are extracted from utterances of variable lengths. Momentum Contrast (MoCo) is a recently proposed unsupervised representation learning framework, and has shown its effectiveness for learning good feature representation for downstream vision tasks. In this work, we apply MoCo to learn speaker embedding from speech segments. We explore MoCo for both unsupervised learning and pretraining settings. In the unsupervised scenario, embedding is learned by MoCo from audio data without using any speaker specific information. On a large scale dataset with $2,500$ speakers, MoCo can achieve EER $4.275%$ trained unsupervisedly, and the EER can decrease further to $3.58%$ if extra unlabelled data are used. In the pretraining scenario, encoder trained by MoCo is used to initialize the downstream supervised training. With finetuning on the MoCo trained model, the equal error rate (EER) reduces $13.7%$ relative ($1.44%$ to $1.242%$) compared to a carefully tuned baseline training from scratch. Comparative study confirms the effectiveness of MoCo learning good speaker embedding.
Tasks	Representation Learning, Speaker Verification, Unsupervised Representation Learning
Published	2020-01-07
URL	https://arxiv.org/abs/2001.01986v1
PDF	https://arxiv.org/pdf/2001.01986v1.pdf
PWC	https://paperswithcode.com/paper/learning-speaker-embedding-with-momentum
Repo
Framework

Automatic image-based identification and biomass estimation of invertebrates


Title	Automatic image-based identification and biomass estimation of invertebrates
Authors	Johanna Ärje, Claus Melvad, Mads Rosenhøj Jeppesen, Sigurd Agerskov Madsen, Jenni Raitoharju, Maria Strandgård Rasmussen, Alexandros Iosifidis, Ville Tirronen, Kristian Meissner, Moncef Gabbouj, Toke Thomas Høye
Abstract	Understanding how biological communities respond to environmental changes is a key challenge in ecology and ecosystem management. The apparent decline of insect populations necessitates more biomonitoring but the time-consuming sorting and identification of taxa pose strong limitations on how many insect samples can be processed. In turn, this affects the scale of efforts to map invertebrate diversity altogether. Given recent advances in computer vision, we propose to replace the standard manual approach of human expert-based sorting and identification with an automatic image-based technology. We describe a robot-enabled image-based identification machine, which can automate the process of invertebrate identification, biomass estimation and sample sorting. We use the imaging device to generate a comprehensive image database of terrestrial arthropod species. We use this database to test the classification accuracy i.e. how well the species identity of a specimen can be predicted from images taken by the machine. We also test sensitivity of the classification accuracy to the camera settings (aperture and exposure time) in order to move forward with the best possible image quality. We use state-of-the-art Resnet-50 and InceptionV3 CNNs for the classification task. The results for the initial dataset are very promising ($\overline{ACC}=0.980$). The system is general and can easily be used for other groups of invertebrates as well. As such, our results pave the way for generating more data on spatial and temporal variation in invertebrate abundance, diversity and biomass.
Tasks
Published	2020-02-05
URL	https://arxiv.org/abs/2002.03807v1
PDF	https://arxiv.org/pdf/2002.03807v1.pdf
PWC	https://paperswithcode.com/paper/automatic-image-based-identification-and
Repo
Framework

Research Directions for Developing and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems


Title	Research Directions for Developing and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems
Authors	Silverio Martínez-Fernández, Xavier Franch, Andreas Jedlitschka, Marc Oriol, Adam Trendowicz
Abstract	Context: Autonomous Systems (ASs) are becoming increasingly pervasive in today’s society. One reason lies in the emergence of sophisticated Artificial Intelligence (AI) solutions that boost the ability of ASs to self-adapt in increasingly complex and dynamic environments. Companies dealing with AI models in ASs face several problems, such as users’ lack of trust in adverse or unknown conditions, and gaps between systems engineering and AI model development and evolution in a continuously changing operational environment. Objective: This vision paper aims to close the gap between the development and operation of trustworthy AI-based ASs by defining a process that coordinates both activities. Method: We synthesize the main challenges of AI-based ASs in industrial settings. To overcome such challenges, we propose a novel, holistic DevOps approach and reflect on the research efforts required to put it into practice. Results: The approach sets up five critical research directions: (a) a trustworthiness score to monitor operational AI-based ASs and identify self-adaptation needs in critical situations; (b) an integrated agile process for the development and continuous evolution of AI models; (c) an infrastructure for gathering key feedback required to address the trustworthiness of AI models at operation time; (d) continuous and seamless deployment of different context-specific instances of AI models in a distributed setting of ASs; and (e) a holistic and effective DevOps-based lifecycle for AI-based ASs. Conclusions: An approach supporting the continuous delivery of evolving AI models and their operation in ASs under adverse conditions would support companies in increasing users’ trust in their products.
Tasks
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05434v1
PDF	https://arxiv.org/pdf/2003.05434v1.pdf
PWC	https://paperswithcode.com/paper/research-directions-for-developing-and
Repo
Framework

Inference with Aggregate Data: An Optimal Transport Approach


Title	Inference with Aggregate Data: An Optimal Transport Approach
Authors	Rahul Singh, Isabel Haasler, Qinsheng Zhang, Johan Karlsson, Yongxin Chen
Abstract	We consider inference problems over probabilistic graphical models with aggregate data. In particular, we propose a new efficient belief propagation type algorithm over tree-structured graphs with polynomial computational complexity as well as a global convergence guarantee. This is in contrast to previous methods that either exhibit prohibitive complexity as the population grows or do not guarantee convergence. Our method is based on optimal transport, or more specifically, multi-marginal optimal transport theory. In particular, the inference problem with aggregate observations we consider in this paper can be seen as a structured multi-marginal optimal transport problem, where the cost function decomposes according to the underlying graph. Consequently, the celebrated Sinkhorn algorithm for multi-marginal optimal transport can be leveraged, together with the standard belief propagation algorithm to establish an efficient inference scheme. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations.
Tasks
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13933v1
PDF	https://arxiv.org/pdf/2003.13933v1.pdf
PWC	https://paperswithcode.com/paper/inference-with-aggregate-data-an-optimal
Repo
Framework