January 30, 2020

3110 words 15 mins read

Paper Group ANR 283

Risk-Averse Planning Under Uncertainty. Deep-learning estimation of band gap with the reading-periodic-table method and periodic convolution layer. Deep learning as optimal control problems: models and numerical methods. Early Action Prediction with Generative Adversarial Networks. Inner-product Kernels are Asymptotically Equivalent to Binary Discr …

Risk-Averse Planning Under Uncertainty


Title	Risk-Averse Planning Under Uncertainty
Authors	Mohamadreza Ahmadi, Masahiro Ono, Michel D. Ingham, Richard M. Murray, Aaron D. Ames
Abstract	We consider the problem of designing policies for partially observable Markov decision processes (POMDPs) with dynamic coherent risk objectives. Synthesizing risk-averse optimal policies for POMDPs requires infinite memory and thus undecidable. To overcome this difficulty, we propose a method based on bounded policy iteration for designing stochastic but finite state (memory) controllers, which takes advantage of standard convex optimization methods. Given a memory budget and optimality criterion, the proposed method modifies the stochastic finite state controller leading to sub-optimal solutions with lower coherent risk.
Tasks
Published	2019-09-27
URL	https://arxiv.org/abs/1909.12499v1
PDF	https://arxiv.org/pdf/1909.12499v1.pdf
PWC	https://paperswithcode.com/paper/risk-averse-planning-under-uncertainty
Repo
Framework

Deep-learning estimation of band gap with the reading-periodic-table method and periodic convolution layer


Title	Deep-learning estimation of band gap with the reading-periodic-table method and periodic convolution layer
Authors	Tomohiko Konno
Abstract	In this study, the deep learning method named reading periodic table, which utilizes deep learning to read the periodic table and the laws of the elements, was extended. The method now also learns the periodicity behind the periodic table, that is, the left- and right-most columns are adjacent to one another behind the table at the learning representation level. While the original method handles the table as it is, the extended method treats the periodic table as if its two edges are connected. This is achieved using novel layers named periodic convolution layers, which can handle inputs having periodicity and may be applied to other problems related to computer vision, time series, and so on if the data possesses some periodicity. In the reading periodic table method, no input of any material feature or descriptor is required. We verified that the method is also applicable for estimating the band gap of materials other than superconductors, for which the method was originally applied. We demonstrated two types of deep learning estimation: methods to estimate the existence of a band gap and those to estimate the value of the band gap given that the materials were known to have one. Finally, we discuss the limitations of the dataset and model evaluation method. We may be unable to distinguish good models based on the random train–test split scheme; thus, we must prepare an appropriate dataset where the training and test data are temporally separate.
Tasks	Band Gap, Time Series
Published	2019-11-16
URL	https://arxiv.org/abs/1912.05916v2
PDF	https://arxiv.org/pdf/1912.05916v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-estimation-of-band-gap-with-the
Repo
Framework

Deep learning as optimal control problems: models and numerical methods


Title	Deep learning as optimal control problems: models and numerical methods
Authors	Martin Benning, Elena Celledoni, Matthias J. Ehrhardt, Brynjulf Owren, Carola-Bibiane Schönlieb
Abstract	We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretisation. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. The differential equation setting lends itself to learning additional parameters such as the time discretisation. We explore this extension alongside natural constraints (e.g. time steps lie in a simplex). We compare these deep learning algorithms numerically in terms of induced flow and generalisation ability.
Tasks
Published	2019-04-11
URL	https://arxiv.org/abs/1904.05657v3
PDF	https://arxiv.org/pdf/1904.05657v3.pdf
PWC	https://paperswithcode.com/paper/deep-learning-as-optimal-control-problems
Repo
Framework

Early Action Prediction with Generative Adversarial Networks


Title	Early Action Prediction with Generative Adversarial Networks
Authors	Dong Wang, Yuan Yuan, Qi Wang
Abstract	Action Prediction is aimed to determine what action is occurring in a video as early as possible, which is crucial to many online applications, such as predicting a traffic accident before it happens and detecting malicious actions in the monitoring system. In this work, we address this problem by developing an end-to-end architecture that improves the discriminability of features of partially observed videos by assimilating them to features from complete videos. For this purpose, the generative adversarial network is introduced for tackling action prediction problem, which improves the recognition accuracy of partially observed videos though narrowing the feature difference of partially observed videos from complete ones. Specifically, its generator comprises of two networks: a CNN for feature extraction and an LSTM for estimating residual error between features of the partially observed videos and complete ones, and then the features from CNN adds the residual error from LSTM, which is regarded as the enhanced feature to fool a competing discriminator. Meanwhile, the generator is trained with an additional perceptual objective, which forces the enhanced features of partially observed videos are discriminative enough for action prediction. Extensive experimental results on UCF101, BIT and UT-Interaction datasets demonstrate that our approach outperforms the state-of-the-art methods, especially for videos that less than 50% portion of frames is observed.
Tasks
Published	2019-04-30
URL	http://arxiv.org/abs/1904.13085v1
PDF	http://arxiv.org/pdf/1904.13085v1.pdf
PWC	https://paperswithcode.com/paper/early-action-prediction-with-generative
Repo
Framework

Inner-product Kernels are Asymptotically Equivalent to Binary Discrete Kernels


Title	Inner-product Kernels are Asymptotically Equivalent to Binary Discrete Kernels
Authors	Zhenyu Liao, Romain Couillet
Abstract	This article investigates the eigenspectrum of the inner product-type kernel matrix $\sqrt{p} \mathbf{K}={f( \mathbf{x}_i^{\sf T} \mathbf{x}j/\sqrt{p})}{i,j=1}^n $ under a binary mixture model in the high dimensional regime where the number of data $n$ and their dimension $p$ are both large and comparable. Based on recent advances in random matrix theory, we show that, for a wide range of nonlinear functions $f$, the eigenspectrum behavior is asymptotically equivalent to that of an (at most) cubic function. This sheds new light on the understanding of nonlinearity in large dimensional problems. As a byproduct, we propose a simple function prototype valued in $ (-1,0,1) $ that, while reducing both storage memory and running time, achieves the same (asymptotic) classification performance as any arbitrary function $f$.
Tasks
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06788v1
PDF	https://arxiv.org/pdf/1909.06788v1.pdf
PWC	https://paperswithcode.com/paper/inner-product-kernels-are-asymptotically
Repo
Framework

Deep Sky Modeling for Single Image Outdoor Lighting Estimation


Title	Deep Sky Modeling for Single Image Outdoor Lighting Estimation
Authors	Yannick Hold-Geoffroy, Akshaya Athawale, Jean-François Lalonde
Abstract	We propose a data-driven learned sky model, which we use for outdoor lighting estimation from a single image. As no large-scale dataset of images and their corresponding ground truth illumination is readily available, we use complementary datasets to train our approach, combining the vast diversity of illumination conditions of SUN360 with the radiometrically calibrated and physically accurate Laval HDR sky database. Our key contribution is to provide a holistic view of both lighting modeling and estimation, solving both problems end-to-end. From a test image, our method can directly estimate an HDR environment map of the lighting without relying on analytical lighting models. We demonstrate the versatility and expressivity of our learned sky model and show that it can be used to recover plausible illumination, leading to visually pleasant virtual object insertions. To further evaluate our method, we capture a dataset of HDR 360{\deg} panoramas and show through extensive validation that we significantly outperform previous state-of-the-art.
Tasks
Published	2019-05-10
URL	https://arxiv.org/abs/1905.03897v1
PDF	https://arxiv.org/pdf/1905.03897v1.pdf
PWC	https://paperswithcode.com/paper/deep-sky-modeling-for-single-image-outdoor
Repo
Framework

KNG: The K-Norm Gradient Mechanism


Title	KNG: The K-Norm Gradient Mechanism
Authors	Matthew Reimherr, Jordan Awan
Abstract	This paper presents a new mechanism for producing sanitized statistical summaries that achieve \emph{differential privacy}, called the \emph{K-Norm Gradient} Mechanism, or KNG. This new approach maintains the strong flexibility of the exponential mechanism, while achieving the powerful utility performance of objective perturbation. KNG starts with an inherent objective function (often an empirical risk), and promotes summaries that are close to minimizing the objective by weighting according to how far the gradient of the objective function is from zero. Working with the gradient instead of the original objective function allows for additional flexibility as one can penalize using different norms. We show that, unlike the exponential mechanism, the noise added by KNG is asymptotically negligible compared to the statistical error for many problems. In addition to theoretical guarantees on privacy and utility, we confirm the utility of KNG empirically in the settings of linear and quantile regression through simulations.
Tasks
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09436v1
PDF	https://arxiv.org/pdf/1905.09436v1.pdf
PWC	https://paperswithcode.com/paper/kng-the-k-norm-gradient-mechanism
Repo
Framework

Global Pixel Transformers for Virtual Staining of Microscopy Images


Title	Global Pixel Transformers for Virtual Staining of Microscopy Images
Authors	Yi Liu, Hao Yuan, Zhengyang Wang, Shuiwang Ji
Abstract	Visualizing the details of different cellular structures is of great importance to elucidate cellular functions. However, it is challenging to obtain high quality images of different structures directly due to complex cellular environments. Fluorescence staining is a popular technique to label different structures but has several drawbacks. In particular, label staining is time consuming and may affect cell morphology, and simultaneous labels are inherently limited. This raises the need of building computational models to learn relationships between unlabeled microscopy images and labeled fluorescence images, and to infer fluorescence labels of other microscopy images excluding the physical staining process. We propose to develop a novel deep model for virtual staining of unlabeled microscopy images. We first propose a novel network layer, known as the global pixel transformer layer, that fuses global information from inputs effectively. The proposed global pixel transformer layer can generate outputs with arbitrary dimensions, and can be employed for all the regular, down-sampling, and up-sampling operators. We then incorporate our proposed global pixel transformer layers and dense blocks to build an U-Net like network. We believe such a design can promote feature reusing between layers. In addition, we propose a multi-scale input strategy to encourage networks to capture features at different scales. We conduct evaluations across various fluorescence image prediction tasks to demonstrate the effectiveness of our approach. Both quantitative and qualitative results show that our method outperforms the state-of-the-art approach significantly. It is also shown that our proposed global pixel transformer layer is useful to improve the fluorescence image prediction results.
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00941v3
PDF	https://arxiv.org/pdf/1907.00941v3.pdf
PWC	https://paperswithcode.com/paper/global-transformer-u-nets-for-label-free
Repo
Framework

Software-Defined FPGA Accelerator Design for Mobile Deep Learning Applications


Title	Software-Defined FPGA Accelerator Design for Mobile Deep Learning Applications
Authors	Panagiotis G. Mousouliotis, Loukas P. Petrou
Abstract	Recently, the field of deep learning has received great attention by the scientific community and it is used to provide improved solutions to many computer vision problems. Convolutional neural networks (CNNs) have been successfully used to attack problems such as object recognition, object detection, semantic segmentation, and scene understanding. The rapid development of deep learning goes hand by hand with the adaptation of GPUs for accelerating its processes, such as network training and inference. Even though FPGA design exists long before the use of GPUs for accelerating computations and despite the fact that high-level synthesis (HLS) tools are getting more attractive, the adaptation of FPGAs for deep learning research and application development is poor due to the requirement of hardware design related expertise. This work presents a workflow for deep learning mobile application acceleration on small low-cost low-power FPGA devices using HLS tools. This workflow eases the design of an improved version of the SqueezeJet accelerator used for the speedup of mobile-friendly low-parameter ImageNet class CNNs, such as the SqueezeNet v1.1 and the ZynqNet. Additionally, the workflow includes the development of an HLS-driven analytical model which is used for performance estimation of the accelerator. This model can be also used to direct the design process and lead to future design improvements and optimizations.
Tasks	Object Detection, Object Recognition, Scene Understanding, Semantic Segmentation
Published	2019-02-08
URL	http://arxiv.org/abs/1902.03192v2
PDF	http://arxiv.org/pdf/1902.03192v2.pdf
PWC	https://paperswithcode.com/paper/software-defined-fpga-accelerator-design-for
Repo
Framework

Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation


Title	Building Task-Oriented Visual Dialog Systems Through Alternative Optimization Between Dialog Policy and Language Generation
Authors	Mingyang Zhou, Josh Arnold, Zhou Yu
Abstract	Reinforcement learning (RL) is an effective approach to learn an optimal dialog policy for task-oriented visual dialog systems. A common practice is to apply RL on a neural sequence-to-sequence (seq2seq) framework with the action space being the output vocabulary in the decoder. However, it is difficult to design a reward function that can achieve a balance between learning an effective policy and generating a natural dialog response. This paper proposes a novel framework that alternatively trains a RL policy for image guessing and a supervised seq2seq model to improve dialog generation quality. We evaluate our framework on the GuessWhich task and the framework achieves the state-of-the-art performance in both task completion and dialog quality.
Tasks	Text Generation, Visual Dialog
Published	2019-09-06
URL	https://arxiv.org/abs/1909.05365v2
PDF	https://arxiv.org/pdf/1909.05365v2.pdf
PWC	https://paperswithcode.com/paper/building-task-oriented-visual-dialog-systems
Repo
Framework

Testing Properties of Multiple Distributions with Few Samples


Title	Testing Properties of Multiple Distributions with Few Samples
Authors	Maryam Aliakbarpour, Sandeep Silwal
Abstract	We propose a new setting for testing properties of distributions while receiving samples from several distributions, but few samples per distribution. Given samples from $s$ distributions, $p_1, p_2, \ldots, p_s$, we design testers for the following problems: (1) Uniformity Testing: Testing whether all the $p_i$'s are uniform or $\epsilon$-far from being uniform in $\ell_1$-distance (2) Identity Testing: Testing whether all the $p_i$'s are equal to an explicitly given distribution $q$ or $\epsilon$-far from $q$ in $\ell_1$-distance, and (3) Closeness Testing: Testing whether all the $p_i$'s are equal to a distribution $q$ which we have sample access to, or $\epsilon$-far from $q$ in $\ell_1$-distance. By assuming an additional natural condition about the source distributions, we provide sample optimal testers for all of these problems.
Tasks
Published	2019-11-17
URL	https://arxiv.org/abs/1911.07324v1
PDF	https://arxiv.org/pdf/1911.07324v1.pdf
PWC	https://paperswithcode.com/paper/testing-properties-of-multiple-distributions
Repo
Framework

Stochastic gradient-free descents


Title	Stochastic gradient-free descents
Authors	Xiaopeng Luo, Xin Xu
Abstract	In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the convergence behavior of these methods under the mean-variance framework, and also provide a theoretical analysis about the inclusion of momentum in stochastic settings which reveals that the momentum term we used adds a deviation of order $\mathcal{O}(1/k)$ but controls the variance at the order $\mathcal{O}(1/k)$ for the $k$th iteration. So it is shown that, when employing a decaying stepsize $\alpha_k=\mathcal{O}(1/k)$, the stochastic gradient-free methods can still maintain the sublinear convergence rate $\mathcal{O}(1/k)$ and the accelerated methods with momentum can achieve a convergence rate $\mathcal{O}(1/k^2)$ in probability for the strongly convex objectives with Lipschitz gradients; and all these methods converge to a solution with a zero expected gradient norm when the objective function is nonconvex, twice differentiable and bounded below.
Tasks	Stochastic Optimization
Published	2019-12-31
URL	https://arxiv.org/abs/1912.13305v5
PDF	https://arxiv.org/pdf/1912.13305v5.pdf
PWC	https://paperswithcode.com/paper/stochastic-gradient-free-descents
Repo
Framework

Particle filter with rejection control and unbiased estimator of the marginal likelihood


Title	Particle filter with rejection control and unbiased estimator of the marginal likelihood
Authors	Jan Kudlicka, Lawrence M. Murray, Thomas B. Schön, Fredrik Lindsten
Abstract	We consider the combined use of resampling and partial rejection control in sequential Monte Carlo methods, also known as particle filters. While the variance reducing properties of rejection control are known, there has not been (to the best of our knowledge) any work on unbiased estimation of the marginal likelihood (also known as the model evidence or the normalizing constant) in this type of particle filter. Being able to estimate the marginal likelihood without bias is highly relevant for model comparison, computation of interpretable and reliable confidence intervals, and in exact approximation methods, such as particle Markov chain Monte Carlo. In the paper we present a particle filter with rejection control that enables unbiased estimation of the marginal likelihood.
Tasks
Published	2019-10-21
URL	https://arxiv.org/abs/1910.09527v2
PDF	https://arxiv.org/pdf/1910.09527v2.pdf
PWC	https://paperswithcode.com/paper/particle-filter-with-rejection-control-and
Repo
Framework

The Supermarket Model with Known and Predicted Service Times


Title	The Supermarket Model with Known and Predicted Service Times
Authors	Michael Mitzenmacher
Abstract	The supermarket model typically refers to a system with a large number of queues, where arriving customers choose $d$ queues at random and join the queue with fewest customers. The supermarket model demonstrates the power of even small amounts of choice, as compared to simply joining a queue chosen uniformly at random, for load balancing systems. In this work we perform simulation-based studies to consider variations where service times for a customer are predicted, as might be done in modern settings using machine learning techniques or related mechanisms. To begin, we start by considering the baseline where service times are known. We find that this allows for significant improvements. In particular, not only can the queue being joined be chosen based on the total work at the queue instead of the number of jobs, but also the jobs in the queue can be served using strategies that take advantage of the service times such as shortest job first or shortest remaining processing time. Such strategies greatly improve performance under high load. We then examine the impact of using predictions in place of true service times. Our main takeaway is that using even seemingly weak predictions of service times can yield significant benefits over blind First In First Out queueing in this context. However, some care must be taken when using predicted service time information to both choose a queue and order elements for service within a queue; while in many cases using the information for both choosing and ordering is beneficial, in many of our simulation settings we find that simply using the number of jobs to choose a queue is better when using predicted service times to order jobs in a queue. Our study leaves many natural open questions for further work.
Tasks
Published	2019-05-23
URL	https://arxiv.org/abs/1905.12155v1
PDF	https://arxiv.org/pdf/1905.12155v1.pdf
PWC	https://paperswithcode.com/paper/190512155
Repo
Framework

Deep Clustering of Compressed Variational Embeddings


Title	Deep Clustering of Compressed Variational Embeddings
Authors	Suya Wu, Enmao Diao, Jie Ding, Vahid Tarokh
Abstract	Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint Variational Autoencoders with Bernoulli mixture models (VAB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by Variational Autoencoders (VAEs) and group data representations by Bernoulli mixture models (BMMs). Once jointly trained for compression and clustering, the model can be decomposed into two parts: a data vendor that encodes the raw data into compressed data, and a data consumer that classifies the received (compressed) data. In this way, the data vendor benefits from data security and communication bandwidth, while the data consumer benefits from low computational complexity. To enable training using the gradient descent algorithm, we propose to use the Gumbel-Softmax distribution to resolve the infeasibility of the back-propagation algorithm when assessing categorical samples.
Tasks
Published	2019-10-23
URL	https://arxiv.org/abs/1910.10341v1
PDF	https://arxiv.org/pdf/1910.10341v1.pdf
PWC	https://paperswithcode.com/paper/deep-clustering-of-compressed-variational
Repo
Framework