January 30, 2020

3321 words 16 mins read

Paper Group ANR 275

SetGAN: Improving the stability and diversity of generative models through a permutation invariant architecture. Technical notes: Syntax-aware Representation Learning With Pointer Networks. Automatic Model Building in GEFCom 2017 Qualifying Match. Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinf …

SetGAN: Improving the stability and diversity of generative models through a permutation invariant architecture


Title	SetGAN: Improving the stability and diversity of generative models through a permutation invariant architecture
Authors	Alessandro Ferrero, Shireen Elhabian, Ross Whitaker
Abstract	Generative adversarial networks (GANs) have proven effective in modeling distributions of high-dimensional data. However, their training instability is a well-known hindrance to convergence, which results in practical challenges in their applications to novel data. Furthermore, even when convergence is reached, GANs can be affected by mode collapse, a phenomenon for which the generator learns to model only a small part of the target distribution, disregarding the vast majority of the data manifold or distribution. This paper addresses these challenges by introducing SetGAN, an adversarial architecture that processes sets of generated and real samples, and discriminates between the origins of these sets (i.e., training versus generated data) in a flexible, permutation invariant manner. We also propose a new metric to quantitatively evaluate GANs that does not require previous knowledge of the application, apart from the data itself. Using the new metric, in conjunction with the state-of-the-art evaluation methods, we show that the proposed architecture, when compared with GAN variants stemming from similar strategies, produces more accurate models of the input data in a way that is also less sensitive to hyperparameter settings.
Tasks
Published	2019-06-28
URL	https://arxiv.org/abs/1907.00109v2
PDF	https://arxiv.org/pdf/1907.00109v2.pdf
PWC	https://paperswithcode.com/paper/setgans-enforcing-distributional-accuracy-in
Repo
Framework

Technical notes: Syntax-aware Representation Learning With Pointer Networks


Title	Technical notes: Syntax-aware Representation Learning With Pointer Networks
Authors	Matteo Grella
Abstract	This is a work-in-progress report, which aims to share preliminary results of a novel sequence-to-sequence schema for dependency parsing that relies on a combination of a BiLSTM and two Pointer Networks (Vinyals et al., 2015), in which the final softmax function has been replaced with the logistic regression. The two pointer networks co-operate to develop a latent syntactic knowledge, by learning the lexical properties of “selection” and the lexical properties of “selectability”, respectively. At the moment and without fine-tuning, the parser implementation gets a UAS of 93.14% on the English Penn-treebank (Marcus et al., 1993) annotated with Stanford Dependencies: 2-3% under the SOTA but yet attractive as a baseline of the approach.
Tasks	Dependency Parsing, Representation Learning
Published	2019-03-17
URL	http://arxiv.org/abs/1903.07161v1
PDF	http://arxiv.org/pdf/1903.07161v1.pdf
PWC	https://paperswithcode.com/paper/technical-notes-syntax-aware-representation
Repo
Framework

Automatic Model Building in GEFCom 2017 Qualifying Match


Title	Automatic Model Building in GEFCom 2017 Qualifying Match
Authors	Ján Dolinský, Mária Starovská, Robert Tóth
Abstract	The Tangent Works team participated in GEFCom 2017 to test its automatic model building strategy for time series known as Tangent Information Modeller (TIM). Model building using TIM combined with historical temperature shuffling resulted in winning the competition. This strategy involved one remaining degree of freedom, a decision on using a trend variable. This paper describes our modelling efforts in the competition, and furthermore outlines a fully automated scenario where the decision on using the trend variable is handled by TIM. The results show that such a setup would also win the competition.
Tasks	Time Series
Published	2019-04-12
URL	http://arxiv.org/abs/1904.12608v1
PDF	http://arxiv.org/pdf/1904.12608v1.pdf
PWC	https://paperswithcode.com/paper/190412608
Repo
Framework

Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning


Title	Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning
Authors	Artur Garcia-Saez, Jordi Riu
Abstract	We present a classical control mechanism for Quantum devices using Reinforcement Learning. Our strategy is applied to the Quantum Approximate Optimization Algorithm (QAOA) in order to optimize an objective function that encodes a solution to a hard combinatorial problem. This method provides optimal control of the Quantum device following a reformulation of QAOA as an environment where an autonomous classical agent interacts and performs actions to achieve higher rewards. This formulation allows a hybrid classical-Quantum device to train itself from previous executions using a continuous formulation of deep Q-learning to control the continuous degrees of freedom of QAOA. Our approach makes a selective use of Quantum measurements to complete the observations of the Quantum state available to the agent. We run tests of this approach on MAXCUT instances of size up to N = 21 obtaining optimal results. We show how this formulation can be used to transfer the knowledge from shorter training episodes to reach a regime of longer executions where QAOA delivers higher results.
Tasks	Continuous Control, Q-Learning
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09682v1
PDF	https://arxiv.org/pdf/1911.09682v1.pdf
PWC	https://paperswithcode.com/paper/quantum-observables-for-continuous-control-of
Repo
Framework

Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization


Title	Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization
Authors	Digvijay Boob, Qi Deng, Guanghui Lan
Abstract	Functional constrained optimization is becoming more and more important in machine learning and operations research. Such problems have potential applications in risk-averse machine learning, semisupervised learning and robust optimization among others. In this paper, we first present a novel Constraint Extrapolation (ConEx) method for solving convex functional constrained problems, which utilizes linear approximations of the constraint functions to define the extrapolation (or acceleration) step. We show that this method is a unified algorithm that achieves the best-known rate of convergence for solving different functional constrained convex composite problems, including convex or strongly convex, and smooth or nonsmooth problems with stochastic objective and/or stochastic constraints. Many of these rates of convergence were in fact obtained for the first time in the literature. In addition, ConEx is a single-loop algorithm that does not involve any penalty subproblems. Contrary to existing dual methods, it does not require the projection of Lagrangian multipliers into a (possibly unknown) bounded set. Second, for nonconvex functional constrained problem, we introduce a new proximal point method which transforms the initial nonconvex problem into a sequence of convex functional constrained subproblems. We establish the convergence and rate of convergence of this algorithm to KKT points under different constraint qualifications. For practical use, we present inexact variants of this algorithm, in which approximate solutions of the subproblems are computed using the aforementioned ConEx method and establish their associated rate of convergence. To the best of our knowledge, most of these convergence and complexity results of the proximal point method for nonconvex problems also seem to be new in the literature.
Tasks
Published	2019-08-07
URL	https://arxiv.org/abs/1908.02734v3
PDF	https://arxiv.org/pdf/1908.02734v3.pdf
PWC	https://paperswithcode.com/paper/proximal-point-methods-for-optimization-with
Repo
Framework

A large-scale field test on word-image classification in large historical document collections using a traditional and two deep-learning methods


Title	A large-scale field test on word-image classification in large historical document collections using a traditional and two deep-learning methods
Authors	Lambert Schomaker
Abstract	This technical report describes a practical field test on word-image classification in a very large collection of more than 300 diverse handwritten historical manuscripts, with 1.6 million unique labeled images and more than 11 million images used in testing. Results indicate that several deep-learning tests completely failed (mean accuracy 83%). In the tests with more than 1000 output units (lexical words) in one-hot encoding for classification, performance steeply drops to almost zero percent accuracy, even with a modest size of the pre-final (i.e., penultimate) layer (150 units). A traditional feature method (BOVW) displays a consistent performance over numbers of classes and numbers of training examples (mean accuracy 87%). Additional tests using nearest mean on the output of the pre-final layer of an Inception V3 network, for each book, only yielded mediocre results (mean accuracy 49%), but was not sensitive to high numbers of classes. Notably, this experiment was only possible on the basis of labels that were harvested on the basis of a traditional method which already works starting from a single labeled image per class. It is expected that the performance of the failed deep learning tests can be repaired, but only on the basis of human handcrafting (sic) of network architecture and hyperparameters. When the failed problematic books are not considered, end-to-end CNN training yields about 95% accuracy. This average is dominated by a large subset of Chinese characters, performances for other script styles being lower.
Tasks	Image Classification
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08421v1
PDF	http://arxiv.org/pdf/1904.08421v1.pdf
PWC	https://paperswithcode.com/paper/a-large-scale-field-test-on-word-image
Repo
Framework

Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures


Title	Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures
Authors	Chankyu Lee, Syed Shakib Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, Kaushik Roy
Abstract	Spiking Neural Networks (SNNs) have recently emerged as a prominent neural computing paradigm. However, the typical shallow SNN architectures have limited capacity for expressing complex representations while training deep SNNs using input spikes has not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-the-shelf trained deep Artificial Neural Networks (ANNs) to SNNs. However, the ANN-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous, non-differentiable nature of the spike generation function. To overcome this problem, we propose an approximate derivative method that accounts for the leaky behavior of LIF neurons. This method enables training deep convolutional SNNs directly (with input spike events) using spike-based backpropagation. Our experiments show the effectiveness of the proposed spike-based learning on deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with a spike-based learning. Moreover, we analyze sparse event-based computations to demonstrate the efficacy of the proposed SNN training method for inference operation in the spiking domain.
Tasks
Published	2019-03-15
URL	https://arxiv.org/abs/1903.06379v4
PDF	https://arxiv.org/pdf/1903.06379v4.pdf
PWC	https://paperswithcode.com/paper/enabling-spike-based-backpropagation-in-state
Repo
Framework

Probabilistic Regressor Chains with Monte Carlo Methods


Title	Probabilistic Regressor Chains with Monte Carlo Methods
Authors	Jesse Read, Luca Martino
Abstract	A large number and diversity of techniques have been offered in the literature in recent years for solving multi-label classification tasks, including classifier chains where predictions are cascaded to other models as additional features. The idea of extending this chaining methodology to multi-output regression has already been suggested and trialed: regressor chains. However, this has so-far been limited to greedy inference and has provided relatively poor results compared to individual models, and of limited applicability. In this paper we identify and discuss the main limitations, including an analysis of different base models, loss functions, explainability, and other desiderata of real-world applications. To overcome the identified limitations we study and develop methods for regressor chains. In particular we present a sequential Monte Carlo scheme in the framework of a probabilistic regressor chain, and we show it can be effective, flexible and useful in several types of data. We place regressor chains in context in general terms of multi-output learning with continuous outputs, and in doing this shed additional light on classifier chains.
Tasks	Multi-Label Classification
Published	2019-07-18
URL	https://arxiv.org/abs/1907.08087v1
PDF	https://arxiv.org/pdf/1907.08087v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-regressor-chains-with-monte
Repo
Framework

Performance Evaluation of Learned 3D Features


Title	Performance Evaluation of Learned 3D Features
Authors	Riccardo Spezialetti, Samuele Salti, Luigi Di Stefano
Abstract	Matching surfaces is a challenging 3D Computer Vision problem typically addressed by local features. Although a variety of 3D feature detectors and descriptors has been proposed in literature, they have seldom been proposed together and it is yet not clear how to identify the most effective detector-descriptor pair for a specific application. A promising solution is to leverage machine learning to learn the optimal 3D detector for any given 3D descriptor [15]. In this paper, we report a performance evaluation of the detector-descriptor pairs obtained by learning a paired 3D detector for the most popular 3D descriptors. In particular, we address experimental settings dealing with object recognition and surface registration.
Tasks	Object Recognition
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06884v1
PDF	https://arxiv.org/pdf/1909.06884v1.pdf
PWC	https://paperswithcode.com/paper/performance-evaluation-of-learned-3d-features
Repo
Framework

Bayesian Feature Pyramid Networks for Automatic Multi-Label Segmentation of Chest X-rays and Assessment of Cardio-Thoratic Ratio


Title	Bayesian Feature Pyramid Networks for Automatic Multi-Label Segmentation of Chest X-rays and Assessment of Cardio-Thoratic Ratio
Authors	Roman Solovyev, Iaroslav Melekhov, Timo Lesonen, Elias Vaattovaara, Osmo Tervonen, Aleksei Tiulpin
Abstract	Cardiothoratic ratio (CTR) estimated from chest radiographs is a marker indicative of cardiomegaly, the presence of which is in the criteria for heart failure diagnosis. Existing methods for automatic assessment of CTR are driven by Deep Learning-based segmentation. However, these techniques produce only point estimates of CTR but clinical decision making typically assumes the uncertainty. In this paper, we propose a novel method for chest X-ray segmentation and CTR assessment in an automatic manner. In contrast to the previous art, we, for the first time, propose to estimate CTR with uncertainty bounds. Our method is based on Deep Convolutional Neural Network with Feature Pyramid Network (FPN) decoder. We propose two modifications of FPN: replace the batch normalization with instance normalization and inject the dropout which allows to obtain the Monte-Carlo estimates of the segmentation maps at test time. Finally, using the predicted segmentation mask samples, we estimate CTR with uncertainty. In our experiments we demonstrate that the proposed method generalizes well to three different test sets. Finally, we make the annotations produced by two radiologists for all our datasets publicly available.
Tasks	Decision Making
Published	2019-08-08
URL	https://arxiv.org/abs/1908.02924v1
PDF	https://arxiv.org/pdf/1908.02924v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-feature-pyramid-networks-for
Repo
Framework

FoodTracker: A Real-time Food Detection Mobile Application by Deep Convolutional Neural Networks


Title	FoodTracker: A Real-time Food Detection Mobile Application by Deep Convolutional Neural Networks
Authors	Jianing Sun, Katarzyna Radecka, Zeljko Zilic
Abstract	We present a mobile application made to recognize food items of multi-object meal from a single image in real-time, and then return the nutrition facts with components and approximate amounts. Our work is organized in two parts. First, we build a deep convolutional neural network merging with YOLO, a state-of-the-art detection strategy, to achieve simultaneous multi-object recognition and localization with nearly 80% mean average precision. Second, we adapt our model into a mobile application with extending function for nutrition analysis. After inferring and decoding the model output in the app side, we present detection results that include bounding box position and class label in either real-time or local mode. Our model is well-suited for mobile devices with negligible inference time and small memory requirements with a deep learning algorithm.
Tasks	Object Recognition
Published	2019-09-13
URL	https://arxiv.org/abs/1909.05994v2
PDF	https://arxiv.org/pdf/1909.05994v2.pdf
PWC	https://paperswithcode.com/paper/foodtracker-a-real-time-food-detection-mobile
Repo
Framework

Scale Invariant Power Iteration


Title	Scale Invariant Power Iteration
Authors	Cheolmin Kim, Youngseok Kim, Diego Klabjan
Abstract	Power iteration has been generalized to solve many interesting problems in machine learning and statistics. Despite its striking success, theoretical understanding of when and how such an algorithm enjoys good convergence property is limited. In this work, we introduce a new class of optimization problems called scale invariant problems and prove that they can be efficiently solved by scale invariant power iteration (SCI-PI) with a generalized convergence guarantee of power iteration. By deriving that a stationary point is an eigenvector of the Hessian evaluated at the point, we show that scale invariant problems indeed resemble the leading eigenvector problem near a local optimum. Also, based on a novel reformulation, we geometrically derive SCI-PI which has a general form of power iteration. The convergence analysis shows that SCI-PI attains local linear convergence with a rate being proportional to the top two eigenvalues of the Hessian at the optimum. Moreover, we discuss some extended settings of scale invariant problems and provide similar convergence results for them. In numerical experiments, we introduce applications to independent component analysis, Gaussian mixtures, and non-negative matrix factorization. Experimental results demonstrate that SCI-PI is competitive to state-of-the-art benchmark algorithms and often yield better solutions.
Tasks
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09882v1
PDF	https://arxiv.org/pdf/1905.09882v1.pdf
PWC	https://paperswithcode.com/paper/scale-invariant-power-iteration
Repo
Framework

CapStore: Energy-Efficient Design and Management of the On-Chip Memory for CapsuleNet Inference Accelerators


Title	CapStore: Energy-Efficient Design and Management of the On-Chip Memory for CapsuleNet Inference Accelerators
Authors	Alberto Marchisio, Muhammad Abdullah Hanif, Mohammad Taghi Teimoori, Muhammad Shafique
Abstract	Deep Neural Networks (DNNs) have been established as the state-of-the-art algorithm for advanced machine learning applications. Recently, CapsuleNets have improved the generalization ability, as compared to DNNs, due to their multi-dimensional capsules. However, they pose high computational and memory requirements, which makes energy-efficient inference a challenging task. In this paper, we perform an extensive analysis to demonstrate their key limitations due to intense memory accesses and large on-chip memory requirements. To enable efficient CaspuleNet inference accelerators, we propose a specialized on-chip memory hierarchy which minimizes the off-chip memory accesses, while efficiently feeding the data to the accelerator. We analyze the on-chip memory requirements for each memory component of the architecture. By leveraging this analysis, we propose a methodology to explore different on-chip memory designs and a power-gating technique to further reduce the energy consumption, depending upon the utilization across different operations of a CapsuleNet. Our memory designs can significantly reduce the energy consumption of the on-chip memory by up to 86%, when compared to a state-of-the-art memory design. Since the power consumption of the memory elements is the major contributor in the power breakdown of the CapsuleNet accelerator, as we will also show in our analyses, the proposed memory design can effectively reduce the overall energy consumption of the complete CapsuleNet accelerator architecture.
Tasks
Published	2019-02-04
URL	http://arxiv.org/abs/1902.01151v2
PDF	http://arxiv.org/pdf/1902.01151v2.pdf
PWC	https://paperswithcode.com/paper/capstore-energy-efficient-design-and
Repo
Framework

Single Image Portrait Relighting


Title	Single Image Portrait Relighting
Authors	Tiancheng Sun, Jonathan T. Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, Ravi Ramamoorthi
Abstract	Lighting plays a central role in conveying the essence and depth of the subject in a portrait photograph. Professional photographers will carefully control the lighting in their studio to manipulate the appearance of their subject, while consumer photographers are usually constrained to the illumination of their environment. Though prior works have explored techniques for relighting an image, their utility is usually limited due to requirements of specialized hardware, multiple images of the subject under controlled or known illuminations, or accurate models of geometry and reflectance. To this end, we present a system for portrait relighting: a neural network that takes as input a single RGB image of a portrait taken with a standard cellphone camera in an unconstrained environment, and from that image produces a relit image of that subject as though it were illuminated according to any provided environment map. Our method is trained on a small database of 18 individuals captured under different directional light sources in a controlled light stage setup consisting of a densely sampled sphere of lights. Our proposed technique produces quantitatively superior results on our dataset’s validation set compared to prior works, and produces convincing qualitative relighting results on a dataset of hundreds of real-world cellphone portraits. Because our technique can produce a 640 $\times$ 640 image in only 160 milliseconds, it may enable interactive user-facing photographic applications in the future.
Tasks
Published	2019-05-02
URL	https://arxiv.org/abs/1905.00824v1
PDF	https://arxiv.org/pdf/1905.00824v1.pdf
PWC	https://paperswithcode.com/paper/single-image-portrait-relighting
Repo
Framework

Reconstruction of 3-D Atomic Distortions from Electron Microscopy with Deep Learning


Title	Reconstruction of 3-D Atomic Distortions from Electron Microscopy with Deep Learning
Authors	Nouamane Laanait, Qian He, Albina Y. Borisevich
Abstract	Deep learning has demonstrated superb efficacy in processing imaging data, yet its suitability in solving challenging inverse problems in scientific imaging has not been fully explored. Of immense interest is the determination of local material properties from atomically-resolved imaging, such as electron microscopy, where such information is encoded in subtle and complex data signatures, and whose recovery and interpretation necessitate intensive numerical simulations subject to the requirement of near-perfect knowledge of the experimental setup. We demonstrate that an end-to-end deep learning model can successfully recover 3-dimensional atomic distortions of a variety of oxide perovskite materials from a single 2-dimensional experimental scanning transmission electron (STEM) micrograph, in the process resolving a longstanding question in the recovery of 3-D atomic distortions from STEM experiments. Our results indicate that deep learning is a promising approach to efficiently address unsolved inverse problems in scientific imaging and to underpin novel material investigations at atomic resolution.
Tasks
Published	2019-02-19
URL	http://arxiv.org/abs/1902.06876v1
PDF	http://arxiv.org/pdf/1902.06876v1.pdf
PWC	https://paperswithcode.com/paper/reconstruction-of-3-d-atomic-distortions-from
Repo
Framework