January 26, 2020

2970 words 14 mins read

Paper Group ANR 1587

CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs. Benchmarking Batch Deep Reinforcement Learning Algorithms. Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization. Scalable Holistic Linear Regression. Real-time Data Driven Precision Estimator for RAVEN-II Surgical Robot End Effector Positio …

CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs


Title	CodeX: Bit-Flexible Encoding for Streaming-based FPGA Acceleration of DNNs
Authors	Mohammad Samragh, Mojan Javaheripi, Farinaz Koushanfar
Abstract	This paper proposes CodeX, an end-to-end framework that facilitates encoding, bitwidth customization, fine-tuning, and implementation of neural networks on FPGA platforms. CodeX incorporates nonlinear encoding to the computation flow of neural networks to save memory. The encoded features demand significantly lower storage compared to the raw full-precision activation values; therefore, the execution flow of CodeX hardware engine is completely performed within the FPGA using on-chip streaming buffers with no access to the off-chip DRAM. We further propose a fully-automated algorithm inspired by reinforcement learning which determines the customized encoding bitwidth across network layers. CodeX full-stack framework comprises of a compiler which takes a high-level Python description of an arbitrary neural network architecture. The compiler then instantiates the corresponding elements from CodeX Hardware library for FPGA implementation. Proof-of-concept evaluations on MNIST, SVHN, and CIFAR-10 datasets demonstrate an average of 4.65x throughput improvement compared to stand-alone weight encoding. We further compare CodeX with six existing full-precision DNN accelerators on ImageNet, showing an average of 3.6x and 2.54x improvement in throughput and performance-per-watt, respectively.
Tasks
Published	2019-01-17
URL	http://arxiv.org/abs/1901.05582v1
PDF	http://arxiv.org/pdf/1901.05582v1.pdf
PWC	https://paperswithcode.com/paper/codex-bit-flexible-encoding-for-streaming
Repo
Framework

Benchmarking Batch Deep Reinforcement Learning Algorithms


Title	Benchmarking Batch Deep Reinforcement Learning Algorithms
Authors	Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, Joelle Pineau
Abstract	Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting–learning from a fixed data set without interaction with the environment. Following this result, there have been several papers showing reasonable performances under a variety of environments and batch settings. In this paper, we benchmark the performance of recent off-policy and batch reinforcement learning algorithms under unified settings on the Atari domain, with data generated by a single partially-trained behavioral policy. We find that under these conditions, many of these algorithms underperform DQN trained online with the same amount of data, as well as the partially-trained behavioral policy. To introduce a strong baseline, we adapt the Batch-Constrained Q-learning algorithm to a discrete-action setting, and show it outperforms all existing algorithms at this task.
Tasks	Q-Learning
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01708v1
PDF	https://arxiv.org/pdf/1910.01708v1.pdf
PWC	https://paperswithcode.com/paper/benchmarking-batch-deep-reinforcement
Repo
Framework

Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization


Title	Non-Asymptotic Analysis of Fractional Langevin Monte Carlo for Non-Convex Optimization
Authors	Thanh Huy Nguyen, Umut Şimşekli, Gaël Richard
Abstract	Recent studies on diffusion-based sampling methods have shown that Langevin Monte Carlo (LMC) algorithms can be beneficial for non-convex optimization, and rigorous theoretical guarantees have been proven for both asymptotic and finite-time regimes. Algorithmically, LMC-based algorithms resemble the well-known gradient descent (GD) algorithm, where the GD recursion is perturbed by an additive Gaussian noise whose variance has a particular form. Fractional Langevin Monte Carlo (FLMC) is a recently proposed extension of LMC, where the Gaussian noise is replaced by a heavy-tailed {\alpha}-stable noise. As opposed to its Gaussian counterpart, these heavy-tailed perturbations can incur large jumps and it has been empirically demonstrated that the choice of {\alpha}-stable noise can provide several advantages in modern machine learning problems, both in optimization and sampling contexts. However, as opposed to LMC, only asymptotic convergence properties of FLMC have been yet established. In this study, we analyze the non-asymptotic behavior of FLMC for non-convex optimization and prove finite-time bounds for its expected suboptimality. Our results show that the weak-error of FLMC increases faster than LMC, which suggests using smaller step-sizes in FLMC. We finally extend our results to the case where the exact gradients are replaced by stochastic gradients and show that similar results hold in this setting as well.
Tasks
Published	2019-01-22
URL	http://arxiv.org/abs/1901.07487v1
PDF	http://arxiv.org/pdf/1901.07487v1.pdf
PWC	https://paperswithcode.com/paper/non-asymptotic-analysis-of-fractional
Repo
Framework

Scalable Holistic Linear Regression


Title	Scalable Holistic Linear Regression
Authors	Dimitris Bertsimas, Michael Lingzhi Li
Abstract	We propose a new scalable algorithm for holistic linear regression building on Bertsimas & King (2016). Specifically, we develop new theory to model significance and multicollinearity as lazy constraints rather than checking the conditions iteratively. The resulting algorithm scales with the number of samples $n$ in the 10,000s, compared to the low 100s in the previous framework. Computational results on real and synthetic datasets show it greatly improves from previous algorithms in accuracy, false detection rate, computational time and scalability.
Tasks
Published	2019-02-08
URL	https://arxiv.org/abs/1902.03272v2
PDF	https://arxiv.org/pdf/1902.03272v2.pdf
PWC	https://paperswithcode.com/paper/accounting-for-significance-and
Repo
Framework

Real-time Data Driven Precision Estimator for RAVEN-II Surgical Robot End Effector Position


Title	Real-time Data Driven Precision Estimator for RAVEN-II Surgical Robot End Effector Position
Authors	Haonan Peng, Xingjian Yang, Yun-Hsuan Su, Blake Hannaford
Abstract	Surgical robots have been introduced to operating rooms over the past few decades due to their high sensitivity, small size, and remote controllability. The cable-driven nature of many surgical robots allows the systems to be dexterous and lightweight, with diameters as low as 5mm. However, due to the slack and stretch of the cables and the backlash of the gears, inevitable uncertainties are brought into the kinematics calculation. Since the reported end effector position of surgical robots like RAVEN-II is directly calculated using the motor encoder measurements and forward kinematics, it may contain relatively large error up to 10mm, whereas semi-autonomous functions being introduced into abdominal surgeries require position inaccuracy of at most 1mm. To resolve the problem, a cost-effective, real-time and data-driven pipeline for robot end effector position precision estimation is proposed and tested on RAVEN-II. Analysis shows an improved end effector position error of around 1mm RMS traversing through the entire robot workspace without high-resolution motion tracker.
Tasks
Published	2019-10-14
URL	https://arxiv.org/abs/1910.06425v1
PDF	https://arxiv.org/pdf/1910.06425v1.pdf
PWC	https://paperswithcode.com/paper/real-time-data-driven-precision-estimator-for
Repo
Framework

Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning


Title	Self-Supervised Visual Terrain Classification from Unsupervised Acoustic Feature Learning
Authors	Jannik Zürn, Wolfram Burgard, Abhinav Valada
Abstract	Mobile robots operating in unknown urban environments encounter a wide range of complex terrains to which they must adapt their planned trajectory for safe and efficient navigation. Most existing approaches utilize supervised learning to classify terrains from either an exteroceptive or a proprioceptive sensor modality. However, this requires a tremendous amount of manual labeling effort for each newly encountered terrain as well as for variations of terrains caused by changing environmental conditions. In this work, we propose a novel terrain classification framework leveraging an unsupervised proprioceptive classifier that learns from vehicle-terrain interaction sounds to self-supervise an exteroceptive classifier for pixel-wise semantic segmentation of images. To this end, we first learn a discriminative embedding space for vehicle-terrain interaction sounds from triplets of audio clips formed using visual features of the corresponding terrain patches and cluster the resulting embeddings. We subsequently use these clusters to label the visual terrain patches by projecting the traversed tracks of the robot into the camera images. Finally, we use the sparsely labeled images to train our semantic segmentation network in a weakly supervised manner. We present extensive quantitative and qualitative results that demonstrate that our proprioceptive terrain classifier exceeds the state-of-the-art among unsupervised methods and our self-supervised exteroceptive semantic segmentation model achieves a comparable performance to supervised learning with manually labeled data.
Tasks	Semantic Segmentation
Published	2019-12-06
URL	https://arxiv.org/abs/1912.03227v1
PDF	https://arxiv.org/pdf/1912.03227v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-visual-terrain-classification
Repo
Framework

Inferring 3D Shapes from Image Collections using Adversarial Networks


Title	Inferring 3D Shapes from Image Collections using Adversarial Networks
Authors	Matheus Gadelha, Aartika Rai, Subhransu Maji, Rui Wang
Abstract	We investigate the problem of learning a probabilistic distribution over three-dimensional shapes given two-dimensional views of multiple objects taken from unknown viewpoints. Our approach called projective generative adversarial network (PrGAN) trains a deep generative model of 3D shapes whose projections (or renderings) match the distributions of the provided 2D distribution. The addition of a differentiable projection module allows us to infer the underlying 3D shape distribution without access to any explicit 3D or viewpoint annotation during the learning phase. We show that our approach produces 3D shapes of comparable quality to GANs trained directly on 3D data. %for a number of shape categoriesincluding chairs, airplanes, and cars. Experiments also show that the disentangled representation of 2D shapes into geometry and viewpoint leads to a good generative model of 2D shapes. The key advantage of our model is that it estimates 3D shape, viewpoint, and generates novel views from an input image in a completely unsupervised manner. We further investigate how the generative models can be improved if additional information such as depth, viewpoint or part segmentations is available at training time. To this end, we present new differentiable projection operators that can be used by PrGAN to learn better 3D generative models. Our experiments show that our method can successfully leverage extra visual cues to create more diverse and accurate shapes.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04910v1
PDF	https://arxiv.org/pdf/1906.04910v1.pdf
PWC	https://paperswithcode.com/paper/inferring-3d-shapes-from-image-collections
Repo
Framework

Forecasting intracranial hypertension using multi-scale waveform metrics


Title	Forecasting intracranial hypertension using multi-scale waveform metrics
Authors	Matthias Hüser, Adrian Kündig, Walter Karlen, Valeria De Luca, Martin Jaggi
Abstract	Objective: Acute intracranial hypertension is an important risk factor of secondary brain damage after traumatic brain injury. Hypertensive episodes are often diagnosed reactively, leading to late detection and lost time for intervention planning. A pro-active approach that predicts critical events several hours ahead of time could assist in directing attention to patients at risk. Approach: We developed a prediction framework that forecasts onsets of acute intracranial hypertension in the next 8 hours. It jointly uses cerebral auto-regulation indices, spectral energies and morphological pulse metrics to describe the neurological state of the patient. One-minute base windows were compressed by computing signal metrics, and then stored in a multi-scale history, from which physiological features were derived. Main results: Our model predicted events up to 8 hours in advance with alarm recall rates of 90% at a precision of 30.3% in the MIMIC-III waveform database, improving upon two baselines from the literature. We found that features derived from high-frequency waveforms substantially improved the prediction performance over simple statistical summaries of low-frequency time series, and each of the three feature classes contributed to the performance gain. The inclusion of long-term history up to 8 hours was especially important. Significance: Our results highlight the importance of information contained in high-frequency waveforms in the neurological intensive care unit. They could motivate future studies on pre-hypertensive patterns and the design of new alarm algorithms for critical events in the injured brain.
Tasks	Time Series
Published	2019-02-25
URL	https://arxiv.org/abs/1902.09499v3
PDF	https://arxiv.org/pdf/1902.09499v3.pdf
PWC	https://paperswithcode.com/paper/forecasting-intracranial-hypertension-using
Repo
Framework

Extra-gradient with player sampling for provable fast convergence in n-player games


Title	Extra-gradient with player sampling for provable fast convergence in n-player games
Authors	Samy Jelassi, Carles Domingo Enrich, Damien Scieur, Arthur Mensch, Joan Bruna
Abstract	Data-driven model training is increasingly relying on finding Nash equilibria with provable techniques, e.g., for GANs and multi-agent RL. In this paper, we analyse a new extra-gradient method, that performs gradient extrapolations and updates on a random subset of players at each iteration. This approach provably exhibits the same rate of convergence as full extra-gradient in non-smooth convex games. We propose an additional variance reduction mechanism for this to hold for smooth convex games. Our approach makes extrapolation amenable to massive multiplayer settings, and brings empirical speed-ups, in particular when using cyclic sampling schemes. We demonstrate the efficiency of player sampling on large-scale non-smooth and non-strictly convex games. We show that the joint use of extrapolation and player sampling allows to train better GANs on CIFAR10.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12363v4
PDF	https://arxiv.org/pdf/1905.12363v4.pdf
PWC	https://paperswithcode.com/paper/extra-gradient-with-player-sampling-for
Repo
Framework

LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese


Title	LSTM Easy-first Dependency Parsing with Pre-trained Word Embeddings and Character-level Word Embeddings in Vietnamese
Authors	Binh Duc Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
Abstract	In Vietnamese dependency parsing, several methods have been proposed. Dependency parser which uses deep neural network model has been reported that achieved state-of-the-art results. In this paper, we proposed a new method which applies LSTM easy-first dependency parsing with pre-trained word embeddings and character-level word embeddings. Our method achieves an accuracy of 80.91% of unlabeled attachment score and 72.98% of labeled attachment score on the Vietnamese Dependency Treebank (VnDT).
Tasks	Dependency Parsing, Word Embeddings
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13732v1
PDF	https://arxiv.org/pdf/1910.13732v1.pdf
PWC	https://paperswithcode.com/paper/lstm-easy-first-dependency-parsing-with-pre
Repo
Framework

Binary Classifier Inspired by Quantum Theory


Title	Binary Classifier Inspired by Quantum Theory
Authors	Prayag Tiwari, Massimo Melucci
Abstract	\ac{ML} helps us to recognize patterns from raw data. \ac{ML} is used in numerous domains i.e. biomedical, agricultural, food technology, etc. Despite recent technological advancements, there is still room for substantial improvement in prediction. Current \ac{ML} models are based on classical theories of probability and statistics, which can now be replaced by \ac{QT} with the aim of improving the effectiveness of \ac{ML}. In this paper, we propose the \ac{BCIQT} model, which outperforms the state of the art classification in terms of recall for every category.
Tasks
Published	2019-03-04
URL	http://arxiv.org/abs/1903.01167v1
PDF	http://arxiv.org/pdf/1903.01167v1.pdf
PWC	https://paperswithcode.com/paper/binary-classifier-inspired-by-quantum-theory
Repo
Framework

DeepObfuscator: Adversarial Training Framework for Privacy-Preserving Image Classification


Title	DeepObfuscator: Adversarial Training Framework for Privacy-Preserving Image Classification
Authors	Ang Li, Jiayi Guo, Huanrui Yang, Yiran Chen
Abstract	Deep learning has been widely utilized in many computer vision applications and achieved remarkable commercial success. However, running deep learning models on mobile devices is generally challenging due to limitation of the available computing resources. It is common to let the users send their service requests to cloud servers that run the large-scale deep learning models to process. Sending the data associated with the service requests to the cloud, however, impose risks on the user data privacy. Some prior arts proposed sending the features extracted from raw data (e.g., images) to the cloud. Unfortunately, these extracted features can still be exploited by attackers to recover raw images and to infer embedded private attributes (e.g., age, gender, etc.). In this paper, we propose an adversarial training framework DeepObfuscator that can prevent extracted features from being utilized to reconstruct raw images and infer private attributes, while retaining the useful information for the intended cloud service (i.e., image classification). DeepObfuscator includes a learnable encoder, namely, obfuscator that is designed to hide privacy-related sensitive information from the features by performingour proposed adversarial training algorithm. Our experiments on CelebAdataset show that the quality of the reconstructed images fromthe obfuscated features of the raw image is dramatically decreased from 0.9458 to 0.3175 in terms of multi-scale structural similarity (MS-SSIM). The person in the reconstructed image, hence, becomes hardly to be re-identified. The classification accuracy of the inferred private attributes that can be achieved by the attacker drops down to a random-guessing level, e.g., the accuracy of gender is reduced from 97.36% to 58.85%. As a comparison, the accuracy of the intended classification tasks performed via the cloud service drops by only 2%
Tasks	Image Classification
Published	2019-09-09
URL	https://arxiv.org/abs/1909.04126v1
PDF	https://arxiv.org/pdf/1909.04126v1.pdf
PWC	https://paperswithcode.com/paper/deepobfuscator-adversarial-training-framework
Repo
Framework

Enhancing magic sets with an application to ontological reasoning


Title	Enhancing magic sets with an application to ontological reasoning
Authors	Mario Alviano, Nicola Leone, Pierfrancesco Veltri, Jessica Zangari
Abstract	Magic sets are a Datalog to Datalog rewriting technique to optimize query answering. The rewritten program focuses on a portion of the stable model(s) of the input program which is sufficient to answer the given query. However, the rewriting may introduce new recursive definitions, which can involve even negation and aggregations, and may slow down program evaluation. This paper enhances the magic set technique by preventing the creation of (new) recursive definitions in the rewritten program. It turns out that the new version of magic sets is closed for Datalog programs with stratified negation and aggregations, which is very convenient to obtain efficient computation of the stable model of the rewritten program. Moreover, the rewritten program is further optimized by the elimination of subsumed rules and by the efficient handling of the cases where binding propagation is lost. The research was stimulated by a challenge on the exploitation of Datalog/\textsc{dlv} for efficient reasoning on large ontologies. All proposed techniques have been hence implemented in the \textsc{dlv} system, and tested for ontological reasoning, confirming their effectiveness. Under consideration for publication in Theory and Practice of Logic Programming.
Tasks
Published	2019-07-19
URL	https://arxiv.org/abs/1907.08424v1
PDF	https://arxiv.org/pdf/1907.08424v1.pdf
PWC	https://paperswithcode.com/paper/enhancing-magic-sets-with-an-application-to
Repo
Framework

Ten-year Survival Prediction for Breast Cancer Patients


Title	Ten-year Survival Prediction for Breast Cancer Patients
Authors	Changmao Li, Han He, Yunze Hao, Caleb Ziems
Abstract	This report assesses different machine learning approaches to 10-year survival prediction of breast cancer patients.
Tasks
Published	2019-11-02
URL	https://arxiv.org/abs/1911.00776v1
PDF	https://arxiv.org/pdf/1911.00776v1.pdf
PWC	https://paperswithcode.com/paper/ten-year-survival-prediction-for-breast
Repo
Framework

Zoom To Learn, Learn To Zoom


Title	Zoom To Learn, Learn To Zoom
Authors	Xuaner Cecilia Zhang, Qifeng Chen, Ren Ng, Vladlen Koltun
Abstract	This paper shows that when applying machine learning to digital zoom for photography, it is beneficial to use real, RAW sensor data for training. Existing learning-based super-resolution methods do not use real sensor data, instead operating on RGB images. In practice, these approaches result in loss of detail and accuracy in their digitally zoomed output when zooming in on distant image regions. We also show that synthesizing sensor data by resampling high-resolution RGB images is an oversimplified approximation of real sensor data and noise, resulting in worse image quality. The key barrier to using real sensor data for training is that ground truth high-resolution imagery is missing. We show how to obtain the ground-truth data with optically zoomed images and contribute a dataset, SR-RAW, for real-world computational zoom. We use SR-RAW to train a deep network with a novel contextual bilateral loss (CoBi) that delivers critical robustness to mild misalignment in input-output image pairs. The trained network achieves state-of-the-art performance in 4X and 8X computational zoom.
Tasks	Super-Resolution
Published	2019-05-13
URL	https://arxiv.org/abs/1905.05169v1
PDF	https://arxiv.org/pdf/1905.05169v1.pdf
PWC	https://paperswithcode.com/paper/zoom-to-learn-learn-to-zoom
Repo
Framework