January 30, 2020

3162 words 15 mins read

Paper Group ANR 232

Architecting Dependable Learning-enabled Autonomous Systems: A Survey. Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation. Deep Iterative Frame Interpolation for Full-frame Video Stabilization. Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different …

Architecting Dependable Learning-enabled Autonomous Systems: A Survey


Title	Architecting Dependable Learning-enabled Autonomous Systems: A Survey
Authors	Chih-Hong Cheng, Dhiraj Gulati, Rongjie Yan
Abstract	We provide a summary over architectural approaches that can be used to construct dependable learning-enabled autonomous systems, with a focus on automated driving. We consider three technology pillars for architecting dependable autonomy, namely diverse redundancy, information fusion, and runtime monitoring. For learning-enabled components, we additionally summarize recent architectural approaches to increase the dependability beyond standard convolutional neural networks. We conclude the study with a list of promising research directions addressing the challenges of existing approaches.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10590v1
PDF	http://arxiv.org/pdf/1902.10590v1.pdf
PWC	https://paperswithcode.com/paper/architecting-dependable-learning-enabled
Repo
Framework

Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation


Title	Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation
Authors	Avishek Ghosh, Ashwin Pananjady, Adityanand Guntuboyina, Kannan Ramchandran
Abstract	Max-affine regression refers to a model where the unknown regression function is modeled as a maximum of $k$ unknown affine functions for a fixed $k \geq 1$. This generalizes linear regression and (real) phase retrieval, and is closely related to convex regression. Working within a non-asymptotic framework, we study this problem in the high-dimensional setting assuming that $k$ is a fixed constant, and focus on estimation of the unknown coefficients of the affine functions underlying the model. We analyze a natural alternating minimization (AM) algorithm for the non-convex least squares objective when the design is random. We show that the AM algorithm, when initialized suitably, converges with high probability and at a geometric rate to a small ball around the optimal coefficients. In order to initialize the algorithm, we propose and analyze a combination of a spectral method and a random search scheme in a low-dimensional space, which may be of independent interest. The final rate that we obtain is near-parametric and minimax optimal (up to a poly-logarithmic factor) as a function of the dimension, sample size, and noise variance. In that sense, our approach should be viewed as a direct and implementable method of enforcing regularization to alleviate the curse of dimensionality in problems of the convex regression type. As a by-product of our analysis, we also obtain guarantees on a classical algorithm for the phase retrieval problem under considerably weaker assumptions on the design distribution than was previously known. Numerical experiments illustrate the sharpness of our bounds in the various problem parameters.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.09255v1
PDF	https://arxiv.org/pdf/1906.09255v1.pdf
PWC	https://paperswithcode.com/paper/max-affine-regression-provable-tractable-and
Repo
Framework

Deep Iterative Frame Interpolation for Full-frame Video Stabilization


Title	Deep Iterative Frame Interpolation for Full-frame Video Stabilization
Authors	Jinsoo Choi, In So Kweon
Abstract	Video stabilization is a fundamental and important technique for higher quality videos. Prior works have extensively explored video stabilization, but most of them involve cropping of the frame boundaries and introduce moderate levels of distortion. We present a novel deep approach to video stabilization which can generate video frames without cropping and low distortion. The proposed framework utilizes frame interpolation techniques to generate in between frames, leading to reduced inter-frame jitter. Once applied in an iterative fashion, the stabilization effect becomes stronger. A major advantage is that our framework is end-to-end trainable in an unsupervised manner. In addition, our method is able to run in near real-time (15 fps). To the best of our knowledge, this is the first work to propose an unsupervised deep approach to full-frame video stabilization. We show the advantages of our method through quantitative and qualitative evaluations comparing to the state-of-the-art methods.
Tasks
Published	2019-09-05
URL	https://arxiv.org/abs/1909.02641v1
PDF	https://arxiv.org/pdf/1909.02641v1.pdf
PWC	https://paperswithcode.com/paper/deep-iterative-frame-interpolation-for-full
Repo
Framework

Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract


Title	Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract
Authors	Marc Aubreville, Miguel Goncalves, Christian Knipfer, Nicolai Oetter, Helmut Neumann, Florian Stelzle, Christopher Bohr, Andreas Maier
Abstract	Squamous Cell Carcinoma (SCC) is the most common cancer type of the epithelium and is often detected at a late stage. Besides invasive diagnosis of SCC by means of biopsy and histo-pathologic assessment, Confocal Laser Endomicroscopy (CLE) has emerged as noninvasive method that was successfully used to diagnose SCC in vivo. For interpretation of CLE images, however, extensive training is required, which limits its applicability and use in clinical practice of the method. To aid diagnosis of SCC in a broader scope, automatic detection methods have been proposed. This work compares two methods with regard to their applicability in a transfer learning sense, i.e. training on one tissue type (from one clinical team) and applying the learnt classification system to another entity (different anatomy, different clinical team). Besides a previously proposed, patch-based method based on convolutional neural networks, a novel classification method on image level (based on a pre-trained Inception V.3 network with dedicated preprocessing and interpretation of class activation maps) is proposed and evaluated. The newly presented approach improves recognition performance, yielding accuracies of 91.63% on the first data set (oral cavity) and 92.63% on a joint data set. The generalization from oral cavity to the second data set (vocal folds) lead to similar area-under-the-ROC curve values than a direct training on the vocal folds data set, indicating good generalization.
Tasks	Transfer Learning
Published	2019-02-24
URL	https://arxiv.org/abs/1902.08985v2
PDF	https://arxiv.org/pdf/1902.08985v2.pdf
PWC	https://paperswithcode.com/paper/transferability-of-deep-learning-algorithms
Repo
Framework


Title	Model enhancement and personalization using weakly supervised learning for multi-modal mobile sensing
Authors	Diyan Teng, Rashmi Kulkarni, Justin McGloin
Abstract	Always-on sensing of mobile device user’s contextual information is critical to many intelligent use cases nowadays such as healthcare, drive assistance, voice UI. State-of-the-art approaches for predicting user context have proved the value to leverage multiple sensing modalities for better accuracy. However, those context inference algorithms that run on application processor nowadays tend to drain heavy amount of power, making them not suitable for an always-on implementation. We claim that not every sensing modality is suitable to be activated all the time and it remains challenging to build an inference engine using power friendly sensing modalities. Meanwhile, due to the diverse population, we find it challenging to learn a context inference model that generalizes well, with limited training data, especially when only using always-on low power sensors. In this work, we propose an approach to leverage the opportunistically-on counterparts in device to improve the always-on prediction model, leading to a personalized solution. We model this problem using a weakly supervised learning framework and provide both theoretical and experimental results to validate our design. The proposed framework achieves satisfying result in the IMU based activity recognition application we considered.
Tasks	Activity Recognition
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13401v1
PDF	https://arxiv.org/pdf/1910.13401v1.pdf
PWC	https://paperswithcode.com/paper/191013401
Repo
Framework

Formal models of Structure Building in Music, Language and Animal Songs


Title	Formal models of Structure Building in Music, Language and Animal Songs
Authors	Willem Zuidema, Dieuwke Hupkes, Geraint Wiggins, Constance Scharff, Martin Rohrmeier
Abstract	Human language, music and a variety of animal vocalisations constitute ways of sonic communication that exhibit remarkable structural complexity. While the complexities of language and possible parallels in animal communication have been discussed intensively, reflections on the complexity of music and animal song, and their comparisons are underrepresented. In some ways, music and animal songs are more comparable to each other than to language, as propositional semantics cannot be used as as indicator of communicative success or well-formedness, and notions of grammaticality are less easily defined. This review brings together accounts of the principles of structure building in language, music and animal song, relating them to the corresponding models in formal language theory, with a special focus on evaluating the benefits of using the Chomsky hierarchy (CH). We further discuss common misunderstandings and shortcomings concerning the CH, as well as extensions or augmentations of it that address some of these issues, and suggest ways to move beyond.
Tasks
Published	2019-01-16
URL	http://arxiv.org/abs/1901.05180v1
PDF	http://arxiv.org/pdf/1901.05180v1.pdf
PWC	https://paperswithcode.com/paper/formal-models-of-structure-building-in-music
Repo
Framework

Provable Tensor Ring Completion


Title	Provable Tensor Ring Completion
Authors	Huyan Huang, Yipeng Liu, Ce Zhu
Abstract	Tensor completion recovers a multi-dimensional array from a limited number of measurements. Using the recently proposed tensor ring (TR) decomposition, in this paper we show that a d-order tensor of dimensional size n and TR rank r can be exactly recovered with high probability by solving a convex optimization program, given n^{d/2} r^2 ln^7(n^{d/2})samples. The proposed TR incoherence condition under which the result holds is similar to the matrix incoherence condition. The experiments on synthetic data verify the recovery guarantee for TR completion. Moreover, the experiments on real-world data show that our method improves the recovery performance compared with the state-of-the-art methods.
Tasks	Matrix Completion
Published	2019-03-08
URL	https://arxiv.org/abs/1903.03315v6
PDF	https://arxiv.org/pdf/1903.03315v6.pdf
PWC	https://paperswithcode.com/paper/low-rank-tensor-completion-via-tensor-ring
Repo
Framework

Bayesian policy selection using active inference


Title	Bayesian policy selection using active inference
Authors	Ozan Çatal, Johannes Nauta, Tim Verbelen, Pieter Simoens, Bart Dhoedt
Abstract	Learning to take actions based on observations is a core requirement for artificial agents to be able to be successful and robust at their task. Reinforcement Learning (RL) is a well-known technique for learning such policies. However, current RL algorithms often have to deal with reward shaping, have difficulties generalizing to other environments and are most often sample inefficient. In this paper, we explore active inference and the free energy principle, a normative theory from neuroscience that explains how self-organizing biological systems operate by maintaining a model of the world and casting action selection as an inference problem. We apply this concept to a typical problem known to the RL community, the mountain car problem, and show how active inference encompasses both RL and learning from demonstrations.
Tasks
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08149v2
PDF	http://arxiv.org/pdf/1904.08149v2.pdf
PWC	https://paperswithcode.com/paper/bayesian-policy-selection-using-active
Repo
Framework

A Generalizable Method for Automated Quality Control of Functional Neuroimaging Datasets


Title	A Generalizable Method for Automated Quality Control of Functional Neuroimaging Datasets
Authors	Matthew Kollada, Qingzhu Gao, Monika S Mellem, Tathagata Banerjee, William J Martin
Abstract	Over the last twenty five years, advances in the collection and analysis of fMRI data have enabled new insights into the brain basis of human health and disease. Individual behavioral variation can now be visualized at a neural level as patterns of connectivity among brain regions. Functional brain imaging is enhancing our understanding of clinical psychiatric disorders by revealing ties between regional and network abnormalities and psychiatric symptoms. Initial success in this arena has recently motivated collection of larger datasets which are needed to leverage fMRI to generate brain-based biomarkers to support development of precision medicines. Despite methodological advances and enhanced computational power, evaluating the quality of fMRI scans remains a critical step in the analytical framework. Before analysis can be performed, expert reviewers visually inspect raw scans and preprocessed derivatives to determine viability of the data. This Quality Control (QC) process is labor intensive, and the inability to automate at large scale has proven to be a limiting factor in clinical neuroscience fMRI research. We present a novel method for automating the QC of fMRI scans. We train machine learning classifiers using features derived from brain MR images to predict the “quality” of those images, based on the ground truth of an expert’s opinion. We emphasize the importance of these classifiers’ ability to generalize their predictions across data from different studies. To address this, we propose a novel approach entitled “FMRI preprocessing Log mining for Automated, Generalizable Quality Control” (FLAG-QC), in which features derived from mining runtime logs are used to train the classifier. We show that classifiers trained on FLAG-QC features perform much better (AUC=0.79) than previously proposed feature sets (AUC=0.56) when testing their ability to generalize across studies.
Tasks
Published	2019-12-20
URL	https://arxiv.org/abs/1912.10127v1
PDF	https://arxiv.org/pdf/1912.10127v1.pdf
PWC	https://paperswithcode.com/paper/a-generalizable-method-for-automated-quality
Repo
Framework

A Non-linear Differential CNN-Rendering Module for 3D Data Enhancement


Title	A Non-linear Differential CNN-Rendering Module for 3D Data Enhancement
Authors	Yonatan Svirsky, Andrei Sharf
Abstract	In this work we introduce a differential rendering module which allows neural networks to efficiently process cluttered data. The module is composed of continuous piecewise differentiable functions defined as a sensor array of cells embedded in 3D space. Our module is learnable and can be easily integrated into neural networks allowing to optimize data rendering towards specific learning tasks using gradient based methods in an end-to-end fashion. Essentially, the module’s sensor cells are allowed to transform independently and locally focus and sense different parts of the 3D data. Thus, through their optimization process, cells learn to focus on important parts of the data, bypassing occlusions, clutter and noise. Since sensor cells originally lie on a grid, this equals to a highly non-linear rendering of the scene into a 2D image. Our module performs especially well in presence of clutter and occlusions. Similarly, it deals well with non-linear deformations and improves classification accuracy through proper rendering of the data. In our experiments, we apply our module to demonstrate efficient localization and classification tasks in cluttered data both 2D and 3D.
Tasks
Published	2019-04-09
URL	http://arxiv.org/abs/1904.04850v1
PDF	http://arxiv.org/pdf/1904.04850v1.pdf
PWC	https://paperswithcode.com/paper/a-non-linear-differential-cnn-rendering
Repo
Framework

High-dimensional Black-box Optimization Under Uncertainty


Title	High-dimensional Black-box Optimization Under Uncertainty
Authors	Hadis Anahideh, Jay Rosenberger, Victoria Chen
Abstract	Limited informative data remains the primary challenge for optimization the expensive complex systems. Learning from limited data and finding the set of variables that optimizes an expected output arise practically in the design problems. In such situations, the underlying function is complex yet unknown, a large number of variables are involved though not all of them are important, and the interactions between the variables are significant. On the other hand, it is usually expensive to collect more data and the outcome is under uncertainty. Unfortunately, despite being real-world challenges, exiting works have not addressed these jointly. We propose a new surrogate optimization approach in this article to tackle these challenges. We design a flexible, non-interpolating, and parsimonious surrogate model using a partitioning technique. The proposed model bends at near-optimal locations and identifies the peaks and valleys for optimization purposes. To discover new candidate points an exploration-exploitation Pareto method is implemented as a sampling strategy. Furthermore, we develop a smart replication approach based on hypothesis testing to overcome the uncertainties associated with the black-box outcome. The Smart-Replication approach identifies promising points to replicate rather than wasting evaluation on less informative data points. We conduct a comprehensive set of experiments on challenging global optimization test functions to evaluate the performance of our proposal.
Tasks
Published	2019-11-06
URL	https://arxiv.org/abs/1911.02457v2
PDF	https://arxiv.org/pdf/1911.02457v2.pdf
PWC	https://paperswithcode.com/paper/high-dimensional-black-box-optimization-under
Repo
Framework

A review on deep learning techniques for 3D sensed data classification


Title	A review on deep learning techniques for 3D sensed data classification
Authors	David Griffiths, Jan Boehm
Abstract	Over the past decade deep learning has driven progress in 2D image understanding. Despite these advancements, techniques for automatic 3D sensed data understanding, such as point clouds, is comparatively immature. However, with a range of important applications from indoor robotics navigation to national scale remote sensing there is a high demand for algorithms that can learn to automatically understand and classify 3D sensed data. In this paper we review the current state-of-the-art deep learning architectures for processing unstructured Euclidean data. We begin by addressing the background concepts and traditional methodologies. We review the current main approaches including; RGB-D, multi-view, volumetric and fully end-to-end architecture designs. Datasets for each category are documented and explained. Finally, we give a detailed discussion about the future of deep learning for 3D sensed data, using literature to justify the areas where future research would be most valuable.
Tasks
Published	2019-07-09
URL	https://arxiv.org/abs/1907.04444v1
PDF	https://arxiv.org/pdf/1907.04444v1.pdf
PWC	https://paperswithcode.com/paper/a-review-on-deep-learning-techniques-for-3d
Repo
Framework

Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging


Title	Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging
Authors	Luis Muñoz-González, Kenneth T. Co, Emil C. Lupu
Abstract	Federated learning enables training collaborative machine learning models at scale with many participants whilst preserving the privacy of their datasets. Standard federated learning techniques are vulnerable to Byzantine failures, biased local datasets, and poisoning attacks. In this paper we introduce Adaptive Federated Averaging, a novel algorithm for robust federated learning that is designed to detect failures, attacks, and bad updates provided by participants in a collaborative model. We propose a Hidden Markov Model to model and learn the quality of model updates provided by each participant during training. In contrast to existing robust federated learning schemes, we propose a robust aggregation rule that detects and discards bad or malicious local model updates at each training iteration. This includes a mechanism that blocks unwanted participants, which also increases the computational and communication efficiency. Our experimental evaluation on 4 real datasets show that our algorithm is significantly more robust to faulty, noisy and malicious participants, whilst being computationally more efficient than other state-of-the-art robust federated learning methods such as Multi-KRUM and coordinate-wise median.
Tasks
Published	2019-09-11
URL	https://arxiv.org/abs/1909.05125v1
PDF	https://arxiv.org/pdf/1909.05125v1.pdf
PWC	https://paperswithcode.com/paper/byzantine-robust-federated-machine-learning
Repo
Framework

Incrementally Learning Functions of the Return


Title	Incrementally Learning Functions of the Return
Authors	Brendan Bennett, Wesley Chung, Muhammad Zaheer, Vincent Liu
Abstract	Temporal difference methods enable efficient estimation of value functions in reinforcement learning in an incremental fashion, and are of broader interest because they correspond learning as observed in biological systems. Standard value functions correspond to the expected value of a sum of discounted returns. While this formulation is often sufficient for many purposes, it would often be useful to be able to represent functions of the return as well. Unfortunately, most such functions cannot be estimated directly using TD methods. We propose a means of estimating functions of the return using its moments, which can be learned online using a modified TD algorithm. The moments of the return are then used as part of a Taylor expansion to approximate analytic functions of the return.
Tasks
Published	2019-07-05
URL	https://arxiv.org/abs/1907.04651v1
PDF	https://arxiv.org/pdf/1907.04651v1.pdf
PWC	https://paperswithcode.com/paper/incrementally-learning-functions-of-the
Repo
Framework

Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic


Title	Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
Authors	Mikael Henaff, Alfredo Canziani, Yann LeCun
Abstract	Learning a policy using only observational data is challenging because the distribution of states it induces at execution time may differ from the distribution observed during training. We propose to train a policy by unrolling a learned model of the environment dynamics over multiple time steps while explicitly penalizing two costs: the original cost the policy seeks to optimize, and an uncertainty cost which represents its divergence from the states it is trained on. We measure this second cost by using the uncertainty of the dynamics model about its own predictions, using recent ideas from uncertainty estimation for deep networks. We evaluate our approach using a large-scale observational dataset of driving behavior recorded from traffic cameras, and show that we are able to learn effective driving policies from purely observational data, with no environment interaction.
Tasks
Published	2019-01-08
URL	http://arxiv.org/abs/1901.02705v1
PDF	http://arxiv.org/pdf/1901.02705v1.pdf
PWC	https://paperswithcode.com/paper/model-predictive-policy-learning-with
Repo
Framework