January 31, 2020

2952 words 14 mins read

Paper Group ANR 196

An Online Sample Based Method for Mode Estimation using ODE Analysis of Stochastic Approximation Algorithms. Instance-wise Depth and Motion Learning from Monocular Videos. Database Meets Deep Learning: Challenges and Opportunities. Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems. Gated Linear Networks. To …

An Online Sample Based Method for Mode Estimation using ODE Analysis of Stochastic Approximation Algorithms


Title	An Online Sample Based Method for Mode Estimation using ODE Analysis of Stochastic Approximation Algorithms
Authors	Chandramouli Kamanchi, Raghuram Bharadwaj Diddigi, Prabuchandran K. J., Shalabh Bhatnagar
Abstract	One of the popular measures of central tendency that provides better representation and interesting insights of the data compared to the other measures like mean and median is the metric mode. If the analytical form of the density function is known, mode is an argument of the maximum value of the density function and one can apply the optimization techniques to find mode. In many of the practical applications, the analytical form of the density is not known and only the samples from the distribution are available. Most of the techniques proposed in the literature for estimating the mode from the samples assume that all the samples are available beforehand. Moreover, some of the techniques employ computationally expensive operations like sorting. In this work we provide a computationally effective, on-line iterative algorithm that estimates the mode of a unimodal smooth density given only the samples generated from the density. Asymptotic convergence of the proposed algorithm using an ordinary differential equation (ODE) based analysis is provided. We also prove the stability of estimates by utilizing the concept of regularization. Experimental results further demonstrate the effectiveness of the proposed algorithm.
Tasks
Published	2019-02-11
URL	https://arxiv.org/abs/1902.03806v2
PDF	https://arxiv.org/pdf/1902.03806v2.pdf
PWC	https://paperswithcode.com/paper/an-online-sample-based-method-for-mode
Repo
Framework

Instance-wise Depth and Motion Learning from Monocular Videos


Title	Instance-wise Depth and Motion Learning from Monocular Videos
Authors	Seokju Lee, Sunghoon Im, Stephen Lin, In So Kweon
Abstract	We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision. The only annotation used in our pipeline is a video instance segmentation map that can be predicted by our new auto-annotation scheme. Our technical contributions are three-fold. First, we propose a differentiable forward rigid projection module that plays a key role in our instance-wise depth and motion learning. Second, we design an instance-wise photometric and geometric consistency loss that effectively decomposes background and moving object regions. Lastly, we introduce an instance-wise mini-batch re-arrangement scheme that does not require additional iterations in training. These proposed elements are validated in a detailed ablation study. Through extensive experiments conducted on the KITTI dataset, our framework is shown to outperform the state-of-the-art depth and motion estimation methods.
Tasks	Instance Segmentation, Motion Estimation, Semantic Segmentation
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09351v1
PDF	https://arxiv.org/pdf/1912.09351v1.pdf
PWC	https://paperswithcode.com/paper/instance-wise-depth-and-motion-learning-from
Repo
Framework

Database Meets Deep Learning: Challenges and Opportunities


Title	Database Meets Deep Learning: Challenges and Opportunities
Authors	Wei Wang, Meihui Zhang, Gang Chen, H. V. Jagadish, Beng Chin Ooi, Kian-Lee Tan
Abstract	Deep learning has recently become very popular on account of its incredible success in many complex data-driven applications, such as image classification and speech recognition. The database community has worked on data-driven applications for many years, and therefore should be playing a lead role in supporting this new wave. However, databases and deep learning are different in terms of both techniques and applications. In this paper, we discuss research problems at the intersection of the two fields. In particular, we discuss possible improvements for deep learning systems from a database perspective, and analyze database applications that may benefit from deep learning techniques.
Tasks	Image Classification, Speech Recognition
Published	2019-06-21
URL	https://arxiv.org/abs/1906.08986v2
PDF	https://arxiv.org/pdf/1906.08986v2.pdf
PWC	https://paperswithcode.com/paper/database-meets-deep-learning-challenges-and
Repo
Framework

Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems


Title	Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
Authors	Elizabeth Qian, Boris Kramer, Benjamin Peherstorfer, Karen Willcox
Abstract	We present Lift & Learn, a physics-informed method for learning low-dimensional models for large-scale dynamical systems. The method exploits knowledge of a system’s governing equations to identify a coordinate transformation in which the system dynamics have quadratic structure. This transformation is called a lifting map because it often adds auxiliary variables to the system state. The lifting map is applied to data obtained by evaluating a model for the original nonlinear system. This lifted data is projected onto its leading principal components, and low-dimensional linear and quadratic matrix operators are fit to the lifted reduced data using a least-squares operator inference procedure. Analysis of our method shows that the Lift & Learn models are able to capture the system physics in the lifted coordinates at least as accurately as traditional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numerical experiments on the FitzHugh-Nagumo neuron activation model and the compressible Euler equations demonstrate the generalizability of our model.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08177v5
PDF	https://arxiv.org/pdf/1912.08177v5.pdf
PWC	https://paperswithcode.com/paper/lift-learn-physics-informed-machine-learning
Repo
Framework

Gated Linear Networks


Title	Gated Linear Networks
Authors	Joel Veness, Tor Lattimore, Avishkar Bhoopchand, David Budden, Christopher Mattern, Agnieszka Grabska-Barwinska, Peter Toth, Simon Schmitt, Marcus Hutter
Abstract	This paper presents a family of backpropagation-free neural architectures, Gated Linear Networks (GLNs),that are well suited to online learning applications where sample efficiency is of paramount importance. The impressive empirical performance of these architectures has long been known within the data compression community, but a theoretically satisfying explanation as to how and why they perform so well has proven difficult. What distinguishes these architectures from other neural systems is the distributed and local nature of their credit assignment mechanism; each neuron directly predicts the target and has its own set of hard-gated weights that are locally adapted via online convex optimization. By providing an interpretation, generalization and subsequent theoretical analysis, we show that sufficiently large GLNs are universal in a strong sense: not only can they model any compactly supported, continuous density function to arbitrary accuracy, but that any choice of no-regret online convex optimization technique will provably converge to the correct solution with enough data. Empirically we show a collection of single-pass learning results on established machine learning benchmarks that are competitive with results obtained with general purpose batch learning techniques.
Tasks
Published	2019-09-30
URL	https://arxiv.org/abs/1910.01526v1
PDF	https://arxiv.org/pdf/1910.01526v1.pdf
PWC	https://paperswithcode.com/paper/gated-linear-networks
Repo
Framework

Topological signature for periodic motion recognition


Title	Topological signature for periodic motion recognition
Authors	Javier Lamar-Leon, Rocio Gonzalez-Diaz, Edel Garcia-Reyes
Abstract	In this paper, we present an algorithm that computes the topological signature for a given periodic motion sequence. Such signature consists of a vector obtained by persistent homology which captures the topological and geometric changes of the object that models the motion. Two topological signatures are compared simply by the angle between the corresponding vectors. With respect to gait recognition, we have tested our method using only the lowest fourth part of the body’s silhouette. In this way, the impact of variations in the upper part of the body, which are very frequent in real scenarios, decreases considerably. We have also tested our method using other periodic motions such as running or jumping. Finally, we formally prove that our method is robust to small perturbations in the input data and does not depend on the number of periods contained in the periodic motion sequence.
Tasks	Gait Recognition
Published	2019-04-11
URL	http://arxiv.org/abs/1904.06210v1
PDF	http://arxiv.org/pdf/1904.06210v1.pdf
PWC	https://paperswithcode.com/paper/topological-signature-for-periodic-motion
Repo
Framework

Marginalized State Distribution Entropy Regularization in Policy Optimization


Title	Marginalized State Distribution Entropy Regularization in Policy Optimization
Authors	Riashat Islam, Zafarali Ahmed, Doina Precup
Abstract	Entropy regularization is used to get improved optimization performance in reinforcement learning tasks. A common form of regularization is to maximize policy entropy to avoid premature convergence and lead to more stochastic policies for exploration through action space. However, this does not ensure exploration in the state space. In this work, we instead consider the distribution of discounted weighting of states, and propose to maximize the entropy of a lower bound approximation to the weighting of a state, based on latent space state representation. We propose entropy regularization based on the marginal state distribution, to encourage the policy to have a more uniform distribution over the state space for exploration. Our approach based on marginal state distribution achieves superior state space coverage on complex gridworld domains, that translate into empirical gains in sparse reward 3D maze navigation and continuous control domains compared to entropy regularization with stochastic policies.
Tasks	Continuous Control
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05128v1
PDF	https://arxiv.org/pdf/1912.05128v1.pdf
PWC	https://paperswithcode.com/paper/marginalized-state-distribution-entropy
Repo
Framework

ML Health: Fitness Tracking for Production Models


Title	ML Health: Fitness Tracking for Production Models
Authors	Sindhu Ghanta, Sriram Subramanian, Lior Khermosh, Swaminathan Sundararaman, Harshil Shah, Yakov Goldberg, Drew Roselli, Nisha Talagala
Abstract	Deployment of machine learning (ML) algorithms in production for extended periods of time has uncovered new challenges such as monitoring and management of real-time prediction quality of a model in the absence of labels. However, such tracking is imperative to prevent catastrophic business outcomes resulting from incorrect predictions. The scale of these deployments makes manual monitoring prohibitive, making automated techniques to track and raise alerts imperative. We present a framework, ML Health, for tracking potential drops in the predictive performance of ML models in the absence of labels. The framework employs diagnostic methods to generate alerts for further investigation. We develop one such method to monitor potential problems when production data patterns do not match training data distributions. We demonstrate that our method performs better than standard “distance metrics”, such as RMSE, KL-Divergence, and Wasserstein at detecting issues with mismatched data sets. Finally, we present a working system that incorporates the ML Health approach to monitor and manage ML deployments within a realistic full production ML lifecycle.
Tasks
Published	2019-02-07
URL	http://arxiv.org/abs/1902.02808v1
PDF	http://arxiv.org/pdf/1902.02808v1.pdf
PWC	https://paperswithcode.com/paper/ml-health-fitness-tracking-for-production
Repo
Framework

Deep autofocus with cone-beam CT consistency constraint


Title	Deep autofocus with cone-beam CT consistency constraint
Authors	Alexander Preuhs, Michael Manhart, Philipp Roser, Bernhard Stimpel, Christopher Syben, Marios Psychogios, Markus Kowarschik, Andreas Maier
Abstract	High quality reconstruction with interventional C-arm cone-beam computed tomography (CBCT) requires exact geometry information. If the geometry information is corrupted, e. g., by unexpected patient or system movement, the measured signal is misplaced in the backprojection operation. With prolonged acquisition times of interventional C-arm CBCT the likelihood of rigid patient motion increases. To adapt the backprojection operation accordingly, a motion estimation strategy is necessary. Recently, a novel learning-based approach was proposed, capable of compensating motions within the acquisition plane. We extend this method by a CBCT consistency constraint, which was proven to be efficient for motions perpendicular to the acquisition plane. By the synergistic combination of these two measures, in and out-plane motion is well detectable, achieving an average artifact suppression of 93 [percent]. This outperforms the entropy-based state-of-the-art autofocus measure which achieves on average an artifact suppression of 54 [percent].
Tasks	Motion Estimation
Published	2019-11-29
URL	https://arxiv.org/abs/1911.13162v3
PDF	https://arxiv.org/pdf/1911.13162v3.pdf
PWC	https://paperswithcode.com/paper/deep-autofocus-with-cone-beam-ct-consistency
Repo
Framework

Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control


Title	Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control
Authors	Inyoung Paik, Sangjun Oh, Tae-Yeong Kwak, Injung Kim
Abstract	To address the issue of catastrophic forgetting in neural networks, we propose a novel, simple, and effective solution called neuron-level plasticity control (NPC). While learning a new task, the proposed method preserves the knowledge for the previous tasks by controlling the plasticity of the network at the neuron level. NPC estimates the importance value of each neuron and consolidates important \textit{neurons} by applying lower learning rates, rather than restricting individual connection weights to stay close to certain values. The experimental results on the incremental MNIST (iMNIST) and incremental CIFAR100 (iCIFAR100) datasets show that neuron-level consolidation is substantially more effective compared to the connection-level consolidation approaches.
Tasks
Published	2019-07-31
URL	https://arxiv.org/abs/1907.13322v1
PDF	https://arxiv.org/pdf/1907.13322v1.pdf
PWC	https://paperswithcode.com/paper/overcoming-catastrophic-forgetting-by-neuron
Repo
Framework

Spatial-Aware GAN for Unsupervised Person Re-identification


Title	Spatial-Aware GAN for Unsupervised Person Re-identification
Authors	Fangneng Zhan, Shijian Lu, Aoran Xiao
Abstract	The recent person re-identification research has achieved great success by learning from a large number of labeled person images. On the other hand, the learned models often experience significant performance drops when applied to images collected in a different environment. Unsupervised domain adaptation (UDA) has been investigated to mitigate this constraint, but most existing systems adapt images at pixel level only and ignore obvious discrepancies at spatial level. This paper presents an innovative UDA-based person re-identification network that is capable of adapting images at both spatial and pixel levels simultaneously. A novel disentangled cycle-consistency loss is designed which guides the learning of spatial-level and pixel-level adaptation in a collaborative manner. In addition, a novel multi-modal mechanism is incorporated which is capable of generating images of different geometry views and augmenting training images effectively. Extensive experiments over a number of public datasets show that the proposed UDA network achieves superior person re-identification performance as compared with the state-of-the-art.
Tasks	Domain Adaptation, Person Re-Identification, Unsupervised Domain Adaptation, Unsupervised Person Re-Identification
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11312v1
PDF	https://arxiv.org/pdf/1911.11312v1.pdf
PWC	https://paperswithcode.com/paper/spatial-aware-gan-for-unsupervised-person-re
Repo
Framework

GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception


Title	GoodNewsEveryone: A Corpus of News Headlines Annotated with Emotions, Semantic Roles, and Reader Perception
Authors	Laura Bostan, Evgeny Kim, Roman Klinger
Abstract	Most research on emotion analysis from text focuses on the task of emotion classification or emotion intensity regression. Fewer works address emotions as a phenomenon to be tackled with structured learning, which can be explained by the lack of relevant datasets. We fill this gap by releasing a dataset of 5000 English news headlines annotated via crowdsourcing with their associated emotions, the corresponding emotion experiencers and textual cues, related emotion causes and targets, as well as the reader’s perception of the emotion of the headline. This annotation task is comparably challenging, given the large number of classes and roles to be identified. We therefore propose a multiphase annotation procedure in which we first find relevant instances with emotional content and then annotate the more fine-grained aspects. Finally, we develop a baseline for the task of automatic prediction of semantic role structures and discuss the results. The corpus we release enables further research on emotion classification, emotion intensity prediction, emotion cause detection, and supports further qualitative studies.
Tasks	Emotion Classification, Emotion Recognition
Published	2019-12-06
URL	https://arxiv.org/abs/1912.03184v3
PDF	https://arxiv.org/pdf/1912.03184v3.pdf
PWC	https://paperswithcode.com/paper/goodnewseveryone-a-corpus-of-news-headlines
Repo
Framework

A Survey on the Use of Preferences for Virtual Machine Placement in Cloud Data Centers


Title	A Survey on the Use of Preferences for Virtual Machine Placement in Cloud Data Centers
Authors	Abdulaziz Alashaikh, Eisa Alanazi, Ala Al-Fuqaha
Abstract	With the rapid development of virtualization techniques, cloud data centers allow for cost effective, flexible, and customizable deployments of applications on virtualized infrastructure. Virtual machine (VM) placement aims to assign each virtual machine to a server in the cloud environment. VM Placement is of paramount importance to the design of cloud data centers. Typically, VM placement involves complex relations and multiple design factors as well as local policies that govern the assignment decisions. It also involves different constituents including cloud administrators and customers that might have disparate preferences while opting for a placement solution. Thus, it is often valuable to not only return an optimized solution to the VM placement problem but also a solution that reflects the given preferences of the constituents. In this paper, we provide a detailed review on the role of preferences in the recent literature on VM placement. We further discuss key challenges and identify possible research opportunities to better incorporate preferences within the context of VM placement.
Tasks
Published	2019-07-17
URL	https://arxiv.org/abs/1907.07778v3
PDF	https://arxiv.org/pdf/1907.07778v3.pdf
PWC	https://paperswithcode.com/paper/a-survey-on-the-use-of-preferences-for
Repo
Framework

Unsupervised Few-shot Learning via Self-supervised Training


Title	Unsupervised Few-shot Learning via Self-supervised Training
Authors	Zilong Ji, Xiaolong Zou, Tiejun Huang, Si Wu
Abstract	Learning from limited exemplars (few-shot learning) is a fundamental, unsolved problem that has been laboriously explored in the machine learning community. However, current few-shot learners are mostly supervised and rely heavily on a large amount of labeled examples. Unsupervised learning is a more natural procedure for cognitive mammals and has produced promising results in many machine learning tasks. In the current study, we develop a method to learn an unsupervised few-shot learner via self-supervised training (UFLST), which can effectively generalize to novel but related classes. The proposed model consists of two alternate processes, progressive clustering and episodic training. The former generates pseudo-labeled training examples for constructing episodic tasks; and the later trains the few-shot learner using the generated episodic tasks which further optimizes the feature representations of data. The two processes facilitate with each other, and eventually produce a high quality few-shot learner. Using the benchmark dataset Omniglot and Mini-ImageNet, we show that our model outperforms other unsupervised few-shot learning methods. Using the benchmark dataset Market1501, we further demonstrate the feasibility of our model to a real-world application on person re-identification.
Tasks	Few-Shot Learning, Omniglot, Person Re-Identification
Published	2019-12-20
URL	https://arxiv.org/abs/1912.12178v1
PDF	https://arxiv.org/pdf/1912.12178v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-few-shot-learning-via-self-1
Repo
Framework

Sound source ranging using a feed-forward neural network with fitting-based early stopping


Title	Sound source ranging using a feed-forward neural network with fitting-based early stopping
Authors	Jing Chi, Xiaolei Li, Haozhong Wang, Dazhi Gao, Peter Gerstoft
Abstract	When a feed-forward neural network (FNN) is trained for source ranging in an ocean waveguide, it is difficult evaluating the range accuracy of the FNN on unlabeled test data. A fitting-based early stopping (FEAST) method is introduced to evaluate the range error of the FNN on test data where the distance of source is unknown. Based on FEAST, when the evaluated range error of the FNN reaches the minimum on test data, stopping training, which will help to improve the ranging accuracy of the FNN on the test data. The FEAST is demonstrated on simulated and experimental data.
Tasks
Published	2019-04-01
URL	http://arxiv.org/abs/1904.00583v1
PDF	http://arxiv.org/pdf/1904.00583v1.pdf
PWC	https://paperswithcode.com/paper/sound-source-ranging-using-a-feed-forward
Repo
Framework