January 27, 2020

3244 words 16 mins read

Paper Group ANR 1320

SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction. Modeling default rate in P2P lending via LSTM. Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I. Deep Dual Relation Modeling for Egocentric Interaction Recognition. On Variational Bounds of Mutual Information. Faster feature selection …

SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction


Title	SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction
Authors	Qichen Li, Jiaxin Pei, Jianding Zhang, Bo Han
Abstract	The typical multi-task learning methods for spatio-temporal data prediction involve low-rank tensor computation. However, such a method have relatively weak performance when the task number is small, and we cannot integrate it into non-linear models. In this paper, we propose a two-step suboptimal unitary method (SUM) to combine a meta-learning strategy into multi-task models. In the first step, it searches for a global pattern by optimising the general parameters with gradient descents under constraints, which is a geological regularizer to enable model learning with less training data. In the second step, we derive an optimised model on each specific task from the global pattern with only a few local training data. Compared with traditional multi-task learning methods, SUM shows advantages of generalisation ability on distant tasks. It can be applied on any multi-task models with the gradient descent as its optimiser regardless if the prediction function is linear or not. Moreover, we can harness the model to enable traditional prediction model to make coKriging. The experiments on public datasets have suggested that our framework, when combined with current multi-task models, has a conspicuously better prediction result when the task number is small compared to low-rank tensor learning, and our model has a quite satisfying outcome when adjusting the current prediction models for coKriging.
Tasks	Meta-Learning, Multi-Task Learning
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05150v1
PDF	https://arxiv.org/pdf/1910.05150v1.pdf
PWC	https://paperswithcode.com/paper/sum-suboptimal-unitary-multi-task-learning
Repo
Framework

Modeling default rate in P2P lending via LSTM


Title	Modeling default rate in P2P lending via LSTM
Authors	Yan Wang, Xuelei Sherry Ni
Abstract	With the fast development of peer to peer (P2P) lending, financial institutions have a substantial challenge from benefit loss due to the delinquent behaviors of the money borrowers. Therefore, having a comprehensive understanding of the changing trend of the default rate in the P2P domain is crucial. In this paper, we comprehensively study the changing trend of the default rate of P2P USA market at the aggregative level from August 2007 to January 2016. From the data visualization perspective, we found that three features, including delinq_2yrs, recoveries, and collection_recovery_fee, could potentially increase the default rate. The long short-term memory (LSTM) approach shows its great potential in modeling the P2P transaction data. Furthermore, incorporating the macroeconomic feature unemp_rate can improve the LSTM performance by decreasing RMSE on both training and testing datasets. Our study can broaden the applications of LSTM approach in the P2P market.
Tasks
Published	2019-02-13
URL	http://arxiv.org/abs/1902.04954v1
PDF	http://arxiv.org/pdf/1902.04954v1.pdf
PWC	https://paperswithcode.com/paper/modeling-default-rate-in-p2p-lending-via-lstm
Repo
Framework

Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I


Title	Distributed Inexact Successive Convex Approximation ADMM: Analysis-Part I
Authors	Sandeep Kumar, Ketan Rajawat, Daniel P. Palomar
Abstract	In this two-part work, we propose an algorithmic framework for solving non-convex problems whose objective function is the sum of a number of smooth component functions plus a convex (possibly non-smooth) or/and smooth (possibly non-convex) regularization function. The proposed algorithm incorporates ideas from several existing approaches such as alternate direction method of multipliers (ADMM), successive convex approximation (SCA), distributed and asynchronous algorithms, and inexact gradient methods. Different from a number of existing approaches, however, the proposed framework is flexible enough to incorporate a class of non-convex objective functions, allow distributed operation with and without a fusion center, and include variance reduced methods as special cases. Remarkably, the proposed algorithms are robust to uncertainties arising from random, deterministic, and adversarial sources. The part I of the paper develops two variants of the algorithm under very mild assumptions and establishes first-order convergence rate guarantees. The proof developed here allows for generic errors and delays, paving the way for different variance-reduced, asynchronous, and stochastic implementations, outlined and evaluated in part II.
Tasks
Published	2019-07-21
URL	https://arxiv.org/abs/1907.08969v1
PDF	https://arxiv.org/pdf/1907.08969v1.pdf
PWC	https://paperswithcode.com/paper/distributed-inexact-successive-convex
Repo
Framework

Deep Dual Relation Modeling for Egocentric Interaction Recognition


Title	Deep Dual Relation Modeling for Egocentric Interaction Recognition
Authors	Haoxin Li, Yijun Cai, Wei-Shi Zheng
Abstract	Egocentric interaction recognition aims to recognize the camera wearer’s interactions with the interactor who faces the camera wearer in egocentric videos. In such a human-human interaction analysis problem, it is crucial to explore the relations between the camera wearer and the interactor. However, most existing works directly model the interactions as a whole and lack modeling the relations between the two interacting persons. To exploit the strong relations for egocentric interaction recognition, we introduce a dual relation modeling framework which learns to model the relations between the camera wearer and the interactor based on the individual action representations of the two persons. Specifically, we develop a novel interactive LSTM module, the key component of our framework, to explicitly model the relations between the two interacting persons based on their individual action representations, which are collaboratively learned with an interactor attention module and a global-local motion module. Experimental results on three egocentric interaction datasets show the effectiveness of our method and advantage over state-of-the-arts.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13586v1
PDF	https://arxiv.org/pdf/1905.13586v1.pdf
PWC	https://paperswithcode.com/paper/deep-dual-relation-modeling-for-egocentric-1
Repo
Framework

On Variational Bounds of Mutual Information


Title	On Variational Bounds of Mutual Information
Authors	Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, George Tucker
Abstract	Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational bounds parameterized by neural networks, but the relationships and tradeoffs between these bounds remains unclear. In this work, we unify these recent developments in a single framework. We find that the existing variational lower bounds degrade when the MI is large, exhibiting either high bias or high variance. To address this problem, we introduce a continuum of lower bounds that encompasses previous bounds and flexibly trades off bias and variance. On high-dimensional, controlled problems, we empirically characterize the bias and variance of the bounds and their gradients and demonstrate the effectiveness of our new bounds for estimation and representation learning.
Tasks	Representation Learning
Published	2019-05-16
URL	https://arxiv.org/abs/1905.06922v1
PDF	https://arxiv.org/pdf/1905.06922v1.pdf
PWC	https://paperswithcode.com/paper/on-variational-bounds-of-mutual-information
Repo
Framework

Faster feature selection with a Dropping Forward-Backward algorithm


Title	Faster feature selection with a Dropping Forward-Backward algorithm
Authors	Thu Nguyen
Abstract	In this era of big data, feature selection techniques, which have long been proven to simplify the model, makes the model more comprehensible, speed up the process of learning, have become more and more important. Among many developed methods, forward and stepwise feature selection regression remained widely used due to their simplicity and efficiency. However, they all involving rescanning all the un-selected features again and again. Moreover, many times, the backward steps in stepwise deem unnecessary, as we will illustrate in our example. These remarks motivate us to introduce a novel algorithm that may boost the speed up to 65.77% compared to the stepwise procedure while maintaining good performance in terms of the number of selected features and error rates. Also, our experiments illustrate that feature selection procedures may be a better choice for high-dimensional problems where the number of features highly exceeds the number of samples.
Tasks	Feature Selection
Published	2019-10-17
URL	https://arxiv.org/abs/1910.08007v3
PDF	https://arxiv.org/pdf/1910.08007v3.pdf
PWC	https://paperswithcode.com/paper/dropping-forward-backward-algorithms-for
Repo
Framework

Predicting Network Controllability Robustness: A Convolutional Neural Network Approach


Title	Predicting Network Controllability Robustness: A Convolutional Neural Network Approach
Authors	Yang Lou, Yaodong He, Lin Wang, Guanrong Chen
Abstract	Network controllability measures how well a networked system can be controlled to a target state, and its robustness reflects how well the system can maintain the controllability against malicious attacks by means of node-removals or edge-removals. The measure of network controllability is quantified by the number of external control inputs needed to recover or to retain the controllability after the occurrence of an unexpected attack. The measure of the network controllability robustness, on the other hand, is quantified by a sequence of values that record the remaining controllability of the network after a sequence of attacks. Traditionally, the controllability robustness is determined by attack simulations, which is computationally time consuming. In this paper, a method to predict the controllability robustness based on machine learning using a convolutional neural network is proposed, motivated by the observations that 1) there is no clear correlation between the topological features and the controllability robustness of a general network, 2) the adjacency matrix of a network can be regarded as a gray-scale image, and 3) the convolutional neural network technique has proved successful in image processing without human intervention. Under the new framework, a fairly large number of training data generated by simulations are used to train a convolutional neural network for predicting the controllability robustness according to the input network-adjacency matrices, without performing conventional attack simulations. Extensive experimental studies were carried out, which demonstrate that the proposed framework for predicting controllability robustness of different network configurations is accurate and reliable with very low overheads.
Tasks
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09471v1
PDF	https://arxiv.org/pdf/1908.09471v1.pdf
PWC	https://paperswithcode.com/paper/predicting-network-controllability-robustness
Repo
Framework

Specificity-Based Sentence Ordering for Multi-Document Extractive Risk Summarization


Title	Specificity-Based Sentence Ordering for Multi-Document Extractive Risk Summarization
Authors	Berk Ekmekci, Eleanor Hagerman, Blake Howald
Abstract	Risk mining technologies seek to find relevant textual extractions that capture entity-risk relationships. However, when high volume data sets are processed, a multitude of relevant extractions can be returned, shifting the focus to how best to present the results. We provide the details of a risk mining multi-document extractive summarization system that produces high quality output by modeling shifts in specificity that are characteristic of well-formed discourses. In particular, we propose a novel selection algorithm that alternates between extracts based on human curated or expanded autoencoded key terms, which exhibit greater specificity or generality as it relates to an entity-risk relationship. Through this extract ordering, and without the need for more complex discourse-aware NLP, we induce felicitous shifts in specificity in the alternating summaries that outperform non-alternating summaries on automatic ROUGE and BLEU scores, and manual understandability and preferences evaluations - achieving no statistically significant difference when compared to human authored summaries.
Tasks	Sentence Ordering
Published	2019-09-23
URL	https://arxiv.org/abs/1909.10393v1
PDF	https://arxiv.org/pdf/1909.10393v1.pdf
PWC	https://paperswithcode.com/paper/190910393
Repo
Framework

How data, synapses and neurons interact with each other: a variational principle marrying gradient ascent and message passing


Title	How data, synapses and neurons interact with each other: a variational principle marrying gradient ascent and message passing
Authors	Haiping Huang
Abstract	Unsupervised learning requiring only raw data is not only a fundamental function of the cerebral cortex, but also a foundation for a next generation of artificial neural networks. However, a unified theoretical framework to treat sensory inputs, synapses and neural activity together is still lacking. The computational obstacle originates from the discrete nature of synapses, and complex interactions among these three essential elements of learning. Here, we propose a variational mean-field theory in which only the distribution of synaptic weight is considered. The unsupervised learning can then be decomposed into two interwoven steps: a maximization step is carried out as a gradient ascent of the lower-bound on the data log-likelihood, and an expectation step is carried out as a message passing procedure on an equivalent or dual neural network whose parameter is specified by the variational parameter of the weight distribution. Therefore, our framework explains how data (or sensory inputs), synapses and neural activities interact with each other to achieve the goal of extracting statistical regularities in sensory inputs. This variational framework is verified in restricted Boltzmann machines with planted synaptic weights and learning handwritten digits.
Tasks
Published	2019-11-11
URL	https://arxiv.org/abs/1911.07662v1
PDF	https://arxiv.org/pdf/1911.07662v1.pdf
PWC	https://paperswithcode.com/paper/how-data-synapses-and-neurons-interact-with
Repo
Framework

An Analysis of Object Embeddings for Image Retrieval


Title	An Analysis of Object Embeddings for Image Retrieval
Authors	Bor-Chun Chen, Larry S. Davis, Ser-Nam Lim
Abstract	We present an analysis of embeddings extracted from different pre-trained models for content-based image retrieval. Specifically, we study embeddings from image classification and object detection models. We discover that even with additional human annotations such as bounding boxes and segmentation masks, the discriminative power of the embeddings based on modern object detection models is significantly worse than their classification counterparts for the retrieval task. At the same time, our analysis also unearths that object detection model can help retrieval task by acting as a hard attention module for extracting object embeddings that focus on salient region from the convolutional feature map. In order to efficiently extract object embeddings, we introduce a simple guided student-teacher training paradigm for learning discriminative embeddings within the object detection framework. We support our findings with strong experimental results.
Tasks	Content-Based Image Retrieval, Image Classification, Image Retrieval, Object Detection
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11903v1
PDF	https://arxiv.org/pdf/1905.11903v1.pdf
PWC	https://paperswithcode.com/paper/an-analysis-of-object-embeddings-for-image
Repo
Framework

Manipulation by Feel: Touch-Based Control with Deep Predictive Models


Title	Manipulation by Feel: Touch-Based Control with Deep Predictive Models
Authors	Stephen Tian, Frederik Ebert, Dinesh Jayaraman, Mayur Mudigonda, Chelsea Finn, Roberto Calandra, Sergey Levine
Abstract	Touch sensing is widely acknowledged to be important for dexterous robotic manipulation, but exploiting tactile sensing for continuous, non-prehensile manipulation is challenging. General purpose control techniques that are able to effectively leverage tactile sensing as well as accurate physics models of contacts and forces remain largely elusive, and it is unclear how to even specify a desired behavior in terms of tactile percepts. In this paper, we take a step towards addressing these issues by combining high-resolution tactile sensing with data-driven modeling using deep neural network dynamics models. We propose deep tactile MPC, a framework for learning to perform tactile servoing from raw tactile sensor inputs, without manual supervision. We show that this method enables a robot equipped with a GelSight-style tactile sensor to manipulate a ball, analog stick, and 20-sided die, learning from unsupervised autonomous interaction and then using the learned tactile predictive model to reposition each object to user-specified configurations, indicated by a goal tactile reading. Videos, visualizations and the code are available here: https://sites.google.com/view/deeptactilempc
Tasks
Published	2019-03-11
URL	http://arxiv.org/abs/1903.04128v1
PDF	http://arxiv.org/pdf/1903.04128v1.pdf
PWC	https://paperswithcode.com/paper/manipulation-by-feel-touch-based-control-with
Repo
Framework

Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments


Title	Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments
Authors	Ethan Manilow, Prem Seetharaman, Bryan Pardo
Abstract	We present a single deep learning architecture that can both separate an audio recording of a musical mixture into constituent single-instrument recordings and transcribe these instruments into a human-readable format at the same time, learning a shared musical representation for both tasks. This novel architecture, which we call Cerberus, builds on the Chimera network for source separation by adding a third “head” for transcription. By training each head with different losses, we are able to jointly learn how to separate and transcribe up to 5 instruments in our experiments with a single network. We show that the two tasks are highly complementary with one another and when learned jointly, lead to Cerberus networks that are better at both separation and transcription and generalize better to unseen mixtures.
Tasks
Published	2019-10-22
URL	https://arxiv.org/abs/1910.12621v2
PDF	https://arxiv.org/pdf/1910.12621v2.pdf
PWC	https://paperswithcode.com/paper/simultaneous-separation-and-transcription-of
Repo
Framework

RC-DARTS: Resource Constrained Differentiable Architecture Search


Title	RC-DARTS: Resource Constrained Differentiable Architecture Search
Authors	Xiaojie Jin, Jiang Wang, Joshua Slocum, Ming-Hsuan Yang, Shengyang Dai, Shuicheng Yan, Jiashi Feng
Abstract	Recent advances show that Neural Architectural Search (NAS) method is able to find state-of-the-art image classification deep architectures. In this paper, we consider the one-shot NAS problem for resource constrained applications. This problem is of great interest because it is critical to choose different architectures according to task complexity when the resource is constrained. Previous techniques are either too slow for one-shot learning or does not take the resource constraint into consideration. In this paper, we propose the resource constrained differentiable architecture search (RC-DARTS) method to learn architectures that are significantly smaller and faster while achieving comparable accuracy. Specifically, we propose to formulate the RC-DARTS task as a constrained optimization problem by adding the resource constraint. An iterative projection method is proposed to solve the given constrained optimization problem. We also propose a multi-level search strategy to enable layers at different depths to adaptively learn different types of neural architectures. Through extensive experiments on the Cifar10 and ImageNet datasets, we show that the RC-DARTS method learns lightweight neural architectures which have smaller model size and lower computational complexity while achieving comparable or better performances than the state-of-the-art methods.
Tasks	Image Classification, One-Shot Learning
Published	2019-12-30
URL	https://arxiv.org/abs/1912.12814v1
PDF	https://arxiv.org/pdf/1912.12814v1.pdf
PWC	https://paperswithcode.com/paper/rc-darts-resource-constrained-differentiable
Repo
Framework

Meta-Transfer Learning through Hard Tasks


Title	Meta-Transfer Learning through Hard Tasks
Authors	Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, Bernt Schiele
Abstract	Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, typical meta-learning models use shallow neural networks, thus limiting its effectiveness. In order to achieve top performance, some recent works tried to use the DNNs pre-trained on large-scale datasets but mostly in straight-forward manners, e.g., (1) taking their weights as a warm start of meta-training, and (2) freezing their convolutional layers as the feature extractor of base-learners. In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks. Specifically, meta refers to training multiple tasks, and transfer is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum that further boosts the learning efficiency of MTL. We conduct few-shot learning experiments and report top performance for five-class few-shot recognition tasks on three challenging benchmarks: miniImageNet, tieredImageNet and Fewshot-CIFAR100 (FC100). Extensive comparisons to related works validate that our MTL approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.
Tasks	Few-Shot Learning, Meta-Learning, Transfer Learning
Published	2019-10-07
URL	https://arxiv.org/abs/1910.03648v1
PDF	https://arxiv.org/pdf/1910.03648v1.pdf
PWC	https://paperswithcode.com/paper/meta-transfer-learning-through-hard-tasks
Repo
Framework

Privacy-Net: An Adversarial Approach for Identity-Obfuscated Segmentation of Medical Images


Title	Privacy-Net: An Adversarial Approach for Identity-Obfuscated Segmentation of Medical Images
Authors	Bach Ngoc Kim, Jose Dolz, Pierre-Marc Jodoin, Christian Desrosiers
Abstract	This paper presents a client/server privacy-preserving network in the context of multicentric medical image analysis. Our approach is based on adversarial learning which encodes images to obfuscate the patient identity while preserving enough information for a target task. Our novel architecture is composed of three components: 1) an encoder network which removes identity-specific features from input medical images, 2) a discriminator network that attempts to identify the subject from the encoded images, 3) a medical image analysis network which analyzes the content of the encoded images (segmentation in our case). By simultaneously fooling the discriminator and optimizing the medical analysis network, the encoder learns to remove privacy-specific features while keeping those essentials for the target task. Our approach is illustrated on the problem of segmenting brain MRI from the large-scale Parkinson Progression Marker Initiative (PPMI) dataset. Using longitudinal data from PPMI, we show that the discriminator learns to heavily distort input images while allowing for highly accurate segmentation results.
Tasks
Published	2019-09-09
URL	https://arxiv.org/abs/1909.04087v2
PDF	https://arxiv.org/pdf/1909.04087v2.pdf
PWC	https://paperswithcode.com/paper/privacy-net-an-adversarial-approach-for
Repo
Framework