January 26, 2020

3139 words 15 mins read

Paper Group ANR 1615

Context-driven Active and Incremental Activity Recognition. Link Prediction with Mutual Attention for Text-Attributed Networks. Federated Learning: Challenges, Methods, and Future Directions. Progressive Gradient Pruning for Classification, Detection and DomainAdaptation. Robust Hierarchical-Optimization RLS Against Sparse Outliers. Few-shot Learni …

Context-driven Active and Incremental Activity Recognition


Title	Context-driven Active and Incremental Activity Recognition
Authors	Gabriele Civitarese, Riccardo Presotto, Claudio Bettini
Abstract	Human activity recognition based on mobile device sensor data has been an active research area in mobile and pervasive computing for several years. While the majority of the proposed techniques are based on supervised learning, semi-supervised approaches are being considered to significantly reduce the size of the training set required to initialize the recognition model. These approaches usually apply self-training or active learning to incrementally refine the model, but their effectiveness seems to be limited to a restricted set of physical activities. We claim that the context which surrounds the user (e.g., semantic location, proximity to transportation routes, time of the day) combined with common knowledge about the relationship between this context and human activities could be effective in significantly increasing the set of recognized activities including those that are difficult to discriminate only considering inertial sensors, and the ones that are highly context-dependent. In this paper, we propose CAVIAR, a novel hybrid semi-supervised and knowledge-based system for real-time activity recognition. Our method applies semantic reasoning to context data to refine the prediction of a semi-supervised classifier. The context-refined predictions are used as new labeled samples to update the classifier combining self-training and active learning techniques. Results on a real dataset obtained from 26 subjects show the effectiveness of the context-aware approach both on the recognition rates and on the number of queries to the subjects generated by the active learning module. In order to evaluate the impact of context reasoning, we also compare CAVIAR with a purely statistical version, considering features computed on context data as part of the machine learning process.
Tasks	Active Learning, Activity Recognition, Human Activity Recognition
Published	2019-06-07
URL	https://arxiv.org/abs/1906.03033v1
PDF	https://arxiv.org/pdf/1906.03033v1.pdf
PWC	https://paperswithcode.com/paper/context-driven-active-and-incremental
Repo
Framework

Link Prediction with Mutual Attention for Text-Attributed Networks


Title	Link Prediction with Mutual Attention for Text-Attributed Networks
Authors	Robin Brochier, Adrien Guille, Julien Velcin
Abstract	In this extended abstract, we present an algorithm that learns a similarity measure between documents from the network topology of a structured corpus. We leverage the Scaled Dot-Product Attention, a recently proposed attention mechanism, to design a mutual attention mechanism between pairs of documents. To train its parameters, we use the network links as supervision. We provide preliminary experiment results with a citation dataset on two prediction tasks, demonstrating the capacity of our model to learn a meaningful textual similarity.
Tasks	Link Prediction
Published	2019-02-28
URL	http://arxiv.org/abs/1902.11054v2
PDF	http://arxiv.org/pdf/1902.11054v2.pdf
PWC	https://paperswithcode.com/paper/link-prediction-with-mutual-attention-for
Repo
Framework

Federated Learning: Challenges, Methods, and Future Directions


Title	Federated Learning: Challenges, Methods, and Future Directions
Authors	Tian Li, Anit Kumar Sahu, Ameet Talwalkar, Virginia Smith
Abstract	Federated learning involves training statistical models over remote devices or siloed data centers, such as mobile phones or hospitals, while keeping data localized. Training in heterogeneous and potentially massive networks introduces novel challenges that require a fundamental departure from standard approaches for large-scale machine learning, distributed optimization, and privacy-preserving data analysis. In this article, we discuss the unique characteristics and challenges of federated learning, provide a broad overview of current approaches, and outline several directions of future work that are relevant to a wide range of research communities.
Tasks	Distributed Optimization
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07873v1
PDF	https://arxiv.org/pdf/1908.07873v1.pdf
PWC	https://paperswithcode.com/paper/190807873
Repo
Framework

Progressive Gradient Pruning for Classification, Detection and DomainAdaptation


Title	Progressive Gradient Pruning for Classification, Detection and DomainAdaptation
Authors	Le Thanh Nguyen-Meidine, Eric Granger, Madhu Kiran, Louis-Antoine Blais-Morin, Marco Pedersoli
Abstract	Although deep neural networks (NNs) have achievedstate-of-the-art accuracy in many visual recognition tasks,the growing computational complexity and energy con-sumption of networks remains an issue, especially for ap-plications on platforms with limited resources and requir-ing real-time processing. Filter pruning techniques haverecently shown promising results for the compression andacceleration of convolutional NNs (CNNs). However, thesetechniques involve numerous steps and complex optimisa-tions because some only prune after training CNNs, whileothers prune from scratch during training by integratingsparsity constraints or modifying the loss function.In this paper we propose a new Progressive GradientPruning (PGP) technique for iterative filter pruning dur-ing training. In contrast to previous progressive pruningtechniques, it relies on a novel filter selection criterion thatmeasures the change in filter weights, uses a new hard andsoft pruning strategy and effectively adapts momentum ten-sors during the backward propagation pass. Experimentalresults obtained after training various CNNs on image datafor classification, object detection and domain adaptationbenchmarks indicate that the PGP technique can achievea better trade-off between classification accuracy and net-work (time and memory) complexity than PSFP and otherstate-of-the-art filter pruning techniques.
Tasks	Object Detection
Published	2019-06-20
URL	https://arxiv.org/abs/1906.08746v4
PDF	https://arxiv.org/pdf/1906.08746v4.pdf
PWC	https://paperswithcode.com/paper/an-improved-trade-off-between-accuracy-and
Repo
Framework

Robust Hierarchical-Optimization RLS Against Sparse Outliers


Title	Robust Hierarchical-Optimization RLS Against Sparse Outliers
Authors	Konstantinos Slavakis, Sinjini Banerjee
Abstract	This paper fortifies the recently introduced hierarchical-optimization recursive least squares (HO-RLS) against outliers which contaminate infrequently linear-regression models. Outliers are modeled as nuisance variables and are estimated together with the linear filter/system variables via a sparsity-inducing (non-)convexly regularized least-squares task. The proposed outlier-robust HO-RLS builds on steepest-descent directions with a constant step size (learning rate), needs no matrix inversion (lemma), accommodates colored nominal noise of known correlation matrix, exhibits small computational footprint, and offers theoretical guarantees, in a probabilistic sense, for the convergence of the system estimates to the solutions of a hierarchical-optimization problem: Minimize a convex loss, which models a-priori knowledge about the unknown system, over the minimizers of the classical ensemble LS loss. Extensive numerical tests on synthetically generated data in both stationary and non-stationary scenarios showcase notable improvements of the proposed scheme over state-of-the-art techniques.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05399v1
PDF	https://arxiv.org/pdf/1910.05399v1.pdf
PWC	https://paperswithcode.com/paper/robust-hierarchical-optimization-rls-against
Repo
Framework

Few-shot Learning with Deep Triplet Networks for Brain Imaging Modality Recognition


Title	Few-shot Learning with Deep Triplet Networks for Brain Imaging Modality Recognition
Authors	Santi Puch, Irina Sánchez, Matt Rowe
Abstract	Image modality recognition is essential for efficient imaging workflows in current clinical environments, where multiple imaging modalities are used to better comprehend complex diseases. Emerging biomarkers from novel, rare modalities are being developed to aid in such understanding, however the availability of these images is often limited. This scenario raises the necessity of recognising new imaging modalities without them being collected and annotated in large amounts. In this work, we present a few-shot learning model for limited training examples based on Deep Triplet Networks. We show that the proposed model is more accurate in distinguishing different modalities than a traditional Convolutional Neural Network classifier when limited samples are available. Furthermore, we evaluate the performance of both classifiers when presented with noisy samples and provide an initial inspection of how the proposed model can incorporate measures of uncertainty to be more robust against out-of-sample examples.
Tasks	Few-Shot Learning
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10266v1
PDF	https://arxiv.org/pdf/1908.10266v1.pdf
PWC	https://paperswithcode.com/paper/few-shot-learning-with-deep-triplet-networks
Repo
Framework

Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables


Title	Justlookup: One Millisecond Deep Feature Extraction for Point Clouds By Lookup Tables
Authors	Hongxin Lin, Zelin Xiao, Yang Tan, Hongyang Chao, Shengyong Ding
Abstract	Deep models are capable of fitting complex high dimensional functions while usually yielding large computation load. There is no way to speed up the inference process by classical lookup tables due to the high-dimensional input and limited memory size. Recently, a novel architecture (PointNet) for point clouds has demonstrated that it is possible to obtain a complicated deep function from a set of 3-variable functions. In this paper, we exploit this property and apply a lookup table to encode these 3-variable functions. This method ensures that the inference time is only determined by the memory access no matter how complicated the deep function is. We conduct extensive experiments on ModelNet and ShapeNet datasets and demonstrate that we can complete the inference process in 1.5 ms on an Intel i7-8700 CPU (single core mode), 32x speedup over the PointNet architecture without any performance degradation.
Tasks
Published	2019-08-14
URL	https://arxiv.org/abs/1908.08996v1
PDF	https://arxiv.org/pdf/1908.08996v1.pdf
PWC	https://paperswithcode.com/paper/justlookup-one-millisecond-deep-feature
Repo
Framework

Runtime Analysis of RLS and (1+1) EA for the Dynamic Weighted Vertex Cover Problem


Title	Runtime Analysis of RLS and (1+1) EA for the Dynamic Weighted Vertex Cover Problem
Authors	Mojgan Pourhassan, Vahid Roostapour, Frank Neumann
Abstract	In this paper, we perform theoretical analyses on the behaviour of an evolutionary algorithm and a randomised search algorithm for the dynamic vertex cover problem based on its dual formulation. The dynamic vertex cover problem has already been theoretically investigated to some extent and it has been shown that using its dual formulation to represent possible solutions can lead to a better approximation behaviour. We improve some of the existing results, i.e. we find a linear expected re-optimization time for a (1+1) EA to re-discover a 2-approximation when edges are dynamically deleted from the graph. Furthermore, we investigate a different setting for applying the dynamism to the problem, in which a dynamic change happens at each step with a probability $P_D$. We also expand these analyses to the weighted vertex cover problem, in which weights are assigned to vertices and the goal is to find a cover set with minimum total weight. Similar to the classical case, the dynamic changes that we consider on the weighted vertex cover problem are adding and removing edges to and from the graph. We aim at finding a maximal solution for the dual problem, which gives a 2-approximate solution for the vertex cover problem. This is equivalent to the maximal matching problem for the classical vertex cover problem.
Tasks
Published	2019-03-06
URL	http://arxiv.org/abs/1903.02195v1
PDF	http://arxiv.org/pdf/1903.02195v1.pdf
PWC	https://paperswithcode.com/paper/runtime-analysis-of-rls-and-11-ea-for-the
Repo
Framework

Corpus Augmentation by Sentence Segmentation for Low-Resource Neural Machine Translation


Title	Corpus Augmentation by Sentence Segmentation for Low-Resource Neural Machine Translation
Authors	Jinyi Zhang, Tadahiro Matsumoto
Abstract	Neural Machine Translation (NMT) has been proven to achieve impressive results. The NMT system translation results depend strongly on the size and quality of parallel corpora. Nevertheless, for many language pairs, no rich-resource parallel corpora exist. As described in this paper, we propose a corpus augmentation method by segmenting long sentences in a corpus using back-translation and generating pseudo-parallel sentence pairs. The experiment results of the Japanese-Chinese and Chinese-Japanese translation with Japanese-Chinese scientific paper excerpt corpus (ASPEC-JC) show that the method improves translation performance.
Tasks	Low-Resource Neural Machine Translation, Machine Translation
Published	2019-05-22
URL	https://arxiv.org/abs/1905.08945v1
PDF	https://arxiv.org/pdf/1905.08945v1.pdf
PWC	https://paperswithcode.com/paper/corpus-augmentation-by-sentence-segmentation
Repo
Framework

Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer


Title	Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer
Authors	Benjamin D. Steel
Abstract	Future developments in deep learning applications requiring large datasets will be limited by power and speed limitations of silicon based Von-Neumann computing architectures. Optical architectures provide a low power and high speed hardware alternative. Recent publications have suggested promising implementations of optical neural networks (ONNs), showing huge orders of magnitude efficiency and speed gains over current state of the art hardware alternatives. In this work, the transmission of the Fabry-Perot Interferometer (FPI) is proposed as a low power, low footprint activation function unit. Numerical simulations of optical CNNs using the FPI based activation functions show accuracies of 98% on the MNIST dataset. An investigation of possible physical implementation of the network shows that an ONN based on current tunable FPIs could be slowed by actuation delays, but rapidly developing optical hardware fabrication techniques could make an integrated approach using the proposed FPI setups a powerful solution for previously inaccessible deep learning applications.
Tasks
Published	2019-11-22
URL	https://arxiv.org/abs/1911.10109v2
PDF	https://arxiv.org/pdf/1911.10109v2.pdf
PWC	https://paperswithcode.com/paper/implementation-of-optical-deep-neural
Repo
Framework

Location-Centered House Price Prediction: A Multi-Task Learning Approach


Title	Location-Centered House Price Prediction: A Multi-Task Learning Approach
Authors	Guangliang Gao, Zhifeng Bao, Jie Cao, A. K. Qin, Timos Sellis, Fellow, IEEE, Zhiang Wu
Abstract	Accurate house prediction is of great significance to various real estate stakeholders such as house owners, buyers, investors, and agents. We propose a location-centered prediction framework that differs from existing work in terms of data profiling and prediction model. Regarding data profiling, we define and capture a fine-grained location profile powered by a diverse range of location data sources, such as transportation profile (e.g., distance to nearest train station), education profile (e.g., school zones and ranking), suburb profile based on census data, facility profile (e.g., nearby hospitals, supermarkets). Regarding the choice of prediction model, we observe that a variety of approaches either consider the entire house data for modeling, or split the entire data and model each partition independently. However, such modeling ignores the relatedness between partitions, and for all prediction scenarios, there may not be sufficient training samples per partition for the latter approach. We address this problem by conducting a careful study of exploiting the Multi-Task Learning (MTL) model. Specifically, we map the strategies for splitting the entire house data to the ways the tasks are defined in MTL, and each partition obtained is aligned with a task. Furthermore, we select specific MTL-based methods with different regularization terms to capture and exploit the relatedness between tasks. Based on real-world house transaction data collected in Melbourne, Australia. We design extensive experimental evaluations, and the results indicate a significant superiority of MTL-based methods over state-of-the-art approaches. Meanwhile, we conduct an in-depth analysis on the impact of task definitions and method selections in MTL on the prediction performance, and demonstrate that the impact of task definitions on prediction performance far exceeds that of method selections.
Tasks	Multi-Task Learning
Published	2019-01-07
URL	http://arxiv.org/abs/1901.01774v1
PDF	http://arxiv.org/pdf/1901.01774v1.pdf
PWC	https://paperswithcode.com/paper/location-centered-house-price-prediction-a
Repo
Framework

Active Adversarial Domain Adaptation


Title	Active Adversarial Domain Adaptation
Authors	Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, Manmohan Chandraker
Abstract	We propose an active learning approach for transferring representations across domains. Our approach, active adversarial domain adaptation (AADA), explores a duality between two related problems: adversarial domain alignment and importance sampling for adapting models across domains. The former uses a domain discriminative model to align domains, while the latter utilizes it to weigh samples to account for distribution shifts. Specifically, our importance weight promotes samples with large uncertainty in classification and diversity from labeled examples, thus serves as a sample selection scheme for active learning. We show that these two views can be unified in one framework for domain adaptation and transfer learning when the source domain has many labeled examples while the target domain does not. AADA provides significant improvements over fine-tuning based approaches and other sampling methods when the two domains are closely related. Results on challenging domain adaptation tasks, e.g., object detection, demonstrate that the advantage over baseline approaches is retained even after hundreds of examples being actively annotated.
Tasks	Active Learning, Domain Adaptation, Object Detection, Transfer Learning
Published	2019-04-16
URL	https://arxiv.org/abs/1904.07848v2
PDF	https://arxiv.org/pdf/1904.07848v2.pdf
PWC	https://paperswithcode.com/paper/active-adversarial-domain-adaptation
Repo
Framework

A Signal Propagation Perspective for Pruning Neural Networks at Initialization


Title	A Signal Propagation Perspective for Pruning Neural Networks at Initialization
Authors	Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, Philip H. S. Torr
Abstract	Network pruning is a promising avenue for compressing deep neural networks. A typical approach to pruning starts by training a model and then removing redundant parameters while minimizing the impact on what is learned. Alternatively, a recent approach shows that pruning can be done at initialization prior to training, based on a saliency criterion called connection sensitivity. However, it remains unclear exactly why pruning an untrained, randomly initialized neural network is effective. In this work, by noting connection sensitivity as a form of gradient, we formally characterize initialization conditions to ensure reliable connection sensitivity measurements, which in turn yields effective pruning results. Moreover, we analyze the signal propagation properties of the resulting pruned networks and introduce a simple, data-free method to improve their trainability. Our modifications to the existing pruning at initialization method lead to improved results on all tested network models for image classification tasks. Furthermore, we empirically study the effect of supervision for pruning and demonstrate that our signal propagation perspective, combined with unsupervised pruning, can be useful in various scenarios where pruning is applied to non-standard arbitrarily-designed architectures.
Tasks	Image Classification, Network Pruning
Published	2019-06-14
URL	https://arxiv.org/abs/1906.06307v2
PDF	https://arxiv.org/pdf/1906.06307v2.pdf
PWC	https://paperswithcode.com/paper/a-signal-propagation-perspective-for-pruning
Repo
Framework

Nonconvex Nonsmooth Low-Rank Minimization for Generalized Image Compressed Sensing via Group Sparse Representation


Title	Nonconvex Nonsmooth Low-Rank Minimization for Generalized Image Compressed Sensing via Group Sparse Representation
Authors	Yunyi Li, Li Liu, Yu Zhao, Xiefeng Cheng, Guan Gui
Abstract	Group sparse representation (GSR) based method has led to great successes in various image recovery tasks, which can be converted into a low-rank matrix minimization problem. As a widely used surrogate function of low-rank, the nuclear norm based convex surrogate usually leads to over-shrinking problem, since the standard soft-thresholding operator shrinks all singular values equally. To improve traditional sparse representation based image compressive sensing (CS) performance, we propose a generalized CS framework based on GSR model, which leads to a nonconvex nonsmooth low-rank minimization problem. The popular L_2-norm and M-estimator are employed for standard image CS and robust CS problem to fit the data respectively. For the better approximation of the rank of group-matrix, a family of nuclear norms are employed to address the over-shrinking problem. Moreover, we also propose a flexible and effective iteratively-weighting strategy to control the weighting and contribution of each singular value. Then we develop an iteratively reweighted nuclear norm algorithm for our generalized framework via an alternating direction method of multipliers framework, namely, GSR-AIR. Experimental results demonstrate that our proposed CS framework can achieve favorable reconstruction performance compared with current state-of-the-art methods and the robust CS framework can suppress the outliers effectively.
Tasks	Compressive Sensing
Published	2019-11-18
URL	https://arxiv.org/abs/1911.08914v2
PDF	https://arxiv.org/pdf/1911.08914v2.pdf
PWC	https://paperswithcode.com/paper/nonconvex-nonsmooth-low-rank-minimization-for
Repo
Framework

P$^2$GNet: Pose-Guided Point Cloud Generating Networks for 6-DoF Object Pose Estimation


Title	P$^2$GNet: Pose-Guided Point Cloud Generating Networks for 6-DoF Object Pose Estimation
Authors	Peiyu Yu, Yongming Rao, Jiwen Lu, Jie Zhou
Abstract	Humans are able to perform fast and accurate object pose estimation even under severe occlusion by exploiting learned object model priors from everyday life. However, most recently proposed pose estimation algorithms neglect to utilize the information of object models, often end up with limited accuracy, and tend to fall short in cluttered scenes. In this paper, we present a novel learning-based model, \underline{P}ose-Guided \underline{P}oint Cloud \underline{G}enerating Networks for 6D Object Pose Estimation (P$^2$GNet), designed to effectively exploit object model priors to facilitate 6D object pose estimation. We achieve this with an end-to-end estimation-by-generation workflow that combines the appearance information from the RGB-D image and the structure knowledge from object point cloud to enable accurate and robust pose estimation. Experiments on two commonly used benchmarks for 6D pose estimation, YCB-Video dataset and LineMOD dataset, demonstrate that P$^2$GNet outperforms the state-of-the-art method by a large margin and shows marked robustness towards heavy occlusion, while achieving real-time inference.
Tasks	6D Pose Estimation, 6D Pose Estimation using RGB, Pose Estimation
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09316v2
PDF	https://arxiv.org/pdf/1912.09316v2.pdf
PWC	https://paperswithcode.com/paper/p2gnet-pose-guided-point-cloud-generating
Repo
Framework