Paper Group ANR 1337
Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views. Transflow Learning: Repurposing Flow Models Without Retraining. On Universal Features for High-Dimensional Learning and Inference. Minimum Uncertainty Based Detection of Adversaries in Deep Neural Networks. Wasserstein Distance Guided Cross-Domain Learning. …
Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views
| Title | Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views |
| Authors | Zhizhong Han, Xinhai Liu, Yu-Shen Liu, Matthias Zwicker |
| Abstract | Deep learning has achieved remarkable results in 3D shape analysis by learning global shape features from the pixel-level over multiple views. Previous methods, however, compute low-level features for entire views without considering part-level information. In contrast, we propose a deep neural network, called Parts4Feature, to learn 3D global features from part-level information in multiple views. We introduce a novel definition of generally semantic parts, which Parts4Feature learns to detect in multiple views from different 3D shape segmentation benchmarks. A key idea of our architecture is that it transfers the ability to detect semantically meaningful parts in multiple views to learn 3D global features. Parts4Feature achieves this by combining a local part detection branch and a global feature learning branch with a shared region proposal module. The global feature learning branch aggregates the detected parts in terms of learned part patterns with a novel multi-attention mechanism, while the region proposal module enables locally and globally discriminative information to be promoted by each other. We demonstrate that Parts4Feature outperforms the state-of-the-art under three large-scale 3D shape benchmarks. |
| Tasks | 3D Shape Analysis |
| Published | 2019-05-18 |
| URL | https://arxiv.org/abs/1905.07506v1 |
| https://arxiv.org/pdf/1905.07506v1.pdf | |
| PWC | https://paperswithcode.com/paper/parts4feature-learning-3d-global-features |
| Repo | |
| Framework | |
Transflow Learning: Repurposing Flow Models Without Retraining
| Title | Transflow Learning: Repurposing Flow Models Without Retraining |
| Authors | Andrew Gambardella, Atılım Güneş Baydin, Philip H. S. Torr |
| Abstract | It is well known that deep generative models have a rich latent space, and that it is possible to smoothly manipulate their outputs by traversing this latent space. Recently, architectures have emerged that allow for more complex manipulations, such as making an image look as though it were from a different class, or painted in a certain style. These methods typically require large amounts of training in order to learn a single class of manipulations. We present Transflow Learning, a method for transforming a pre-trained generative model so that its outputs more closely resemble data that we provide afterwards. In contrast to previous methods, Transflow Learning does not require any training at all, and instead warps the probability distribution from which we sample latent vectors using Bayesian inference. Transflow Learning can be used to solve a wide variety of tasks, such as neural style transfer and few-shot classification. |
| Tasks | Bayesian Inference, Style Transfer |
| Published | 2019-11-29 |
| URL | https://arxiv.org/abs/1911.13270v2 |
| https://arxiv.org/pdf/1911.13270v2.pdf | |
| PWC | https://paperswithcode.com/paper/transflow-learning-repurposing-flow-models |
| Repo | |
| Framework | |
On Universal Features for High-Dimensional Learning and Inference
| Title | On Universal Features for High-Dimensional Learning and Inference |
| Authors | Shao-Lun Huang, Anuran Makur, Gregory W. Wornell, Lizhong Zheng |
| Abstract | We consider the problem of identifying universal low-dimensional features from high-dimensional data for inference tasks in settings involving learning. For such problems, we introduce natural notions of universality and we show a local equivalence among them. Our analysis is naturally expressed via information geometry, and represents a conceptually and computationally useful analysis. The development reveals the complementary roles of the singular value decomposition, Hirschfeld-Gebelein-R'enyi maximal correlation, the canonical correlation and principle component analyses of Hotelling and Pearson, Tishby’s information bottleneck, Wyner’s common information, Ky Fan $k$-norms, and Brieman and Friedman’s alternating conditional expectations algorithm. We further illustrate how this framework facilitates understanding and optimizing aspects of learning systems, including multinomial logistic (softmax) regression and the associated neural network architecture, matrix factorization methods for collaborative filtering and other applications, rank-constrained multivariate linear regression, and forms of semi-supervised learning. |
| Tasks | |
| Published | 2019-11-20 |
| URL | https://arxiv.org/abs/1911.09105v1 |
| https://arxiv.org/pdf/1911.09105v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-universal-features-for-high-dimensional |
| Repo | |
| Framework | |
Minimum Uncertainty Based Detection of Adversaries in Deep Neural Networks
| Title | Minimum Uncertainty Based Detection of Adversaries in Deep Neural Networks |
| Authors | Fatemeh Sheikholeslami, Swayambhoo Jain, Georgios B. Giannakis |
| Abstract | Despite their unprecedented performance in various domains, utilization of Deep Neural Networks (DNNs) in safety-critical environments is severely limited in the presence of even small adversarial perturbations. The present work develops a randomized approach to detecting such perturbations based on minimum uncertainty metrics that rely on sampling at the hidden layers during the DNN inference stage. The sampling probabilities are designed for effective detection of the adversarially corrupted inputs. Being modular, the novel detector of adversaries can be conveniently employed by any pre-trained DNN at no extra training overhead. Selecting which units to sample per hidden layer entails quantifying the amount of DNN output uncertainty from the viewpoint of Bayesian neural networks, where the overall uncertainty is expressed in terms of its layer-wise components - what also promotes scalability. Sampling probabilities are then sought by minimizing uncertainty measures layer-by-layer, leading to a novel convex optimization problem that admits an exact solver with superlinear convergence rate. By simplifying the objective function, low-complexity approximate solvers are also developed. In addition to valuable insights, these approximations link the novel approach with state-of-the-art randomized adversarial detectors. The effectiveness of the novel detectors in the context of competing alternatives is highlighted through extensive tests for various types of adversarial attacks with variable levels of strength. |
| Tasks | |
| Published | 2019-04-05 |
| URL | http://arxiv.org/abs/1904.02841v1 |
| http://arxiv.org/pdf/1904.02841v1.pdf | |
| PWC | https://paperswithcode.com/paper/minimum-uncertainty-based-detection-of |
| Repo | |
| Framework | |
Wasserstein Distance Guided Cross-Domain Learning
| Title | Wasserstein Distance Guided Cross-Domain Learning |
| Authors | Jie Su |
| Abstract | Domain adaptation aims to generalise a high-performance learner on target domain (non-labelled data) by leveraging the knowledge from source domain (rich labelled data) which comes from a different but related distribution. Assuming the source and target domains data(e.g. images) come from a joint distribution but follow on different marginal distributions, the domain adaptation work aims to infer the joint distribution from the source and target domain to learn the domain invariant features. Therefore, in this study, I extend the existing state-of-the-art approach to solve the domain adaptation problem. In particular, I propose a new approach to infer the joint distribution of images from different distributions, namely Wasserstein Distance Guided Cross-Domain Learning (WDGCDL). WDGCDL applies the Wasserstein distance to estimate the divergence between the source and target distribution which provides good gradient property and promising generalisation bound. Moreover, to tackle the training difficulty of the proposed framework, I propose two different training schemes for stable training. Qualitative results show that this new approach is superior to the existing state-of-the-art methods in the standard domain adaptation benchmark. |
| Tasks | Domain Adaptation |
| Published | 2019-10-14 |
| URL | https://arxiv.org/abs/1910.07676v1 |
| https://arxiv.org/pdf/1910.07676v1.pdf | |
| PWC | https://paperswithcode.com/paper/wasserstein-distance-guided-cross-domain |
| Repo | |
| Framework | |
Unsupervised Videographic Analysis of Rodent Behaviour
| Title | Unsupervised Videographic Analysis of Rodent Behaviour |
| Authors | Anthony Bourached, Parashkev Nachev |
| Abstract | Animal behaviour is complex and the amount of data in the form of video, if extracted, is copious. Manual analysis of behaviour is massively limited by two insurmountable obstacles, the complexity of the behavioural patterns and human bias. Automated visual analysis has the potential to eliminate both of these issues and also enable continuous analysis allowing a much higher bandwidth of data collection which is vital to capture complex behaviour at many different time scales. Behaviour is not confined to a finite set modules and thus we can only model it by inferring the generative distribution. In this way unpredictable, anomalous behaviour may be considered. Here we present a method of unsupervised behavioural analysis from nothing but high definition video recordings taken from a single, fixed perspective. We demonstrate that the identification of stereotyped rodent behaviour can be extracted in this way. |
| Tasks | |
| Published | 2019-10-22 |
| URL | https://arxiv.org/abs/1910.11065v2 |
| https://arxiv.org/pdf/1910.11065v2.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-videographic-analysis-of-rodent |
| Repo | |
| Framework | |
Adaptive Feature Processing for Robust Human Activity Recognition on a Novel Multi-Modal Dataset
| Title | Adaptive Feature Processing for Robust Human Activity Recognition on a Novel Multi-Modal Dataset |
| Authors | Mirco Moencks, Varuna De Silva, Jamie Roche, Ahmet Kondoz |
| Abstract | Human Activity Recognition (HAR) is a key building block of many emerging applications such as intelligent mobility, sports analytics, ambient-assisted living and human-robot interaction. With robust HAR, systems will become more human-aware, leading towards much safer and empathetic autonomous systems. While human pose detection has made significant progress with the dawn of deep convolutional neural networks (CNNs), the state-of-the-art research has almost exclusively focused on a single sensing modality, especially video. However, in safety critical applications it is imperative to utilize multiple sensor modalities for robust operation. To exploit the benefits of state-of-the-art machine learning techniques for HAR, it is extremely important to have multimodal datasets. In this paper, we present a novel, multi-modal sensor dataset that encompasses nine indoor activities, performed by 16 participants, and captured by four types of sensors that are commonly used in indoor applications and autonomous vehicles. This multimodal dataset is the first of its kind to be made openly available and can be exploited for many applications that require HAR, including sports analytics, healthcare assistance and indoor intelligent mobility. We propose a novel data preprocessing algorithm to enable adaptive feature extraction from the dataset to be utilized by different machine learning algorithms. Through rigorous experimental evaluations, this paper reviews the performance of machine learning approaches to posture recognition, and analyses the robustness of the algorithms. When performing HAR with the RGB-Depth data from our new dataset, machine learning algorithms such as a deep neural network reached a mean accuracy of up to 96.8% for classification across all stationary and dynamic activities |
| Tasks | Activity Recognition, Autonomous Vehicles, Human Activity Recognition, Multimodal Activity Recognition |
| Published | 2019-01-09 |
| URL | http://arxiv.org/abs/1901.02858v1 |
| http://arxiv.org/pdf/1901.02858v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-feature-processing-for-robust-human |
| Repo | |
| Framework | |
Accelerated Sparsified SGD with Error Feedback
| Title | Accelerated Sparsified SGD with Error Feedback |
| Authors | Tomoya Murata, Taiji Suzuki |
| Abstract | We study a stochastic gradient method for synchronous distributed optimization. For reducing communication cost, we are interested in utilizing compression of communicated gradients. Our main focus is a {\it{sparsified}} stochastic gradient method with {\it{error feedback}} scheme combined with {\it{Nesterov’s acceleration}}. Strong theoretical analysis of sparsified SGD with error feedback in parallel computing settings and an application of acceleration scheme to sparsified SGD with error feedback are new. It is shown that (i) our method asymptotically achieves the same iteration complexity of non-sparsified SGD even in parallel computing settings; (ii) Nesterov’s acceleration can improve the iteration complexity of non-accelerated methods in convex and even in nonconvex optimization problems for moderate optimization accuracy. |
| Tasks | Distributed Optimization |
| Published | 2019-05-29 |
| URL | https://arxiv.org/abs/1905.12224v1 |
| https://arxiv.org/pdf/1905.12224v1.pdf | |
| PWC | https://paperswithcode.com/paper/accelerated-sparsified-sgd-with-error |
| Repo | |
| Framework | |
Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models
| Title | Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models |
| Authors | Yunfei Teng, Wenbo Gao, Francois Chalus, Anna Choromanska, Donald Goldfarb, Adrian Weller |
| Abstract | We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a regular gradient step, and a corrective direction dictated by the currently best-performing worker (leader). Our method differs from the parameter-averaging scheme EASGD in a number of ways: (i) our objective formulation does not change the location of stationary points compared to the original optimization problem; (ii) we avoid convergence decelerations caused by pulling local workers descending to different local minima to each other (i.e. to the average of their parameters); (iii) our update by design breaks the curse of symmetry (the phenomenon of being trapped in poorly generalizing sub-optimal solutions in symmetric non-convex landscapes); and (iv) our approach is more communication efficient since it broadcasts only parameters of the leader rather than all workers. We provide theoretical analysis of the batch version of the proposed algorithm, which we call Leader Gradient Descent (LGD), and its stochastic variant (LSGD). Finally, we implement an asynchronous version of our algorithm and extend it to the multi-leader setting, where we form groups of workers, each represented by its own local leader (the best performer in a group), and update each worker with a corrective direction comprised of two attractive forces: one to the local, and one to the global leader (the best performer among all workers). The multi-leader setting is well-aligned with current hardware architecture, where local workers forming a group lie within a single computational node and different groups correspond to different nodes. For training convolutional neural networks, we empirically demonstrate that our approach compares favorably to state-of-the-art baselines. |
| Tasks | Distributed Optimization |
| Published | 2019-05-24 |
| URL | https://arxiv.org/abs/1905.10395v1 |
| https://arxiv.org/pdf/1905.10395v1.pdf | |
| PWC | https://paperswithcode.com/paper/leader-stochastic-gradient-descent-for |
| Repo | |
| Framework | |
Parallel genetic algorithm for planning safe and optimal route for ship
| Title | Parallel genetic algorithm for planning safe and optimal route for ship |
| Authors | Ivan Yanchin, Oleg Petrov |
| Abstract | The paper represents an algorithm for planning safe and optimal routes for transport facilities with unrestricted movement direction that travel within areas with obstacles. Paper explains the algorithm using a ship as an example of such a transport facility. This paper also provides a survey of several existing solutions for the problem. The method employs an evolutionary algorithm to plan several locally optimal routes and a parallel genetic algorithm to create the final route by optimising the abovementioned set of routes. The routes are optimized against the arrival time, assuming that the optimal route is the route with the lowermost arrival time. It is also possible to apply additional restriction to the routes. |
| Tasks | |
| Published | 2019-05-14 |
| URL | https://arxiv.org/abs/1905.05478v1 |
| https://arxiv.org/pdf/1905.05478v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-genetic-algorithm-for-planning-safe |
| Repo | |
| Framework | |
Differentially Private Consensus-Based Distributed Optimization
| Title | Differentially Private Consensus-Based Distributed Optimization |
| Authors | Mehrdad Showkatbakhsh, Can Karakus, Suhas Diggavi |
| Abstract | Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under data privacy constraints. Consensus-based optimization consists of a set of computational nodes arranged in a graph, each having a local objective that depends on their local data, where in every step nodes take a linear combination of their neighbors’ messages, as well as taking a new gradient step. Since the algorithm requires exchanging messages that depend on local data, private information gets leaked at every step. Taking $(\epsilon, \delta)$-differential privacy (DP) as our criterion, we consider the strategy where the nodes add random noise to their messages before broadcasting it, and show that the method achieves convergence with a bounded mean-squared error, while satisfying $(\epsilon, \delta)$-DP. By relaxing the more stringent $\epsilon$-DP requirement in previous work, we strengthen a known convergence result in the literature. We conclude the paper with numerical results demonstrating the effectiveness of our methods for mean estimation. |
| Tasks | Distributed Optimization |
| Published | 2019-03-19 |
| URL | http://arxiv.org/abs/1903.07792v1 |
| http://arxiv.org/pdf/1903.07792v1.pdf | |
| PWC | https://paperswithcode.com/paper/differentially-private-consensus-based |
| Repo | |
| Framework | |
3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention
| Title | 3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention |
| Authors | Zhizhong Han, Xiyang Wang, Chi-Man Vong, Yu-Shen Liu, Matthias Zwicker, C. L. Philip Chen |
| Abstract | Learning global features by aggregating information over multiple views has been shown to be effective for 3D shape analysis. For view aggregation in deep learning models, pooling has been applied extensively. However, pooling leads to a loss of the content within views, and the spatial relationship among views, which limits the discriminability of learned features. We propose 3DViewGraph to resolve this issue, which learns 3D global features by more effectively aggregating unordered views with attention. Specifically, unordered views taken around a shape are regarded as view nodes on a view graph. 3DViewGraph first learns a novel latent semantic mapping to project low-level view features into meaningful latent semantic embeddings in a lower dimensional space, which is spanned by latent semantic patterns. Then, the content and spatial information of each pair of view nodes are encoded by a novel spatial pattern correlation, where the correlation is computed among latent semantic patterns. Finally, all spatial pattern correlations are integrated with attention weights learned by a novel attention mechanism. This further increases the discriminability of learned features by highlighting the unordered view nodes with distinctive characteristics and depressing the ones with appearance ambiguity. We show that 3DViewGraph outperforms state-of-the-art methods under three large-scale benchmarks. |
| Tasks | 3D Shape Analysis |
| Published | 2019-05-17 |
| URL | https://arxiv.org/abs/1905.07503v1 |
| https://arxiv.org/pdf/1905.07503v1.pdf | |
| PWC | https://paperswithcode.com/paper/3dviewgraph-learning-global-features-for-3d |
| Repo | |
| Framework | |
A Provably Communication-Efficient Asynchronous Distributed Inference Method for Convex and Nonconvex Problems
| Title | A Provably Communication-Efficient Asynchronous Distributed Inference Method for Convex and Nonconvex Problems |
| Authors | Jineng Ren, Jarvis Haupt |
| Abstract | This paper proposes and analyzes a communication-efficient distributed optimization framework for general nonconvex nonsmooth signal processing and machine learning problems under an asynchronous protocol. At each iteration, worker machines compute gradients of a known empirical loss function using their own local data, and a master machine solves a related minimization problem to update the current estimate. We prove that for nonconvex nonsmooth problems, the proposed algorithm converges with a sublinear rate over the number of communication rounds, coinciding with the best theoretical rate that can be achieved for this class of problems. Linear convergence is established without any statistical assumptions of the local data for problems characterized by composite loss functions whose smooth parts are strongly convex. Extensive numerical experiments verify that the performance of the proposed approach indeed improves – sometimes significantly – over other state-of-the-art algorithms in terms of total communication efficiency. |
| Tasks | Distributed Optimization |
| Published | 2019-03-16 |
| URL | http://arxiv.org/abs/1903.06871v1 |
| http://arxiv.org/pdf/1903.06871v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-provably-communication-efficient |
| Repo | |
| Framework | |
Robust Navigation with Language Pretraining and Stochastic Sampling
| Title | Robust Navigation with Language Pretraining and Stochastic Sampling |
| Authors | Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi |
| Abstract | Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments. In this paper, we report two simple but highly effective methods to address these challenges and lead to a new state-of-the-art performance. First, we adapt large-scale pretrained language models to learn text representations that generalize better to previously unseen instructions. Second, we propose a stochastic sampling scheme to reduce the considerable gap between the expert actions in training and sampled actions in test, so that the agent can learn to correct its own mistakes during long sequential action decoding. Combining the two techniques, we achieve a new state of the art on the Room-to-Room benchmark with 6% absolute gain over the previous best result (47% -> 53%) on the Success Rate weighted by Path Length metric. |
| Tasks | |
| Published | 2019-09-05 |
| URL | https://arxiv.org/abs/1909.02244v1 |
| https://arxiv.org/pdf/1909.02244v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-navigation-with-language-pretraining |
| Repo | |
| Framework | |
Minimum Enclosing Ball Revisited: Stability, Sub-linear Time Algorithms, and Extension
| Title | Minimum Enclosing Ball Revisited: Stability, Sub-linear Time Algorithms, and Extension |
| Authors | Hu Ding |
| Abstract | In this paper, we revisit the Minimum Enclosing Ball (MEB) problem and its robust version, MEB with outliers, in Euclidean space $\mathbb{R}^d$. Though the problem has been extensively studied before, most of the existing algorithms need at least linear time (in the number of input points $n$ and the dimensionality $d$) to achieve a $(1+\epsilon)$-approximation. Motivated by some recent developments on beyond worst-case analysis, we introduce the notion of stability for MEB (with outliers), which is natural and easy to understand. Under the stability assumption, we present two sampling algorithms for computing approximate MEB with sample complexities independent of the number of input points; further, we achieve the first sub-linear time single-criterion approximation algorithm for the MEB with outliers problem. Our result can be viewed as a new step along the direction of beyond worst-case analysis. We also show that our ideas can be extended to be more general techniques, a novel uniform-adaptive sampling method and a sandwich lemma, for solving the general case of MEB with outliers ({\em i.e.,} without the stability assumption) and the problem of $k$-center clustering with outliers. We achieve sub-linear time bi-criteria approximation algorithms for these problems respectively; the algorithms have sample sizes independent of the number of points $n$ and the dimensionality $d$, which significantly improve the time complexities of existing algorithms. We expect that our technique will be applicable to design sub-linear time algorithms for other shape fitting with outliers problems. |
| Tasks | |
| Published | 2019-04-08 |
| URL | https://arxiv.org/abs/1904.03796v2 |
| https://arxiv.org/pdf/1904.03796v2.pdf | |
| PWC | https://paperswithcode.com/paper/minimum-enclosing-ball-revisited-stability |
| Repo | |
| Framework | |
