Paper Group ANR 60
Partial Weight Adaptation for Robust DNN Inference. A Hybrid 3DCNN and 3DC-LSTM based model for 4D Spatio-temporal fMRI data: An ABIDE Autism Classification study. Reducing Noise from Competing Neighbours: Word Retrieval with Lateral Inhibition in Multilink. Disturbance-immune Weight Sharing for Neural Architecture Search. Active Learning Approach …
Partial Weight Adaptation for Robust DNN Inference
| Title | Partial Weight Adaptation for Robust DNN Inference |
| Authors | Xiufeng Xie, Kyu-Han Kim |
| Abstract | Mainstream video analytics uses a pre-trained DNN model with an assumption that inference input and training data follow the same probability distribution. However, this assumption does not always hold in the wild: autonomous vehicles may capture video with varying brightness; unstable wireless bandwidth calls for adaptive bitrate streaming of video; and, inference servers may serve inputs from heterogeneous IoT devices/cameras. In such situations, the level of input distortion changes rapidly, thus reshaping the probability distribution of the input. We present GearNN, an adaptive inference architecture that accommodates heterogeneous DNN inputs. GearNN employs an optimization algorithm to identify a small set of “distortion-sensitive” DNN parameters, given a memory budget. Based on the distortion level of the input, GearNN then adapts only the distortion-sensitive parameters, while reusing the rest of constant parameters across all input qualities. In our evaluation of DNN inference with dynamic input distortions, GearNN improves the accuracy (mIoU) by an average of 18.12% over a DNN trained with the undistorted dataset and 4.84% over stability training from Google, with only 1.8% extra memory overhead. |
| Tasks | Autonomous Vehicles |
| Published | 2020-03-13 |
| URL | https://arxiv.org/abs/2003.06131v1 |
| https://arxiv.org/pdf/2003.06131v1.pdf | |
| PWC | https://paperswithcode.com/paper/partial-weight-adaptation-for-robust-dnn |
| Repo | |
| Framework | |
A Hybrid 3DCNN and 3DC-LSTM based model for 4D Spatio-temporal fMRI data: An ABIDE Autism Classification study
| Title | A Hybrid 3DCNN and 3DC-LSTM based model for 4D Spatio-temporal fMRI data: An ABIDE Autism Classification study |
| Authors | Ahmed El-Gazzar, Mirjam Quaak, Leonardo Cerliani, Peter Bloem, Guido van Wingen, Rajat Mani Thomas |
| Abstract | Functional Magnetic Resonance Imaging (fMRI) captures the temporal dynamics of neural activity as a function of spatial location in the brain. Thus, fMRI scans are represented as 4-Dimensional (3-space + 1-time) tensors. And it is widely believed that the spatio-temporal patterns in fMRI manifests as behaviour and clinical symptoms. Because of the high dimensionality ($\sim$ 1 Million) of fMRI, and the added constraints of limited cardinality of data sets, extracting such patterns are challenging. A standard approach to overcome these hurdles is to reduce the dimensionality of the data by either summarizing activation over time or space at the expense of possible loss of useful information. Here, we introduce an end-to-end algorithm capable of extracting spatiotemporal features from the full 4-D data using 3-D CNNs and 3-D Convolutional LSTMs. We evaluate our proposed model on the publicly available ABIDE dataset to demonstrate the capability of our model to classify Autism Spectrum Disorder (ASD) from resting-state fMRI data. Our results show that the proposed model achieves state of the art results on single sites with F1-scores of 0.78 and 0.7 on NYU and UM sites, respectively. |
| Tasks | |
| Published | 2020-02-14 |
| URL | https://arxiv.org/abs/2002.05981v1 |
| https://arxiv.org/pdf/2002.05981v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-hybrid-3dcnn-and-3dc-lstm-based-model-for |
| Repo | |
| Framework | |
Reducing Noise from Competing Neighbours: Word Retrieval with Lateral Inhibition in Multilink
| Title | Reducing Noise from Competing Neighbours: Word Retrieval with Lateral Inhibition in Multilink |
| Authors | Aaron van Geffen |
| Abstract | Multilink is a computational model for word retrieval in monolingual and multilingual individuals under different task circumstances (Dijkstra et al., 2018). In the present study, we added lateral inhibition to Multilink’s lexical network. Parameters were fit on the basis of reaction times from the English, British, and Dutch Lexicon Projects. We found a maximum correlation of 0.643 (N=1,205) on these data sets as a whole. Furthermore, the simulations themselves became faster as a result of adding lateral inhibition. We tested the fitted model to stimuli from a neighbourhood study (Mulder et al., 2018). Lateral inhibition was found to improve Multilink’s correlations for this study, yielding an overall correlation of 0.67. Next, we explored the role of lateral inhibition as part of the model’s task/decision system by running simulations on data from two studies concerning interlingual homographs (Vanlangendonck et al., in press; Goertz, 2018). We found that, while lateral inhibition plays a substantial part in the word selection process, this alone is not enough to result in a correct response selection. To solve this problem, we added a new task component to Multilink, especially designed to account for the translation process of interlingual homographs, cognates, and language-specific control words. The subsequent simulation results showed patterns remarkably similar to those in the Goertz study. The isomorphicity of the simulated data to the empirical data was further attested by an overall correlation of 0.538 (N=254) between reaction times and simulated model cycle times, as well as a condition pattern correlation of 0.853 (N=8). We conclude that Multilink yields an excellent fit to empirical data, particularly when a task-specific setting of the inhibition parameters is allowed. |
| Tasks | |
| Published | 2020-01-25 |
| URL | https://arxiv.org/abs/2002.00730v1 |
| https://arxiv.org/pdf/2002.00730v1.pdf | |
| PWC | https://paperswithcode.com/paper/reducing-noise-from-competing-neighbours-word |
| Repo | |
| Framework | |
Disturbance-immune Weight Sharing for Neural Architecture Search
| Title | Disturbance-immune Weight Sharing for Neural Architecture Search |
| Authors | Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yong Guo, Peilin Zhao, Junzhou Huang, Mingkui Tan |
| Abstract | Neural architecture search (NAS) has gained increasing attention in the community of architecture design. One of the key factors behind the success lies in the training efficiency created by the weight sharing (WS) technique. However, WS-based NAS methods often suffer from a performance disturbance (PD) issue. That is, the training of subsequent architectures inevitably disturbs the performance of previously trained architectures due to the partially shared weights. This leads to inaccurate performance estimation for the previous architectures, which makes it hard to learn a good search strategy. To alleviate the performance disturbance issue, we propose a new disturbance-immune update strategy for model updating. Specifically, to preserve the knowledge learned by previous architectures, we constrain the training of subsequent architectures in an orthogonal space via orthogonal gradient descent. Equipped with this strategy, we propose a novel disturbance-immune training scheme for NAS. We theoretically analyze the effectiveness of our strategy in alleviating the PD risk. Extensive experiments on CIFAR-10 and ImageNet verify the superiority of our method. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-29 |
| URL | https://arxiv.org/abs/2003.13089v1 |
| https://arxiv.org/pdf/2003.13089v1.pdf | |
| PWC | https://paperswithcode.com/paper/disturbance-immune-weight-sharing-for-neural |
| Repo | |
| Framework | |
Active Learning Approach to Optimization of Experimental Control
| Title | Active Learning Approach to Optimization of Experimental Control |
| Authors | Yadong Wu, Zengming Meng, Kai Wen, Chengdong Mi, Jing Zhang, Hui Zhai |
| Abstract | In this work we present a general machine learning based scheme to optimize experimental control. The method utilizes the neural network to learn the relation between the control parameters and the control goal, with which the optimal control parameters can be obtained. The main challenge of this approach is that the labeled data obtained from experiments are not abundant. The central idea of our scheme is to use the active learning to overcome this difficulty. As a demonstration example, we apply our method to control evaporative cooling experiments in cold atoms. We have first tested our method with simulated data and then applied our method to real experiments. We demonstrate that our method can successfully reach the best performance within hundreds of experimental runs. Our method does not require knowledge of the experimental system as a prior and is universal for experimental control in different systems. |
| Tasks | Active Learning |
| Published | 2020-03-26 |
| URL | https://arxiv.org/abs/2003.11804v2 |
| https://arxiv.org/pdf/2003.11804v2.pdf | |
| PWC | https://paperswithcode.com/paper/active-learning-approach-to-optimization-of |
| Repo | |
| Framework | |
Integrating Informativeness, Representativeness and Diversity in Pool-Based Sequential Active Learning for Regression
| Title | Integrating Informativeness, Representativeness and Diversity in Pool-Based Sequential Active Learning for Regression |
| Authors | Ziang Liu, Dongrui Wu |
| Abstract | In many real-world machine learning applications, unlabeled samples are easy to obtain, but it is expensive and/or time-consuming to label them. Active learning is a common approach for reducing this data labeling effort. It optimally selects the best few samples to label, so that a better machine learning model can be trained from the same number of labeled samples. This paper considers active learning for regression (ALR) problems. Three essential criteria – informativeness, representativeness, and diversity – have been proposed for ALR. However, very few approaches in the literature have considered all three of them simultaneously. We propose three new ALR approaches, with different strategies for integrating the three criteria. Extensive experiments on 12 datasets in various domains demonstrated their effectiveness. |
| Tasks | Active Learning |
| Published | 2020-03-26 |
| URL | https://arxiv.org/abs/2003.11786v1 |
| https://arxiv.org/pdf/2003.11786v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-informativeness |
| Repo | |
| Framework | |
Pool-Based Unsupervised Active Learning for Regression Using Iterative Representativeness-Diversity Maximization (iRDM)
| Title | Pool-Based Unsupervised Active Learning for Regression Using Iterative Representativeness-Diversity Maximization (iRDM) |
| Authors | Ziang Liu, Xue Jiang, Hanbin Luo, Weili Fang, Jiajing Liu, Dongrui Wu |
| Abstract | Active learning (AL) selects the most beneficial unlabeled samples to label, and hence a better machine learning model can be trained from the same number of labeled samples. Most existing active learning for regression (ALR) approaches are supervised, which means the sampling process must use some label information, or an existing regression model. This paper considers completely unsupervised ALR, i.e., how to select the samples to label without knowing any true label information. We propose a novel unsupervised ALR approach, iterative representativeness-diversity maximization (iRDM), to optimally balance the representativeness and the diversity of the selected samples. Experiments on 12 datasets from various domains demonstrated its effectiveness. Our iRDM can be applied to both linear regression and kernel regression, and it even significantly outperforms supervised ALR when the number of labeled samples is small. |
| Tasks | Active Learning |
| Published | 2020-03-17 |
| URL | https://arxiv.org/abs/2003.07658v2 |
| https://arxiv.org/pdf/2003.07658v2.pdf | |
| PWC | https://paperswithcode.com/paper/pool-based-unsupervised-active-learning-for |
| Repo | |
| Framework | |
MUTATT: Visual-Textual Mutual Guidance for Referring Expression Comprehension
| Title | MUTATT: Visual-Textual Mutual Guidance for Referring Expression Comprehension |
| Authors | Shuai Wang, Fan Lyu, Wei Feng, Song Wang |
| Abstract | Referring expression comprehension (REC) aims to localize a text-related region in a given image by a referring expression in natural language. Existing methods focus on how to build convincing visual and language representations independently, which may significantly isolate visual and language information. In this paper, we argue that for REC the referring expression and the target region are semantically correlated and subject, location and relationship consistency exist between vision and language.On top of this, we propose a novel approach called MutAtt to construct mutual guidance between vision and language, which treat vision and language equally thus yield compact information matching. Specifically, for each module of subject, location and relationship, MutAtt builds two kinds of attention-based mutual guidance strategies. One strategy is to generate vision-guided language embedding for the sake of matching relevant visual feature. The other reversely generates language-guided visual feature to match relevant language embedding. This mutual guidance strategy can effectively guarantees the vision-language consistency in three modules. Experiments on three popular REC datasets demonstrate that the proposed approach outperforms the current state-of-the-art methods. |
| Tasks | |
| Published | 2020-03-18 |
| URL | https://arxiv.org/abs/2003.08027v2 |
| https://arxiv.org/pdf/2003.08027v2.pdf | |
| PWC | https://paperswithcode.com/paper/mutatt-visual-textual-mutual-guidance-for |
| Repo | |
| Framework | |
Topological Effects on Attacks Against Vertex Classification
| Title | Topological Effects on Attacks Against Vertex Classification |
| Authors | Benjamin A. Miller, Mustafa Çamurcu, Alexander J. Gomez, Kevin Chan, Tina Eliassi-Rad |
| Abstract | Vertex classification is vulnerable to perturbations of both graph topology and vertex attributes, as shown in recent research. As in other machine learning domains, concerns about robustness to adversarial manipulation can prevent potential users from adopting proposed methods when the consequence of action is very high. This paper considers two topological characteristics of graphs and explores the way these features affect the amount the adversary must perturb the graph in order to be successful. We show that, if certain vertices are included in the training set, it is possible to substantially an adversary’s required perturbation budget. On four citation datasets, we demonstrate that if the training set includes high degree vertices or vertices that ensure all unlabeled nodes have neighbors in the training set, we show that the adversary’s budget often increases by a substantial factor—often a factor of 2 or more—over random training for the Nettack poisoning attack. Even for especially easy targets (those that are misclassified after just one or two perturbations), the degradation of performance is much slower, assigning much lower probabilities to the incorrect classes. In addition, we demonstrate that this robustness either persists when recently proposed defenses are applied, or is competitive with the resulting performance improvement for the defender. |
| Tasks | |
| Published | 2020-03-12 |
| URL | https://arxiv.org/abs/2003.05822v1 |
| https://arxiv.org/pdf/2003.05822v1.pdf | |
| PWC | https://paperswithcode.com/paper/topological-effects-on-attacks-against-vertex |
| Repo | |
| Framework | |
Off-Road Drivable Area Extraction Using 3D LiDAR Data
| Title | Off-Road Drivable Area Extraction Using 3D LiDAR Data |
| Authors | Biao Gao, Anran Xu, Yancheng Pan, Xijun Zhao, Wen Yao, Huijing Zhao |
| Abstract | We propose a method for off-road drivable area extraction using 3D LiDAR data with the goal of autonomous driving application. A specific deep learning framework is designed to deal with the ambiguous area, which is one of the main challenges in the off-road environment. To reduce the considerable demand for human-annotated data for network training, we utilize the information from vast quantities of vehicle paths and auto-generated obstacle labels. Using these autogenerated annotations, the proposed network can be trained using weakly supervised or semi-supervised methods, which can achieve better performance with fewer human annotations. The experiments on our dataset illustrate the reasonability of our framework and the validity of our weakly and semi-supervised methods. |
| Tasks | Autonomous Driving |
| Published | 2020-03-10 |
| URL | https://arxiv.org/abs/2003.04780v1 |
| https://arxiv.org/pdf/2003.04780v1.pdf | |
| PWC | https://paperswithcode.com/paper/off-road-drivable-area-extraction-using-3d |
| Repo | |
| Framework | |
Voxel Map for Visual SLAM
| Title | Voxel Map for Visual SLAM |
| Authors | Manasi Muglikar, Zichao Zhang, Davide Scaramuzza |
| Abstract | In modern visual SLAM systems, it is a standard practice to retrieve potential candidate map points from overlapping keyframes for further feature matching or direct tracking. In this work, we argue that keyframes are not the optimal choice for this task, due to several inherent limitations, such as weak geometric reasoning and poor scalability. We propose a voxel-map representation to efficiently retrieve map points for visual SLAM. In particular, we organize the map points in a regular voxel grid. Visible points from a camera pose are queried by sampling the camera frustum in a raycasting manner, which can be done in constant time using an efficient voxel hashing method. Compared with keyframes, the retrieved points using our method are geometrically guaranteed to fall in the camera field-of-view, and occluded points can be identified and removed to a certain extend. This method also naturally scales up to large scenes and complicated multicamera configurations. Experimental results show that our voxel map representation is as efficient as a keyframe map with 5 keyframes and provides significantly higher localization accuracy (average 46% improvement in RMSE) on the EuRoC dataset. The proposed voxel-map representation is a general approach to a fundamental functionality in visual SLAM and widely applicable. |
| Tasks | |
| Published | 2020-03-04 |
| URL | https://arxiv.org/abs/2003.02247v1 |
| https://arxiv.org/pdf/2003.02247v1.pdf | |
| PWC | https://paperswithcode.com/paper/voxel-map-for-visual-slam |
| Repo | |
| Framework | |
Temporal Embeddings and Transformer Models for Narrative Text Understanding
| Title | Temporal Embeddings and Transformer Models for Narrative Text Understanding |
| Authors | Vani K, Simone Mellace, Alessandro Antonucci |
| Abstract | We present two deep learning approaches to narrative text understanding for character relationship modelling. The temporal evolution of these relations is described by dynamic word embeddings, that are designed to learn semantic changes over time. An empirical analysis of the corresponding character trajectories shows that such approaches are effective in depicting dynamic evolution. A supervised learning approach based on the state-of-the-art transformer model BERT is used instead to detect static relations between characters. The empirical validation shows that such events (e.g., two characters belonging to the same family) might be spotted with good accuracy, even when using automatically annotated data. This provides a deeper understanding of narrative plots based on the identification of key facts. Standard clustering techniques are finally used for character de-aliasing, a necessary pre-processing step for both approaches. Overall, deep learning models appear to be suitable for narrative text understanding, while also providing a challenging and unexploited benchmark for general natural language understanding. |
| Tasks | De-aliasing, Word Embeddings |
| Published | 2020-03-19 |
| URL | https://arxiv.org/abs/2003.08811v1 |
| https://arxiv.org/pdf/2003.08811v1.pdf | |
| PWC | https://paperswithcode.com/paper/temporal-embeddings-and-transformer-models |
| Repo | |
| Framework | |
Graph Neural Networks for Decentralized Controllers
| Title | Graph Neural Networks for Decentralized Controllers |
| Authors | Fernando Gama, Ekaterina Tolstaya, Alejandro Ribeiro |
| Abstract | Dynamical systems comprised of autonomous agents arise in many relevant problems such as multi-agent robotics, smart grids, or smart cities. Controlling these systems is of paramount importance to guarantee a successful deployment. Optimal centralized controllers are readily available but face limitations in terms of scalability and practical implementation. Optimal decentralized controllers, on the other hand, are difficult to find. In this paper, we use graph neural networks (GNNs) to learn decentralized controllers from data. GNNs are well-suited for the task since they are naturally distributed architectures. Furthermore, they are equivariant and stable, leading to good scalability and transferability properties. The problem of flocking is explored to illustrate the power of GNNs in learning decentralized controllers. |
| Tasks | |
| Published | 2020-03-23 |
| URL | https://arxiv.org/abs/2003.10280v1 |
| https://arxiv.org/pdf/2003.10280v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-neural-networks-for-decentralized |
| Repo | |
| Framework | |
MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference
| Title | MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference |
| Authors | Achille Thin, Nikita Kotelevskii, Jean-Stanislas Denain, Leo Grinsztajn, Alain Durmus, Maxim Panov, Eric Moulines |
| Abstract | In this contribution, we propose a new computationally efficient method to combine Variational Inference (VI) with Markov Chain Monte Carlo (MCMC). This approach can be used with generic MCMC kernels, but is especially well suited to \textit{MetFlow}, a novel family of MCMC algorithms we introduce, in which proposals are obtained using Normalizing Flows. The marginal distribution produced by such MCMC algorithms is a mixture of flow-based distributions, thus drastically increasing the expressivity of the variational family. Unlike previous methods following this direction, our approach is amenable to the reparametrization trick and does not rely on computationally expensive reverse kernels. Extensive numerical experiments show clear computational and performance improvements over state-of-the-art methods. |
| Tasks | |
| Published | 2020-02-27 |
| URL | https://arxiv.org/abs/2002.12253v1 |
| https://arxiv.org/pdf/2002.12253v1.pdf | |
| PWC | https://paperswithcode.com/paper/metflow-a-new-efficient-method-for-bridging |
| Repo | |
| Framework | |
Deep Learning for Asset Bubbles Detection
| Title | Deep Learning for Asset Bubbles Detection |
| Authors | Oksana Bashchenko, Alexis Marchal |
| Abstract | We develop a methodology for detecting asset bubbles using a neural network. We rely on the theory of local martingales in continuous-time and use a deep network to estimate the diffusion coefficient of the price process more accurately than the current estimator, obtaining an improved detection of bubbles. We show the outperformance of our algorithm over the existing statistical method in a laboratory created with simulated data. We then apply the network classification to real data and build a zero net exposure trading strategy that exploits the risky arbitrage emanating from the presence of bubbles in the US equity market from 2006 to 2008. The profitability of the strategy provides an estimation of the economical magnitude of bubbles as well as support for the theoretical assumptions relied on. |
| Tasks | |
| Published | 2020-02-15 |
| URL | https://arxiv.org/abs/2002.06405v1 |
| https://arxiv.org/pdf/2002.06405v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-for-asset-bubbles-detection |
| Repo | |
| Framework | |
