April 2, 2020

3078 words 15 mins read

Paper Group ANR 169

Voxel-Based Indoor Reconstruction From HoloLens Triangle Meshes. Motif Difference Field: A Simple and Effective Image Representation of Time Series for Classification. Deep Reinforcement Learning for Autonomous Driving: A Survey. Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers. Unconstrained Bi …

Voxel-Based Indoor Reconstruction From HoloLens Triangle Meshes


Title	Voxel-Based Indoor Reconstruction From HoloLens Triangle Meshes
Authors	P. Hübner, M. Weinmann, S. Wursthorn
Abstract	Current mobile augmented reality devices are often equipped with range sensors. The Microsoft HoloLens for instance is equipped with a Time-Of-Flight (ToF) range camera providing coarse triangle meshes that can be used in custom applications. We suggest to use the triangle meshes for the automatic generation of indoor models that can serve as basis for augmenting their physical counterpart with location-dependent information. In this paper, we present a novel voxel-based approach for automated indoor reconstruction from unstructured three-dimensional geometries like triangle meshes. After an initial voxelization of the input data, rooms are detected in the resulting voxel grid by segmenting connected voxel components of ceiling candidates and extruding them downwards to find floor candidates. Semantic class labels like ‘Wall’, ‘Wall Opening’, ‘Interior Object’ and ‘Empty Interior’ are then assigned to the room voxels in-between ceiling and floor by a rule-based voxel sweep algorithm. Finally, the geometry of the detected walls and their openings is refined in voxel representation. The proposed approach is not restricted to Manhattan World scenarios and does not rely on room surfaces being planar.
Tasks
Published	2020-02-18
URL	https://arxiv.org/abs/2002.07689v1
PDF	https://arxiv.org/pdf/2002.07689v1.pdf
PWC	https://paperswithcode.com/paper/voxel-based-indoor-reconstruction-from
Repo
Framework

Motif Difference Field: A Simple and Effective Image Representation of Time Series for Classification


Title	Motif Difference Field: A Simple and Effective Image Representation of Time Series for Classification
Authors	Yadong Zhang, Xin Chen
Abstract	Time series motifs play an important role in the time series analysis. The motif-based time series clustering is used for the discovery of higher-order patterns or structures in time series data. Inspired by the convolutional neural network (CNN) classifier based on the image representations of time series, motif difference field (MDF) is proposed. Compared to other image representations of time series, MDF is simple and easy to construct. With the Fully Convolution Network (FCN) as the classifier, MDF demonstrates the superior performance on the UCR time series dataset in benchmark with other time series classification methods. It is interesting to find that the triadic time series motifs give the best result in the test. Due to the motif clustering reflected in MDF, the significant motifs are detected with the help of the Gradient-weighted Class Activation Mapping (Grad-CAM). The areas in MDF with high weight in Grad-CAM have a high contribution from the significant motifs with the desired ordinal patterns associated with the signature patterns in time series. However, the signature patterns cannot be identified with the neural network classifiers directly based on the time series.
Tasks	Time Series, Time Series Analysis, Time Series Classification, Time Series Clustering
Published	2020-01-21
URL	https://arxiv.org/abs/2001.07582v1
PDF	https://arxiv.org/pdf/2001.07582v1.pdf
PWC	https://paperswithcode.com/paper/motif-difference-field-a-simple-and-effective
Repo
Framework

Deep Reinforcement Learning for Autonomous Driving: A Survey


Title	Deep Reinforcement Learning for Autonomous Driving: A Survey
Authors	B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A. Al Sallab, Senthil Yogamani, Patrick Pérez
Abstract	With the development of deep representation learning, the domain of reinforcement learning (RL) has become a powerful learning framework now capable of learning complex policies in high dimensional environments. This review summarises deep reinforcement learning (DRL) algorithms, provides a taxonomy of automated driving tasks where (D)RL methods have been employed, highlights the key challenges algorithmically as well as in terms of deployment of real world autonomous driving agents, the role of simulators in training agents, and finally methods to evaluate, test and robustifying existing solutions in RL and imitation learning.
Tasks	Autonomous Driving, Imitation Learning, Representation Learning
Published	2020-02-02
URL	https://arxiv.org/abs/2002.00444v1
PDF	https://arxiv.org/pdf/2002.00444v1.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-for-autonomous-2
Repo
Framework

Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers


Title	Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
Authors	Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez
Abstract	Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference. We study the impact of model size in this setting, focusing on Transformer models for NLP tasks that are limited by compute: self-supervised pretraining and high-resource machine translation. We first show that even though smaller Transformer models execute faster per iteration, wider and deeper models converge in significantly fewer steps. Moreover, this acceleration in convergence typically outpaces the additional computational overhead of using larger models. Therefore, the most compute-efficient training strategy is to counterintuitively train extremely large models but stop after a small number of iterations. This leads to an apparent trade-off between the training efficiency of large Transformer models and the inference efficiency of small Transformer models. However, we show that large models are more robust to compression techniques such as quantization and pruning than small models. Consequently, one can get the best of both worlds: heavily compressed, large models achieve higher accuracy than lightly compressed, small models.
Tasks	Machine Translation, Quantization
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11794v1
PDF	https://arxiv.org/pdf/2002.11794v1.pdf
PWC	https://paperswithcode.com/paper/train-large-then-compress-rethinking-model
Repo
Framework

Unconstrained Biometric Recognition: Summary of Recent SOCIA Lab. Research


Title	Unconstrained Biometric Recognition: Summary of Recent SOCIA Lab. Research
Authors	Varsha Balakrishnan
Abstract	The development of biometric recognition solutions able to work in visual surveillance conditions, i.e., in unconstrained data acquisition conditions and under covert protocols has been motivating growing efforts from the research community. Among the various laboratories, schools and research institutes concerned about this problem, the SOCIA: Soft Computing and Image Analysis Lab., of the University of Beira Interior, Portugal, has been among the most active in pursuing disruptive solutions for obtaining such extremely ambitious kind of automata. This report summarises the research works published by elements of the SOCIA Lab. in the last decade in the scope of biometric recognition in unconstrained conditions. The idea is that it can be used as basis for someone wishing to entering in this research topic.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2001.09703v2
PDF	https://arxiv.org/pdf/2001.09703v2.pdf
PWC	https://paperswithcode.com/paper/unconstrained-biometric-recognition-summary
Repo
Framework

Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training


Title	Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training
Authors	An Xu, Zhouyuan Huo, Heng Huang
Abstract	The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications. In particular, the growing size of deep learning models leads to higher communication overheads that defy the ideal linear training speedup regarding the number of devices. Gradient quantization is one of the common methods to reduce communication costs. However, it can lead to quantization error in the training and result in model performance degradation. In this work, we deduce the optimal condition of both the binary and multi-level gradient quantization for \textbf{ANY} gradient distribution. Based on the optimal condition, we develop two novel quantization schemes: biased BinGrad and unbiased ORQ for binary and multi-level gradient quantization respectively, which dynamically determine the optimal quantization levels. Extensive experimental results on CIFAR and ImageNet datasets with several popular convolutional neural networks show the superiority of our proposed methods.
Tasks	Quantization
Published	2020-02-25
URL	https://arxiv.org/abs/2002.11082v1
PDF	https://arxiv.org/pdf/2002.11082v1.pdf
PWC	https://paperswithcode.com/paper/optimal-gradient-quantization-condition-for
Repo
Framework

A Formal Approach to Explainability


Title	A Formal Approach to Explainability
Authors	Lior Wolf, Tomer Galanti, Tamir Hazan
Abstract	We regard explanations as a blending of the input sample and the model’s output and offer a few definitions that capture various desired properties of the function that generates these explanations. We study the links between these properties and between explanation-generating functions and intermediate representations of learned models and are able to show, for example, that if the activations of a given layer are consistent with an explanation, then so do all other subsequent layers. In addition, we study the intersection and union of explanations as a way to construct new explanations.
Tasks
Published	2020-01-15
URL	https://arxiv.org/abs/2001.05207v1
PDF	https://arxiv.org/pdf/2001.05207v1.pdf
PWC	https://paperswithcode.com/paper/a-formal-approach-to-explainability
Repo
Framework

Learning to rank for uplift modeling


Title	Learning to rank for uplift modeling
Authors	Floris Devriendt, Tias Guns, Wouter Verbeke
Abstract	Uplift modeling has effectively been used in fields such as marketing and customer retention, to target those customers that are most likely to respond due to the campaign or treatment. Uplift models produce uplift scores which are then used to essentially create a ranking. We instead investigate to learn to rank directly by looking into the potential of learning-to-rank techniques in the context of uplift modeling. We propose a unified formalisation of different global uplift modeling measures in use today and explore how these can be integrated into the learning-to-rank framework. Additionally, we introduce a new metric for learning-to-rank that focusses on optimizing the area under the uplift curve called the promoted cumulative gain (PCG). We employ the learning-to-rank technique LambdaMART to optimize the ranking according to PCG and show improved results over standard learning-to-rank metrics and equal to improved results when compared with state-of-the-art uplift modeling. Finally, we show how learning-to-rank models can learn to optimize a certain targeting depth, however, these results do not generalize on the test set.
Tasks	Learning-To-Rank
Published	2020-02-14
URL	https://arxiv.org/abs/2002.05897v1
PDF	https://arxiv.org/pdf/2002.05897v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-rank-for-uplift-modeling
Repo
Framework

Open Domain Question Answering Using Web Tables


Title	Open Domain Question Answering Using Web Tables
Authors	Kaushik Chakrabarti, Zhimin Chen, Siamak Shakeri, Guihong Cao
Abstract	Tables extracted from web documents can be used to directly answer many web search queries. Previous works on question answering (QA) using web tables have focused on factoid queries, i.e., those answerable with a short string like person name or a number. However, many queries answerable using tables are non-factoid in nature. In this paper, we develop an open-domain QA approach using web tables that works for both factoid and non-factoid queries. Our key insight is to combine deep neural network-based semantic similarity between the query and the table with features that quantify the dominance of the table in the document as well as the quality of the information in the table. Our experiments on real-life web search queries show that our approach significantly outperforms state-of-the-art baseline approaches. Our solution is used in production in a major commercial web search engine and serves direct answers for tens of millions of real user queries per month.
Tasks	Open-Domain Question Answering, Question Answering, Semantic Similarity, Semantic Textual Similarity
Published	2020-01-10
URL	https://arxiv.org/abs/2001.03272v1
PDF	https://arxiv.org/pdf/2001.03272v1.pdf
PWC	https://paperswithcode.com/paper/open-domain-question-answering-using-web
Repo
Framework

Sampling Prediction-Matching Examples in Neural Networks: A Probabilistic Programming Approach


Title	Sampling Prediction-Matching Examples in Neural Networks: A Probabilistic Programming Approach
Authors	Serena Booth, Ankit Shah, Yilun Zhou, Julie Shah
Abstract	Though neural network models demonstrate impressive performance, we do not understand exactly how these black-box models make individual predictions. This drawback has led to substantial research devoted to understand these models in areas such as robustness, interpretability, and generalization ability. In this paper, we consider the problem of exploring the prediction level sets of a classifier using probabilistic programming. We define a prediction level set to be the set of examples for which the predictor has the same specified prediction confidence with respect to some arbitrary data distribution. Notably, our sampling-based method does not require the classifier to be differentiable, making it compatible with arbitrary classifiers. As a specific instantiation, if we take the classifier to be a neural network and the data distribution to be that of the training data, we can obtain examples that will result in specified predictions by the neural network. We demonstrate this technique with experiments on a synthetic dataset and MNIST. Such level sets in classification may facilitate human understanding of classification behaviors.
Tasks	Probabilistic Programming
Published	2020-01-09
URL	https://arxiv.org/abs/2001.03076v1
PDF	https://arxiv.org/pdf/2001.03076v1.pdf
PWC	https://paperswithcode.com/paper/sampling-prediction-matching-examples-in
Repo
Framework

SeMemNN: A Semantic Matrix-Based Memory Neural Network for Text Classification


Title	SeMemNN: A Semantic Matrix-Based Memory Neural Network for Text Classification
Authors	Changzeng Fu, Chaoran Liu, Carlos Toshinori Ishi, Yuichiro Yoshikawa, Hiroshi Ishiguro
Abstract	Text categorization is the task of assigning labels to documents written in a natural language, and it has numerous real-world applications including sentiment analysis as well as traditional topic assignment tasks. In this paper, we propose 5 different configurations for the semantic matrix-based memory neural network with end-to-end learning manner and evaluate our proposed method on two corpora of news articles (AG news, Sogou news). The best performance of our proposed method outperforms the baseline VDCNN models on the text classification task and gives a faster speed for learning semantics. Moreover, we also evaluate our model on small scale datasets. The results show that our proposed method can still achieve better results in comparison to VDCNN on the small scale dataset. This paper is to appear in the Proceedings of the 2020 IEEE 14th International Conference on Semantic Computing (ICSC 2020), San Diego, California, 2020.
Tasks	Sentiment Analysis, Text Categorization, Text Classification
Published	2020-03-04
URL	https://arxiv.org/abs/2003.01857v1
PDF	https://arxiv.org/pdf/2003.01857v1.pdf
PWC	https://paperswithcode.com/paper/sememnn-a-semantic-matrix-based-memory-neural
Repo
Framework

MCMC Guided CNN Training and Segmentation for Pancreas Extraction


Title	MCMC Guided CNN Training and Segmentation for Pancreas Extraction
Authors	Jinchan He, Xiaxia Yu, Chudong Cai, Yi Gao
Abstract	Efficient organ segmentation is the precondition of various quantitative analysis. Segmenting the pancreas from abdominal CT images is a challenging task because of its high anatomical variability in shape, size and location. What’s more, the pancreas only occupies a small portion in abdomen, and the organ border is very fuzzy. All these factors make the segmentation methods of other organs less suitable for the pancreas segmentation. In this report, we propose a Markov Chain Monte Carlo (MCMC) sampling guided convolutional neural network (CNN) approach, in order to handle such difficulties in morphological and photometric variabilities. Specifically, the proposed method mainly contains three steps: First, registration is carried out to mitigate the body weight and location variability. Then, an MCMC sampling is employed to guide the sampling of 3D patches, which are fed to the CNN for training. At the same time, the pancreas distribution is also learned for the subsequent segmentation. Third, sampled from the learned distribution, an MCMC process guides the segmentation process. Lastly, the patches based segmentation is fused using a Bayesian voting scheme. This method is evaluated on the NIH pancreatic datasets which contains 82 abdominal contrast-enhanced CT volumes. Finally, we achieved a competitive result of 78.13% Dice Similarity Coefficient value and 82.65% Recall value in testing data.
Tasks	Pancreas Segmentation
Published	2020-03-09
URL	https://arxiv.org/abs/2003.03938v1
PDF	https://arxiv.org/pdf/2003.03938v1.pdf
PWC	https://paperswithcode.com/paper/mcmc-guided-cnn-training-and-segmentation-for
Repo
Framework

KryptoOracle: A Real-Time Cryptocurrency Price Prediction Platform Using Twitter Sentiments


Title	KryptoOracle: A Real-Time Cryptocurrency Price Prediction Platform Using Twitter Sentiments
Authors	Shubhankar Mohapatra, Nauman Ahmed, Paulo Alencar
Abstract	Cryptocurrencies, such as Bitcoin, are becoming increasingly popular, having been widely used as an exchange medium in areas such as financial transaction and asset transfer verification. However, there has been a lack of solutions that can support real-time price prediction to cope with high currency volatility, handle massive heterogeneous data volumes, including social media sentiments, while supporting fault tolerance and persistence in real time, and provide real-time adaptation of learning algorithms to cope with new price and sentiment data. In this paper we introduce KryptoOracle, a novel real-time and adaptive cryptocurrency price prediction platform based on Twitter sentiments. The integrative and modular platform is based on (i) a Spark-based architecture which handles the large volume of incoming data in a persistent and fault tolerant way; (ii) an approach that supports sentiment analysis which can respond to large amounts of natural language processing queries in real time; and (iii) a predictive method grounded on online learning in which a model adapts its weights to cope with new prices and sentiments. Besides providing an architectural design, the paper also describes the KryptoOracle platform implementation and experimental evaluation. Overall, the proposed platform can help accelerate decision-making, uncover new opportunities and provide more timely insights based on the available and ever-larger financial data volume and variety.
Tasks	Decision Making, Sentiment Analysis
Published	2020-02-21
URL	https://arxiv.org/abs/2003.04967v1
PDF	https://arxiv.org/pdf/2003.04967v1.pdf
PWC	https://paperswithcode.com/paper/kryptooracle-a-real-time-cryptocurrency-price
Repo
Framework

Quality Diversity for Multi-task Optimization


Title	Quality Diversity for Multi-task Optimization
Authors	Jean-Baptiste Mouret, Glenn Maguire
Abstract	Quality Diversity (QD) algorithms are a recent family of optimization algorithms that search for a large set of diverse but high-performing solutions. Interestingly, they can solve multiple tasks at once. For instance, they can find the joint positions required for a robotic arm to reach a set of points, which can also be solved by running a classic optimizer for each target point. However, they cannot solve multiple tasks when the fitness needs to be evaluated independently for each task (e.g., optimizing policies to grasp many different objects). In this paper, we propose an extension of the MAP-Elites algorithm, called Multi-task MAP-Elites, that solves multiple tasks when the fitness function depends on the task. We evaluate it on a simulated parametrized planar arm (10-dimensional search space; 5000 tasks) and on a 6-legged robot with legs of different lengths (36-dimensional search space; 2000 tasks). The results show that in both cases our algorithm outperforms the optimization of each task separately with the CMA-ES algorithm.
Tasks
Published	2020-03-09
URL	https://arxiv.org/abs/2003.04407v1
PDF	https://arxiv.org/pdf/2003.04407v1.pdf
PWC	https://paperswithcode.com/paper/quality-diversity-for-multi-task-optimization
Repo
Framework

Neural Architecture Search over Decentralized Data


Title	Neural Architecture Search over Decentralized Data
Authors	Mengwei Xu, Yuxin Zhao, Kaigui Bian, Gang Huang, Qiaozhu Mei, Xuanzhe Liu
Abstract	To preserve user privacy while enabling mobile intelligence, techniques have been proposed to train deep neural networks on decentralized data. However, training over decentralized data makes the design of neural architecture quite difficult as it already was. Such difficulty is further amplified when designing and deploying different neural architectures for heterogeneous mobile platforms. In this work, we propose an automatic neural architecture search into the decentralized training, as a new DNN training paradigm called Federated Neural Architecture Search, namely federated NAS. To deal with the primary challenge of limited on-client computational and communication resources, we present FedNAS, a highly optimized framework for efficient federated NAS. FedNAS fully exploits the key opportunity of insufficient model candidate re-training during the architecture search process, and incorporates three key optimizations: parallel candidates training on partial clients, early dropping candidates with inferior performance, and dynamic round numbers. Tested on large-scale datasets and typical CNN architectures, FedNAS achieves comparable model accuracy as state-of-the-art NAS algorithm that trains models with centralized data, and also reduces the client cost by up to two orders of magnitude compared to a straightforward design of federated NAS.
Tasks	Neural Architecture Search
Published	2020-02-15
URL	https://arxiv.org/abs/2002.06352v2
PDF	https://arxiv.org/pdf/2002.06352v2.pdf
PWC	https://paperswithcode.com/paper/neural-architecture-search-over-decentralized
Repo
Framework