April 1, 2020

3574 words 17 mins read

Paper Group ANR 438

ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions. Approximate MMAP by Marginal Search. Curriculum DeepSDF. Data Curves Clustering Using Common Patterns Detection. Drift-Adjusted And Arbitrated Ensemble Framework For Time Series Forecasting. High-Dimensional Independence Testing and Maximum Marginal Correlation. …

ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions


Title	ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
Authors	Zechun Liu, Zhiqiang Shen, Marios Savvides, Kwang-Ting Cheng
Abstract	In this paper, we propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost. We first construct a baseline network by modifying and binarizing a compact real-valued network with parameter-free shortcuts, bypassing all the intermediate convolutional layers including the downsampling layers. This baseline network strikes a good trade-off between accuracy and efficiency, achieving superior performance than most of existing binary networks at approximately half of the computational cost. Through extensive experiments and analysis, we observed that the performance of binary networks is sensitive to activation distribution variations. Based on this important observation, we propose to generalize the traditional Sign and PReLU functions, denoted as RSign and RPReLU for the respective generalized functions, to enable explicit learning of the distribution reshape and shift at near-zero extra cost. Lastly, we adopt a distributional loss to further enforce the binary network to learn similar output distributions as those of a real-valued network. We show that after incorporating all these ideas, the proposed ReActNet outperforms all the state-of-the-arts by a large margin. Specifically, it outperforms Real-to-Binary Net and MeliusNet29 by 4.0% and 3.6% respectively for the top-1 accuracy and also reduces the gap to its real-valued counterpart to within 3.0% top-1 accuracy on ImageNet dataset.
Tasks
Published	2020-03-07
URL	https://arxiv.org/abs/2003.03488v1
PDF	https://arxiv.org/pdf/2003.03488v1.pdf
PWC	https://paperswithcode.com/paper/reactnet-towards-precise-binary-neural
Repo
Framework

Approximate MMAP by Marginal Search


Title	Approximate MMAP by Marginal Search
Authors	Alessandro Antonucci, Thomas Tiotto
Abstract	We present a heuristic strategy for marginal MAP (MMAP) queries in graphical models. The algorithm is based on a reduction of the task to a polynomial number of marginal inference computations. Given an input evidence, the marginals mass functions of the variables to be explained are computed. Marginal information gain is used to decide the variables to be explained first, and their most probable marginal states are consequently moved to the evidence. The sequential iteration of this procedure leads to a MMAP explanation and the minimum information gain obtained during the process can be regarded as a confidence measure for the explanation. Preliminary experiments show that the proposed confidence measure is properly detecting instances for which the algorithm is accurate and, for sufficiently high confidence levels, the algorithm gives the exact solution or an approximation whose Hamming distance from the exact one is small.
Tasks
Published	2020-02-12
URL	https://arxiv.org/abs/2002.04827v1
PDF	https://arxiv.org/pdf/2002.04827v1.pdf
PWC	https://paperswithcode.com/paper/approximate-mmap-by-marginal-search
Repo
Framework

Curriculum DeepSDF


Title	Curriculum DeepSDF
Authors	Yueqi Duan, Haidong Zhu, He Wang, Li Yi, Ram Nevatia, Leonidas J. Guibas
Abstract	When learning to sketch, beginners start with simple and flexible shapes, and then gradually strive for more complex and accurate ones in the subsequent training sessions. In this paper, we design a “shape curriculum” for learning continuous Signed Distance Function (SDF) on shapes, namely Curriculum DeepSDF. Inspired by how humans learn, Curriculum DeepSDF organizes the learning task in ascending order of difficulty according to the following two criteria: surface accuracy and sample difficulty. The former considers stringency in supervising with ground truth, while the latter regards the weights of hard training samples near complex geometry and fine structure. More specifically, Curriculum DeepSDF learns to reconstruct coarse shapes at first, and then gradually increases the accuracy and focuses more on complex local details. Experimental results show that a carefully-designed curriculum leads to significantly better shape reconstructions with the same training data, training epochs and network architecture as DeepSDF. We believe that the application of shape curricula can benefit the training process of a wide variety of 3D shape representation learning methods.
Tasks	3D Shape Representation, Representation Learning
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08593v1
PDF	https://arxiv.org/pdf/2003.08593v1.pdf
PWC	https://paperswithcode.com/paper/curriculum-deepsdf
Repo
Framework

Data Curves Clustering Using Common Patterns Detection


Title	Data Curves Clustering Using Common Patterns Detection
Authors	Konstantinos F. Xylogiannopoulos
Abstract	For the past decades we have experienced an enormous expansion of the accumulated data that humanity produces. Daily a numerous number of smart devices, usually interconnected over internet, produce vast, real-values datasets. Time series representing datasets from completely irrelevant domains such as finance, weather, medical applications, traffic control etc. become more and more crucial in human day life. Analyzing and clustering these time series, or in general any kind of curves, could be critical for several human activities. In the current paper, the new Curves Clustering Using Common Patterns (3CP) methodology is introduced, which applies a repeated pattern detection algorithm in order to cluster sequences according to their shape and the similarities of common patterns between time series, data curves and eventually any kind of discrete sequences. For this purpose, the Longest Expected Repeated Pattern Reduced Suffix Array (LERP-RSA) data structure has been used in combination with the All Repeated Patterns Detection (ARPaD) algorithm in order to perform highly accurate and efficient detection of similarities among data curves that can be used for clustering purposes and which also provides additional flexibility and features.
Tasks	Time Series
Published	2020-01-05
URL	https://arxiv.org/abs/2001.02095v1
PDF	https://arxiv.org/pdf/2001.02095v1.pdf
PWC	https://paperswithcode.com/paper/data-curves-clustering-using-common-patterns
Repo
Framework

Drift-Adjusted And Arbitrated Ensemble Framework For Time Series Forecasting


Title	Drift-Adjusted And Arbitrated Ensemble Framework For Time Series Forecasting
Authors	Anirban Chatterjee, Subhadip Paul, Uddipto Dutta, Smaranya Dey
Abstract	Time Series Forecasting is at the core of many practical applications such as sales forecasting for business, rainfall forecasting for agriculture and many others. Though this problem has been extensively studied for years, it is still considered a challenging problem due to complex and evolving nature of time series data. Typical methods proposed for time series forecasting modeled linear or non-linear dependencies between data observations. However it is a generally accepted notion that no one method is universally effective for all kinds of time series data. Attempts have been made to use dynamic and weighted combination of heterogeneous and independent forecasting models and it has been found to be a promising direction to tackle this problem. This method is based on the assumption that different forecasters have different specialization and varying performance for different distribution of data and weights are dynamically assigned to multiple forecasters accordingly. However in many practical time series data-set, the distribution of data slowly evolves with time. We propose to employ a re-weighting based method to adjust the assigned weights to various forecasters in order to account for such distribution-drift. An exhaustive testing was performed against both real-world and synthesized time-series. Experimental results show the competitiveness of the method in comparison to state-of-the-art approaches for combining forecasters and handling drift.
Tasks	Time Series, Time Series Forecasting
Published	2020-03-16
URL	https://arxiv.org/abs/2003.09311v1
PDF	https://arxiv.org/pdf/2003.09311v1.pdf
PWC	https://paperswithcode.com/paper/drift-adjusted-and-arbitrated-ensemble
Repo
Framework

High-Dimensional Independence Testing and Maximum Marginal Correlation


Title	High-Dimensional Independence Testing and Maximum Marginal Correlation
Authors	Cencheng Shen
Abstract	A number of universally consistent dependence measures have been recently proposed for testing independence, such as distance correlation, kernel correlation, multiscale graph correlation, etc. They provide a satisfactory solution for dependence testing in low-dimensions, but often exhibit decreasing power for high-dimensional data, a phenomenon that has been recognized but remains mostly unchartered. In this paper, we aim to better understand the high-dimensional testing scenarios and explore a procedure that is robust against increasing dimension. To that end, we propose the maximum marginal correlation method and characterize high-dimensional dependence structures via the notion of dependent dimensions. We prove that the maximum method can be valid and universally consistent for testing high-dimensional dependence under regularity conditions, and demonstrate when and how the maximum method may outperform other methods. The methodology can be implemented by most existing dependence measures, has a superior testing power in a variety of common high-dimensional settings, and is computationally efficient for big data analysis when using the distance correlation chi-square test.
Tasks
Published	2020-01-04
URL	https://arxiv.org/abs/2001.01095v1
PDF	https://arxiv.org/pdf/2001.01095v1.pdf
PWC	https://paperswithcode.com/paper/high-dimensional-independence-testing-and
Repo
Framework

Time series and machine learning to forecast the water quality from satellite data


Title	Time series and machine learning to forecast the water quality from satellite data
Authors	Maryam R. Al Shehhi, Abdullah Kaya
Abstract	Managing the quality of water for present and future generations of coastal regions should be a central concern of both citizens and public officials. Remote sensing can contribute to the management and monitoring of coastal water and pollutants. Algal blooms are a coastal pollutant that is a cause of concern. Many satellite data, such as MODIS, have been used to generate water-quality products to detect the blooms such as chlorophyll a (Chl-a), a photosynthesis index called fluorescence line height (FLH), and sea surface temperature (SST). It is important to characterize the spatial and temporal variations of these water quality products by using the mathematical models of these products. However, for monitoring, pollution control boards will need nowcasts and forecasts of any pollution. Therefore, we aim to predict the future values of the MODIS Chl-a, FLH, and SST of the water. This will not be limited to one type of water but, rather, will cover different types of water varying in depth and turbidity. This is very significant because the temporal trend of Chl-a, FLH, and SST is dependent on the geospatial and water properties. For this purpose, we will decompose the time series of each pixel into several components: trend, intra-annual variations, seasonal cycle, and stochastic stationary. We explore three such time series machine learning models that can characterize the non-stationary time series data and predict future values, including the Seasonal ARIMA (Auto Regressive Integrated Moving Average) (SARIMA), regression, and neural network. The results indicate that all these methods are effective at modelling Chl-a, FLH, and SST time series and predicting the values reasonably well. However, regression and neural network are found to be the best at predicting Chl-a in all types of water (turbid and shallow). Meanwhile, the SARIMA model provides the best prediction of FLH and SST.
Tasks	Time Series
Published	2020-03-16
URL	https://arxiv.org/abs/2003.11923v1
PDF	https://arxiv.org/pdf/2003.11923v1.pdf
PWC	https://paperswithcode.com/paper/time-series-and-machine-learning-to-forecast
Repo
Framework

General linear-time inference for Gaussian Processes on one dimension


Title	General linear-time inference for Gaussian Processes on one dimension
Authors	Jackson Loper, David Blei, John P. Cunningham, Liam Paninski
Abstract	Gaussian Processes (GPs) provide a powerful probabilistic framework for interpolation, forecasting, and smoothing, but have been hampered by computational scaling issues. Here we prove that for data sampled on one dimension (e.g., a time series sampled at arbitrarily-spaced intervals), approximate GP inference at any desired level of accuracy requires computational effort that scales linearly with the number of observations; this new theorem enables inference on much larger datasets than was previously feasible. To achieve this improved scaling we propose a new family of stationary covariance kernels: the Latent Exponentially Generated (LEG) family, which admits a convenient stable state-space representation that allows linear-time inference. We prove that any continuous integrable stationary kernel can be approximated arbitrarily well by some member of the LEG family. The proof draws connections to Spectral Mixture Kernels, providing new insight about the flexibility of this popular family of kernels. We propose parallelized algorithms for performing inference and learning in the LEG model, test the algorithm on real and synthetic data, and demonstrate scaling to datasets with billions of samples.
Tasks	Gaussian Processes, Time Series
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05554v1
PDF	https://arxiv.org/pdf/2003.05554v1.pdf
PWC	https://paperswithcode.com/paper/general-linear-time-inference-for-gaussian
Repo
Framework

Human-in-the-Loop Design Cycles – A Process Framework that Integrates Design Sprints, Agile Processes, and Machine Learning with Humans


Title	Human-in-the-Loop Design Cycles – A Process Framework that Integrates Design Sprints, Agile Processes, and Machine Learning with Humans
Authors	Chaehan So
Abstract	Demands on more transparency of the backbox nature of machine learning models have led to the recent rise of human-in-the-loop in machine learning, i.e. processes that integrate humans in the training and application of machine learning models. The present work argues that this process requirement does not represent an obstacle but an opportunity to optimize the design process. Hence, this work proposes a new process framework, Human-in-the-learning-loop (HILL) Design Cycles - a design process that integrates the structural elements of agile and design thinking process, and controls the training of a machine learning model by the human in the loop. The HILL Design Cycles process replaces the qualitative user testing by a quantitative psychometric measurement instrument for design perception. The generated user feedback serves to train a machine learning model and to instruct the subsequent design cycle along four design dimensions (novelty, energy, simplicity, tool). Mapping the four-dimensional user feedback into user stories and priorities, the design sprint thus transforms the user feedback directly into the implementation process. The human in the loop is a quality engineer who scrutinizes the collected user feedback to prevents invalid data to enter machine learning model training.
Tasks
Published	2020-02-29
URL	https://arxiv.org/abs/2003.05268v1
PDF	https://arxiv.org/pdf/2003.05268v1.pdf
PWC	https://paperswithcode.com/paper/human-in-the-loop-design-cycles-a-process
Repo
Framework

Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection


Title	Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection
Authors	Guansong Pang, Cheng Yan, Chunhua Shen, Anton van den Hengel, Xiao Bai
Abstract	Video anomaly detection is of critical practical importance to a variety of real applications because it allows human attention to be focused on events that are likely to be of interest, in spite of an otherwise overwhelming volume of video. We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods, namely, 1) being highly dependent on manually labeled normal training data; and 2) sub-optimal feature learning. By formulating a surrogate two-class ordinal regression task we devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data. Experiments on eight real-world video scenes show that our proposed method outperforms state-of-the-art methods that require no labeled training data by a substantial margin, and enables easy and accurate localization of the identified anomalies. Furthermore, we demonstrate that our method offers effective human-in-the-loop anomaly detection which can be critical in applications where anomalies are rare and the false-negative cost is high.
Tasks	Anomaly Detection, Representation Learning
Published	2020-03-15
URL	https://arxiv.org/abs/2003.06780v1
PDF	https://arxiv.org/pdf/2003.06780v1.pdf
PWC	https://paperswithcode.com/paper/self-trained-deep-ordinal-regression-for-end
Repo
Framework

Active Perception and Representation for Robotic Manipulation


Title	Active Perception and Representation for Robotic Manipulation
Authors	Youssef Zaky, Gaurav Paruthi, Bryan Tripp, James Bergstra
Abstract	The vast majority of visual animals actively control their eyes, heads, and/or bodies to direct their gaze toward different parts of their environment. In contrast, recent applications of reinforcement learning in robotic manipulation employ cameras as passive sensors. These are carefully placed to view a scene from a fixed pose. Active perception allows animals to gather the most relevant information about the world and focus their computational resources where needed. It also enables them to view objects from different distances and viewpoints, providing a rich visual experience from which to learn abstract representations of the environment. Inspired by the primate visual-motor system, we present a framework that leverages the benefits of active perception to accomplish manipulation tasks. Our agent uses viewpoint changes to localize objects, to learn state representations in a self-supervised manner, and to perform goal-directed actions. We apply our model to a simulated grasping task with a 6-DoF action space. Compared to its passive, fixed-camera counterpart, the active model achieves 8% better performance in targeted grasping. Compared to vanilla deep Q-learning algorithms, our model is at least four times more sample-efficient, highlighting the benefits of both active perception and representation learning.
Tasks	Q-Learning, Representation Learning
Published	2020-03-15
URL	https://arxiv.org/abs/2003.06734v1
PDF	https://arxiv.org/pdf/2003.06734v1.pdf
PWC	https://paperswithcode.com/paper/active-perception-and-representation-for
Repo
Framework

Mission-Aware Spatio-Temporal Deep Learning Model for UAS Instantaneous Density Prediction


Title	Mission-Aware Spatio-Temporal Deep Learning Model for UAS Instantaneous Density Prediction
Authors	Ziyi Zhao, Zhao Jin, Wentian Bai, Wentan Bai, Carlos Caicedo, M. Cenk Gursoy, Qinru Qiu
Abstract	The number of daily sUAS operations in uncontrolled low altitude airspace is expected to reach into the millions in a few years. Therefore, UAS density prediction has become an emerging and challenging problem. In this paper, a deep learning-based UAS instantaneous density prediction model is presented. The model takes two types of data as input: 1) the historical density generated from the historical data, and 2) the future sUAS mission information. The architecture of our model contains four components: Historical Density Formulation module, UAS Mission Translation module, Mission Feature Extraction module, and Density Map Projection module. The training and testing data are generated by a python based simulator which is inspired by the multi-agent air traffic resource usage simulator (MATRUS) framework. The quality of prediction is measured by the correlation score and the Area Under the Receiver Operating Characteristics (AUROC) between the predicted value and simulated value. The experimental results demonstrate outstanding performance of the deep learning-based UAS density predictor. Compared to the baseline models, for simplified traffic scenario where no-fly zones and safe distance among sUASs are not considered, our model improves the prediction accuracy by more than 15.2% and its correlation score reaches 0.947. In a more realistic scenario, where the no-fly zone avoidance and the safe distance among sUASs are maintained using A* routing algorithm, our model can still achieve 0.823 correlation score. Meanwhile, the AUROC can reach 0.951 for the hot spot prediction.
Tasks
Published	2020-03-22
URL	https://arxiv.org/abs/2003.09785v1
PDF	https://arxiv.org/pdf/2003.09785v1.pdf
PWC	https://paperswithcode.com/paper/mission-aware-spatio-temporal-deep-learning
Repo
Framework

A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching


Title	A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching
Authors	Venkata Vamsikrishna Meduri, Lucian Popa, Prithviraj Sen, Mohamed Sarwat
Abstract	Entity Matching (EM) is a core data cleaning task, aiming to identify different mentions of the same real-world entity. Active learning is one way to address the challenge of scarce labeled data in practice, by dynamically collecting the necessary examples to be labeled by an Oracle and refining the learned model (classifier) upon them. In this paper, we build a unified active learning benchmark framework for EM that allows users to easily combine different learning algorithms with applicable example selection algorithms. The goal of the framework is to enable concrete guidelines for practitioners as to what active learning combinations will work well for EM. Towards this, we perform comprehensive experiments on publicly available EM datasets from product and publication domains to evaluate active learning methods, using a variety of metrics including EM quality, #labels and example selection latencies. Our most surprising result finds that active learning with fewer labels can learn a classifier of comparable quality as supervised learning. In fact, for several of the datasets, we show that there is an active learning combination that beats the state-of-the-art supervised learning result. Our framework also includes novel optimizations that improve the quality of the learned model by roughly 9% in terms of F1-score and reduce example selection latencies by up to 10x without affecting the quality of the model.
Tasks	Active Learning
Published	2020-03-29
URL	https://arxiv.org/abs/2003.13114v1
PDF	https://arxiv.org/pdf/2003.13114v1.pdf
PWC	https://paperswithcode.com/paper/a-comprehensive-benchmark-framework-for
Repo
Framework

Integrating Crowdsourcing and Active Learning for Classification of Work-Life Events from Tweets


Title	Integrating Crowdsourcing and Active Learning for Classification of Work-Life Events from Tweets
Authors	Yunpeng Zhao, Mattia Prosperi, Tianchen Lyu, Yi Guo, Jing Bian
Abstract	Social media, especially Twitter, is being increasingly used for research with predictive analytics. In social media studies, natural language processing (NLP) techniques are used in conjunction with expert-based, manual and qualitative analyses. However, social media data are unstructured and must undergo complex manipulation for research use. The manual annotation is the most resource and time-consuming process that multiple expert raters have to reach consensus on every item, but is essential to create gold-standard datasets for training NLP-based machine learning classifiers. To reduce the burden of the manual annotation, yet maintaining its reliability, we devised a crowdsourcing pipeline combined with active learning strategies. We demonstrated its effectiveness through a case study that identifies job loss events from individual tweets. We used Amazon Mechanical Turk platform to recruit annotators from the Internet and designed a number of quality control measures to assure annotation accuracy. We evaluated 4 different active learning strategies (i.e., least confident, entropy, vote entropy, and Kullback-Leibler divergence). The active learning strategies aim at reducing the number of tweets needed to reach a desired performance of automated classification. Results show that crowdsourcing is useful to create high-quality annotations and active learning helps in reducing the number of required tweets, although there was no substantial difference among the strategies tested.
Tasks	Active Learning
Published	2020-03-26
URL	https://arxiv.org/abs/2003.12139v1
PDF	https://arxiv.org/pdf/2003.12139v1.pdf
PWC	https://paperswithcode.com/paper/integrating-crowdsourcing-and-active-learning
Repo
Framework

Top-1 Solution of Multi-Moments in Time Challenge 2019


Title	Top-1 Solution of Multi-Moments in Time Challenge 2019
Authors	Manyuan Zhang, Hao Shao, Guanglu Song, Yu Liu, Junjie Yan
Abstract	In this technical report, we briefly introduce the solutions of our team ‘Efficient’ for the Multi-Moments in Time challenge in ICCV 2019. We first conduct several experiments with popular Image-Based action recognition methods TRN, TSN, and TSM. Then a novel temporal interlacing network is proposed towards fast and accurate recognition. Besides, the SlowFast network and its variants are explored. Finally, we ensemble all the above models and achieve 67.22% on the validation set and 60.77% on the test set, which ranks 1st on the final leaderboard. In addition, we release a new code repository for video understanding which unifies state-of-the-art 2D and 3D methods based on PyTorch. The solution of the challenge is also included in the repository, which is available at https://github.com/Sense-X/X-Temporal.
Tasks	Video Understanding
Published	2020-03-12
URL	https://arxiv.org/abs/2003.05837v2
PDF	https://arxiv.org/pdf/2003.05837v2.pdf
PWC	https://paperswithcode.com/paper/top-1-solution-of-multi-moments-in-time
Repo
Framework