January 27, 2020

3251 words 16 mins read

Paper Group ANR 1239

Machine Learning and the Internet of Things Enable Steam Flood Optimization for Improved Oil Production. “Bring Your Own Greedy"+Max: Near-Optimal $1/2$-Approximations for Submodular Knapsack. Graph heat mixture model learning. Robot-Friendly Cities. Emergence of Compositional Language with Deep Generational Transmission. Sentiment Analysis for Ara …

Machine Learning and the Internet of Things Enable Steam Flood Optimization for Improved Oil Production


Title	Machine Learning and the Internet of Things Enable Steam Flood Optimization for Improved Oil Production
Authors	Mi Yan, Jonathan C. MacDonald, Chris T. Reaume, Wesley Cobb, Tamas Toth, Sarah S. Karthigan
Abstract	Recently developed machine learning techniques, in association with the Internet of Things (IoT) allow for the implementation of a method of increasing oil production from heavy-oil wells. Steam flood injection, a widely used enhanced oil recovery technique, uses thermal and gravitational potential to mobilize and dilute heavy oil in situ to increase oil production. In contrast to traditional steam flood simulations based on principles of classic physics, we introduce here an approach using cutting-edge machine learning techniques that have the potential to provide a better way to describe the performance of steam flood. We propose a workflow to address a category of time-series data that can be analyzed with supervised machine learning algorithms and IoT. We demonstrate the effectiveness of the technique for forecasting oil production in steam flood scenarios. Moreover, we build an optimization system that recommends an optimal steam allocation plan, and show that it leads to a 3% improvement in oil production. We develop a minimum viable product on a cloud platform that can implement real-time data collection, transfer, and storage, as well as the training and implementation of a cloud-based machine learning model. This workflow also offers an applicable solution to other problems with similar time-series data structures, like predictive maintenance.
Tasks	Time Series
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11319v2
PDF	https://arxiv.org/pdf/1908.11319v2.pdf
PWC	https://paperswithcode.com/paper/machine-learning-and-the-internet-of-things
Repo
Framework

“Bring Your Own Greedy"+Max: Near-Optimal $1/2$-Approximations for Submodular Knapsack


Title	“Bring Your Own Greedy"+Max: Near-Optimal $1/2$-Approximations for Submodular Knapsack
Authors	Dmitrii Avdiukhin, Grigory Yaroslavtsev, Samson Zhou
Abstract	The problem of selecting a small-size representative summary of a large dataset is a cornerstone of machine learning, optimization and data science. Motivated by applications to recommendation systems and other scenarios with query-limited access to vast amounts of data, we propose a new rigorous algorithmic framework for a standard formulation of this problem as a submodular maximization subject to a linear (knapsack) constraint. Our framework is based on augmenting all partial Greedy solutions with the best additional item. It can be instantiated with negligible overhead in any model of computation, which allows the classic \greedy algorithm and its variants to be implemented. We give such instantiations in the offline (Greedy+Max), multi-pass streaming (Sieve+Max) and distributed (Distributed+Max) settings. Our algorithms give ($1/2-\epsilon$)-approximation with most other key parameters of interest being near-optimal. Our analysis is based on a new set of first-order linear differential inequalities and their robust approximate versions. Experiments on typical datasets (movie recommendations, influence maximization) confirm scalability and high quality of solutions obtained via our framework. Instance-specific approximations are typically in the 0.6-0.7 range and frequently beat even the $(1-1/e) \approx 0.63$ worst-case barrier for polynomial-time algorithms.
Tasks	Recommendation Systems
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05646v1
PDF	https://arxiv.org/pdf/1910.05646v1.pdf
PWC	https://paperswithcode.com/paper/bring-your-own-greedymax-near-optimal-12
Repo
Framework

Graph heat mixture model learning


Title	Graph heat mixture model learning
Authors	Hermina Petric Maretic, Mireille El Gheche, Pascal Frossard
Abstract	Graph inference methods have recently attracted a great interest from the scientific community, due to the large value they bring in data interpretation and analysis. However, most of the available state-of-the-art methods focus on scenarios where all available data can be explained through the same graph, or groups corresponding to each graph are known a priori. In this paper, we argue that this is not always realistic and we introduce a generative model for mixed signals following a heat diffusion process on multiple graphs. We propose an expectation-maximisation algorithm that can successfully separate signals into corresponding groups, and infer multiple graphs that govern their behaviour. We demonstrate the benefits of our method on both synthetic and real data.
Tasks
Published	2019-01-24
URL	http://arxiv.org/abs/1901.08585v1
PDF	http://arxiv.org/pdf/1901.08585v1.pdf
PWC	https://paperswithcode.com/paper/graph-heat-mixture-model-learning
Repo
Framework

Robot-Friendly Cities


Title	Robot-Friendly Cities
Authors	Seng W. Loke
Abstract	Robots are increasingly tested in public spaces, towards a future where urban environments are not only for humans but for autonomous systems. While robots are promising, for convenience and efficiency, there are challenges associated with building cities crowded with machines. This paper provides an overview of the problems and some solutions, and calls for greater attention on this matter.
Tasks
Published	2019-10-22
URL	https://arxiv.org/abs/1910.10258v1
PDF	https://arxiv.org/pdf/1910.10258v1.pdf
PWC	https://paperswithcode.com/paper/robot-friendly-cities
Repo
Framework

Emergence of Compositional Language with Deep Generational Transmission


Title	Emergence of Compositional Language with Deep Generational Transmission
Authors	Michael Cogswell, Jiasen Lu, Stefan Lee, Devi Parikh, Dhruv Batra
Abstract	Consider a collaborative task that requires communication. Two agents are placed in an environment and must create a language from scratch in order to coordinate. Recent work has been interested in what kinds of languages emerge when deep reinforcement learning agents are put in such a situation, and in particular in the factors that cause language to be compositional-i.e. meaning is expressed by combining words which themselves have meaning. Evolutionary linguists have also studied the emergence of compositional language for decades, and they find that in addition to structural priors like those already studied in deep learning, the dynamics of transmitting language from generation to generation contribute significantly to the emergence of compositionality. In this paper, we introduce these cultural evolutionary dynamics into language emergence by periodically replacing agents in a population to create a knowledge gap, implicitly inducing cultural transmission of language. We show that this implicit cultural transmission encourages the resulting languages to exhibit better compositional generalization and suggest how elements of cultural dynamics can be further integrated into populations of deep agents.
Tasks
Published	2019-04-19
URL	http://arxiv.org/abs/1904.09067v1
PDF	http://arxiv.org/pdf/1904.09067v1.pdf
PWC	https://paperswithcode.com/paper/emergence-of-compositional-language-with-deep
Repo
Framework


Title	Sentiment Analysis for Arabic in Social Media Network: A Systematic Mapping Study
Authors	Mohamed Elhag M. Abo, Ram Gopal Raj, Atika Qazi, Abubakar Zakari
Abstract	With the expansion in tenders on the Internet and social media, Arabic Sentiment Analysis (ASA) has assumed a significant position in the field of text mining study and has since remained used to explore the sentiments of users about services, various products or topics conversed over the Internet. This mapping paper designs to comprehensively investigate the papers demographics, fertility, and directions of the ASA research domain. Furthermore, plans to analyze current ASA techniques and find movements in the research. This paper describes a systematic mapping study (SMS) of 51 primary selected studies (PSS) is handled with the approval of an evidence-based systematic method to ensure handling of all related papers. The analyzed results showed the increase of both the ASA research area and numbers of publications per year since 2015. Three main research facets were found, i.e. validation, solution, and evaluation research, with solution research becoming more treatment than another research type. Therefore numerous contribution facets were singled out. In totality, the general demographics of the ASA research field were highlighted and discussed
Tasks	Arabic Sentiment Analysis, Sentiment Analysis
Published	2019-10-26
URL	https://arxiv.org/abs/1911.05483v1
PDF	https://arxiv.org/pdf/1911.05483v1.pdf
PWC	https://paperswithcode.com/paper/sentiment-analysis-for-arabic-in-social-media
Repo
Framework

Action Recognition via Pose-Based Graph Convolutional Networks with Intermediate Dense Supervision


Title	Action Recognition via Pose-Based Graph Convolutional Networks with Intermediate Dense Supervision
Authors	Lei Shi, Yifan Zhang, Jian Cheng, Hanqing Lu
Abstract	Pose-based action recognition has drawn considerable attention recently. Existing methods exploit the joint positions to extract the body-part features from the activation map of the convolutional networks to assist human action recognition. However, these features are simply concatenated or max-pooled in previous works. The structured correlations among the body parts, which are essential for understanding complex human actions, are not fully exploited. To address the problem, we propose a pose-based graph convolutional network (PGCN), which encodes the body-part features into a human-based spatiotemporal graph, and explicitly models their correlations with a novel light-weight adaptive graph convolutional module to produce a highly discriminative representation for human action recognition. Besides, we discover that the backbone network tends to identify patterns from the most discriminative areas of the input regardless of the others. Thus the features pooled by the joint positions from other areas are less informative, which consequently hampers the performance of the followed aggregation process for recognizing actions. To alleviate this issue, we introduce a simple intermediate dense supervision mechanism for the backbone network, which adequately addresses the problem with no extra computation cost during inference. We evaluate the proposed approach on three popular benchmarks for pose-based action recognition tasks, i.e., Sub-JHMDB, PennAction and NTU-RGBD, where our approach significantly outperforms state-of-the-arts without the bells and whistles.
Tasks	Skeleton Based Action Recognition, Temporal Action Localization
Published	2019-11-28
URL	https://arxiv.org/abs/1911.12509v1
PDF	https://arxiv.org/pdf/1911.12509v1.pdf
PWC	https://paperswithcode.com/paper/action-recognition-via-pose-based-graph
Repo
Framework

Automated Composition of Picture-Synched Music Soundtracks for Movies


Title	Automated Composition of Picture-Synched Music Soundtracks for Movies
Authors	Vansh Dassani, Jon Bird, Dave Cliff
Abstract	We describe the implementation of and early results from a system that automatically composes picture-synched musical soundtracks for videos and movies. We use the phrase “picture-synched” to mean that the structure of the automatically composed music is determined by visual events in the input movie, i.e. the final music is synchronised to visual events and features such as cut transitions or within-shot key-frame events. Our system combines automated video analysis and computer-generated music-composition techniques to create unique soundtracks in response to the video input, and can be thought of as an initial step in creating a computerised replacement for a human composer writing music to fit the picture-locked edit of a movie. Working only from the video information in the movie, key features are extracted from the input video, using video analysis techniques, which are then fed into a machine-learning-based music generation tool, to compose a piece of music from scratch. The resulting soundtrack is tied to video features, such as scene transition markers and scene-level energy values, and is unique to the input video. Although the system we describe here is only a preliminary proof-of-concept, user evaluations of the output of the system have been positive.
Tasks	Music Generation
Published	2019-10-19
URL	https://arxiv.org/abs/1910.08773v1
PDF	https://arxiv.org/pdf/1910.08773v1.pdf
PWC	https://paperswithcode.com/paper/automated-composition-of-picture-synched
Repo
Framework

Stochastic Region Pooling: Make Attention More Expressive


Title	Stochastic Region Pooling: Make Attention More Expressive
Authors	Mingnan Luo, Guihua Wen, Yang Hu, Dan Dai, Yingxue Xu
Abstract	Global Average Pooling (GAP) is used by default on the channel-wise attention mechanism to extract channel descriptors. However, the simple global aggregation method of GAP is easy to make the channel descriptors have homogeneity, which weakens the detail distinction between feature maps, thus affecting the performance of the attention mechanism. In this work, we propose a novel method for channel-wise attention network, called Stochastic Region Pooling (SRP), which makes the channel descriptors more representative and diversity by encouraging the feature map to have more or wider important feature responses. Also, SRP is the general method for the attention mechanisms without any additional parameters or computation. It can be widely applied to attention networks without modifying the network structure. Experimental results on image recognition datasets including CIAFR-10/100, ImageNet and three Fine-grained datasets (CUB-200-2011, Stanford Cars and Stanford Dogs) show that SRP brings the significant improvements of the performance over efficient CNNs and achieves the state-of-the-art results.
Tasks
Published	2019-04-22
URL	http://arxiv.org/abs/1904.09853v1
PDF	http://arxiv.org/pdf/1904.09853v1.pdf
PWC	https://paperswithcode.com/paper/190409853
Repo
Framework

Stein Bridging: Enabling Mutual Reinforcement between Explicit and Implicit Generative Models


Title	Stein Bridging: Enabling Mutual Reinforcement between Explicit and Implicit Generative Models
Authors	Qitian Wu, Rui Gao, Hongyuan Zha
Abstract	Deep generative models are generally categorized into explicit models and implicit models. The former defines an explicit density form, whose normalizing constant is often unknown; while the latter, including generative adversarial networks (GANs), generates samples without explicitly defining a density function. In spite of substantial recent advances demonstrating the power of the two classes of generative models in many applications, both of them, when used alone, suffer from respective limitations and drawbacks. To mitigate these issues, we propose Stein Bridging, a novel joint training framework that connects an explicit density estimator and an implicit sample generator with Stein discrepancy. We show that the Stein Bridge induces new regularization schemes for both explicit and implicit models. Convergence analysis and extensive experiments demonstrate that the Stein Bridging i) improves the stability and sample quality of the GAN training, and ii) facilitates the density estimator to seek more modes in data and alleviate the mode-collapse issue. Additionally, we discuss several applications of Stein Bridging and useful tricks in practical implementation used in our experiments.
Tasks
Published	2019-09-28
URL	https://arxiv.org/abs/1909.13035v2
PDF	https://arxiv.org/pdf/1909.13035v2.pdf
PWC	https://paperswithcode.com/paper/stein-bridging-enabling-mutual-reinforcement-1
Repo
Framework

Distilling Transformers into Simple Neural Networks with Unlabeled Transfer Data


Title	Distilling Transformers into Simple Neural Networks with Unlabeled Transfer Data
Authors	Subhabrata Mukherjee, Ahmed Hassan Awadallah
Abstract	Recent advances in pre-training huge models on large amounts of text through self supervision have obtained state-of-the-art results in various natural language processing tasks. However, these huge and expensive models are difficult to use in practise for downstream tasks. Some recent efforts use knowledge distillation to compress these models. However, we see a gap between the performance of the smaller student models as compared to that of the large teacher. In this work, we leverage large amounts of in-domain unlabeled transfer data in addition to a limited amount of labeled training instances to bridge this gap. We show that simple RNN based student models even with hard distillation can perform at par with the huge teachers given the transfer set. The student performance can be further improved with soft distillation and leveraging teacher intermediate representations. We show that our student models can compress the huge teacher by up to 26x while still matching or even marginally exceeding the teacher performance in low-resource settings with small amount of labeled data.
Tasks
Published	2019-10-04
URL	https://arxiv.org/abs/1910.01769v1
PDF	https://arxiv.org/pdf/1910.01769v1.pdf
PWC	https://paperswithcode.com/paper/distilling-transformers-into-simple-neural
Repo
Framework

Towards Quantifying Intrinsic Generalization of Deep ReLU Networks


Title	Towards Quantifying Intrinsic Generalization of Deep ReLU Networks
Authors	Shaeke Salman, Canlin Zhang, Xiuwen Liu, Washington Mio
Abstract	Understanding the underlying mechanisms that enable the empirical successes of deep neural networks is essential for further improving their performance and explaining such networks. Towards this goal, a specific question is how to explain the “surprising” behavior of the same over-parametrized deep neural networks that can generalize well on real datasets and at the same time “memorize” training samples when the labels are randomized. In this paper, we demonstrate that deep ReLU networks generalize from training samples to new points via piece-wise linear interpolation. We provide a quantified analysis on the generalization ability of a deep ReLU network: Given a fixed point $\mathbf{x}$ and a fixed direction in the input space $\mathcal{S}$, there is always a segment such that any point on the segment will be classified the same as the fixed point $\mathbf{x}$. We call this segment the $generalization \ interval$. We show that the generalization intervals of a ReLU network behave similarly along pairwise directions between samples of the same label in both real and random cases on the MNIST and CIFAR-10 datasets. This result suggests that the same interpolation mechanism is used in both cases. Additionally, for datasets using real labels, such networks provide a good approximation of the underlying manifold in the data, where the changes are much smaller along tangent directions than along normal directions. On the other hand, however, for datasets with random labels, generalization intervals along mid-lines of triangles with the same label are much smaller than those on the datasets with real labels, suggesting different behaviors along other directions. Our systematic experiments demonstrate for the first time that such deep neural networks generalize through the same interpolation and explain the differences between their performance on datasets with real and random labels.
Tasks
Published	2019-10-18
URL	https://arxiv.org/abs/1910.08581v1
PDF	https://arxiv.org/pdf/1910.08581v1.pdf
PWC	https://paperswithcode.com/paper/towards-quantifying-intrinsic-generalization
Repo
Framework

Efficient single input-output layer spiking neural classifier with time-varying weight model


Title	Efficient single input-output layer spiking neural classifier with time-varying weight model
Authors	Abeegithan Jeyasothy, Savitha Ramasamy, Suresh Sundaram
Abstract	This paper presents a supervised learning algorithm, namely, the Synaptic Efficacy Function with Meta-neuron based learning algorithm (SEF-M) for a spiking neural network with a time-varying weight model. For a given pattern, SEF-M uses the learning algorithm derived from meta-neuron based learning algorithm to determine the change in weights corresponding to each presynaptic spike times. The changes in weights modulate the amplitude of a Gaussian function centred at the same presynaptic spike times. The sum of amplitude modulated Gaussian functions represents the synaptic efficacy functions (or time-varying weight models). The performance of SEF-M is evaluated against state-of-the-art spiking neural network learning algorithms on 10 benchmark datasets from UCI machine learning repository. Performance studies show superior generalization ability of SEF-M. An ablation study on time-varying weight model is conducted using JAFFE dataset. The results of the ablation study indicate that using a time-varying weight model instead of single weight model improves the classification accuracy by 14%. Thus, it can be inferred that a single input-output layer spiking neural network with time-varying weight model is computationally more efficient than a multi-layer spiking neural network with long-term or short-term weight model.
Tasks
Published	2019-03-21
URL	http://arxiv.org/abs/1904.10400v1
PDF	http://arxiv.org/pdf/1904.10400v1.pdf
PWC	https://paperswithcode.com/paper/190410400
Repo
Framework

Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification


Title	Beyond Intra-modality: A Survey of Heterogeneous Person Re-identification
Authors	Zheng Wang, Zhixiang Wang, Yinqiang Zheng, Yang Wu, Shin’ichi Satoh
Abstract	An effective and efficient person re-identification (ReID) algorithm alleviates painful video watching and accelerates the investigation progress. Recently, with the explosive requirements of practical applications, a lot of research efforts have been dedicated to heterogeneous person re-identification (He-ReID). In this paper, we review the state-of-the-art methods comprehensively with respect to four main application scenarios – low-resolution, infrared, sketch and text. We begin with a comparison between He-ReID and the general Homogeneous ReID (Ho-ReID) task. Available existing datasets for performing evaluation are briefly described. We then survey the models that have been widely employed in He-ReID. We also summarize and compare the representative approaches. Finally, we discuss some future research directions.
Tasks	Person Re-Identification
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10048v3
PDF	https://arxiv.org/pdf/1905.10048v3.pdf
PWC	https://paperswithcode.com/paper/beyond-intra-modality-discrepancy-a
Repo
Framework

Application of Time Series Analysis to Traffic Accidents in Los Angeles


Title	Application of Time Series Analysis to Traffic Accidents in Los Angeles
Authors	Qinghao Ye, Kaiyuan Hu, Yizhe Wang
Abstract	With the improvements of Los Angeles in many aspects, people in mounting numbers tend to live or travel to the city. The primary objective of this paper is to apply a set of methods for the time series analysis of traffic accidents in Los Angeles in the past few years. The number of traffic accidents, collected from 2010 to 2019 monthly reveals that the traffic accident happens seasonally and increasing with fluctuation. This paper utilizes the ensemble methods to combine several different methods to model the data from various perspectives, which can lead to better forecasting accuracy. The IMA(1, 1), ETS(A, N, A), and two models with Fourier items are failed in independence assumption checking. However, the Online Gradient Descent (OGD) model generated by the ensemble method shows the perfect fit in the data modeling, which is the state-of-the-art model among our candidate models. Therefore, it can be easier to accurately forecast future traffic accidents based on previous data through our model, which can help designers to make better plans.
Tasks	Time Series, Time Series Analysis
Published	2019-11-28
URL	https://arxiv.org/abs/1911.12813v1
PDF	https://arxiv.org/pdf/1911.12813v1.pdf
PWC	https://paperswithcode.com/paper/application-of-time-series-analysis-to
Repo
Framework