October 20, 2019

3300 words 16 mins read

Paper Group ANR 67

Probabilistic Ensemble of Collaborative Filters. Time is of the Essence: a Joint Hierarchical RNN and Point Process Model for Time and Item Predictions. Probabilistic Prediction of Vehicle Semantic Intention and Motion. Natural Language Generation by Hierarchical Decoding with Linguistic Patterns. Adversarial Domain Adaptation for Variational Neura …

Probabilistic Ensemble of Collaborative Filters


Title	Probabilistic Ensemble of Collaborative Filters
Authors	Zhiyu Min, Dahua Lin
Abstract	Collaborative filtering is an important technique for recommendation. Whereas it has been repeatedly shown to be effective in previous work, its performance remains unsatisfactory in many real-world applications, especially those where the items or users are highly diverse. In this paper, we explore an ensemble-based framework to enhance the capability of a recommender in handling diverse data. Specifically, we formulate a probabilistic model which integrates the items, the users, as well as the associations between them into a generative process. On top of this formulation, we further derive a progressive algorithm to construct an ensemble of collaborative filters. In each iteration, a new filter is derived from re-weighted entries and incorporated into the ensemble. It is noteworthy that while the algorithmic procedure of our algorithm is apparently similar to boosting, it is derived from an essentially different formulation and thus differs in several key technical aspects. We tested the proposed method on three large datasets, and observed substantial improvement over the state of the art, including L2Boost, an effective method based on boosting.
Tasks
Published	2018-06-26
URL	http://arxiv.org/abs/1808.03298v2
PDF	http://arxiv.org/pdf/1808.03298v2.pdf
PWC	https://paperswithcode.com/paper/probabilistic-ensemble-of-collaborative
Repo
Framework

Time is of the Essence: a Joint Hierarchical RNN and Point Process Model for Time and Item Predictions


Title	Time is of the Essence: a Joint Hierarchical RNN and Point Process Model for Time and Item Predictions
Authors	Bjørnar Vassøy, Massimiliano Ruocco, Eliezer de Souza da Silva, Erlend Aune
Abstract	In recent years session-based recommendation has emerged as an increasingly applicable type of recommendation. As sessions consist of sequences of events, this type of recommendation is a natural fit for Recurrent Neural Networks (RNNs). Several additions have been proposed for extending such models in order to handle specific problems or data. Two such extensions are 1.) modeling of inter-session relations for catching long term dependencies over user sessions, and 2.) modeling temporal aspects of user-item interactions. The former allows the session-based recommendation to utilize extended session history and inter-session information when providing new recommendations. The latter has been used to both provide state-of-the-art predictions for when the user will return to the service and also for improving recommendations. In this work we combine these two extensions in a joint model for the tasks of recommendation and return-time prediction. The model consists of a Hierarchical RNN for the inter-session and intra-session items recommendation extended with a Point Process model for the time-gaps between the sessions. The experimental results indicate that the proposed model improves recommendations significantly on two datasets over a strong baseline, while simultaneously improving return-time predictions over a baseline return-time prediction model.
Tasks	Session-Based Recommendations
Published	2018-12-04
URL	http://arxiv.org/abs/1812.01276v1
PDF	http://arxiv.org/pdf/1812.01276v1.pdf
PWC	https://paperswithcode.com/paper/time-is-of-the-essence-a-joint-hierarchical
Repo
Framework

Probabilistic Prediction of Vehicle Semantic Intention and Motion


Title	Probabilistic Prediction of Vehicle Semantic Intention and Motion
Authors	Yeping Hu, Wei Zhan, Masayoshi Tomizuka
Abstract	Accurately predicting the possible behaviors of traffic participants is an essential capability for future autonomous vehicles. The majority of current researches fix the number of driving intentions by considering only a specific scenario. However, distinct driving environments usually contain various possible driving maneuvers. Therefore, a intention prediction method that can adapt to different traffic scenarios is needed. To further improve the overall vehicle prediction performance, motion information is usually incorporated with classified intentions. As suggested in some literature, the methods that directly predict possible goal locations can achieve better performance for long-term motion prediction than other approaches due to their automatic incorporation of environment constraints. Moreover, by obtaining the temporal information of the predicted destinations, the optimal trajectories for predicted vehicles as well as the desirable path for ego autonomous vehicle could be easily generated. In this paper, we propose a Semantic-based Intention and Motion Prediction (SIMP) method, which can be adapted to any driving scenarios by using semantic-defined vehicle behaviors. It utilizes a probabilistic framework based on deep neural network to estimate the intentions, final locations, and the corresponding time information for surrounding vehicles. An exemplar real-world scenario was used to implement and examine the proposed method.
Tasks	Autonomous Vehicles, motion prediction
Published	2018-04-10
URL	http://arxiv.org/abs/1804.03629v1
PDF	http://arxiv.org/pdf/1804.03629v1.pdf
PWC	https://paperswithcode.com/paper/probabilistic-prediction-of-vehicle-semantic
Repo
Framework

Natural Language Generation by Hierarchical Decoding with Linguistic Patterns


Title	Natural Language Generation by Hierarchical Decoding with Linguistic Patterns
Authors	Shang-Yu Su, Kai-Ling Lo, Yi-Ting Yeh, Yun-Nung Chen
Abstract	Natural language generation (NLG) is a critical component in spoken dialogue systems. Classic NLG can be divided into two phases: (1) sentence planning: deciding on the overall sentence structure, (2) surface realization: determining specific word forms and flattening the sentence structure into a string. Many simple NLG models are based on recurrent neural networks (RNN) and sequence-to-sequence (seq2seq) model, which basically contains an encoder-decoder structure; these NLG models generate sentences from scratch by jointly optimizing sentence planning and surface realization using a simple cross entropy loss training criterion. However, the simple encoder-decoder architecture usually suffers from generating complex and long sentences, because the decoder has to learn all grammar and diction knowledge. This paper introduces a hierarchical decoding NLG model based on linguistic patterns in different levels, and shows that the proposed method outperforms the traditional one with a smaller model size. Furthermore, the design of the hierarchical decoding is flexible and easily-extensible in various NLG systems.
Tasks	Spoken Dialogue Systems, Text Generation
Published	2018-08-08
URL	http://arxiv.org/abs/1808.02747v2
PDF	http://arxiv.org/pdf/1808.02747v2.pdf
PWC	https://paperswithcode.com/paper/natural-language-generation-by-hierarchical
Repo
Framework

Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems


Title	Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems
Authors	Van-Khanh Tran, Le-Minh Nguyen
Abstract	Domain Adaptation arises when we aim at learning from source domain a model that can per- form acceptably well on a different target domain. It is especially crucial for Natural Language Generation (NLG) in Spoken Dialogue Systems when there are sufficient annotated data in the source domain, but there is a limited labeled data in the target domain. How to effectively utilize as much of existing abilities from source domains is a crucial issue in domain adaptation. In this paper, we propose an adversarial training procedure to train a Variational encoder-decoder based language generator via multiple adaptation steps. In this procedure, a model is first trained on a source domain data and then fine-tuned on a small set of target domain utterances under the guidance of two proposed critics. Experimental results show that the proposed method can effec- tively leverage the existing knowledge in the source domain to adapt to another related domain by using only a small amount of in-domain data.
Tasks	Domain Adaptation, Spoken Dialogue Systems, Text Generation
Published	2018-08-08
URL	http://arxiv.org/abs/1808.02586v1
PDF	http://arxiv.org/pdf/1808.02586v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-domain-adaptation-for-variational
Repo
Framework

Logical Semantics and Commonsense Knowledge: Where Did we Go Wrong, and How to Go Forward, Again


Title	Logical Semantics and Commonsense Knowledge: Where Did we Go Wrong, and How to Go Forward, Again
Authors	Walid S. Saba
Abstract	We argue that logical semantics might have faltered due to its failure in distinguishing between two fundamentally very different types of concepts: ontological concepts, that should be types in a strongly-typed ontology, and logical concepts, that are predicates corresponding to properties of and relations between objects of various ontological types. We will then show that accounting for these differences amounts to the integration of lexical and compositional semantics in one coherent framework, and to an embedding in our logical semantics of a strongly-typed ontology that reflects our commonsense view of the world and the way we talk about it in ordinary language. We will show that in such a framework a number of challenges in natural language semantics can be adequately and systematically treated.
Tasks
Published	2018-08-06
URL	http://arxiv.org/abs/1808.01741v2
PDF	http://arxiv.org/pdf/1808.01741v2.pdf
PWC	https://paperswithcode.com/paper/logical-semantics-and-commonsense-knowledge
Repo
Framework

An Alternative View: When Does SGD Escape Local Minima?


Title	An Alternative View: When Does SGD Escape Local Minima?
Authors	Robert Kleinberg, Yuanzhi Li, Yang Yuan
Abstract	Stochastic gradient descent (SGD) is widely used in machine learning. Although being commonly viewed as a fast but not accurate version of gradient descent (GD), it always finds better solutions than GD for modern neural networks. In order to understand this phenomenon, we take an alternative view that SGD is working on the convolved (thus smoothed) version of the loss function. We show that, even if the function $f$ has many bad local minima or saddle points, as long as for every point $x$, the weighted average of the gradients of its neighborhoods is one point convex with respect to the desired solution $x^$, SGD will get close to, and then stay around $x^$ with constant probability. More specifically, SGD will not get stuck at “sharp” local minima with small diameters, as long as the neighborhoods of these regions contain enough gradient information. The neighborhood size is controlled by step size and gradient noise. Our result identifies a set of functions that SGD provably works, which is much larger than the set of convex functions. Empirically, we observe that the loss surface of neural networks enjoys nice one point convexity properties locally, therefore our theorem helps explain why SGD works so well for neural networks.
Tasks
Published	2018-02-17
URL	http://arxiv.org/abs/1802.06175v2
PDF	http://arxiv.org/pdf/1802.06175v2.pdf
PWC	https://paperswithcode.com/paper/an-alternative-view-when-does-sgd-escape
Repo
Framework

Online Learning: Sufficient Statistics and the Burkholder Method


Title	Online Learning: Sufficient Statistics and the Burkholder Method
Authors	Dylan J. Foster, Alexander Rakhlin, Karthik Sridharan
Abstract	We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain “sufficient statistics” for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these sufficient statistics, not the entire data sequence, so that the online strategy is only required to keep the sufficient statistics in memory. This characterization is achieved by bringing the full power of the Burkholder Method — originally developed for certifying probabilistic martingale inequalities — to bear on the online learning setting. To demonstrate the scope and effectiveness of the Burkholder method, we develop a novel online strategy for matrix prediction that attains a regret bound corresponding to the variance term in matrix concentration inequalities. We also present a linear-time/space prediction strategy for parameter free supervised learning with linear classes and general smooth norms.
Tasks
Published	2018-03-20
URL	http://arxiv.org/abs/1803.07617v1
PDF	http://arxiv.org/pdf/1803.07617v1.pdf
PWC	https://paperswithcode.com/paper/online-learning-sufficient-statistics-and-the
Repo
Framework

OxIOD: The Dataset for Deep Inertial Odometry


Title	OxIOD: The Dataset for Deep Inertial Odometry
Authors	Changhao Chen, Peijun Zhao, Chris Xiaoxuan Lu, Wei Wang, Andrew Markham, Niki Trigoni
Abstract	Advances in micro-electro-mechanical (MEMS) techniques enable inertial measurements units (IMUs) to be small, cheap, energy efficient, and widely used in smartphones, robots, and drones. Exploiting inertial data for accurate and reliable navigation and localization has attracted significant research and industrial interest, as IMU measurements are completely ego-centric and generally environment agnostic. Recent studies have shown that the notorious issue of drift can be significantly alleviated by using deep neural networks (DNNs), e.g. IONet. However, the lack of sufficient labelled data for training and testing various architectures limits the proliferation of adopting DNNs in IMU-based tasks. In this paper, we propose and release the Oxford Inertial Odometry Dataset (OxIOD), a first-of-its-kind data collection for inertial-odometry research, with all sequences having ground-truth labels. Our dataset contains 158 sequences totalling more than 42 km in total distance, much larger than previous inertial datasets. Another notable feature of this dataset lies in its diversity, which can reflect the complex motions of phone-based IMUs in various everyday usage. The measurements were collected with four different attachments (handheld, in the pocket, in the handbag and on the trolley), four motion modes (halting, walking slowly, walking normally, and running), five different users, four types of off-the-shelf consumer phones, and large-scale localization from office buildings. Deep inertial tracking experiments were conducted to show the effectiveness of our dataset in training deep neural network models and evaluate learning-based and model-based algorithms. The OxIOD Dataset is available at: http://deepio.cs.ox.ac.uk
Tasks
Published	2018-09-20
URL	http://arxiv.org/abs/1809.07491v1
PDF	http://arxiv.org/pdf/1809.07491v1.pdf
PWC	https://paperswithcode.com/paper/oxiod-the-dataset-for-deep-inertial-odometry
Repo
Framework

Graph of brain structures grading for early detection of Alzheimer’s disease


Title	Graph of brain structures grading for early detection of Alzheimer’s disease
Authors	Kilian Hett, Vinh-Thong Ta, Jose Vicente Manjon, Pierrick Coupé
Abstract	Alzheimer’s disease is the most common dementia leading to an irreversible neurodegenerative process. To date, subject revealed advanced brain structural alterations when the diagnosis is established. Therefore, an earlier diagnosis of this dementia is crucial although it is a challenging task. Recently, many studies have proposed biomarkers to perform early detection of Alzheimer’s disease. Some of them have proposed methods based on inter-subject similarity while other approaches have investigated framework using intra-subject variability. In this work, we propose a novel framework combining both approaches within an efficient graph of brain structures grading. Subsequently, we demonstrate the competitive performance of the proposed method compared to state-of-the-art methods.
Tasks
Published	2018-07-06
URL	http://arxiv.org/abs/1807.03173v1
PDF	http://arxiv.org/pdf/1807.03173v1.pdf
PWC	https://paperswithcode.com/paper/graph-of-brain-structures-grading-for-early
Repo
Framework

Anomaly Detection for imbalanced datasets with Deep Generative Models


Title	Anomaly Detection for imbalanced datasets with Deep Generative Models
Authors	Nazly Rocio Santos Buitrago, Loek Tonnaer, Vlado Menkovski, Dimitrios Mavroeidis
Abstract	Many important data analysis applications present with severely imbalanced datasets with respect to the target variable. A typical example is medical image analysis, where positive samples are scarce, while performance is commonly estimated against the correct detection of these positive examples. We approach this challenge by formulating the problem as anomaly detection with generative models. We train a generative model without supervision on the `negative' (common) datapoints and use this model to estimate the likelihood of unseen data. A successful model allows us to detect the` positive’ case as low likelihood datapoints. In this position paper, we present the use of state-of-the-art deep generative models (GAN and VAE) for the estimation of a likelihood of the data. Our results show that on the one hand both GANs and VAEs are able to separate the `positive' and` negative’ samples in the MNIST case. On the other hand, for the NLST case, neither GANs nor VAEs were able to capture the complexity of the data and discriminate anomalies at the level that this task requires. These results show that even though there are a number of successes presented in the literature for using generative models in similar applications, there remain further challenges for broad successful implementation.
Tasks	Anomaly Detection
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00986v1
PDF	http://arxiv.org/pdf/1811.00986v1.pdf
PWC	https://paperswithcode.com/paper/anomaly-detection-for-imbalanced-datasets
Repo
Framework

Hardware design of LIF with Latency neuron model with memristive STDP synapses


Title	Hardware design of LIF with Latency neuron model with memristive STDP synapses
Authors	Simone Acciarito, Gian Carlo Cardarilli, Alessandro Cristini, Luca Di Nunzio, Rocco Fazzolari, Gaurav Mani Khanal, Marco Re, Gianluca Susi
Abstract	In this paper, the hardware implementation of a neuromorphic system is presented. This system is composed of a Leaky Integrate-and-Fire with Latency (LIFL) neuron and a Spike-Timing Dependent Plasticity (STDP) synapse. LIFL neuron model allows to encode more information than the common Integrate-and-Fire models, typically considered for neuromorphic implementations. In our system LIFL neuron is implemented using CMOS circuits while memristor is used for the implementation of the STDP synapse. A description of the entire circuit is provided. Finally, the capabilities of the proposed architecture have been evaluated by simulating a motif composed of three neurons and two synapses. The simulation results confirm the validity of the proposed system and its suitability for the design of more complex spiking neural networks
Tasks
Published	2018-03-31
URL	http://arxiv.org/abs/1804.00149v1
PDF	http://arxiv.org/pdf/1804.00149v1.pdf
PWC	https://paperswithcode.com/paper/hardware-design-of-lif-with-latency-neuron
Repo
Framework

Sleep-like slow oscillations improve visual classification through synaptic homeostasis and memory association in a thalamo-cortical model


Title	Sleep-like slow oscillations improve visual classification through synaptic homeostasis and memory association in a thalamo-cortical model
Authors	Cristiano Capone, Elena Pastorelli, Bruno Golosio, Pier Stanislao Paolucci
Abstract	The occurrence of sleep passed through the evolutionary sieve and is widespread in animal species. Sleep is known to be beneficial to cognitive and mnemonic tasks, while chronic sleep deprivation is detrimental. Despite the importance of the phenomenon, a complete understanding of its functions and underlying mechanisms is still lacking. In this paper, we show interesting effects of deep-sleep-like slow oscillation activity on a simplified thalamo-cortical model which is trained to encode, retrieve and classify images of handwritten digits. During slow oscillations, spike-timing-dependent-plasticity (STDP) produces a differential homeostatic process. It is characterized by both a specific unsupervised enhancement of connections among groups of neurons associated to instances of the same class (digit) and a simultaneous down-regulation of stronger synapses created by the training. This hierarchical organization of post-sleep internal representations favours higher performances in retrieval and classification tasks. The mechanism is based on the interaction between top-down cortico-thalamic predictions and bottom-up thalamo-cortical projections during deep-sleep-like slow oscillations. Indeed, when learned patterns are replayed during sleep, cortico-thalamo-cortical connections favour the activation of other neurons coding for similar thalamic inputs, promoting their association. Such mechanism hints at possible applications to artificial learning systems.
Tasks	Image Classification
Published	2018-10-24
URL	https://arxiv.org/abs/1810.10498v5
PDF	https://arxiv.org/pdf/1810.10498v5.pdf
PWC	https://paperswithcode.com/paper/sleep-like-slow-oscillations-improve-visual
Repo
Framework

Computing the Strategy to Commit to in Polymatrix Games (Extended Version)


Title	Computing the Strategy to Commit to in Polymatrix Games (Extended Version)
Authors	Giuseppe De Nittis, Alberto Marchesi, Nicola Gatti
Abstract	Leadership games provide a powerful paradigm to model many real-world settings. Most literature focuses on games with a single follower who acts optimistically, breaking ties in favour of the leader. Unfortunately, for real-world applications, this is unlikely. In this paper, we look for efficiently solvable games with multiple followers who play either optimistically or pessimistically, i.e., breaking ties in favour or against the leader. We study the computational complexity of finding or approximating an optimistic or pessimistic leader-follower equilibrium in specific classes of succinct games—polymatrix like—which are equivalent to 2-player Bayesian games with uncertainty over the follower, with interdependent or independent types. Furthermore, we provide an exact algorithm to find a pessimistic equilibrium for those game classes. Finally, we show that in general polymatrix games the computation is harder even when players are forced to play pure strategies.
Tasks
Published	2018-07-31
URL	http://arxiv.org/abs/1807.11914v1
PDF	http://arxiv.org/pdf/1807.11914v1.pdf
PWC	https://paperswithcode.com/paper/computing-the-strategy-to-commit-to-in
Repo
Framework

Relational Network for Skeleton-Based Action Recognition


Title	Relational Network for Skeleton-Based Action Recognition
Authors	Wu Zheng, Lin Li, Zhaoxiang Zhang, Yan Huang, Liang Wang
Abstract	With the fast development of effective and low-cost human skeleton capture systems, skeleton-based action recognition has attracted much attention recently. Most existing methods use Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to extract spatio-temporal information embedded in the skeleton sequences for action recognition. However, these approaches are limited in the ability of relational modeling in a single skeleton, due to the loss of important structural information when converting the raw skeleton data to adapt to the input format of CNN or RNN. In this paper, we propose an Attentional Recurrent Relational Network-LSTM (ARRN-LSTM) to simultaneously model spatial configurations and temporal dynamics in skeletons for action recognition. We introduce the Recurrent Relational Network to learn the spatial features in a single skeleton, followed by a multi-layer LSTM to learn the temporal features in the skeleton sequences. Between the two modules, we design an adaptive attentional module to focus attention on the most discriminative parts in the single skeleton. To exploit the complementarity from different geometries in the skeleton for sufficient relational modeling, we design a two-stream architecture to learn the structural features among joints and lines simultaneously. Extensive experiments are conducted on several popular skeleton datasets and the results show that the proposed approach achieves better results than most mainstream methods.
Tasks	Skeleton Based Action Recognition, Temporal Action Localization
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02556v4
PDF	http://arxiv.org/pdf/1805.02556v4.pdf
PWC	https://paperswithcode.com/paper/skeleton-based-relational-modeling-for-action
Repo
Framework