Paper Group ANR 169
Refinements of Barndorff-Nielsen and Shephard model: an analysis of crude oil price with machine learning. Streaming Adaptive Nonparametric Variational Autoencoder. Effective Data Augmentation with Multi-Domain Learning GANs. Knowledge Query Network: How Knowledge Interacts with Skills. Assessing the Impact of a User-Item Collaborative Attack on Cl …
Refinements of Barndorff-Nielsen and Shephard model: an analysis of crude oil price with machine learning
| Title | Refinements of Barndorff-Nielsen and Shephard model: an analysis of crude oil price with machine learning |
| Authors | Indranil SenGupta, William Nganje, Erik Hanson |
| Abstract | A commonly used stochastic model for derivative and commodity market analysis is the Barndorff-Nielsen and Shephard (BN-S) model. Though this model is very efficient and analytically tractable, it suffers from the absence of long range dependence and many other issues. For this paper, the analysis is restricted to crude oil price dynamics. A simple way of improving the BN-S model with the implementation of various machine learning algorithms is proposed. This refined BN-S model is more efficient and has fewer parameters than other models which are used in practice as improvements of the BN-S model. The procedure and the model show the application of data science for extracting a “deterministic component” out of processes that are usually considered to be completely stochastic. Empirical applications validate the efficacy of the proposed model for long range dependence. |
| Tasks | |
| Published | 2019-11-29 |
| URL | https://arxiv.org/abs/1911.13300v3 |
| https://arxiv.org/pdf/1911.13300v3.pdf | |
| PWC | https://paperswithcode.com/paper/refinements-of-barndorff-nielsen-and-shephard |
| Repo | |
| Framework | |
Streaming Adaptive Nonparametric Variational Autoencoder
| Title | Streaming Adaptive Nonparametric Variational Autoencoder |
| Authors | Tingting Zhao, Zifeng Wang, Aria Masoomi, Jennifer G. Dy |
| Abstract | We develop a data driven approach to perform clustering and end-to-end feature learning simultaneously for streaming data that can adaptively detect novel clusters in emerging data. Our approach, Adaptive Nonparametric Variational Autoencoder (AdapVAE), learns the cluster membership through a Bayesian Nonparametric (BNP) modeling framework with Deep Neural Networks (DNNs) for feature learning. We develop a joint online variational inference algorithm to learn feature representations and clustering assignments simultaneously via iteratively optimizing the Evidence Lower Bound (ELBO). We resolve the catastrophic forgetting \citep{kirkpatrick2017overcoming} challenges with streaming data by adopting generative samples from the trained AdapVAE using previous data, which avoids the need of storing and reusing past data. We demonstrate the advantages of our model including adaptive novel cluster detection without discarding useful information learned from past data, high quality sample generation and comparable clustering performance as end-to-end batch mode clustering methods on both image and text corpora benchmark datasets. |
| Tasks | Feature Engineering |
| Published | 2019-06-07 |
| URL | https://arxiv.org/abs/1906.03288v2 |
| https://arxiv.org/pdf/1906.03288v2.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-nonparametric-variational |
| Repo | |
| Framework | |
Effective Data Augmentation with Multi-Domain Learning GANs
| Title | Effective Data Augmentation with Multi-Domain Learning GANs |
| Authors | Shin’ya Yamaguchi, Sekitoshi Kanai, Takeharu Eda |
| Abstract | For deep learning applications, the massive data development (e.g., collecting, labeling), which is an essential process in building practical applications, still incurs seriously high costs. In this work, we propose an effective data augmentation method based on generative adversarial networks (GANs), called Domain Fusion. Our key idea is to import the knowledge contained in an outer dataset to a target model by using a multi-domain learning GAN. The multi-domain learning GAN simultaneously learns the outer and target dataset and generates new samples for the target tasks. The simultaneous learning process makes GANs generate the target samples with high fidelity and variety. As a result, we can obtain accurate models for the target tasks by using these generated samples even if we only have an extremely low volume target dataset. We experimentally evaluate the advantages of Domain Fusion in image classification tasks on 3 target datasets: CIFAR-100, FGVC-Aircraft, and Indoor Scene Recognition. When trained on each target dataset reduced the samples to 5,000 images, Domain Fusion achieves better classification accuracy than the data augmentation using fine-tuned GANs. Furthermore, we show that Domain Fusion improves the quality of generated samples, and the improvements can contribute to higher accuracy. |
| Tasks | Data Augmentation, Image Classification, Scene Recognition |
| Published | 2019-12-25 |
| URL | https://arxiv.org/abs/1912.11597v1 |
| https://arxiv.org/pdf/1912.11597v1.pdf | |
| PWC | https://paperswithcode.com/paper/effective-data-augmentation-with-multi-domain |
| Repo | |
| Framework | |
Knowledge Query Network: How Knowledge Interacts with Skills
| Title | Knowledge Query Network: How Knowledge Interacts with Skills |
| Authors | Jinseok Lee, Dit-Yan Yeung |
| Abstract | Knowledge Tracing (KT) is to trace the knowledge of students as they solve a sequence of problems represented by their related skills. This involves abstract concepts of students’ states of knowledge and the interactions between those states and skills. Therefore, a KT model is designed to predict whether students will give correct answers and to describe such abstract concepts. However, existing methods either give relatively low prediction accuracy or fail to explain those concepts intuitively. In this paper, we propose a new model called Knowledge Query Network (KQN) to solve these problems. KQN uses neural networks to encode student learning activities into knowledge state and skill vectors, and models the interactions between the two types of vectors with the dot product. Through this, we introduce a novel concept called \textit{probabilistic skill similarity} that relates the pairwise cosine and Euclidean distances between skill vectors to the odds ratios of the corresponding skills, which makes KQN interpretable and intuitive. On four public datasets, we have carried out experiments to show the following: 1. KQN outperforms all the existing KT models based on prediction accuracy. 2. The interaction between the knowledge state and skills can be visualized for interpretation. 3. Based on probabilistic skill similarity, a skill domain can be analyzed with clustering using the distances between the skill vectors of KQN. 4. For different values of the vector space dimensionality, KQN consistently exhibits high prediction accuracy and a strong positive correlation between the distance matrices of the skill vectors. |
| Tasks | Knowledge Tracing |
| Published | 2019-08-03 |
| URL | https://arxiv.org/abs/1908.02146v2 |
| https://arxiv.org/pdf/1908.02146v2.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-query-network-how-knowledge |
| Repo | |
| Framework | |
Assessing the Impact of a User-Item Collaborative Attack on Class of Users
| Title | Assessing the Impact of a User-Item Collaborative Attack on Class of Users |
| Authors | Yashar Deldjoo, Tommaso Di Noia, Felice Antonio Merra |
| Abstract | Collaborative Filtering (CF) models lie at the core of most recommendation systems due to their state-of-the-art accuracy. They are commonly adopted in e-commerce and online services for their impact on sales volume and/or diversity, and their impact on companies’ outcome. However, CF models are only as good as the interaction data they work with. As these models rely on outside sources of information, counterfeit data such as user ratings or reviews can be injected by attackers to manipulate the underlying data and alter the impact of resulting recommendations, thus implementing a so-called shilling attack. While previous works have focused on evaluating shilling attack strategies from a global perspective paying particular attention to the effect of the size of attacks and attacker’s knowledge, in this work we explore the effectiveness of shilling attacks under novel aspects. First, we investigate the effect of attack strategies crafted on a target user in order to push the recommendation of a low-ranking item to a higher position, referred to as user-item attack. Second, we evaluate the effectiveness of attacks in altering the impact of different CF models by contemplating the class of the target user, from the perspective of the richness of her profile (i.e., cold v.s. warm user). Finally, similar to previous work we contemplate the size of attack (i.e., the amount of fake profiles injected) in examining their success. The results of experiments on two widely used datasets in business and movie domains, namely Yelp and MovieLens, suggest that warm and cold users exhibit contrasting behaviors in datasets with different characteristics. |
| Tasks | Recommendation Systems |
| Published | 2019-08-21 |
| URL | https://arxiv.org/abs/1908.07968v1 |
| https://arxiv.org/pdf/1908.07968v1.pdf | |
| PWC | https://paperswithcode.com/paper/190807968 |
| Repo | |
| Framework | |
Learning Joint Acoustic-Phonetic Word Embeddings
| Title | Learning Joint Acoustic-Phonetic Word Embeddings |
| Authors | Mohamed El-Geish |
| Abstract | Most speech recognition tasks pertain to mapping words across two modalities: acoustic and orthographic. In this work, we suggest learning encoders that map variable-length, acoustic or phonetic, sequences that represent words into fixed-dimensional vectors in a shared latent space; such that the distance between two word vectors represents how closely the two words sound. Instead of directly learning the distances between word vectors, we employ weak supervision and model a binary classification task to predict whether two inputs, one of each modality, represent the same word given a distance threshold. We explore various deep-learning models, bimodal contrastive losses, and techniques for mining hard negative examples such as the semi-supervised technique of self-labeling. Our best model achieves an $F_1$ score of 0.95 for the binary classification task. |
| Tasks | Speech Recognition, Word Embeddings |
| Published | 2019-08-01 |
| URL | https://arxiv.org/abs/1908.00493v1 |
| https://arxiv.org/pdf/1908.00493v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-joint-acoustic-phonetic-word |
| Repo | |
| Framework | |
Deep Reinforcement Learning with Decorrelation
| Title | Deep Reinforcement Learning with Decorrelation |
| Authors | Borislav Mavrin, Hengshuai Yao, Linglong Kong |
| Abstract | Learning an effective representation for high-dimensional data is a challenging problem in reinforcement learning (RL). Deep reinforcement learning (DRL) such as Deep Q networks (DQN) achieves remarkable success in computer games by learning deeply encoded representation from convolution networks. In this paper, we propose a simple yet very effective method for representation learning with DRL algorithms. Our key insight is that features learned by DRL algorithms are highly correlated, which interferes with learning. By adding a regularized loss that penalizes correlation in latent features (with only slight computation), we decorrelate features represented by deep neural networks incrementally. On 49 Atari games, with the same regularization factor, our decorrelation algorithms perform $70%$ in terms of human-normalized scores, which is $40%$ better than DQN. In particular, ours performs better than DQN on 39 games with 4 close ties and lost only slightly on $6$ games. Empirical results also show that the decorrelation method applies to Quantile Regression DQN (QR-DQN) and significantly boosts performance. Further experiments on the losing games show that our decorelation algorithms can win over DQN and QR-DQN with a fined tuned regularization factor. |
| Tasks | Atari Games, Representation Learning |
| Published | 2019-03-18 |
| URL | https://arxiv.org/abs/1903.07765v3 |
| https://arxiv.org/pdf/1903.07765v3.pdf | |
| PWC | https://paperswithcode.com/paper/deep-reinforcement-learning-with |
| Repo | |
| Framework | |
VIABLE: Fast Adaptation via Backpropagating Learned Loss
| Title | VIABLE: Fast Adaptation via Backpropagating Learned Loss |
| Authors | Leo Feng, Luisa Zintgraf, Bei Peng, Shimon Whiteson |
| Abstract | In few-shot learning, typically, the loss function which is applied at test time is the one we are ultimately interested in minimising, such as the mean-squared-error loss for a regression problem. However, given that we have few samples at test time, we argue that the loss function that we are interested in minimising is not necessarily the loss function most suitable for computing gradients in a few-shot setting. We propose VIABLE, a generic meta-learning extension that builds on existing meta-gradient-based methods by learning a differentiable loss function, replacing the pre-defined inner-loop loss function in performing task-specific updates. We show that learning a loss function capable of leveraging relational information between samples reduces underfitting, and significantly improves performance and sample efficiency on a simple regression task. Furthermore, we show VIABLE is scalable by evaluating on the Mini-Imagenet dataset. |
| Tasks | Few-Shot Learning, Meta-Learning |
| Published | 2019-11-29 |
| URL | https://arxiv.org/abs/1911.13159v1 |
| https://arxiv.org/pdf/1911.13159v1.pdf | |
| PWC | https://paperswithcode.com/paper/viable-fast-adaptation-via-backpropagating |
| Repo | |
| Framework | |
Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding
| Title | Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding |
| Authors | Guokan Shang, Antoine Jean-Pierre Tixier, Michalis Vazirgiannis, Jean-Pierre Lorré |
| Abstract | Abstractive community detection is an important spoken language understanding task, whose goal is to group utterances in a conversation according to whether they can be jointly summarized by a common abstractive sentence. This paper provides a novel approach to this task. We first introduce a neural contextual utterance encoder featuring three types of self-attention mechanisms. We then train it using the siamese and triplet energy-based meta-architectures. Experiments on the AMI corpus show that our system outperforms multiple energy-based and non-energy based baselines from the state-of-the-art. Code and data are publicly available. |
| Tasks | Community Detection, Spoken Language Understanding |
| Published | 2019-04-20 |
| URL | https://arxiv.org/abs/1904.09491v2 |
| https://arxiv.org/pdf/1904.09491v2.pdf | |
| PWC | https://paperswithcode.com/paper/energy-based-self-attentive-learning-of |
| Repo | |
| Framework | |
Solving Optical Tomography with Deep Learning
| Title | Solving Optical Tomography with Deep Learning |
| Authors | Yuwei Fan, Lexing Ying |
| Abstract | This paper presents a neural network approach for solving two-dimensional optical tomography (OT) problems based on the radiative transfer equation. The mathematical problem of OT is to recover the optical properties of an object based on the albedo operator that is accessible from boundary measurements. Both the forward map from the optical properties to the albedo operator and the inverse map are high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this, we propose effective neural network architectures for the forward and inverse maps based on convolution layers, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks. |
| Tasks | |
| Published | 2019-10-10 |
| URL | https://arxiv.org/abs/1910.04756v1 |
| https://arxiv.org/pdf/1910.04756v1.pdf | |
| PWC | https://paperswithcode.com/paper/solving-optical-tomography-with-deep-learning |
| Repo | |
| Framework | |
Learning Controllable Disentangled Representations with Decorrelation Regularization
| Title | Learning Controllable Disentangled Representations with Decorrelation Regularization |
| Authors | Zengjie Song, Oluwasanmi Koyejo, Jiangshe Zhang |
| Abstract | A crucial problem in learning disentangled image representations is controlling the degree of disentanglement during image editing, while preserving the identity of objects. In this work, we propose a simple yet effective model with the encoder-decoder architecture to address this challenge. To encourage disentanglement, we devise a distance covariance based decorrelation regularization. Further, for the reconstruction step, our model leverages a soft target representation combined with the latent image code. By exploiting the real-valued space of the soft target representations, we are able to synthesize novel images with the designated properties. We also design a classification based protocol to quantitatively evaluate the disentanglement strength of our model. Experimental results show that the proposed model competently disentangles factors of variation, and is able to manipulate face images to synthesize the desired attributes. |
| Tasks | |
| Published | 2019-12-25 |
| URL | https://arxiv.org/abs/1912.11675v1 |
| https://arxiv.org/pdf/1912.11675v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-controllable-disentangled |
| Repo | |
| Framework | |
Location Forensics Analysis Using ENF Sequences Extracted from Power and Audio Recordings
| Title | Location Forensics Analysis Using ENF Sequences Extracted from Power and Audio Recordings |
| Authors | Dhiman Chowdhury, Mrinmoy Sarkar |
| Abstract | Electrical network frequency (ENF) is the signature of a power distribution grid which represents the nominal frequency (50 or 60 Hz) of a power system network. Due to load variations in a power grid, ENF sequences experience fluctuations. These ENF variations are inherently located in a multimedia signal which is recorded close to the grid or directly from the mains power line. Therefore, a multimedia recording can be localized by analyzing the ENF sequences of that signal in absence of the concurrent power signal. In this paper, a novel approach to analyze location forensics using ENF sequences extracted from a number of power and audio recordings is proposed. The digital recordings are collected from different grid locations around the world. Potential feature components are determined from the ENF sequences. Then, a multi-class support vector machine (SVM) classification model is developed to validate the location authenticity of the recordings. The performance assessments affirm the efficacy of the presented work. |
| Tasks | |
| Published | 2019-12-18 |
| URL | https://arxiv.org/abs/1912.09428v1 |
| https://arxiv.org/pdf/1912.09428v1.pdf | |
| PWC | https://paperswithcode.com/paper/location-forensics-analysis-using-enf |
| Repo | |
| Framework | |
Machine Translation Evaluation with BERT Regressor
| Title | Machine Translation Evaluation with BERT Regressor |
| Authors | Hiroki Shimanaka, Tomoyuki Kajiwara, Mamoru Komachi |
| Abstract | We introduce the metric using BERT (Bidirectional Encoder Representations from Transformers) (Devlin et al., 2019) for automatic machine translation evaluation. The experimental results of the WMT-2017 Metrics Shared Task dataset show that our metric achieves state-of-the-art performance in segment-level metrics task for all to-English language pairs. |
| Tasks | Machine Translation |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12679v1 |
| https://arxiv.org/pdf/1907.12679v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-translation-evaluation-with-bert |
| Repo | |
| Framework | |
Creativity in Robot Manipulation with Deep Reinforcement Learning
| Title | Creativity in Robot Manipulation with Deep Reinforcement Learning |
| Authors | Juan Carlos Vargas, Malhar Bhoite, Amir Barati Farimani |
| Abstract | Deep Reinforcement Learning (DRL) has emerged as a powerful control technique in robotic science. In contrast to control theory, DRL is more robust in the thorough exploration of the environment. This capability of DRL generates more human-like behaviour and intelligence when applied to the robots. To explore this capability, we designed challenging manipulation tasks to observe robots strategy to handle complex scenarios. We observed that robots not only perform tasks successfully, but also transpire a creative and non intuitive solution. We also observed robot’s persistence in tasks that are close to success and its striking ability in discerning to continue or give up. |
| Tasks | |
| Published | 2019-10-16 |
| URL | https://arxiv.org/abs/1910.07459v1 |
| https://arxiv.org/pdf/1910.07459v1.pdf | |
| PWC | https://paperswithcode.com/paper/creativity-in-robot-manipulation-with-deep |
| Repo | |
| Framework | |
Towards De-identification of Legal Texts
| Title | Towards De-identification of Legal Texts |
| Authors | Diego Garat, Dina Wonsever |
| Abstract | In many countries, personal information that can be published or shared between organizations is regulated and, therefore, documents must undergo a process of de-identification to eliminate or obfuscate confidential data. Our work focuses on the de-identification of legal texts, where the goal is to hide the names of the actors involved in a lawsuit without losing the sense of the story. We present a first evaluation on our corpus of NLP tools in tasks such as segmentation, tokenization and recognition of named entities, and we analyze several evaluation measures for our de-identification task. Results are meager: 84% of the documents have at least one name not covered by NER tools, something that might lead to the re-identification of involved names. We conclude that tools must be strongly adapted for processing texts of this particular domain. |
| Tasks | Tokenization |
| Published | 2019-10-09 |
| URL | https://arxiv.org/abs/1910.03739v1 |
| https://arxiv.org/pdf/1910.03739v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-de-identification-of-legal-texts |
| Repo | |
| Framework | |
