October 20, 2019

3294 words 16 mins read

Paper Group AWR 191

Random Warping Series: A Random Features Method for Time-Series Embedding. ClusterGAN : Latent Space Clustering in Generative Adversarial Networks. Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts. Generalization and Regularization in DQN. Temporal Difference Variational Auto-Encoder. Learning Plannable Representatio …

Random Warping Series: A Random Features Method for Time-Series Embedding


Title	Random Warping Series: A Random Features Method for Time-Series Embedding
Authors	Lingfei Wu, Ian En-Hsu Yen, Jinfeng Yi, Fangli Xu, Qi Lei, Michael Witbrock
Abstract	Time series data analytics has been a problem of substantial interests for decades, and Dynamic Time Warping (DTW) has been the most widely adopted technique to measure dissimilarity between time series. A number of global-alignment kernels have since been proposed in the spirit of DTW to extend its use to kernel-based estimation method such as support vector machine. However, those kernels suffer from diagonal dominance of the Gram matrix and a quadratic complexity w.r.t. the sample size. In this work, we study a family of alignment-aware positive definite (p.d.) kernels, with its feature embedding given by a distribution of \emph{Random Warping Series (RWS)}. The proposed kernel does not suffer from the issue of diagonal dominance while naturally enjoys a \emph{Random Features} (RF) approximation, which reduces the computational complexity of existing DTW-based techniques from quadratic to linear in terms of both the number and the length of time-series. We also study the convergence of the RF approximation for the domain of time series of unbounded length. Our extensive experiments on 16 benchmark datasets demonstrate that RWS outperforms or matches state-of-the-art classification and clustering methods in both accuracy and computational time. Our code and data is available at { \url{https://github.com/IBM/RandomWarpingSeries}}.
Tasks	Time Series
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05259v1
PDF	http://arxiv.org/pdf/1809.05259v1.pdf
PWC	https://paperswithcode.com/paper/random-warping-series-a-random-features
Repo	https://github.com/IBM/RandomWarpingSeries
Framework	none

ClusterGAN : Latent Space Clustering in Generative Adversarial Networks


Title	ClusterGAN : Latent Space Clustering in Generative Adversarial Networks
Authors	Sudipto Mukherjee, Himanshu Asnani, Eugene Lin, Sreeram Kannan
Abstract	Generative Adversarial networks (GANs) have obtained remarkable success in many unsupervised learning tasks and unarguably, clustering is an important unsupervised learning problem. While one can potentially exploit the latent-space back-projection in GANs to cluster, we demonstrate that the cluster structure is not retained in the GAN latent space. In this paper, we propose ClusterGAN as a new mechanism for clustering using GANs. By sampling latent variables from a mixture of one-hot encoded variables and continuous latent variables, coupled with an inverse network (which projects the data to the latent space) trained jointly with a clustering specific loss, we are able to achieve clustering in the latent space. Our results show a remarkable phenomenon that GANs can preserve latent space interpolation across categories, even though the discriminator is never exposed to such vectors. We compare our results with various clustering baselines and demonstrate superior performance on both synthetic and real datasets.
Tasks
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03627v2
PDF	http://arxiv.org/pdf/1809.03627v2.pdf
PWC	https://paperswithcode.com/paper/clustergan-latent-space-clustering-in
Repo	https://github.com/zhampel/clusterGAN
Framework	pytorch

Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts


Title	Textual Analogy Parsing: What’s Shared and What’s Compared among Analogous Facts
Authors	Matthew Lamm, Arun Tejasvi Chaganty, Christopher D. Manning, Dan Jurafsky, Percy Liang
Abstract	To understand a sentence like “whereas only 10% of White Americans live at or below the poverty line, 28% of African Americans do” it is important not only to identify individual facts, e.g., poverty rates of distinct demographic groups, but also the higher-order relations between them, e.g., the disparity between them. In this paper, we propose the task of Textual Analogy Parsing (TAP) to model this higher-order meaning. The output of TAP is a frame-style meaning representation which explicitly specifies what is shared (e.g., poverty rates) and what is compared (e.g., White Americans vs. African Americans, 10% vs. 28%) between its component facts. Such a meaning representation can enable new applications that rely on discourse understanding such as automated chart generation from quantitative text. We present a new dataset for TAP, baselines, and a model that successfully uses an ILP to enforce the structural constraints of the problem.
Tasks	Textual Analogy Parsing
Published	2018-09-07
URL	http://arxiv.org/abs/1809.02700v1
PDF	http://arxiv.org/pdf/1809.02700v1.pdf
PWC	https://paperswithcode.com/paper/textual-analogy-parsing-whats-shared-and
Repo	https://github.com/leduoyang/IRIE_project_2018_relation_extraction
Framework	none

Generalization and Regularization in DQN


Title	Generalization and Regularization in DQN
Authors	Jesse Farebrother, Marlos C. Machado, Michael Bowling
Abstract	Deep reinforcement learning algorithms have shown an impressive ability to learn complex control policies in high-dimensional tasks. However, despite the ever-increasing performance on popular benchmarks, policies learned by deep reinforcement learning algorithms can struggle to generalize when evaluated in remarkably similar environments. In this paper we propose a protocol to evaluate generalization in reinforcement learning through different modes of Atari 2600 games. With that protocol we assess the generalization capabilities of DQN, one of the most traditional deep reinforcement learning algorithms, and we provide evidence suggesting that DQN overspecializes to the training environment. We then comprehensively evaluate the impact of dropout and $\ell_2$ regularization, as well as the impact of reusing learned representations to improve the generalization capabilities of DQN. Despite regularization being largely underutilized in deep reinforcement learning, we show that it can, in fact, help DQN learn more general features. These features can be reused and fine-tuned on similar tasks, considerably improving DQN’s sample efficiency.
Tasks	Atari Games
Published	2018-09-29
URL	https://arxiv.org/abs/1810.00123v3
PDF	https://arxiv.org/pdf/1810.00123v3.pdf
PWC	https://paperswithcode.com/paper/generalization-and-regularization-in-dqn
Repo	https://github.com/jessefarebro/dqn-ale
Framework	tf

Temporal Difference Variational Auto-Encoder


Title	Temporal Difference Variational Auto-Encoder
Authors	Karol Gregor, George Papamakarios, Frederic Besse, Lars Buesing, Theophane Weber
Abstract	To act and plan in complex environments, we posit that agents should have a mental simulator of the world with three characteristics: (a) it should build an abstract state representing the condition of the world; (b) it should form a belief which represents uncertainty on the world; (c) it should go beyond simple step-by-step simulation, and exhibit temporal abstraction. Motivated by the absence of a model satisfying all these requirements, we propose TD-VAE, a generative sequence model that learns representations containing explicit beliefs about states several steps into the future, and that can be rolled out directly without single-step transitions. TD-VAE is trained on pairs of temporally separated time points, using an analogue of temporal difference learning used in reinforcement learning.
Tasks
Published	2018-06-08
URL	http://arxiv.org/abs/1806.03107v3
PDF	http://arxiv.org/pdf/1806.03107v3.pdf
PWC	https://paperswithcode.com/paper/temporal-difference-variational-auto-encoder
Repo	https://github.com/UDrink/dynamic_vae
Framework	tf

Learning Plannable Representations with Causal InfoGAN


Title	Learning Plannable Representations with Causal InfoGAN
Authors	Thanard Kurutach, Aviv Tamar, Ge Yang, Stuart Russell, Pieter Abbeel
Abstract	In recent years, deep generative models have been shown to ‘imagine’ convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans – a plausible sequence of observations that transition a dynamical system from its current configuration to a desired goal state, which can later be used as a reference trajectory for control. We focus on systems with high-dimensional observations, such as images, and propose an approach that naturally combines representation learning and planning. Our framework learns a generative model of sequential observations, where the generative process is induced by a transition in a low-dimensional planning model, and an additional noise. By maximizing the mutual information between the generated observations and the transition in the planning model, we obtain a low-dimensional representation that best explains the causal nature of the data. We structure the planning model to be compatible with efficient planning algorithms, and we propose several such models based on either discrete or continuous states. Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations. We demonstrate our method on imagining plausible visual plans of rope manipulation.
Tasks	Representation Learning
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09341v1
PDF	http://arxiv.org/pdf/1807.09341v1.pdf
PWC	https://paperswithcode.com/paper/learning-plannable-representations-with
Repo	https://github.com/thanard/causal-infogan
Framework	pytorch

Stack-Pointer Networks for Dependency Parsing


Title	Stack-Pointer Networks for Dependency Parsing
Authors	Xuezhe Ma, Zecong Hu, Jingzhou Liu, Nanyun Peng, Graham Neubig, Eduard Hovy
Abstract	We introduce a novel architecture for dependency parsing: \emph{stack-pointer networks} (\textbf{\textsc{StackPtr}}). Combining pointer networks~\citep{vinyals2015pointer} with an internal stack, the proposed model first reads and encodes the whole sentence, then builds the dependency tree top-down (from root-to-leaf) in a depth-first fashion. The stack tracks the status of the depth-first search and the pointer networks select one child for the word at the top of the stack at each step. The \textsc{StackPtr} parser benefits from the information of the whole sentence and all previously derived subtree structures, and removes the left-to-right restriction in classical transition-based parsers. Yet, the number of steps for building any (including non-projective) parse tree is linear in the length of the sentence just as other transition-based parsers, yielding an efficient decoding algorithm with $O(n^2)$ time complexity. We evaluate our model on 29 treebanks spanning 20 languages and different dependency annotation schemas, and achieve state-of-the-art performance on 21 of them.
Tasks	Dependency Parsing
Published	2018-05-03
URL	http://arxiv.org/abs/1805.01087v1
PDF	http://arxiv.org/pdf/1805.01087v1.pdf
PWC	https://paperswithcode.com/paper/stack-pointer-networks-for-dependency-parsing
Repo	https://github.com/XuezheMax/NeuroNLP2
Framework	pytorch

On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data


Title	On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data
Authors	Nan Lu, Gang Niu, Aditya Krishna Menon, Masashi Sugiyama
Abstract	Empirical risk minimization (ERM), with proper loss function and regularization, is the common practice of supervised classification. In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM. We prove that it is impossible to estimate the risk of an arbitrary binary classifier in an unbiased manner given a single set of U data, but it becomes possible given two sets of U data with different class priors. These two facts answer a fundamental question—what the minimal supervision is for training any binary classifier from only U data. Following these findings, we propose an ERM-based learning method from two sets of U data, and then prove it is consistent. Experiments demonstrate the proposed method could train deep models and outperform state-of-the-art methods for learning from two sets of U data.
Tasks
Published	2018-08-31
URL	http://arxiv.org/abs/1808.10585v4
PDF	http://arxiv.org/pdf/1808.10585v4.pdf
PWC	https://paperswithcode.com/paper/on-the-minimal-supervision-for-training-any
Repo	https://github.com/lunanbit/UUlearning
Framework	tf

ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content


Title	ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content
Authors	Demetris Marnerides, Thomas Bashford-Rogers, Jonathan Hatchett, Kurt Debattista
Abstract	High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end-to-end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.
Tasks	Quantization
Published	2018-03-06
URL	https://arxiv.org/abs/1803.02266v2
PDF	https://arxiv.org/pdf/1803.02266v2.pdf
PWC	https://paperswithcode.com/paper/expandnet-a-deep-convolutional-neural-network
Repo	https://github.com/dmarnerides/hdr-expandnet
Framework	pytorch

The UCR Time Series Archive


Title	The UCR Time Series Archive
Authors	Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Eamonn Keogh
Abstract	The UCR Time Series Archive - introduced in 2002, has become an important resource in the time series data mining community, with at least one thousand published papers making use of at least one data set from the archive. The original incarnation of the archive had sixteen data sets but since that time, it has gone through periodic expansions. The last expansion took place in the summer of 2015 when the archive grew from 45 to 85 data sets. This paper introduces and will focus on the new data expansion from 85 to 128 data sets. Beyond expanding this valuable resource, this paper offers pragmatic advice to anyone who may wish to evaluate a new algorithm on the archive. Finally, this paper makes a novel and yet actionable claim: of the hundreds of papers that show an improvement over the standard baseline (1-nearest neighbor classification), a large fraction may be mis-attributing the reasons for their improvement. Moreover, they may have been able to achieve the same improvement with a much simpler modification, requiring just a single line of code.
Tasks	Time Series
Published	2018-10-17
URL	https://arxiv.org/abs/1810.07758v2
PDF	https://arxiv.org/pdf/1810.07758v2.pdf
PWC	https://paperswithcode.com/paper/the-ucr-time-series-archive
Repo	https://github.com/FlorentF9/DeepTemporalClustering
Framework	tf

LSTD: A Low-Shot Transfer Detector for Object Detection


Title	LSTD: A Low-Shot Transfer Detector for Object Detection
Authors	Hao Chen, Yali Wang, Guoyou Wang, Yu Qiao
Abstract	Recent advances in object detection are mainly driven by deep learning with large-scale detection benchmarks. However, the fully-annotated training set is often limited for a target detection task, which may deteriorate the performance of deep detectors. To address this challenge, we propose a novel low-shot transfer detector (LSTD) in this paper, where we leverage rich source-domain knowledge to construct an effective target-domain detector with very few training examples. The main contributions are described as follows. First, we design a flexible deep architecture of LSTD to alleviate transfer difficulties in low-shot detection. This architecture can integrate the advantages of both SSD and Faster RCNN in a unified deep framework. Second, we introduce a novel regularized transfer learning framework for low-shot detection, where the transfer knowledge (TK) and background depression (BD) regularizations are proposed to leverage object knowledge respectively from source and target domains, in order to further enhance fine-tuning with a few target images. Finally, we examine our LSTD on a number of challenging low-shot detection experiments, where LSTD outperforms other state-of-the-art approaches. The results demonstrate that LSTD is a preferable deep detector for low-shot scenarios.
Tasks	Object Detection, Transfer Learning
Published	2018-03-05
URL	http://arxiv.org/abs/1803.01529v1
PDF	http://arxiv.org/pdf/1803.01529v1.pdf
PWC	https://paperswithcode.com/paper/lstd-a-low-shot-transfer-detector-for-object
Repo	https://github.com/Cassie94/LSTD
Framework	none

Bayesian inference for PCA and MUSIC algorithms with unknown number of sources


Title	Bayesian inference for PCA and MUSIC algorithms with unknown number of sources
Authors	Viet Hung Tran, Wenwu Wang
Abstract	Principal component analysis (PCA) is a popular method for projecting data onto uncorrelated components in lower dimension, although the optimal number of components is not specified. Likewise, multiple signal classification (MUSIC) algorithm is a popular PCA-based method for estimating directions of arrival (DOAs) of sinusoidal sources, yet it requires the number of sources to be known a priori. The accurate estimation of the number of sources is hence a crucial issue for performance of these algorithms. In this paper, we will show that both PCA and MUSIC actually return the exact joint maximum-a-posteriori (MAP) estimate for uncorrelated steering vectors, although they can only compute this MAP estimate approximately in correlated case. We then use Bayesian method to, for the first time, compute the MAP estimate for the number of sources in PCA and MUSIC algorithms. Intuitively, this MAP estimate corresponds to the highest probability that signal-plus-noise’s variance still dominates projected noise’s variance on signal subspace. In simulations of overlapping multi-tone sources for linear sensor array, our exact MAP estimate is far superior to the asymptotic Akaike information criterion (AIC), which is a popular method for estimating the number of components in PCA and MUSIC algorithms.
Tasks	Bayesian Inference
Published	2018-09-26
URL	http://arxiv.org/abs/1809.10168v1
PDF	http://arxiv.org/pdf/1809.10168v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-inference-for-pca-and-music
Repo	https://github.com/VietTran86/Copula-Variational-Bayes
Framework	none

Sequential adaptive elastic net approach for single-snapshot source localization


Title	Sequential adaptive elastic net approach for single-snapshot source localization
Authors	Muhammad Naveed Tabassum, Esa Ollila
Abstract	This paper proposes efficient algorithms for accurate recovery of direction-of-arrival (DoA) of sources from single-snapshot measurements using compressed beamforming (CBF). In CBF, the conventional sensor array signal model is cast as an underdetermined complex-valued linear regression model and sparse signal recovery methods are used for solving the DoA finding problem. We develop a complex-valued pathwise weighted elastic net (c-PW-WEN) algorithm that finds solutions at knots of penalty parameter values over a path (or grid) of EN tuning parameter values. c-PW-WEN also computes Lasso or weighted Lasso in its path. We then propose a sequential adaptive EN (SAEN) method that is based on c-PW-WEN algorithm with adaptive weights that depend on the previous solution. Extensive simulation studies illustrate that SAEN improves the probability of exact recovery of true support compared to conventional sparse signal recovery approaches such as Lasso, elastic net or orthogonal matching pursuit in several challenging multiple target scenarios. The effectiveness of SAEN is more pronounced in the presence of high mutual coherence.
Tasks
Published	2018-05-19
URL	http://arxiv.org/abs/1805.07575v1
PDF	http://arxiv.org/pdf/1805.07575v1.pdf
PWC	https://paperswithcode.com/paper/sequential-adaptive-elastic-net-approach-for
Repo	https://github.com/mntabassm/SAEN-LARS
Framework	none

Understanding Membership Inferences on Well-Generalized Learning Models


Title	Understanding Membership Inferences on Well-Generalized Learning Models
Authors	Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A. Gunter, Kai Chen
Abstract	Membership Inference Attack (MIA) determines the presence of a record in a machine learning model’s training data by querying the model. Prior work has shown that the attack is feasible when the model is overfitted to its training data or when the adversary controls the training algorithm. However, when the model is not overfitted and the adversary does not control the training algorithm, the threat is not well understood. In this paper, we report a study that discovers overfitting to be a sufficient but not a necessary condition for an MIA to succeed. More specifically, we demonstrate that even a well-generalized model contains vulnerable instances subject to a new generalized MIA (GMIA). In GMIA, we use novel techniques for selecting vulnerable instances and detecting their subtle influences ignored by overfitting metrics. Specifically, we successfully identify individual records with high precision in real-world datasets by querying black-box machine learning models. Further we show that a vulnerable record can even be indirectly attacked by querying other related records and existing generalization techniques are found to be less effective in protecting the vulnerable instances. Our findings sharpen the understanding of the fundamental cause of the problem: the unique influences the training instance may have on the model.
Tasks	Inference Attack
Published	2018-02-13
URL	http://arxiv.org/abs/1802.04889v1
PDF	http://arxiv.org/pdf/1802.04889v1.pdf
PWC	https://paperswithcode.com/paper/understanding-membership-inferences-on-well
Repo	https://github.com/BielStela/membership_inference
Framework	none

Generating 3D faces using Convolutional Mesh Autoencoders


Title	Generating 3D faces using Convolutional Mesh Autoencoders
Authors	Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black
Abstract	Learned 3D representations of human faces are useful for computer vision problems such as 3D face tracking and reconstruction from images, as well as graphics applications such as character generation and animation. Traditional models learn a latent representation of a face using linear subspaces or higher-order tensor generalizations. Due to this linearity, they can not capture extreme deformations and non-linear expressions. To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface. We introduce mesh sampling operations that enable a hierarchical mesh representation that captures non-linear variations in shape and expression at multiple scales within the model. In a variational setting, our model samples diverse realistic 3D faces from a multivariate Gaussian distribution. Our training data consists of 20,466 meshes of extreme expressions captured over 12 different subjects. Despite limited training data, our trained model outperforms state-of-the-art face models with 50% lower reconstruction error, while using 75% fewer parameters. We also show that, replacing the expression space of an existing state-of-the-art face model with our autoencoder, achieves a lower reconstruction error. Our data, model and code are available at http://github.com/anuragranj/coma
Tasks
Published	2018-07-26
URL	http://arxiv.org/abs/1807.10267v3
PDF	http://arxiv.org/pdf/1807.10267v3.pdf
PWC	https://paperswithcode.com/paper/generating-3d-faces-using-convolutional-mesh
Repo	https://github.com/sw-gong/spiralnet_plus
Framework	pytorch