January 30, 2020

3333 words 16 mins read

Paper Group ANR 369

Self-Assignment Flows for Unsupervised Data Labeling on Graphs. Manifold Fitting under Unbounded Noise. Who is in Your Top Three? Optimizing Learning in Elections with Many Candidates. Fast and accurate reconstruction of HARDI using a 1D encoder-decoder convolutional network. Interpreting Distortions in Dimensionality Reduction by Superimposing Nei …

Self-Assignment Flows for Unsupervised Data Labeling on Graphs


Title	Self-Assignment Flows for Unsupervised Data Labeling on Graphs
Authors	Matthias Zisler, Artjom Zern, Stefania Petra, Christoph Schnörr
Abstract	This paper extends the recently introduced assignment flow approach for supervised image labeling to unsupervised scenarios where no labels are given. The resulting self-assignment flow takes a pairwise data affinity matrix as input data and maximizes the correlation with a low-rank matrix that is parametrized by the variables of the assignment flow, which entails an assignment of the data to themselves through the formation of latent labels (feature prototypes). A single user parameter, the neighborhood size for the geometric regularization of assignments, drives the entire process. By smooth geodesic interpolation between different normalizations of self-assignment matrices on the positive definite matrix manifold, a one-parameter family of self-assignment flows is defined. Accordingly, our approach can be characterized from different viewpoints, e.g. as performing spatially regularized, rank-constrained discrete optimal transport, or as computing spatially regularized normalized spectral cuts. Regarding combinatorial optimization, our approach successfully determines completely positive factorizations of self-assignments in large-scale scenarios, subject to spatial regularization. Various experiments including the unsupervised learning of patch dictionaries using a locally invariant distance function, illustrate the properties of the approach.
Tasks	Combinatorial Optimization
Published	2019-11-08
URL	https://arxiv.org/abs/1911.03472v2
PDF	https://arxiv.org/pdf/1911.03472v2.pdf
PWC	https://paperswithcode.com/paper/self-assignment-flows-for-unsupervised-data
Repo
Framework

Manifold Fitting under Unbounded Noise


Title	Manifold Fitting under Unbounded Noise
Authors	Zhigang Yao, Yuqing Xia
Abstract	There has been an emerging trend in non-Euclidean dimension reduction of aiming to recover a low dimensional structure, namely a manifold, underlying the high dimensional data. Recovering the manifold requires the noise to be of certain concentration. Existing methods address this problem by constructing an output manifold based on the tangent space estimation at each sample point. Although theoretical convergence for these methods is guaranteed, either the samples are noiseless or the noise is bounded. However, if the noise is unbounded, which is a common scenario, the tangent space estimation of the noisy samples will be blurred, thereby breaking the manifold fitting. In this paper, we introduce a new manifold-fitting method, by which the output manifold is constructed by directly estimating the tangent spaces at the projected points on the underlying manifold, rather than at the sample points, to decrease the error caused by the noise. Our new method provides theoretical convergence, in terms of the upper bound on the Hausdorff distance between the output and underlying manifold and the lower bound on the reach of the output manifold, when the noise is unbounded. Numerical simulations are provided to validate our theoretical findings and demonstrate the advantages of our method over other relevant methods. Finally, our method is applied to real data examples.
Tasks	Dimensionality Reduction
Published	2019-09-23
URL	https://arxiv.org/abs/1909.10228v1
PDF	https://arxiv.org/pdf/1909.10228v1.pdf
PWC	https://paperswithcode.com/paper/190910228
Repo
Framework

Who is in Your Top Three? Optimizing Learning in Elections with Many Candidates


Title	Who is in Your Top Three? Optimizing Learning in Elections with Many Candidates
Authors	Nikhil Garg, Lodewijk Gelauff, Sukolsak Sakshuwong, Ashish Goel
Abstract	Elections and opinion polls often have many candidates, with the aim to either rank the candidates or identify a small set of winners according to voters’ preferences. In practice, voters do not provide a full ranking; instead, each voter provides their favorite K candidates, potentially in ranked order. The election organizer must choose K and an aggregation rule. We provide a theoretical framework to make these choices. Each K-Approval or K-partial ranking mechanism (with a corresponding positional scoring rule) induces a learning rate for the speed at which the election correctly recovers the asymptotic outcome. Given the voter choice distribution, the election planner can thus identify the rate optimal mechanism. Earlier work in this area provides coarse order-of-magnitude guaranties which are not sufficient to make such choices. Our framework further resolves questions of when randomizing between multiple mechanisms may improve learning, for arbitrary voter noise models. Finally, we use data from 5 large participatory budgeting elections that we organized across several US cities, along with other ranking data, to demonstrate the utility of our methods. In particular, we find that historically such elections have set K too low and that picking the right mechanism can be the difference between identifying the ultimate winner with only a 80% probability or a 99.9% probability after 400 voters.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08160v2
PDF	https://arxiv.org/pdf/1906.08160v2.pdf
PWC	https://paperswithcode.com/paper/who-is-in-your-top-three-optimizing-learning
Repo
Framework

Fast and accurate reconstruction of HARDI using a 1D encoder-decoder convolutional network


Title	Fast and accurate reconstruction of HARDI using a 1D encoder-decoder convolutional network
Authors	Shi Yin, Zhengqiang Zhang, Qinmu Peng, Xinge You
Abstract	High angular resolution diffusion imaging (HARDI) demands a lager amount of data measurements compared to diffusion tensor imaging, restricting its use in practice. In this work, we explore a learning-based approach to reconstruct HARDI from a smaller number of measurements in q-space. The approach aims to directly learn the mapping relationship between the measured and HARDI signals from the collecting HARDI acquisitions of other subjects. Specifically, the mapping is represented as a 1D encoder-decoder convolutional neural network under the guidance of the compressed sensing (CS) theory for HARDI reconstruction. The proposed network architecture mainly consists of two parts: an encoder network produces the sparse coefficients and a decoder network yields a reconstruction result. Experiment results demonstrate we can robustly reconstruct HARDI signals with the accurate results and fast speed.
Tasks
Published	2019-03-21
URL	http://arxiv.org/abs/1903.09272v1
PDF	http://arxiv.org/pdf/1903.09272v1.pdf
PWC	https://paperswithcode.com/paper/fast-and-accurate-reconstruction-of-hardi
Repo
Framework

Interpreting Distortions in Dimensionality Reduction by Superimposing Neighbourhood Graphs


Title	Interpreting Distortions in Dimensionality Reduction by Superimposing Neighbourhood Graphs
Authors	Benoît Colange, Laurent Vuillon, Sylvain Lespinats, Denys Dutykh
Abstract	To perform visual data exploration, many dimensionality reduction methods have been developed. These tools allow data analysts to represent multidimensional data in a 2D or 3D space, while preserving as much relevant information as possible. Yet, they cannot preserve all structures simultaneously and they induce some unavoidable distortions. Hence, many criteria have been introduced to evaluate a map’s overall quality, mostly based on the preservation of neighbourhoods. Such global indicators are currently used to compare several maps, which helps to choose the most appropriate mapping method and its hyperparameters. However, those aggregated indicators tend to hide the local repartition of distortions. Thereby, they need to be supplemented by local evaluation to ensure correct interpretation of maps. In this paper, we describe a new method, called MING, for `Map Interpretation using Neighbourhood Graphs’. It offers a graphical interpretation of pairs of map quality indicators, as well as local evaluation of the distortions. This is done by displaying on the map the nearest neighbours graphs computed in the data space and in the embedding. Shared and unshared edges exhibit reliable and unreliable neighbourhood information conveyed by the mapping. By this mean, analysts may determine whether proximity (or remoteness) of points on the map faithfully represents similarity (or dissimilarity) of original data, within the meaning of a chosen map quality criteria. We apply this approach to two pairs of widespread indicators: precision/recall and trustworthiness/continuity, chosen for their wide use in the community, which will allow an easy handling by users. \|
Tasks	Dimensionality Reduction
Published	2019-09-20
URL	https://arxiv.org/abs/1909.12902v1
PDF	https://arxiv.org/pdf/1909.12902v1.pdf
PWC	https://paperswithcode.com/paper/interpreting-distortions-in-dimensionality
Repo
Framework

Data Mapping and Finite Difference Learning


Title	Data Mapping and Finite Difference Learning
Authors	Jiangsheng You
Abstract	Restricted Boltzmann machine (RBM) is a two-layer neural network constructed as a probabilistic model and its training is to maximize a product of probabilities by the contrastive divergence (CD) scheme. In this paper a data mapping is proposed to describe the relationship between the visible and hidden layers and the training is to minimize a squared error on the visible layer by a finite difference learning. This paper presents three new properties in using the RBM: 1) nodes on the visible and hidden layers can take real-valued matrix data without a probabilistic interpretation; 2) the famous CD1 is a finite difference approximation of the gradient descent; 3) the activation can take non-sigmoid functions such as identity, relu and softsign. The data mapping provides a unified framework on the dimensionality reduction, the feature extraction and the data representation pioneered and developed by Hinton and his colleagues. As an approximation of the gradient descent, the finite difference learning is applicable to both directed and undirected graphs. Numerical experiments are performed to verify these new properties on the very low dimensionality reduction, the collinearity of timer series data and the use of flexible activations.
Tasks	Dimensionality Reduction
Published	2019-09-18
URL	https://arxiv.org/abs/1909.08210v3
PDF	https://arxiv.org/pdf/1909.08210v3.pdf
PWC	https://paperswithcode.com/paper/data-mapping-for-restricted-boltzmann-machine
Repo
Framework

Forecasting Future Sequence of Actions to Complete an Activity


Title	Forecasting Future Sequence of Actions to Complete an Activity
Authors	Yan Bin Ng, Basura Fernando
Abstract	Future human action forecasting from partial observations of activities is an important problem in many practical applications such as assistive robotics, video surveillance and security. We present a method to forecast actions for the unseen future of the video using a neural machine translation technique that uses encoder-decoder architecture. The input to this model is the observed RGB video, and the target is to generate the future symbolic action sequence. Unlike most methods that predict frame or clip level predictions for some unseen percentage of video, we predict the complete action sequence that is required to accomplish the activity. To cater for two types of uncertainty in the future predictions, we propose a novel loss function. We show a combination of optimal transport and future uncertainty losses help to boost results. We evaluate our model in three challenging video datasets (Charades, MPII cooking and Breakfast). We outperform other state-of-the art techniques for frame based action forecasting task by 5.06% on average across several action forecasting setups.
Tasks	Machine Translation
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04608v1
PDF	https://arxiv.org/pdf/1912.04608v1.pdf
PWC	https://paperswithcode.com/paper/forecasting-future-sequence-of-actions-to
Repo
Framework

Wasserstein GAN Can Perform PCA


Title	Wasserstein GAN Can Perform PCA
Authors	Jaewoong Cho, Changho Suh
Abstract	Generative Adversarial Networks (GANs) have become a powerful framework to learn generative models that arise across a wide variety of domains. While there has been a recent surge in the development of numerous GAN architectures with distinct optimization metrics, we are still lacking in our understanding on how far away such GANs are from optimality. In this paper, we make progress on a theoretical understanding of the GANs under a simple linear-generator Gaussian-data setting where the optimal maximum-likelihood generator is known to perform Principal Component Analysis (PCA). We find that the original GAN by Goodfellow et. al. fails to recover the optimal PCA solution. On the other hand, we show that Wasserstein GAN can approach the PCA solution in the limit of sample size, and hence it may serve as a basis for an optimal GAN architecture that yields the optimal generator for a wide range of data settings.
Tasks
Published	2019-02-25
URL	https://arxiv.org/abs/1902.09073v2
PDF	https://arxiv.org/pdf/1902.09073v2.pdf
PWC	https://paperswithcode.com/paper/wasserstein-gan-can-perform-pca
Repo
Framework

Relating Simple Sentence Representations in Deep Neural Networks and the Brain


Title	Relating Simple Sentence Representations in Deep Neural Networks and the Brain
Authors	Sharmistha Jat, Hao Tang, Partha Talukdar, Tom Mitchell
Abstract	What is the relationship between sentence representations learned by deep recurrent models against those encoded by the brain? Is there any correspondence between hidden layers of these recurrent models and brain regions when processing sentences? Can these deep models be used to synthesize brain data which can then be utilized in other extrinsic tasks? We investigate these questions using sentences with simple syntax and semantics (e.g., The bone was eaten by the dog.). We consider multiple neural network architectures, including recently proposed ELMo and BERT. We use magnetoencephalography (MEG) brain recording data collected from human subjects when they were reading these simple sentences. Overall, we find that BERT’s activations correlate the best with MEG brain data. We also find that the deep network representation can be used to generate brain data from new sentences to augment existing brain data. To the best of our knowledge, this is the first work showing that the MEG brain recording when reading a word in a sentence can be used to distinguish earlier words in the sentence. Our exploration is also the first to use deep neural network representations to generate synthetic brain data and to show that it helps in improving subsequent stimuli decoding task accuracy.
Tasks
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11861v1
PDF	https://arxiv.org/pdf/1906.11861v1.pdf
PWC	https://paperswithcode.com/paper/relating-simple-sentence-representations-in
Repo
Framework

Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking


Title	Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking
Authors	Jae Shin Yoon, Takaaki Shiratori, Shoou-I Yu, Hyun Soo Park
Abstract	Improvements in data-capture and face modeling techniques have enabled us to create high-fidelity realistic face models. However, driving these realistic face models requires special input data, e.g. 3D meshes and unwrapped textures. Also, these face models expect clean input data taken under controlled lab environments, which is very different from data collected in the wild. All these constraints make it challenging to use the high-fidelity models in tracking for commodity cameras. In this paper, we propose a self-supervised domain adaptation approach to enable the animation of high-fidelity face models from a commodity camera. Our approach first circumvents the requirement for special input data by training a new network that can directly drive a face model just from a single 2D image. Then, we overcome the domain mismatch between lab and uncontrolled environments by performing self-supervised domain adaptation based on “consecutive frame texture consistency” based on the assumption that the appearance of the face is consistent over consecutive frames, avoiding the necessity of modeling the new environment such as lighting or background. Experiments show that we are able to drive a high-fidelity face model to perform complex facial motion from a cellphone camera without requiring any labeled data from the new domain.
Tasks	Domain Adaptation
Published	2019-07-25
URL	https://arxiv.org/abs/1907.10815v1
PDF	https://arxiv.org/pdf/1907.10815v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-adaptation-of-high-fidelity-1
Repo
Framework

A Single Scalable LSTM Model for Short-Term Forecasting of Disaggregated Electricity Loads


Title	A Single Scalable LSTM Model for Short-Term Forecasting of Disaggregated Electricity Loads
Authors	Andrés M. Alonso, F. Javier Nogales, Carlos Ruiz
Abstract	Most electricity systems worldwide are deploying advanced metering infrastructures to collect relevant operational data. In particular, smart meters allow tracking electricity load consumption at a very disaggregated level and at high frequency rates. This data opens the possibility of developing new forecasting models with a potential positive impact in electricity systems. We present a general methodology that is able to process and forecast a large number of smart meter time series. Instead of using traditional and univariate approaches, we develop a single but complex recurrent neural-network model with long short-term memory that can capture individual consumption patterns and also consumptions from different households. The resulting model can accurately predict future loads (short-term) of individual consumers, even if these were not included in the original training set. This entails a great potential for large scale applications as once the single network is trained, accurate individual forecast for new consumers can be obtained at almost no computational cost. The proposed model is tested under a large set of numerical experiments by using a real-world dataset with thousands of disaggregated electricity consumption time series. Furthermore, we explore how geo-demographic segmentation of consumers may impact the forecasting accuracy of the model.
Tasks	Time Series
Published	2019-10-15
URL	https://arxiv.org/abs/1910.06640v2
PDF	https://arxiv.org/pdf/1910.06640v2.pdf
PWC	https://paperswithcode.com/paper/a-single-scalable-lstm-model-for-short-term
Repo
Framework

The column measure and Gradient-Free Gradient Boosting


Title	The column measure and Gradient-Free Gradient Boosting
Authors	Tino Werner, Peter Ruckdeschel
Abstract	Sparse model selection by structural risk minimization leads to a set of a few predictors, ideally a subset of the true predictors. This selection clearly depends on the underlying loss function $\tilde L$. For linear regression with square loss, the particular (functional) Gradient Boosting variant $L_2-$Boosting excels for its computational efficiency even for very large predictor sets, while still providing suitable estimation consistency. For more general loss functions, functional gradients are not always easily accessible or, like in the case of continuous ranking, need not even exist. To close this gap, starting from column selection frequencies obtained from $L_2-$Boosting, we introduce a loss-dependent ‘‘column measure’’ $\nu^{(\tilde L)}$ which mathematically describes variable selection. The fact that certain variables relevant for a particular loss $\tilde L$ never get selected by $L_2-$Boosting is reflected by a respective singular part of $\nu^{(\tilde L)}$ w.r.t. $\nu^{(L_2)}$. With this concept at hand, it amounts to a suitable change of measure (accounting for singular parts) to make $L_2-$Boosting select variables according to a different loss $\tilde L$. As a consequence, this opens the bridge to applications of simulational techniques such as various resampling techniques, or rejection sampling, to achieve this change of measure in an algorithmic way.
Tasks	Model Selection
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10960v1
PDF	https://arxiv.org/pdf/1909.10960v1.pdf
PWC	https://paperswithcode.com/paper/the-column-measure-and-gradient-free-gradient
Repo
Framework

Improve variational autoEncoder with auxiliary softmax multiclassifier


Title	Improve variational autoEncoder with auxiliary softmax multiclassifier
Authors	Yao Li
Abstract	As a general-purpose generative model architecture, VAE has been widely used in the field of image and natural language processing. VAE maps high dimensional sample data into continuous latent variables with unsupervised learning. Sampling in the latent variable space of the feature, VAE can construct new image or text data. As a general-purpose generation model, the vanilla VAE can not fit well with various data sets and neural networks with different structures. Because of the need to balance the accuracy of reconstruction and the convenience of latent variable sampling in the training process, VAE often has problems known as “posterior collapse”. images reconstructed by VAE are also often blurred. In this paper, we analyze the main cause of these problem, which is the lack of mutual information between the sample variable and the latent feature variable during the training process. To maintain mutual information in model training, we propose to use the auxiliary softmax multi-classification network structure to improve the training effect of VAE, named VAE-AS. We use MNIST and Omniglot data sets to test the VAE-AS model. Based on the test results, It can be show that VAE-AS has obvious effects on the mutual information adjusting and solving the posterior collapse problem.
Tasks	Omniglot
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06966v3
PDF	https://arxiv.org/pdf/1908.06966v3.pdf
PWC	https://paperswithcode.com/paper/improve-variational-autoencoder-with
Repo
Framework

Energy-aware Goal Selection and Path Planning of UAV Systems via Reinforcement Learning


Title	Energy-aware Goal Selection and Path Planning of UAV Systems via Reinforcement Learning
Authors	A. E. Niaraki Asli, J. Roghair, A. Jannesari
Abstract	Smart data collection via UAV systems is an attractive topic in various disciplines. Disturbances such as intense wind can significantly hinder the operational time of drones. This work demonstrates a reinforcement learning approach for the optimization of power consumption in a UAV system for data collection in sparse locations. Two common reinforcement learning algorithms, Q-learning and SARSA, are implemented in a simulation environment, utilizing a combination of robot operating system (ROS) and Gazebo. The effect of time-varying wind fields and time-dependency of the tasks were simulated and the developed framework showed reliable adaptability in various scenarios. This framework can result in 30% power consumption improvement for intense wind conditions in comparison to na"ive control algorithms.
Tasks	Q-Learning
Published	2019-09-26
URL	https://arxiv.org/abs/1909.12217v2
PDF	https://arxiv.org/pdf/1909.12217v2.pdf
PWC	https://paperswithcode.com/paper/a-simulation-of-uav-power-optimization-via
Repo
Framework

Introduction to Online Convex Optimization


Title	Introduction to Online Convex Optimization
Authors	Elad Hazan
Abstract	This manuscript portrays optimization as a process. In many practical applications the environment is so complex that it is infeasible to lay out a comprehensive theoretical model and use classical algorithmic theory and mathematical optimization. It is necessary as well as beneficial to take a robust approach, by applying an optimization method that learns as one goes along, learning from experience as more aspects of the problem are observed. This view of optimization as a process has become prominent in varied fields and has led to some spectacular success in modeling and systems that are now part of our daily lives.
Tasks
Published	2019-09-07
URL	https://arxiv.org/abs/1909.05207v1
PDF	https://arxiv.org/pdf/1909.05207v1.pdf
PWC	https://paperswithcode.com/paper/introduction-to-online-convex-optimization
Repo
Framework