January 27, 2020

3211 words 16 mins read

Paper Group ANR 1123

A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization. Robustness of accelerated first-order algorithms for strongly convex optimization problems. AETv2: AutoEncoding Transformations for Self-Supervised Representation Learning by Minimizing Geodesic Distances in Lie Groups. Generative Adversarial Networks for geometric …

A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization


Title	A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization
Authors	Guangzeng Xie, Luo Luo, Zhihua Zhang
Abstract	This paper studies the lower bound complexity for the optimization problem whose objective function is the average of $n$ individual smooth convex functions. We consider the algorithm which gets access to gradient and proximal oracle for each individual component. For the strongly-convex case, we prove such an algorithm can not reach an $\varepsilon$-suboptimal point in fewer than $\Omega((n+\sqrt{\kappa n})\log(1/\varepsilon))$ iterations, where $\kappa$ is the condition number of the objective function. This lower bound is tighter than previous results and perfectly matches the upper bound of the existing proximal incremental first-order oracle algorithm Point-SAGA. We develop a novel construction to show the above result, which partitions the tridiagonal matrix of classical examples into $n$ groups. This construction is friendly to the analysis of proximal oracle and also could be used to general convex and average smooth cases naturally.
Tasks
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08394v1
PDF	https://arxiv.org/pdf/1908.08394v1.pdf
PWC	https://paperswithcode.com/paper/a-general-analysis-framework-of-lower
Repo
Framework

Robustness of accelerated first-order algorithms for strongly convex optimization problems


Title	Robustness of accelerated first-order algorithms for strongly convex optimization problems
Authors	Hesameddin Mohammadi, Meisam Razaviyayn, Mihailo R. Jovanović
Abstract	We study the robustness of accelerated first-order algorithms to stochastic uncertainties in gradient evaluation. Specifically, for unconstrained, smooth, strongly convex optimization problems, we examine the mean-squared error in the optimization variable when the iterates are perturbed by additive white noise. This type of uncertainty may arise in situations where an approximation of the gradient is sought through measurements of a real system or in a distributed computation over a network. Even though the underlying dynamics of first-order algorithms for this class of problems are nonlinear, we establish upper bounds on the mean-squared deviation from the optimal solution that are tight up to constant factors. Our analysis quantifies fundamental trade-offs between noise amplification and convergence rates obtained via any acceleration scheme similar to Nesterov’s or heavy-ball methods. To gain additional analytical insight, for strongly convex quadratic problems, we explicitly evaluate the steady-state variance of the optimization variable in terms of the eigenvalues of the Hessian of the objective function. We demonstrate that the entire spectrum of the Hessian, rather than just the extreme eigenvalues, influence robustness of noisy algorithms. We specialize this result to the problem of distributed averaging over undirected networks and examine the role of network size and topology on the robustness of noisy accelerated algorithms.
Tasks
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11011v2
PDF	https://arxiv.org/pdf/1905.11011v2.pdf
PWC	https://paperswithcode.com/paper/robustness-of-accelerated-first-order
Repo
Framework

AETv2: AutoEncoding Transformations for Self-Supervised Representation Learning by Minimizing Geodesic Distances in Lie Groups


Title	AETv2: AutoEncoding Transformations for Self-Supervised Representation Learning by Minimizing Geodesic Distances in Lie Groups
Authors	Feng Lin, Haohang Xu, Houqiang Li, Hongkai Xiong, Guo-Jun Qi
Abstract	Self-supervised learning by predicting transformations has demonstrated outstanding performances in both unsupervised and (semi-)supervised tasks. Among the state-of-the-art methods is the AutoEncoding Transformations (AET) by decoding transformations from the learned representations of original and transformed images. Both deterministic and probabilistic AETs rely on the Euclidean distance to measure the deviation of estimated transformations from their groundtruth counterparts. However, this assumption is questionable as a group of transformations often reside on a curved manifold rather staying in a flat Euclidean space. For this reason, we should use the geodesic to characterize how an image transform along the manifold of a transformation group, and adopt its length to measure the deviation between transformations. Particularly, we present to autoencode a Lie group of homography transformations PG(2) to learn image representations. For this, we make an estimate of the intractable Riemannian logarithm by projecting PG(2) to a subgroup of rotation transformations SO(3) that allows the closed-form expression of geodesic distances. Experiments demonstrate the proposed AETv2 model outperforms the previous version as well as the other state-of-the-art self-supervised models in multiple tasks.
Tasks	Representation Learning
Published	2019-11-16
URL	https://arxiv.org/abs/1911.07004v1
PDF	https://arxiv.org/pdf/1911.07004v1.pdf
PWC	https://paperswithcode.com/paper/aetv2-autoencoding-transformations-for-self
Repo
Framework

Generative Adversarial Networks for geometric surfaces prediction in injection molding


Title	Generative Adversarial Networks for geometric surfaces prediction in injection molding
Authors	Pierre Nagorny, Thomas Lacombe, Hugues Favreliere, Maurice Pillet, Eric Pairel, Ronan Le Goff, Marlene Wali, Jerome Loureaux, Patrice Kiener
Abstract	Geometrical and appearance quality requirements set the limits of the current industrial performance in injection molding. To guarantee the product’s quality, it is necessary to adjust the process settings in a closed loop. Those adjustments cannot rely on the final quality because a part takes days to be geometrically stable. Thus, the final part geometry must be predicted from measurements on hot parts. In this paper, we use recent success of Generative Adversarial Networks (GAN) with the pix2pix network architecture to predict the final part geometry, using only hot parts thermographic images, measured right after production. Our dataset is really small, and the GAN learns to translate thermography to geometry. We firstly study prediction performances using different image similarity comparison algorithms. Moreover, we introduce the innovative use of Discrete Modal Decomposition (DMD) to analyze network predictions. The DMD is a geometrical parameterization technique using a modal space projection to geometrically describe surfaces. We study GAN performances to retrieve geometrical parameterization of surfaces.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10178v1
PDF	http://arxiv.org/pdf/1901.10178v1.pdf
PWC	https://paperswithcode.com/paper/generative-adversarial-networks-for-geometric
Repo
Framework

Emotion Recognition from Speech


Title	Emotion Recognition from Speech
Authors	Kannan Venkataramanan, Haresh Rengaraj Rajamohan
Abstract	In this work, we conduct an extensive comparison of various approaches to speech based emotion recognition systems. The analyses were carried out on audio recordings from Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). After pre-processing the raw audio files, features such as Log-Mel Spectrogram, Mel-Frequency Cepstral Coefficients (MFCCs), pitch and energy were considered. The significance of these features for emotion classification was compared by applying methods such as Long Short Term Memory (LSTM), Convolutional Neural Networks (CNNs), Hidden Markov Models (HMMs) and Deep Neural Networks (DNNs). On the 14-class (2 genders x 7 emotions) classification task, an accuracy of 68% was achieved with a 4-layer 2 dimensional CNN using the Log-Mel Spectrogram features. We also observe that, in emotion recognition, the choice of audio features impacts the results much more than the model complexity.
Tasks	Emotion Classification, Emotion Recognition
Published	2019-12-22
URL	https://arxiv.org/abs/1912.10458v1
PDF	https://arxiv.org/pdf/1912.10458v1.pdf
PWC	https://paperswithcode.com/paper/emotion-recognition-from-speech
Repo
Framework

Disentangling Latent Emotions of Word Embeddings on Complex Emotional Narratives


Title	Disentangling Latent Emotions of Word Embeddings on Complex Emotional Narratives
Authors	Zhengxuan Wu, Yueyi Jiang
Abstract	Word embedding models such as GloVe are widely used in natural language processing (NLP) research to convert words into vectors. Here, we provide a preliminary guide to probe latent emotions in text through GloVe word vectors. First, we trained a neural network model to predict continuous emotion valence ratings by taking linguistic inputs from Stanford Emotional Narratives Dataset (SEND). After interpreting the weights in the model, we found that only a few dimensions of the word vectors contributed to expressing emotions in text, and words were clustered on the basis of their emotional polarities. Furthermore, we performed a linear transformation that projected high dimensional embedded vectors into an emotion space. Based on NRC Emotion Lexicon (EmoLex), we visualized the entanglement of emotions in the lexicon by using both projected and raw GloVe word vectors. We showed that, in the proposed emotion space, we were able to better disentangle emotions than using raw GloVe vectors alone. In addition, we found that the sum vectors of different pairs of emotion words successfully captured expressed human feelings in the EmoLex. For example, the sum of two embedded word vectors expressing Joy and Trust which express Love shared high similarity (similarity score .62) with the embedded vector expressing Optimism. On the contrary, this sum vector was dissimilar (similarity score -.19) with the the embedded vector expressing Remorse. In this paper, we argue that through the proposed emotion space, arithmetic of emotions is preserved in the word vectors. The affective representation uncovered in emotion vector space could shed some light on how to help machines to disentangle emotion expressed in word embeddings.
Tasks	Word Embeddings
Published	2019-08-15
URL	https://arxiv.org/abs/1908.07817v1
PDF	https://arxiv.org/pdf/1908.07817v1.pdf
PWC	https://paperswithcode.com/paper/190807817
Repo
Framework

A Novel Efficient Approach with Data-Adaptive Capability for OMP-based Sparse Subspace Clustering


Title	A Novel Efficient Approach with Data-Adaptive Capability for OMP-based Sparse Subspace Clustering
Authors	Jiaqiyu Zhan, Zhiqiang Bai, Yuesheng Zhu
Abstract	Orthogonal Matching Pursuit (OMP) plays an important role in data science and its applications such as sparse subspace clustering and image processing. However, the existing OMP-based approaches lack of data adaptiveness so that the data cannot be represented well enough and may lose the accuracy. This paper proposes a novel approach to enhance the data-adaptive capability for OMP-based sparse subspace clustering. In our method a parameter selection process is developed to adjust the parameters based on the data distribution for information representation. Our theoretical analysis indicates that the parameter selection process can efficiently coordinate with any OMP-based methods to improve the clustering performance. Also a new Self-Expressive-Affinity (SEA) ratio metric is defined to measure the sparse representation conversion efficiency for spectral clustering to obtain data segmentations. Our experiments show that proposed approach can achieve better performances compared with other OMP-based sparse subspace clustering algorithms in terms of clustering accuracy, SEA ratio and representation quality, also keep the time efficiency and anti-noise ability.
Tasks
Published	2019-03-05
URL	https://arxiv.org/abs/1903.01734v2
PDF	https://arxiv.org/pdf/1903.01734v2.pdf
PWC	https://paperswithcode.com/paper/a-novel-efficient-approach-with-data-adaptive
Repo
Framework

Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data


Title	Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
Authors	Sergio Casas, Cole Gulino, Renjie Liao, Raquel Urtasun
Abstract	In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.
Tasks
Published	2019-10-18
URL	https://arxiv.org/abs/1910.08233v1
PDF	https://arxiv.org/pdf/1910.08233v1.pdf
PWC	https://paperswithcode.com/paper/spatially-aware-graph-neural-networks-for
Repo
Framework

Comprehensive Soccer Video Understanding: Towards Human-comparable Video Understanding System in Constrained Environment


Title	Comprehensive Soccer Video Understanding: Towards Human-comparable Video Understanding System in Constrained Environment
Authors	Yudong Jiang, Kaixu Cui, Leilei Chen, Canjin Wang, Chen Wang, Hui Liu, Changliang Xu
Abstract	Comprehensive video understanding, a challenging task in computer vision to understand videos like humans, has been explored in ways including object detection and tracking, action classification. However, most works for video understanding mainly focus on isolated aspects of video analysis, yet ignore the inner correlation among those tasks. Sports games videos can serve as a perfect research object with restrictive conditions, while complex and challenging enough to study the core problems in computer vision comprehensively. In this paper, we propose a new soccer video database named SoccerDB with the benchmark of object detection, action recognition, temporal action detection, and highlight detection. We further survey a collection of strong baselines on SoccerDB, which have demonstrated state-of-the-art performance on each independent task in recent years. We believe that the release of SoccerDB will tremendously advance researches of combining different tasks in closed form around the comprehensive video understanding problem. Our dataset and code will be published after the paper accepted.
Tasks	Action Classification, Action Detection, Object Detection, Video Understanding
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04465v2
PDF	https://arxiv.org/pdf/1912.04465v2.pdf
PWC	https://paperswithcode.com/paper/comprehensive-soccer-video-understanding
Repo
Framework

Simulation of hyperelastic materials in real-time using Deep Learning


Title	Simulation of hyperelastic materials in real-time using Deep Learning
Authors	Andrea Mendizabal, Pablo Márquez-Neila, Stéphane Cotin
Abstract	The finite element method (FEM) is among the most commonly used numerical methods for solving engineering problems. Due to its computational cost, various ideas have been introduced to reduce computation times, such as domain decomposition, parallel computing, adaptive meshing, and model order reduction. In this paper we present U-Mesh: a data-driven method based on a U-Net architecture that approximates the non-linear relation between a contact force and the displacement field computed by a FEM algorithm. We show that deep learning, one of the latest machine learning methods based on artificial neural networks, can enhance computational mechanics through its ability to encode highly non-linear models in a compact form. Our method is applied to two benchmark examples: a cantilever beam and an L-shape subject to moving punctual loads. A comparison between our method and proper orthogonal decomposition (POD) is done through the paper. The results show that U-Mesh can perform very fast simulations on various geometries, mesh resolutions and number of input forces with very small errors.
Tasks	Cantilever Beam
Published	2019-04-10
URL	https://arxiv.org/abs/1904.06197v2
PDF	https://arxiv.org/pdf/1904.06197v2.pdf
PWC	https://paperswithcode.com/paper/190406197
Repo
Framework

Video Extrapolation with an Invertible Linear Embedding


Title	Video Extrapolation with an Invertible Linear Embedding
Authors	Robert Pottorff, Jared Nielsen, David Wingate
Abstract	We predict future video frames from complex dynamic scenes, using an invertible neural network as the encoder of a nonlinear dynamic system with latent linear state evolution. Our invertible linear embedding (ILE) demonstrates successful learning, prediction and latent state inference. In contrast to other approaches, ILE does not use any explicit reconstruction loss or simplistic pixel-space assumptions. Instead, it leverages invertibility to optimize the likelihood of image sequences exactly, albeit indirectly. Comparison with a state-of-the-art method demonstrates the viability of our approach.
Tasks	Predict Future Video Frames
Published	2019-03-01
URL	http://arxiv.org/abs/1903.00133v1
PDF	http://arxiv.org/pdf/1903.00133v1.pdf
PWC	https://paperswithcode.com/paper/video-extrapolation-with-an-invertible-linear
Repo
Framework

Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges


Title	Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges
Authors	Solmaz Niknam, Harpreet S. Dhillon, Jeffery H. Reed
Abstract	There is a growing interest in the wireless communications community to complement the traditional model-based design approaches with data-driven machine learning (ML)-based solutions. While conventional ML approaches rely on the assumption of having the data and processing heads in a central entity, this is not always feasible in wireless communications applications because of the inaccessibility of private data and large communication overhead required to transmit raw data to central ML processors. As a result, decentralized ML approaches that keep the data where it is generated are much more appealing. Owing to its privacy-preserving nature, federated learning is particularly relevant for many wireless applications, especially in the context of fifth generation (5G) networks. In this article, we provide an accessible introduction to the general idea of federated learning, discuss several possible applications in 5G networks, and describe key technical challenges and open problems for future research on federated learning in the context of wireless communications.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1908.06847v3
PDF	https://arxiv.org/pdf/1908.06847v3.pdf
PWC	https://paperswithcode.com/paper/federated-learning-for-wireless
Repo
Framework

‘Place-cell’ emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space


Title	‘Place-cell’ emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight space
Authors	Moshir Harsh, Jérôme Tubiana, Simona Cocco, Remi Monasson
Abstract	Distributions of data or sensory stimuli often enjoy underlying invariances. How and to what extent those symmetries are captured by unsupervised learning methods is a relevant question in machine learning and in computational neuroscience. We study here, through a combination of numerical and analytical tools, the learning dynamics of Restricted Boltzmann Machines (RBM), a neural network paradigm for representation learning. As learning proceeds from a random configuration of the network weights, we show the existence of, and characterize a symmetry-breaking phenomenon, in which the latent variables acquire receptive fields focusing on limited parts of the invariant manifold supporting the data. The symmetry is restored at large learning times through the diffusion of the receptive field over the invariant manifold; hence, the RBM effectively spans a continuous attractor in the space of network weights. This symmetry-breaking phenomenon takes place only if the amount of data available for training exceeds some critical value, depending on the network size and the intensity of symmetry-induced correlations in the data; below this ‘retarded-learning’ threshold, the network weights are essentially noisy and overfit the data.
Tasks	Representation Learning
Published	2019-12-30
URL	https://arxiv.org/abs/1912.12942v1
PDF	https://arxiv.org/pdf/1912.12942v1.pdf
PWC	https://paperswithcode.com/paper/place-cell-emergence-and-learning-of
Repo
Framework

Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters


Title	Adaptive Dynamic Programming for Model-free Tracking of Trajectories with Time-varying Parameters
Authors	Florian Köpf, Simon Ramsteiner, Michael Flad, Sören Hohmann
Abstract	In order to autonomously learn to control unknown systems optimally w.r.t. an objective function, Adaptive Dynamic Programming (ADP) is well-suited to adapt controllers based on experience from interaction with the system. In recent years, many researchers focused on the tracking case, where the aim is to follow a desired trajectory. So far, ADP tracking controllers assume that the reference trajectory follows time-invariant exo-system dynamics-an assumption that does not hold for many applications. In order to overcome this limitation, we propose a new Q-function which explicitly incorporates a parametrized approximation of the reference trajectory. This allows to learn to track a general class of trajectories by means of ADP. Once our Q-function has been learned, the associated controller copes with time-varying reference trajectories without need of further training and independent of exo-system dynamics. After proposing our general model-free off-policy tracking method, we provide analysis of the important special case of linear quadratic tracking. We conclude our paper with an example which demonstrates that our new method successfully learns the optimal tracking controller and outperforms existing approaches in terms of tracking error and cost.
Tasks
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07239v5
PDF	https://arxiv.org/pdf/1909.07239v5.pdf
PWC	https://paperswithcode.com/paper/adaptive-dynamic-programming-for-model-free
Repo
Framework

Network Creation Games with Local Information and Edge Swaps


Title	Network Creation Games with Local Information and Edge Swaps
Authors	Shotaro Yoshimura, Yukiko Yamauchi
Abstract	In the swap game (SG) selfish players, each of which is associated to a vertex, form a graph by edge swaps, i.e., a player changes its strategy by simultaneously removing an adjacent edge and forming a new edge (Alon et al., 2013). The cost of a player considers the average distance to all other players or the maximum distance to other players. Any SG by $n$ players starting from a tree converges to an equilibrium with a constant Price of Anarchy (PoA) within $O(n^3)$ edge swaps (Lenzner, 2011). We focus on SGs where each player knows the subgraph induced by players within distance $k$. Therefore, each player cannot compute its cost nor a best response. We first consider pessimistic players who consider the worst-case global graph. We show that any SG starting from a tree (i) always converges to an equilibrium within $O(n^3)$ edge swaps irrespective of the value of $k$, (ii) the PoA is $\Theta(n)$ for $k=1,2,3$, and (iii) the PoA is constant for $k \geq 4$. We then introduce weakly pessimistic players and optimistic players and show that these less pessimistic players achieve constant PoA for $k \leq 3$ at the cost of best response cycles.
Tasks
Published	2019-11-12
URL	https://arxiv.org/abs/1911.04743v1
PDF	https://arxiv.org/pdf/1911.04743v1.pdf
PWC	https://paperswithcode.com/paper/network-creation-games-with-local-information
Repo
Framework