October 17, 2019

2938 words 14 mins read

Paper Group ANR 931

Day-ahead time series forecasting: application to capacity planning. C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs. Fast and Accurate Reconstruction of Compressed Color Light Field. Online Regularized Nonlinear Acceleration. MoDL-MUSSELS: Model-Based Deep Learning for Multi-Shot Sensitivity Encoded Diffusion MRI. …

Day-ahead time series forecasting: application to capacity planning


Title	Day-ahead time series forecasting: application to capacity planning
Authors	Colin Leverger, Vincent Lemaire, Simon Malinowski, Thomas Guyet, Laurence Rozé
Abstract	In the context of capacity planning, forecasting the evolution of informatics servers usage enables companies to better manage their computational resources. We address this problem by collecting key indicator time series and propose to forecast their evolution a day-ahead. Our method assumes that data is structured by a daily seasonality, but also that there is typical evolution of indicators within a day. Then, it uses the combination of a clustering algorithm and Markov Models to produce day-ahead forecasts. Our experiments on real datasets show that the data satisfies our assumption and that, in the case study, our method outperforms classical approaches (AR, Holt-Winters).
Tasks	Time Series, Time Series Forecasting
Published	2018-11-06
URL	http://arxiv.org/abs/1811.02215v1
PDF	http://arxiv.org/pdf/1811.02215v1.pdf
PWC	https://paperswithcode.com/paper/day-ahead-time-series-forecasting-application
Repo
Framework

C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs


Title	C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs
Authors	Shuo Wang, Zhe Li, Caiwen Ding, Bo Yuan, Yanzhi Wang, Qinru Qiu, Yun Liang
Abstract	Recently, significant accuracy improvement has been achieved for acoustic recognition systems by increasing the model size of Long Short-Term Memory (LSTM) networks. Unfortunately, the ever-increasing size of LSTM model leads to inefficient designs on FPGAs due to the limited on-chip resources. The previous work proposes to use a pruning based compression technique to reduce the model size and thus speedups the inference on FPGAs. However, the random nature of the pruning technique transforms the dense matrices of the model to highly unstructured sparse ones, which leads to unbalanced computation and irregular memory accesses and thus hurts the overall performance and energy efficiency. In contrast, we propose to use a structured compression technique which could not only reduce the LSTM model size but also eliminate the irregularities of computation and memory accesses. This approach employs block-circulant instead of sparse matrices to compress weight matrices and reduces the storage requirement from $\mathcal{O}(k^2)$ to $\mathcal{O}(k)$. Fast Fourier Transform algorithm is utilized to further accelerate the inference by reducing the computational complexity from $\mathcal{O}(k^2)$ to $\mathcal{O}(k\text{log}k)$. The datapath and activation functions are quantized as 16-bit to improve the resource utilization. More importantly, we propose a comprehensive framework called C-LSTM to automatically optimize and implement a wide range of LSTM variants on FPGAs. According to the experimental results, C-LSTM achieves up to 18.8X and 33.5X gains for performance and energy efficiency compared with the state-of-the-art LSTM implementation under the same experimental setup, and the accuracy degradation is very small.
Tasks
Published	2018-03-14
URL	http://arxiv.org/abs/1803.06305v1
PDF	http://arxiv.org/pdf/1803.06305v1.pdf
PWC	https://paperswithcode.com/paper/c-lstm-enabling-efficient-lstm-using
Repo
Framework

Fast and Accurate Reconstruction of Compressed Color Light Field


Title	Fast and Accurate Reconstruction of Compressed Color Light Field
Authors	Ofir Nabati, David Mendlovic, Raja Giryes
Abstract	Light field photography has been studied thoroughly in recent years. One of its drawbacks is the need for multi-lens in the imaging. To compensate that, compressed light field photography has been proposed to tackle the trade-offs between the spatial and angular resolutions. It obtains by only one lens, a compressed version of the regular multi-lens system. The acquisition system consists of a dedicated hardware followed by a decompression algorithm, which usually suffers from high computational time. In this work, we propose a computationally efficient neural network that recovers a high-quality color light field from a single coded image. Unlike previous works, we compress the color channels as well, removing the need for a CFA in the imaging system. Our approach outperforms existing solutions in terms of recovery quality and computational complexity. We propose also a neural network for depth map extraction based on the decompressed light field, which is trained in an unsupervised manner without the ground truth depth map.
Tasks
Published	2018-01-31
URL	http://arxiv.org/abs/1801.10351v2
PDF	http://arxiv.org/pdf/1801.10351v2.pdf
PWC	https://paperswithcode.com/paper/fast-and-accurate-reconstruction-of
Repo
Framework

Online Regularized Nonlinear Acceleration


Title	Online Regularized Nonlinear Acceleration
Authors	Damien Scieur, Edouard Oyallon, Alexandre d’Aspremont, Francis Bach
Abstract	Regularized nonlinear acceleration (RNA) estimates the minimum of a function by post-processing iterates from an algorithm such as the gradient method. It can be seen as a regularized version of Anderson acceleration, a classical acceleration scheme from numerical analysis. The new scheme provably improves the rate of convergence of fixed step gradient descent, and its empirical performance is comparable to that of quasi-Newton methods. However, RNA cannot accelerate faster multistep algorithms like Nesterov’s method and often diverges in this context. Here, we adapt RNA to overcome these issues, so that our scheme can be used on fast algorithms such as gradient methods with momentum. We show optimal complexity bounds for quadratics and asymptotically optimal rates on general convex minimization problems. Moreover, this new scheme works online, i.e., extrapolated solution estimates can be reinjected at each iteration, significantly improving numerical performance over classical accelerated methods.
Tasks
Published	2018-05-24
URL	https://arxiv.org/abs/1805.09639v2
PDF	https://arxiv.org/pdf/1805.09639v2.pdf
PWC	https://paperswithcode.com/paper/nonlinear-acceleration-of-deep-neural
Repo
Framework

MoDL-MUSSELS: Model-Based Deep Learning for Multi-Shot Sensitivity Encoded Diffusion MRI


Title	MoDL-MUSSELS: Model-Based Deep Learning for Multi-Shot Sensitivity Encoded Diffusion MRI
Authors	Hemant Kumar Aggarwal, Merry P. Mani, Mathews Jacob
Abstract	We introduce a model-based deep learning architecture termed MoDL-MUSSELS for the correction of phase errors in multishot diffusion-weighted echo-planar MRI images. The proposed algorithm is a generalization of existing MUSSELS algorithm with similar performance but with significantly reduced computational complexity. In this work, we show that an iterative re-weighted least-squares implementation of MUSSELS alternates between a multichannel filter bank and the enforcement of data consistency. The multichannel filter bank projects the data to the signal subspace thus exploiting the phase relations between shots. Due to the high computational complexity of self-learned filter bank, we propose to replace it with a convolutional neural network (CNN) whose parameters are learned from exemplary data. The proposed CNN is a hybrid model involving a multichannel CNN in the k-space and another CNN in the image space. The k-space CNN exploits the phase relations between the shot images, while the image domain network is used to project the data to an image manifold. The experiments show that the proposed scheme can yield reconstructions that are comparable to state of the art methods while offering several orders of magnitude reduction in run-time.
Tasks
Published	2018-12-19
URL	https://arxiv.org/abs/1812.08115v3
PDF	https://arxiv.org/pdf/1812.08115v3.pdf
PWC	https://paperswithcode.com/paper/multi-shot-sensitivity-encoded-diffusion-mri
Repo
Framework

Towards Explainable NLP: A Generative Explanation Framework for Text Classification


Title	Towards Explainable NLP: A Generative Explanation Framework for Text Classification
Authors	Hui Liu, Qingyu Yin, William Yang Wang
Abstract	Building explainable systems is a critical problem in the field of Natural Language Processing (NLP), since most machine learning models provide no explanations for the predictions. Existing approaches for explainable machine learning systems tend to focus on interpreting the outputs or the connections between inputs and outputs. However, the fine-grained information is often ignored, and the systems do not explicitly generate the human-readable explanations. To better alleviate this problem, we propose a novel generative explanation framework that learns to make classification decisions and generate fine-grained explanations at the same time. More specifically, we introduce the explainable factor and the minimum risk training approach that learn to generate more reasonable explanations. We construct two new datasets that contain summaries, rating scores, and fine-grained reasons. We conduct experiments on both datasets, comparing with several strong neural network baseline systems. Experimental results show that our method surpasses all baselines on both datasets, and is able to generate concise explanations at the same time.
Tasks	Text Classification
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00196v2
PDF	https://arxiv.org/pdf/1811.00196v2.pdf
PWC	https://paperswithcode.com/paper/towards-explainable-nlp-a-generative
Repo
Framework

A Simple Machine Learning Method for Commonsense Reasoning? A Short Commentary on Trinh & Le (2018)


Title	A Simple Machine Learning Method for Commonsense Reasoning? A Short Commentary on Trinh & Le (2018)
Authors	Walid S. Saba
Abstract	This is a short Commentary on Trinh & Le (2018) (“A Simple Method for Commonsense Reasoning”) that outlines three serious flaws in the cited paper and discusses why data-driven approaches cannot be considered as serious models for the commonsense reasoning needed in natural language understanding in general, and in reference resolution, in particular.
Tasks
Published	2018-10-01
URL	http://arxiv.org/abs/1810.00521v1
PDF	http://arxiv.org/pdf/1810.00521v1.pdf
PWC	https://paperswithcode.com/paper/a-simple-machine-learning-method-for
Repo
Framework

Learning Hidden Markov Models from Pairwise Co-occurrences with Application to Topic Modeling


Title	Learning Hidden Markov Models from Pairwise Co-occurrences with Application to Topic Modeling
Authors	Kejun Huang, Xiao Fu, Nicholas D. Sidiropoulos
Abstract	We present a new algorithm for identifying the transition and emission probabilities of a hidden Markov model (HMM) from the emitted data. Expectation-maximization becomes computationally prohibitive for long observation records, which are often required for identification. The new algorithm is particularly suitable for cases where the available sample size is large enough to accurately estimate second-order output probabilities, but not higher-order ones. We show that if one is only able to obtain a reliable estimate of the pairwise co-occurrence probabilities of the emissions, it is still possible to uniquely identify the HMM if the emission probability is \emph{sufficiently scattered}. We apply our method to hidden topic Markov modeling, and demonstrate that we can learn topics with higher quality if documents are modeled as observations of HMMs sharing the same emission (topic) probability, compared to the simple but widely used bag-of-words model.
Tasks
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06894v2
PDF	http://arxiv.org/pdf/1802.06894v2.pdf
PWC	https://paperswithcode.com/paper/learning-hidden-markov-models-from-pairwise
Repo
Framework

Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks


Title	Robot Localisation and 3D Position Estimation Using a Free-Moving Camera and Cascaded Convolutional Neural Networks
Authors	Justinas Miseikis, Patrick Knobelreiter, Inka Brijacak, Saeed Yahyanejad, Kyrre Glette, Ole Jakob Elle, Jim Torresen
Abstract	Many works in collaborative robotics and human-robot interaction focuses on identifying and predicting human behaviour while considering the information about the robot itself as given. This can be the case when sensors and the robot are calibrated in relation to each other and often the reconfiguration of the system is not possible, or extra manual work is required. We present a deep learning based approach to remove the constraint of having the need for the robot and the vision sensor to be fixed and calibrated in relation to each other. The system learns the visual cues of the robot body and is able to localise it, as well as estimate the position of robot joints in 3D space by just using a 2D color image. The method uses a cascaded convolutional neural network, and we present the structure of the network, describe our own collected dataset, explain the network training and achieved results. A fully trained system shows promising results in providing an accurate mask of where the robot is located and a good estimate of its joints positions in 3D. The accuracy is not good enough for visual servoing applications yet, however, it can be sufficient for general safety and some collaborative tasks not requiring very high precision. The main benefit of our method is the possibility of the vision sensor to move freely. This allows it to be mounted on moving objects, for example, a body of the person or a mobile robot working in the same environment as the robots are operating in.
Tasks
Published	2018-01-06
URL	http://arxiv.org/abs/1801.02025v2
PDF	http://arxiv.org/pdf/1801.02025v2.pdf
PWC	https://paperswithcode.com/paper/robot-localisation-and-3d-position-estimation
Repo
Framework

DeepProbLog: Neural Probabilistic Logic Programming


Title	DeepProbLog: Neural Probabilistic Logic Programming
Authors	Robin Manhaeve, Sebastijan Dumančić, Angelika Kimmig, Thomas Demeester, Luc De Raedt
Abstract	We introduce DeepProbLog, a probabilistic logic programming language that incorporates deep learning by means of neural predicates. We show how existing inference and learning techniques can be adapted for the new language. Our experiments demonstrate that DeepProbLog supports both symbolic and subsymbolic representations and inference, 1) program induction, 2) probabilistic (logic) programming, and 3) (deep) learning from examples. To the best of our knowledge, this work is the first to propose a framework where general-purpose neural networks and expressive probabilistic-logical modeling and reasoning are integrated in a way that exploits the full expressiveness and strengths of both worlds and can be trained end-to-end based on examples.
Tasks
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10872v2
PDF	http://arxiv.org/pdf/1805.10872v2.pdf
PWC	https://paperswithcode.com/paper/deepproblog-neural-probabilistic-logic
Repo
Framework

Balancing Two-Player Stochastic Games with Soft Q-Learning


Title	Balancing Two-Player Stochastic Games with Soft Q-Learning
Authors	Jordi Grau-Moya, Felix Leibfried, Haitham Bou-Ammar
Abstract	Within the context of video games the notion of perfectly rational agents can be undesirable as it leads to uninteresting situations, where humans face tough adversarial decision makers. Current frameworks for stochastic games and reinforcement learning prohibit tuneable strategies as they seek optimal performance. In this paper, we enable such tuneable behaviour by generalising soft Q-learning to stochastic games, where more than one agent interact strategically. We contribute both theoretically and empirically. On the theory side, we show that games with soft Q-learning exhibit a unique value and generalise team games and zero-sum games far beyond these two extremes to cover a continuous spectrum of gaming behaviour. Experimentally, we show how tuning agents’ constraints affect performance and demonstrate, through a neural network architecture, how to reliably balance games with high-dimensional representations.
Tasks	Q-Learning
Published	2018-02-09
URL	http://arxiv.org/abs/1802.03216v2
PDF	http://arxiv.org/pdf/1802.03216v2.pdf
PWC	https://paperswithcode.com/paper/balancing-two-player-stochastic-games-with
Repo
Framework

A Complementary Tracking Model with Multiple Features


Title	A Complementary Tracking Model with Multiple Features
Authors	Peng Gao, Yipeng Ma, Chao Li, Ke Song, Fei Wang, Liyi Xiao
Abstract	Discriminative Correlation Filters based tracking algorithms exploiting conventional handcrafted features have achieved impressive results both in terms of accuracy and robustness. Template handcrafted features have shown excellent performance, but they perform poorly when the appearance of target changes rapidly such as fast motions and fast deformations. In contrast, statistical handcrafted features are insensitive to fast states changes, but they yield inferior performance in the scenarios of illumination variations and background clutters. In this work, to achieve an efficient tracking performance, we propose a novel visual tracking algorithm, named MFCMT, based on a complementary ensemble model with multiple features, including Histogram of Oriented Gradients (HOGs), Color Names (CNs) and Color Histograms (CHs). Additionally, to improve tracking results and prevent targets drift, we introduce an effective fusion method by exploiting relative entropy to coalesce all basic response maps and get an optimal response. Furthermore, we suggest a simple but efficient update strategy to boost tracking performance. Comprehensive evaluations are conducted on two tracking benchmarks demonstrate and the experimental results demonstrate that our method is competitive with numerous state-of-the-art trackers. Our tracker achieves impressive performance with faster speed on these benchmarks.
Tasks	Visual Tracking
Published	2018-04-20
URL	http://arxiv.org/abs/1804.07459v3
PDF	http://arxiv.org/pdf/1804.07459v3.pdf
PWC	https://paperswithcode.com/paper/a-complementary-tracking-model-with-multiple
Repo
Framework

A Parallel MOEA with Criterion-based Selection Applied to the Knapsack Problem


Title	A Parallel MOEA with Criterion-based Selection Applied to the Knapsack Problem
Authors	Kantour Nedjmeddine, Bouroubi Sadek, Chaabane Djamel
Abstract	In this paper, we propose a parallel multiobjective evolutionary algorithm called Parallel Criterion-based Partitioning MOEA (PCPMOEA), with an application to the Mutliobjective Knapsack Problem (MOKP). The suggested search strategy is based on a periodic partitioning of potentially efficient solutions, which are distributed to multiple multiobjective evolutionary algorithms (MOEAs). Each MOEA is dedicated to a sole objective, in which it combines both criterion-based and dominance-based approaches. The suggested algorithm addresses two main sub-objectives: minimizing the distance between the current non-dominated solutions and the ideal point, and ensuring the spread of the potentially efficient solutions. Experimental results are included, where we assess the performance of the suggested algorithm against the above mentioned sub-objectives, compared with state-of-the-art results using well-known multi-objective metaheuristics.
Tasks
Published	2018-11-06
URL	http://arxiv.org/abs/1811.02271v1
PDF	http://arxiv.org/pdf/1811.02271v1.pdf
PWC	https://paperswithcode.com/paper/a-parallel-moea-with-criterion-based
Repo
Framework

Model-Free Information Extraction in Enriched Nonlinear Phase-Space


Title	Model-Free Information Extraction in Enriched Nonlinear Phase-Space
Authors	Bin Li, Yueheng Lan, Weisi Guo, Chenglin Zhao
Abstract	Detecting anomalies and discovering driving signals is an essential component of scientific research and industrial practice. Often the underlying mechanism is highly complex, involving hidden evolving nonlinear dynamics and noise contamination. When representative physical models and large labeled data sets are unavailable, as is the case with most real-world applications, model-dependent Bayesian approaches would yield misleading results, and most supervised learning machines would also fail to reliably resolve the intricately evolving systems. Here, we propose an unsupervised machine-learning approach that operates in a well-constructed function space, whereby the evolving nonlinear dynamics are captured through a linear functional representation determined by the Koopman operator. This breakthrough leverages on the time-feature embedding and the ensuing reconstruction of a phase-space representation of the dynamics, thereby permitting the reliable identification of critical global signatures from the whole trajectory. This dramatically improves over commonly used static local features, which are vulnerable to unknown transitions or noise. Thanks to its data-driven nature, our method excludes any prior models and training corpus. We benchmark the astonishing accuracy of our method on three diverse and challenging problems in: biology, medicine, and engineering. In all cases, it outperforms existing state-of-the-art methods. As a new unsupervised information processing paradigm, it is suitable for ubiquitous nonlinear dynamical systems or end-users with little expertise, which permits an unbiased excavation of underlying working principles or intrinsic correlations submerged in unlabeled data flows.
Tasks
Published	2018-04-14
URL	http://arxiv.org/abs/1804.05170v2
PDF	http://arxiv.org/pdf/1804.05170v2.pdf
PWC	https://paperswithcode.com/paper/model-free-information-extraction-in-enriched
Repo
Framework

Gradient descent in some simple settings


Title	Gradient descent in some simple settings
Authors	Y. Cooper
Abstract	In this note, we observe the behavior of gradient flow and discrete and noisy gradient descent in some simple settings. It is commonly noted that addition of noise to gradient descent can affect the trajectory of gradient descent. Here, we run some computer experiments for gradient descent on some simple functions, and observe this principle in some concrete examples.
Tasks
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04839v2
PDF	http://arxiv.org/pdf/1808.04839v2.pdf
PWC	https://paperswithcode.com/paper/discrete-gradient-descent-differs
Repo
Framework