Paper Group ANR 1625
Large scale representation learning from triplet comparisons. High Sensitivity Snapshot Spectrometer Based on Deep Network Unmixing. Trading via Image Classification. Clone Swarms: Learning to Predict and Control Multi-Robot Systems by Imitation. Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularization. Mul …
Large scale representation learning from triplet comparisons
| Title | Large scale representation learning from triplet comparisons |
| Authors | Siavash Haghiri, Leena Chennuru Vankadara, Ulrike von Luxburg |
| Abstract | In this paper, we discuss the fundamental problem of representation learning from a new perspective. It has been observed in many supervised/unsupervised DNNs that the final layer of the network often provides an informative representation for many tasks, even though the network has been trained to perform a particular task. The common ingredient in all previous studies is a low-level feature representation for items, for example, RGB values of images in the image context. In the present work, we assume that no meaningful representation of the items is given. Instead, we are provided with the answers to some triplet comparisons of the following form: Is item A more similar to item B or item C? We provide a fast algorithm based on DNNs that constructs a Euclidean representation for the items, using solely the answers to the above-mentioned triplet comparisons. This problem has been studied in a sub-community of machine learning by the name “Ordinal Embedding”. Previous approaches to the problem are painfully slow and cannot scale to larger datasets. We demonstrate that our proposed approach is significantly faster than available methods, and can scale to real-world large datasets. Thereby, we also draw attention to the less explored idea of using neural networks to directly, approximately solve non-convex, NP-hard optimization problems that arise naturally in unsupervised learning problems. |
| Tasks | Representation Learning |
| Published | 2019-12-03 |
| URL | https://arxiv.org/abs/1912.01666v1 |
| https://arxiv.org/pdf/1912.01666v1.pdf | |
| PWC | https://paperswithcode.com/paper/large-scale-representation-learning-from-3 |
| Repo | |
| Framework | |
High Sensitivity Snapshot Spectrometer Based on Deep Network Unmixing
| Title | High Sensitivity Snapshot Spectrometer Based on Deep Network Unmixing |
| Authors | XiaoYu Chen, Xu Wang, Lianfa Bai, Jing Han, Zhuang Zhao |
| Abstract | In this paper, we present a convolution neural network based method to recover the light intensity distribution from the overlapped dispersive spectra instead of adding an extra light path to capture it directly for the first time. Then, we construct a single-path sub-Hadamard snapshot spectrometer based on our previous dual-path snapshot spectrometer. In the proposed single-path spectrometer, we use the reconstructed light intensity as the original light intensity and recover high signal-to-noise ratio spectra successfully. Compared with dual-path snapshot spectrometer, the network based single-path spectrometer has a more compact structure and maintains snapshot and high sensitivity. Abundant simulated and experimental results have demonstrated that the proposed method can obtain a better reconstructed signal-to-noise ratio spectrum than the dual-path sub-Hadamard spectrometer because of its higher light throughput. |
| Tasks | |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.00209v1 |
| https://arxiv.org/pdf/1907.00209v1.pdf | |
| PWC | https://paperswithcode.com/paper/high-sensitivity-snapshot-spectrometer-based |
| Repo | |
| Framework | |
Trading via Image Classification
| Title | Trading via Image Classification |
| Authors | Naftali Cohen, Tucker Balch, Manuela Veloso |
| Abstract | The art of systematic financial trading evolved with an array of approaches, ranging from simple strategies to complex algorithms all relying, primary, on aspects of time-series analysis. Recently, after visiting the trading floor of a leading financial institution, we noticed that traders always execute their trade orders while observing images of financial time-series on their screens. In this work, we built upon the success in image recognition and examine the value in transforming the traditional time-series analysis to that of image classification. We create a large sample of financial time-series images encoded as candlestick (Box and Whisker) charts and label the samples following three algebraically-defined binary trade strategies. Using the images, we train over a dozen machine-learning classification models and find that the algorithms are very efficient in recovering the complicated, multiscale label-generating rules when the data is represented visually. We suggest that the transformation of continuous numeric time-series classification problem to a vision problem is useful for recovering signals typical of technical analysis. |
| Tasks | Image Classification, Time Series, Time Series Analysis, Time Series Classification |
| Published | 2019-07-23 |
| URL | https://arxiv.org/abs/1907.10046v2 |
| https://arxiv.org/pdf/1907.10046v2.pdf | |
| PWC | https://paperswithcode.com/paper/trading-via-image-classification |
| Repo | |
| Framework | |
Clone Swarms: Learning to Predict and Control Multi-Robot Systems by Imitation
| Title | Clone Swarms: Learning to Predict and Control Multi-Robot Systems by Imitation |
| Authors | Siyu Zhou, Mariano J. Phielipp, Jorge A. Sefair, Sara I. Walker, Heni Ben Amor |
| Abstract | In this paper, we propose SwarmNet – a neural network architecture that can learn to predict and imitate the behavior of an observed swarm of agents in a centralized manner. Tested on artificially generated swarm motion data, the network achieves high levels of prediction accuracy and imitation authenticity. We compare our model to previous approaches for modelling interaction systems and show how modifying components of other models gradually approaches the performance of ours. Finally, we also discuss an extension of SwarmNet that can deal with nondeterministic, noisy, and uncertain environments, as often found in robotics applications. |
| Tasks | |
| Published | 2019-12-05 |
| URL | https://arxiv.org/abs/1912.02811v1 |
| https://arxiv.org/pdf/1912.02811v1.pdf | |
| PWC | https://paperswithcode.com/paper/clone-swarms-learning-to-predict-and-control |
| Repo | |
| Framework | |
Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularization
| Title | Learning Interpretable Shapelets for Time Series Classification through Adversarial Regularization |
| Authors | Yichang Wang, Rémi Emonet, Elisa Fromont, Simon Malinowski, Etienne Menager, Loïc Mosser, Romain Tavenard |
| Abstract | Times series classification can be successfully tackled by jointly learning a shapelet-based representation of the series in the dataset and classifying the series according to this representation. However, although the learned shapelets are discriminative, they are not always similar to pieces of a real series in the dataset. This makes it difficult to interpret the decision, i.e. difficult to analyze if there are particular behaviors in a series that triggered the decision. In this paper, we make use of a simple convolutional network to tackle the time series classification task and we introduce an adversarial regularization to constrain the model to learn more interpretable shapelets. Our classification results on all the usual time series benchmarks are comparable with the results obtained by similar state-of-the-art algorithms but our adversarially regularized method learns shapelets that are, by design, interpretable. |
| Tasks | Time Series, Time Series Classification |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.00917v2 |
| https://arxiv.org/pdf/1906.00917v2.pdf | |
| PWC | https://paperswithcode.com/paper/190600917 |
| Repo | |
| Framework | |
Multi-scale Embedded CNN for Music Tagging (MsE-CNN)
| Title | Multi-scale Embedded CNN for Music Tagging (MsE-CNN) |
| Authors | Nima Hamidi, Mohsen Vahidzadeh, Stephen Baek |
| Abstract | Convolutional neural networks (CNN) recently gained notable attraction in a variety of machine learning tasks: including music classification and style tagging. In this work, we propose implementing intermediate connections to the CNN architecture to facilitate the transfer of multi-scale/level knowledge between different layers. Our novel model for music tagging shows significant improvement in comparison to the proposed approaches in the literature, due to its ability to carry low-level timbral features to the last layer. |
| Tasks | Music Classification |
| Published | 2019-06-16 |
| URL | https://arxiv.org/abs/1906.06746v1 |
| https://arxiv.org/pdf/1906.06746v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-scale-embedded-cnn-for-music-tagging |
| Repo | |
| Framework | |
RTN: Reparameterized Ternary Network
| Title | RTN: Reparameterized Ternary Network |
| Authors | Yuhang Li, Xin Dong, Sai Qian Zhang, Haoli Bai, Yuanpeng Chen, Wei Wang |
| Abstract | To deploy deep neural networks on resource-limited devices, quantization has been widely explored. In this work, we study the extremely low-bit networks which have tremendous speed-up, memory saving with quantized activation and weights. We first bring up three omitted issues in extremely low-bit networks: the squashing range of quantized values; the gradient vanishing during backpropagation and the unexploited hardware acceleration of ternary networks. By reparameterizing quantized activation and weights vector with full precision scale and offset for fixed ternary vector, we decouple the range and magnitude from the direction to extenuate the three issues. Learnable scale and offset can automatically adjust the range of quantized values and sparsity without gradient vanishing. A novel encoding and computation pat-tern are designed to support efficient computing for our reparameterized ternary network (RTN). Experiments on ResNet-18 for ImageNet demonstrate that the proposed RTN finds a much better efficiency between bitwidth and accuracy, and achieves up to 26.76% relative accuracy improvement compared with state-of-the-art methods. Moreover, we validate the proposed computation pattern on Field Programmable Gate Arrays (FPGA), and it brings 46.46x and 89.17x savings on power and area respectively compared with the full precision convolution. |
| Tasks | Quantization |
| Published | 2019-12-04 |
| URL | https://arxiv.org/abs/1912.02057v2 |
| https://arxiv.org/pdf/1912.02057v2.pdf | |
| PWC | https://paperswithcode.com/paper/rtn-reparameterized-ternary-network |
| Repo | |
| Framework | |
3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting
| Title | 3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting |
| Authors | Bing Yu, Mengzhang Li, Jiyong Zhang, Zhanxing Zhu |
| Abstract | Spatio-temporal prediction plays an important role in many application areas especially in traffic domain. However, due to complicated spatio-temporal dependency and high non-linear dynamics in road networks, traffic prediction task is still challenging. Existing works either exhibit heavy training cost or fail to accurately capture the spatio-temporal patterns, also ignore the correlation between distant roads that share the similar patterns. In this paper, we propose a novel deep learning framework to overcome these issues: 3D Temporal Graph Convolutional Networks (3D-TGCN). Two novel components of our model are introduced. (1) Instead of constructing the road graph based on spatial information, we learn it by comparing the similarity between time series for each road, thus providing a spatial information free framework. (2) We propose an original 3D graph convolution model to model the spatio-temporal data more accurately. Empirical results show that 3D-TGCN could outperform state-of-the-art baselines. |
| Tasks | Time Series, Traffic Prediction |
| Published | 2019-03-03 |
| URL | http://arxiv.org/abs/1903.00919v1 |
| http://arxiv.org/pdf/1903.00919v1.pdf | |
| PWC | https://paperswithcode.com/paper/3d-graph-convolutional-networks-with-temporal |
| Repo | |
| Framework | |
Collaborative Filtering with Label Consistent Restricted Boltzmann Machine
| Title | Collaborative Filtering with Label Consistent Restricted Boltzmann Machine |
| Authors | Sagar Verma, Prince Patel, Angshul Majumdar |
| Abstract | The possibility of employing restricted Boltzmann machine (RBM) for collaborative filtering has been known for about a decade. However, there has been hardly any work on this topic since 2007. This work revisits the application of RBM in recommender systems. RBM based collaborative filtering only used the rating information; this is an unsupervised architecture. This work adds supervision by exploiting user demographic information and item metadata. A network is learned from the representation layer to the labels (metadata). The proposed label consistent RBM formulation improves significantly on the existing RBM based approach and yield results at par with the state-of-the-art latent factor based models. |
| Tasks | Recommendation Systems |
| Published | 2019-10-17 |
| URL | https://arxiv.org/abs/1910.07724v1 |
| https://arxiv.org/pdf/1910.07724v1.pdf | |
| PWC | https://paperswithcode.com/paper/collaborative-filtering-with-label-consistent |
| Repo | |
| Framework | |
Weighted Tensor Completion for Time-Series Causal Inference
| Title | Weighted Tensor Completion for Time-Series Causal Inference |
| Authors | Debmalya Mandal, David Parkes |
| Abstract | Marginal Structural Models (MSM) {Robins, 2000} are the most popular models for causal inference from time-series observational data. However, they have two main drawbacks: (a) they do not capture subject heterogeneity, and (b) they only consider fixed time intervals and do not scale gracefully with longer intervals. In this work, we propose a new family of MSMs to address these two concerns. We model the potential outcomes as a three-dimensional tensor of low rank, where the three dimensions correspond to the agents, time periods and the set of possible histories. Unlike the traditional MSM, we allow the dimensions of the tensor to increase with the number of agents and time periods. We set up a weighted tensor completion problem as our estimation procedure, and show that the solution to this problem converges to the true model in an appropriate sense. Then we show how to solve the estimation problem, providing conditions under which we can approximately and efficiently solve the estimation problem. Finally, we propose an algorithm based on projected gradient descent, which is easy to implement and evaluate its performance on a simulated dataset. |
| Tasks | Causal Inference, Time Series |
| Published | 2019-02-12 |
| URL | http://arxiv.org/abs/1902.04646v2 |
| http://arxiv.org/pdf/1902.04646v2.pdf | |
| PWC | https://paperswithcode.com/paper/weighted-tensor-completion-for-time-series |
| Repo | |
| Framework | |
Parallel and Communication Avoiding Least Angle Regression
| Title | Parallel and Communication Avoiding Least Angle Regression |
| Authors | K. Fountoulakis, S. Das, L. Grigori, M. W. Mahoney, J. Demmel |
| Abstract | We are interested in parallelizing the Least Angle Regression (LARS) algorithm for fitting linear regression models to high-dimensional data. We consider two parallel and communication avoiding versions of the basic LARS algorithm. The two algorithms apply to data that have different layout patterns (one is appropriate for row-partitioned data, and the other is appropriate for column-partitioned data), and they have different asymptotic costs and practical performance. The first is bLARS, a block version of LARS algorithm, where we update b columns at each iteration. Assuming that the data are row-partitioned, bLARS reduces the number of arithmetic operations, latency, and bandwidth by a factor of b. The second is Tournament-bLARS (T-bLARS), a tournament version of LARS, in which case processors compete, by running several LARS computations in parallel, to choose b new columns to be added into the solution. Assuming that the data are column-partitioned, T-bLARS reduces latency by a factor of b. Similarly to LARS, our proposed methods generate a sequence of linear models. We present extensive numerical experiments that illustrate speed-ups up to 25x compared to LARS. |
| Tasks | |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1905.11340v2 |
| https://arxiv.org/pdf/1905.11340v2.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-and-communication-avoiding-least |
| Repo | |
| Framework | |
Interpretable Conservation Law Estimation by Deriving the Symmetries of Dynamics from Trained Deep Neural Networks
| Title | Interpretable Conservation Law Estimation by Deriving the Symmetries of Dynamics from Trained Deep Neural Networks |
| Authors | Yoh-ichi Mototake |
| Abstract | As deep neural networks (DNN) have the ability to model the distribution of datasets as a low-dimensional manifold, we propose a method to extract the coordinate transformation that makes a dataset distribution invariant by sampling DNNs using the replica exchange Monte-Carlo method. In addition, we derive the relation between the canonical transformation that makes the Hamiltonian invariant (a necessary condition for Noether’s theorem) and the symmetry of the manifold structure of the time series data of the dynamical system. By integrating this knowledge with the method described above, we propose a method to estimate the interpretable conservation laws from the time-series data. Furthermore, we verified the efficiency of the proposed methods in primitive cases and large scale collective motion in metastable state. |
| Tasks | Time Series |
| Published | 2019-12-31 |
| URL | https://arxiv.org/abs/2001.00111v1 |
| https://arxiv.org/pdf/2001.00111v1.pdf | |
| PWC | https://paperswithcode.com/paper/interpretable-conservation-law-estimation-by |
| Repo | |
| Framework | |
Computational Phase Transition Signature in Gibbs Sampling
| Title | Computational Phase Transition Signature in Gibbs Sampling |
| Authors | H. Philathong, V. Akshay, I. Zacharov, J. Biamonte |
| Abstract | Gibbs sampling is fundamental to a wide range of computer algorithms. Such algorithms are set to be replaced by physics based processors$-$be it quantum or stochastic annealing devices$-$which embed problem instances and evolve a physical system into an ensemble to recover a probability distribution. At a critical constraint to variable ratio, decision problems$-$such as propositional satisfiability$-$appear to statistically exhibit an abrupt transition in required computational resources. This so called, algorithmic or computational phase transition signature, has yet-to-be observed in contemporary physics based processors. We found that the computational phase transition admits a signature in Gibbs’ distributions and hence we predict and prescribe the physical observation of this effect. We simulate such an experiment, that when realized experimentally, we believe would represent a milestone in the physical theory of computation. |
| Tasks | |
| Published | 2019-06-25 |
| URL | https://arxiv.org/abs/1906.10705v1 |
| https://arxiv.org/pdf/1906.10705v1.pdf | |
| PWC | https://paperswithcode.com/paper/computational-phase-transition-signature-in |
| Repo | |
| Framework | |
End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis
| Title | End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis |
| Authors | Lin Xu, Qixian Zhou, Ke Gong, Xiaodan Liang, Jianheng Tang, Liang Lin |
| Abstract | Beyond current conversational chatbots or task-oriented dialogue systems that have attracted increasing attention, we move forward to develop a dialogue system for automatic medical diagnosis that converses with patients to collect additional symptoms beyond their self-reports and automatically makes a diagnosis. Besides the challenges for conversational dialogue systems (e.g. topic transition coherency and question understanding), automatic medical diagnosis further poses more critical requirements for the dialogue rationality in the context of medical knowledge and symptom-disease relations. Existing dialogue systems (Madotto, Wu, and Fung 2018; Wei et al. 2018; Li et al. 2017) mostly rely on data-driven learning and cannot be able to encode extra expert knowledge graph. In this work, we propose an End-to-End Knowledge-routed Relational Dialogue System (KR-DS) that seamlessly incorporates rich medical knowledge graph into the topic transition in dialogue management, and makes it cooperative with natural language understanding and natural language generation. A novel Knowledge-routed Deep Q-network (KR-DQN) is introduced to manage topic transitions, which integrates a relational refinement branch for encoding relations among different symptoms and symptom-disease pairs, and a knowledge-routed graph branch for topic decision-making. Extensive experiments on a public medical dialogue dataset show our KR-DS significantly beats state-of-the-art methods (by more than 8% in diagnosis accuracy). We further show the superiority of our KR-DS on a newly collected medical dialogue system dataset, which is more challenging retaining original self-reports and conversational data between patients and doctors. |
| Tasks | Decision Making, Dialogue Management, Medical Diagnosis, Task-Oriented Dialogue Systems, Text Generation |
| Published | 2019-01-30 |
| URL | http://arxiv.org/abs/1901.10623v2 |
| http://arxiv.org/pdf/1901.10623v2.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-knowledge-routed-relational |
| Repo | |
| Framework | |
Online Learning Algorithms for Quaternion ARMA Model
| Title | Online Learning Algorithms for Quaternion ARMA Model |
| Authors | Xiaokun Pu, Chunguang Li |
| Abstract | In this paper, we address the problem of adaptive learning for autoregressive moving average (ARMA) model in the quaternion domain. By transforming the original learning problem into a full information optimization task without explicit noise terms, and then solving the optimization problem using the gradient descent and the Newton analogues, we obtain two online learning algorithms for the quaternion ARMA. Furthermore, regret bound analysis accounting for the specific properties of quaternion algebra is presented, which proves that the performance of the online algorithms asymptotically approaches that of the best quaternion ARMA model in hindsight. |
| Tasks | |
| Published | 2019-04-26 |
| URL | http://arxiv.org/abs/1904.11830v1 |
| http://arxiv.org/pdf/1904.11830v1.pdf | |
| PWC | https://paperswithcode.com/paper/online-learning-algorithms-for-quaternion |
| Repo | |
| Framework | |
