Paper Group ANR 422
Learning Sparse Wavelet Representations. Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation. A Fast Hierarchically Preconditioned Eigensolver Based On Multiresolution Matrix Decomposition. Tetris: Re-architecting Convolutional Neural Network Computation for Machine Learning Accelerators. Automatic Classifiers as Scientific Ins …
Learning Sparse Wavelet Representations
| Title | Learning Sparse Wavelet Representations |
| Authors | Daniel Recoskie, Richard Mann |
| Abstract | In this work we propose a method for learning wavelet filters directly from data. We accomplish this by framing the discrete wavelet transform as a modified convolutional neural network. We introduce an autoencoder wavelet transform network that is trained using gradient descent. We show that the model is capable of learning structured wavelet filters from synthetic and real data. The learned wavelets are shown to be similar to traditional wavelets that are derived using Fourier methods. Our method is simple to implement and easily incorporated into neural network architectures. A major advantage to our model is that we can learn from raw audio data. |
| Tasks | |
| Published | 2018-02-08 |
| URL | http://arxiv.org/abs/1802.02961v1 |
| http://arxiv.org/pdf/1802.02961v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-sparse-wavelet-representations |
| Repo | |
| Framework | |
Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation
| Title | Augmented Cyclic Adversarial Learning for Low Resource Domain Adaptation |
| Authors | Ehsan Hosseini-Asl, Yingbo Zhou, Caiming Xiong, Richard Socher |
| Abstract | Training a model to perform a task typically requires a large amount of data from the domains in which the task will be applied. However, it is often the case that data are abundant in some domains but scarce in others. Domain adaptation deals with the challenge of adapting a model trained from a data-rich source domain to perform well in a data-poor target domain. In general, this requires learning plausible mappings between domains. CycleGAN is a powerful framework that efficiently learns to map inputs from one domain to another using adversarial training and a cycle-consistency constraint. However, the conventional approach of enforcing cycle-consistency via reconstruction may be overly restrictive in cases where one or more domains have limited training data. In this paper, we propose an augmented cyclic adversarial learning model that enforces the cycle-consistency constraint via an external task specific model, which encourages the preservation of task-relevant content as opposed to exact reconstruction. We explore digit classification in a low-resource setting in supervised, semi and unsupervised situation, as well as high resource unsupervised. In low-resource supervised setting, the results show that our approach improves absolute performance by 14% and 4% when adapting SVHN to MNIST and vice versa, respectively, which outperforms unsupervised domain adaptation methods that require high-resource unlabeled target domain. Moreover, using only few unsupervised target data, our approach can still outperforms many high-resource unsupervised models. In speech domains, we similarly adopt a speech recognition model from each domain as the task specific model. Our approach improves absolute performance of speech recognition by 2% for female speakers in the TIMIT dataset, where the majority of training samples are from male voices. |
| Tasks | Domain Adaptation, Speech Recognition, Unsupervised Domain Adaptation |
| Published | 2018-07-01 |
| URL | http://arxiv.org/abs/1807.00374v4 |
| http://arxiv.org/pdf/1807.00374v4.pdf | |
| PWC | https://paperswithcode.com/paper/augmented-cyclic-adversarial-learning-for-low |
| Repo | |
| Framework | |
A Fast Hierarchically Preconditioned Eigensolver Based On Multiresolution Matrix Decomposition
| Title | A Fast Hierarchically Preconditioned Eigensolver Based On Multiresolution Matrix Decomposition |
| Authors | Thomas Y. Hou, De Huang, Ka Chun Lam, Ziyun Zhang |
| Abstract | In this paper we propose a new iterative method to hierarchically compute a relatively large number of leftmost eigenpairs of a sparse symmetric positive matrix under the multiresolution operator compression framework. We exploit the well-conditioned property of every decomposition components by integrating the multiresolution framework into the Implicitly restarted Lanczos method. We achieve this combination by proposing an extension-refinement iterative scheme, in which the intrinsic idea is to decompose the target spectrum into several segments such that the corresponding eigenproblem in each segment is well-conditioned. Theoretical analysis and numerical illustration are also reported to illustrate the efficiency and effectiveness of this algorithm. |
| Tasks | |
| Published | 2018-04-10 |
| URL | http://arxiv.org/abs/1804.03415v2 |
| http://arxiv.org/pdf/1804.03415v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-fast-hierarchically-preconditioned |
| Repo | |
| Framework | |
Tetris: Re-architecting Convolutional Neural Network Computation for Machine Learning Accelerators
| Title | Tetris: Re-architecting Convolutional Neural Network Computation for Machine Learning Accelerators |
| Authors | Hang Lu, Xin Wei, Ning Lin, Guihai Yan, and Xiaowei Li |
| Abstract | Inference efficiency is the predominant consideration in designing deep learning accelerators. Previous work mainly focuses on skipping zero values to deal with remarkable ineffectual computation, while zero bits in non-zero values, as another major source of ineffectual computation, is often ignored. The reason lies on the difficulty of extracting essential bits during operating multiply-and-accumulate (MAC) in the processing element. Based on the fact that zero bits occupy as high as 68.9% fraction in the overall weights of modern deep convolutional neural network models, this paper firstly proposes a weight kneading technique that could eliminate ineffectual computation caused by either zero value weights or zero bits in non-zero weights, simultaneously. Besides, a split-and-accumulate (SAC) computing pattern in replacement of conventional MAC, as well as the corresponding hardware accelerator design called Tetris are proposed to support weight kneading at the hardware level. Experimental results prove that Tetris could speed up inference up to 1.50x, and improve power efficiency up to 5.33x compared with the state-of-the-art baselines. |
| Tasks | |
| Published | 2018-11-14 |
| URL | http://arxiv.org/abs/1811.06841v1 |
| http://arxiv.org/pdf/1811.06841v1.pdf | |
| PWC | https://paperswithcode.com/paper/tetris-re-architecting-convolutional-neural |
| Repo | |
| Framework | |
Automatic Classifiers as Scientific Instruments: One Step Further Away from Ground-Truth
| Title | Automatic Classifiers as Scientific Instruments: One Step Further Away from Ground-Truth |
| Authors | Jacob Whitehill, Anand Ramakrishnan |
| Abstract | Automatic machine learning-based detectors of various psychological and social phenomena (e.g., emotion, stress, engagement) have great potential to advance basic science. However, when a detector $d$ is trained to approximate an existing measurement tool (e.g., a questionnaire, observation protocol), then care must be taken when interpreting measurements collected using $d$ since they are one step further removed from the underlying construct. We examine how the accuracy of $d$, as quantified by the correlation $q$ of $d$'s outputs with the ground-truth construct $U$, impacts the estimated correlation between $U$ (e.g., stress) and some other phenomenon $V$ (e.g., academic performance). In particular: (1) We show that if the true correlation between $U$ and $V$ is $r$, then the expected sample correlation, over all vectors $\mathcal{T}^n$ whose correlation with $U$ is $q$, is $qr$. (2) We derive a formula for the probability that the sample correlation (over $n$ subjects) using $d$ is positive given that the true correlation is negative (and vice-versa); this probability can be substantial (around $20-30%$) for values of $n$ and $q$ that have been used in recent affective computing studies. %We also show that this probability decreases monotonically in $n$ and in $q$. (3) With the goal to reduce the variance of correlations estimated by an automatic detector, we show that training multiple neural networks $d^{(1)},\ldots,d^{(m)}$ using different training architectures and hyperparameters for the same detection task provides only limited ``coverage’’ of $\mathcal{T}^n$. | |
| Tasks | |
| Published | 2018-12-19 |
| URL | https://arxiv.org/abs/1812.08255v2 |
| https://arxiv.org/pdf/1812.08255v2.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-classifiers-as-scientific |
| Repo | |
| Framework | |
Fair Algorithms for Learning in Allocation Problems
| Title | Fair Algorithms for Learning in Allocation Problems |
| Authors | Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, Zachary Schutzman |
| Abstract | Settings such as lending and policing can be modeled by a centralized agent allocating a resource (loans or police officers) amongst several groups, in order to maximize some objective (loans given that are repaid or criminals that are apprehended). Often in such problems fairness is also a concern. A natural notion of fairness, based on general principles of equality of opportunity, asks that conditional on an individual being a candidate for the resource, the probability of actually receiving it is approximately independent of the individual’s group. In lending this means that equally creditworthy individuals in different racial groups have roughly equal chances of receiving a loan. In policing it means that two individuals committing the same crime in different districts would have roughly equal chances of being arrested. We formalize this fairness notion for allocation problems and investigate its algorithmic consequences. Our main technical results include an efficient learning algorithm that converges to an optimal fair allocation even when the frequency of candidates (creditworthy individuals or criminals) in each group is unknown. The algorithm operates in a censored feedback model in which only the number of candidates who received the resource in a given allocation can be observed, rather than the true number of candidates. This models the fact that we do not learn the creditworthiness of individuals we do not give loans to nor learn about crimes committed if the police presence in a district is low. As an application of our framework, we consider the predictive policing problem. The learning algorithm is trained on arrest data gathered from its own deployments on previous days, resulting in a potential feedback loop that our algorithm provably overcomes. We empirically investigate the performance of our algorithm on the Philadelphia Crime Incidents dataset. |
| Tasks | |
| Published | 2018-08-30 |
| URL | http://arxiv.org/abs/1808.10549v2 |
| http://arxiv.org/pdf/1808.10549v2.pdf | |
| PWC | https://paperswithcode.com/paper/fair-algorithms-for-learning-in-allocation |
| Repo | |
| Framework | |
N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
| Title | N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials |
| Authors | Risi Kondor |
| Abstract | We describe N-body networks, a neural network architecture for learning the behavior and properties of complex many body physical systems. Our specific application is to learn atomic potential energy surfaces for use in molecular dynamics simulations. Our architecture is novel in that (a) it is based on a hierarchical decomposition of the many body system into subsytems, (b) the activations of the network correspond to the internal state of each subsystem, (c) the “neurons” in the network are constructed explicitly so as to guarantee that each of the activations is covariant to rotations, (d) the neurons operate entirely in Fourier space, and the nonlinearities are realized by tensor products followed by Clebsch-Gordan decompositions. As part of the description of our network, we give a characterization of what way the weights of the network may interact with the activations so as to ensure that the covariance property is maintained. |
| Tasks | |
| Published | 2018-03-05 |
| URL | http://arxiv.org/abs/1803.01588v1 |
| http://arxiv.org/pdf/1803.01588v1.pdf | |
| PWC | https://paperswithcode.com/paper/n-body-networks-a-covariant-hierarchical |
| Repo | |
| Framework | |
A Variational Topological Neural Model for Cascade-based Diffusion in Networks
| Title | A Variational Topological Neural Model for Cascade-based Diffusion in Networks |
| Authors | Sylvain Lamprier |
| Abstract | Many works have been proposed in the literature to capture the dynamics of diffusion in networks. While some of them define graphical markovian models to extract temporal relationships between node infections in networks, others consider diffusion episodes as sequences of infections via recurrent neural models. In this paper we propose a model at the crossroads of these two extremes, which embeds the history of diffusion in infected nodes as hidden continuous states. Depending on the trajectory followed by the content before reaching a given node, the distribution of influence probabilities may vary. However, content trajectories are usually hidden in the data, which induces challenging learning problems. We propose a topological recurrent neural model which exhibits good experimental performances for diffusion modelling and prediction. |
| Tasks | |
| Published | 2018-12-28 |
| URL | http://arxiv.org/abs/1812.10962v1 |
| http://arxiv.org/pdf/1812.10962v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-variational-topological-neural-model-for |
| Repo | |
| Framework | |
Identifying Domain Adjacent Instances for Semantic Parsers
| Title | Identifying Domain Adjacent Instances for Semantic Parsers |
| Authors | James Ferguson, Janara Christensen, Edward Li, Edgar Gonzàlez |
| Abstract | When the semantics of a sentence are not representable in a semantic parser’s output schema, parsing will inevitably fail. Detection of these instances is commonly treated as an out-of-domain classification problem. However, there is also a more subtle scenario in which the test data is drawn from the same domain. In addition to formalizing this problem of domain-adjacency, we present a comparison of various baselines that could be used to solve it. We also propose a new simple sentence representation that emphasizes words which are unexpected. This approach improves the performance of a downstream semantic parser run on in-domain and domain-adjacent instances. |
| Tasks | |
| Published | 2018-08-26 |
| URL | http://arxiv.org/abs/1808.08626v1 |
| http://arxiv.org/pdf/1808.08626v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-domain-adjacent-instances-for |
| Repo | |
| Framework | |
COBRAS-TS: A new approach to Semi-Supervised Clustering of Time Series
| Title | COBRAS-TS: A new approach to Semi-Supervised Clustering of Time Series |
| Authors | Toon Van Craenendonck, Wannes Meert, Sebastijan Dumancic, Hendrik Blockeel |
| Abstract | Clustering is ubiquitous in data analysis, including analysis of time series. It is inherently subjective: different users may prefer different clusterings for a particular dataset. Semi-supervised clustering addresses this by allowing the user to provide examples of instances that should (not) be in the same cluster. This paper studies semi-supervised clustering in the context of time series. We show that COBRAS, a state-of-the-art semi-supervised clustering method, can be adapted to this setting. We refer to this approach as COBRAS-TS. An extensive experimental evaluation supports the following claims: (1) COBRAS-TS far outperforms the current state of the art in semi-supervised clustering for time series, and thus presents a new baseline for the field; (2) COBRAS-TS can identify clusters with separated components; (3) COBRAS-TS can identify clusters that are characterized by small local patterns; (4) a small amount of semi-supervision can greatly improve clustering quality for time series; (5) the choice of the clustering algorithm matters (contrary to earlier claims in the literature). |
| Tasks | Time Series |
| Published | 2018-05-02 |
| URL | http://arxiv.org/abs/1805.00779v1 |
| http://arxiv.org/pdf/1805.00779v1.pdf | |
| PWC | https://paperswithcode.com/paper/cobras-ts-a-new-approach-to-semi-supervised |
| Repo | |
| Framework | |
Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model
| Title | Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model |
| Authors | Ke Wang, Junbo Zhang, Yujun Wang, Lei Xie |
| Abstract | Speaker adaptation aims to estimate a speaker specific acoustic model from a speaker independent one to minimize the mismatch between the training and testing conditions arisen from speaker variabilities. A variety of neural network adaptation methods have been proposed since deep learning models have become the main stream. But there still lacks an experimental comparison between different methods, especially when DNN-based acoustic models have been advanced greatly. In this paper, we aim to close this gap by providing an empirical evaluation of three typical speaker adaptation methods: LIN, LHUC and KLD. Adaptation experiments, with different size of adaptation data, are conducted on a strong TDNN-LSTM acoustic model. More challengingly, here, the source and target we are concerned with are standard Mandarin speaker model and accented Mandarin speaker model. We compare the performances of different methods and their combinations. Speaker adaptation performance is also examined by speaker’s accent degree. |
| Tasks | |
| Published | 2018-03-27 |
| URL | http://arxiv.org/abs/1803.10146v3 |
| http://arxiv.org/pdf/1803.10146v3.pdf | |
| PWC | https://paperswithcode.com/paper/empirical-evaluation-of-speaker-adaptation-on |
| Repo | |
| Framework | |
Angular-Based Word Meta-Embedding Learning
| Title | Angular-Based Word Meta-Embedding Learning |
| Authors | James O’ Neill, Danushka Bollegala |
| Abstract | Ensembling word embeddings to improve distributed word representations has shown good success for natural language processing tasks in recent years. These approaches either carry out straightforward mathematical operations over a set of vectors or use unsupervised learning to find a lower-dimensional representation. This work compares meta-embeddings trained for different losses, namely loss functions that account for angular distance between the reconstructed embedding and the target and those that account normalized distances based on the vector length. We argue that meta-embeddings are better to treat the ensemble set equally in unsupervised learning as the respective quality of each embedding is unknown for upstream tasks prior to meta-embedding. We show that normalization methods that account for this such as cosine and KL-divergence objectives outperform meta-embedding trained on standard $\ell_1$ and $\ell_2$ loss on \textit{defacto} word similarity and relatedness datasets and find it outperforms existing meta-learning strategies. |
| Tasks | Meta-Learning, Word Embeddings |
| Published | 2018-08-13 |
| URL | http://arxiv.org/abs/1808.04334v1 |
| http://arxiv.org/pdf/1808.04334v1.pdf | |
| PWC | https://paperswithcode.com/paper/angular-based-word-meta-embedding-learning |
| Repo | |
| Framework | |
Sequential Learning of Movement Prediction in Dynamic Environments using LSTM Autoencoder
| Title | Sequential Learning of Movement Prediction in Dynamic Environments using LSTM Autoencoder |
| Authors | Meenakshi Sarkar, Debasish Ghose |
| Abstract | Predicting movement of objects while the action of learning agent interacts with the dynamics of the scene still remains a key challenge in robotics. We propose a multi-layer Long Short Term Memory (LSTM) autoendocer network that predicts future frames for a robot navigating in a dynamic environment with moving obstacles. The autoencoder network is composed of a state and action conditioned decoder network that reconstructs the future frames of video, conditioned on the action taken by the agent. The input image frames are first transformed into low dimensional feature vectors with a pre-trained encoder network and then reconstructed with the LSTM autoencoder network to generate the future frames. A virtual environment, based on the OpenAi-Gym framework for robotics, is used to gather training data and test the proposed network. The initial experiments show promising results indicating that these predicted frames can be used by an appropriate reinforcement learning framework in future to navigate around dynamic obstacles. |
| Tasks | |
| Published | 2018-10-12 |
| URL | http://arxiv.org/abs/1810.05394v1 |
| http://arxiv.org/pdf/1810.05394v1.pdf | |
| PWC | https://paperswithcode.com/paper/sequential-learning-of-movement-prediction-in |
| Repo | |
| Framework | |
Learning Discriminative Motion Features Through Detection
| Title | Learning Discriminative Motion Features Through Detection |
| Authors | Gedas Bertasius, Christoph Feichtenhofer, Du Tran, Jianbo Shi, Lorenzo Torresani |
| Abstract | Despite huge success in the image domain, modern detection models such as Faster R-CNN have not been used nearly as much for video analysis. This is arguably due to the fact that detection models are designed to operate on single frames and as a result do not have a mechanism for learning motion representations directly from video. We propose a learning procedure that allows detection models such as Faster R-CNN to learn motion features directly from the RGB video data while being optimized with respect to a pose estimation task. Given a pair of video frames—Frame A and Frame B—we force our model to predict human pose in Frame A using the features from Frame B. We do so by leveraging deformable convolutions across space and time. Our network learns to spatially sample features from Frame B in order to maximize pose detection accuracy in Frame A. This naturally encourages our network to learn motion offsets encoding the spatial correspondences between the two frames. We refer to these motion offsets as DiMoFs (Discriminative Motion Features). In our experiments we show that our training scheme helps learn effective motion cues, which can be used to estimate and localize salient human motion. Furthermore, we demonstrate that as a byproduct, our model also learns features that lead to improved pose detection in still-images, and better keypoint tracking. Finally, we show how to leverage our learned model for the tasks of spatiotemporal action localization and fine-grained action recognition. |
| Tasks | Action Localization, Pose Estimation, Temporal Action Localization |
| Published | 2018-12-11 |
| URL | http://arxiv.org/abs/1812.04172v1 |
| http://arxiv.org/pdf/1812.04172v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-discriminative-motion-features |
| Repo | |
| Framework | |
One-Shot Observation Learning Using Visual Activity Features
| Title | One-Shot Observation Learning Using Visual Activity Features |
| Authors | Leo Pauly, Wisdom C. Agboh, David C. Hogg, Raul Fuentes |
| Abstract | Observation learning is the process of learning a task by observing an expert demonstrator. Our principal contribution is a one-shot learning method for robot manipulation tasks in which only a single demonstration is required. The key idea is to encode the demonstration in an activity space defined as part of a previously trained activity classifier. The distance between this encoding and equivalent encodings from trials of a robot performing the same task provides a reward function supporting iterative learning of task completion by the robotic manipulator. We use reinforcement learning for experiments with a simulated robotic manipulator, and stochastic trajectory optimisation for experiments with a real robotic manipulator. We show that the proposed method can be used to learn tasks from a single demonstration under varying viewpoint of observation, object properties, scene background and morphology of the manipulator. Videos of all results, including demonstrations, can be found on: https://tinyurl.com/s2l-stage1 |
| Tasks | One-Shot Learning |
| Published | 2018-10-17 |
| URL | https://arxiv.org/abs/1810.07483v2 |
| https://arxiv.org/pdf/1810.07483v2.pdf | |
| PWC | https://paperswithcode.com/paper/one-shot-observation-learning |
| Repo | |
| Framework | |
