Paper Group ANR 671
Effective Strategies in Zero-Shot Neural Machine Translation. SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation. Blind nonnegative source separation using biological neural networks. Pushing the envelope in deep visual recognition for mobile platforms. Predictive State Recurrent Neural Networks. Cross-Platform Emoji Interp …
Effective Strategies in Zero-Shot Neural Machine Translation
| Title | Effective Strategies in Zero-Shot Neural Machine Translation |
| Authors | Thanh-Le Ha, Jan Niehues, Alexander Waibel |
| Abstract | In this paper, we proposed two strategies which can be applied to a multilingual neural machine translation system in order to better tackle zero-shot scenarios despite not having any parallel corpus. The experiments show that they are effective in terms of both performance and computing resources, especially in multilingual translation of unbalanced data in real zero-resourced condition when they alleviate the language bias problem. |
| Tasks | Machine Translation |
| Published | 2017-11-21 |
| URL | http://arxiv.org/abs/1711.07893v2 |
| http://arxiv.org/pdf/1711.07893v2.pdf | |
| PWC | https://paperswithcode.com/paper/effective-strategies-in-zero-shot-neural |
| Repo | |
| Framework | |
SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation
| Title | SenseGen: A Deep Learning Architecture for Synthetic Sensor Data Generation |
| Authors | Moustafa Alzantot, Supriyo Chakraborty, Mani B. Srivastava |
| Abstract | Our ability to synthesize sensory data that preserves specific statistical properties of the real data has had tremendous implications on data privacy and big data analytics. The synthetic data can be used as a substitute for selective real data segments,that are sensitive to the user, thus protecting privacy and resulting in improved analytics.However, increasingly adversarial roles taken by data recipients such as mobile apps, or other cloud-based analytics services, mandate that the synthetic data, in addition to preserving statistical properties, should also be difficult to distinguish from the real data. Typically, visual inspection has been used as a test to distinguish between datasets. But more recently, sophisticated classifier models (discriminators), corresponding to a set of events, have also been employed to distinguish between synthesized and real data. The model operates on both datasets and the respective event outputs are compared for consistency. In this paper, we take a step towards generating sensory data that can pass a deep learning based discriminator model test, and make two specific contributions: first, we present a deep learning based architecture for synthesizing sensory data. This architecture comprises of a generator model, which is a stack of multiple Long-Short-Term-Memory (LSTM) networks and a Mixture Density Network. second, we use another LSTM network based discriminator model for distinguishing between the true and the synthesized data. Using a dataset of accelerometer traces, collected using smartphones of users doing their daily activities, we show that the deep learning based discriminator model can only distinguish between the real and synthesized traces with an accuracy in the neighborhood of 50%. |
| Tasks | |
| Published | 2017-01-31 |
| URL | http://arxiv.org/abs/1701.08886v1 |
| http://arxiv.org/pdf/1701.08886v1.pdf | |
| PWC | https://paperswithcode.com/paper/sensegen-a-deep-learning-architecture-for |
| Repo | |
| Framework | |
Blind nonnegative source separation using biological neural networks
| Title | Blind nonnegative source separation using biological neural networks |
| Authors | Cengiz Pehlevan, Sreyas Mohan, Dmitri B. Chklovskii |
| Abstract | Blind source separation, i.e. extraction of independent sources from a mixture, is an important problem for both artificial and natural signal processing. Here, we address a special case of this problem when sources (but not the mixing matrix) are known to be nonnegative, for example, due to the physical nature of the sources. We search for the solution to this problem that can be implemented using biologically plausible neural networks. Specifically, we consider the online setting where the dataset is streamed to a neural network. The novelty of our approach is that we formulate blind nonnegative source separation as a similarity matching problem and derive neural networks from the similarity matching objective. Importantly, synaptic weights in our networks are updated according to biologically plausible local learning rules. |
| Tasks | |
| Published | 2017-06-01 |
| URL | http://arxiv.org/abs/1706.00382v1 |
| http://arxiv.org/pdf/1706.00382v1.pdf | |
| PWC | https://paperswithcode.com/paper/blind-nonnegative-source-separation-using |
| Repo | |
| Framework | |
Pushing the envelope in deep visual recognition for mobile platforms
| Title | Pushing the envelope in deep visual recognition for mobile platforms |
| Authors | Lorenzo Alvino |
| Abstract | Image classification is the task of assigning to an input image a label from a fixed set of categories. One of its most important applicative fields is that of robotics, in particular the needing of a robot to be aware of what’s around and the consequent exploitation of that information as a benefit for its tasks. In this work we consider the problem of a robot that enters a new environment and wants to understand visual data coming from its camera, so to extract knowledge from them. As main novelty we want to overcome the needing of a physical robot, as it could be expensive and unhandy, so to hopefully enhance, speed up and ease the research in this field. That’s why we propose to develop an application for a mobile platform that wraps several deep visual recognition tasks. First we deal with a simple Image classification, testing a model obtained from an AlexNet trained on the ILSVRC 2012 dataset. Several photo settings are considered to better understand which factors affect most the quality of classification. For the same purpose we are interested to integrate the classification task with an extra module dealing with segmentation of the object inside the image. In particular we propose a technique for extracting the object shape and moving out all the background, so to focus the classification only on the region occupied by the object. Another significant task that is included is that of object discovery. Its purpose is to simulate the situation in which the robot needs a certain object to complete one of its activities. It starts searching for what it needs by looking around and trying to understand the location of the object by scanning the surrounding environment. Finally we provide a tool for dealing with the creation of customized task-specific databases, meant to better suit to one’s needing in a particular vision task. |
| Tasks | Image Classification |
| Published | 2017-10-16 |
| URL | http://arxiv.org/abs/1710.05982v2 |
| http://arxiv.org/pdf/1710.05982v2.pdf | |
| PWC | https://paperswithcode.com/paper/pushing-the-envelope-in-deep-visual |
| Repo | |
| Framework | |
Predictive State Recurrent Neural Networks
| Title | Predictive State Recurrent Neural Networks |
| Authors | Carlton Downey, Ahmed Hefny, Boyue Li, Byron Boots, Geoffrey Gordon |
| Abstract | We present a new model, Predictive State Recurrent Neural Networks (PSRNNs), for filtering and prediction in dynamical systems. PSRNNs draw on insights from both Recurrent Neural Networks (RNNs) and Predictive State Representations (PSRs), and inherit advantages from both types of models. Like many successful RNN architectures, PSRNNs use (potentially deeply composed) bilinear transfer functions to combine information from multiple sources. We show that such bilinear functions arise naturally from state updates in Bayes filters like PSRs, in which observations can be viewed as gating belief states. We also show that PSRNNs can be learned effectively by combining Backpropogation Through Time (BPTT) with an initialization derived from a statistically consistent learning algorithm for PSRs called two-stage regression (2SR). Finally, we show that PSRNNs can be factorized using tensor decomposition, reducing model size and suggesting interesting connections to existing multiplicative architectures such as LSTMs. We applied PSRNNs to 4 datasets, and showed that we outperform several popular alternative approaches to modeling dynamical systems in all cases. |
| Tasks | |
| Published | 2017-05-25 |
| URL | http://arxiv.org/abs/1705.09353v2 |
| http://arxiv.org/pdf/1705.09353v2.pdf | |
| PWC | https://paperswithcode.com/paper/predictive-state-recurrent-neural-networks |
| Repo | |
| Framework | |
Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications
| Title | Cross-Platform Emoji Interpretation: Analysis, a Solution, and Applications |
| Authors | Fred Morstatter, Kai Shu, Suhang Wang, Huan Liu |
| Abstract | Most social media platforms are largely based on text, and users often write posts to describe where they are, what they are seeing, and how they are feeling. Because written text lacks the emotional cues of spoken and face-to-face dialogue, ambiguities are common in written language. This problem is exacerbated in the short, informal nature of many social media posts. To bypass this issue, a suite of special characters called “emojis,” which are small pictograms, are embedded within the text. Many emojis are small depictions of facial expressions designed to help disambiguate the emotional meaning of the text. However, a new ambiguity arises in the way that emojis are rendered. Every platform (Windows, Mac, and Android, to name a few) renders emojis according to their own style. In fact, it has been shown that some emojis can be rendered so differently that they look “happy” on some platforms, and “sad” on others. In this work, we use real-world data to verify the existence of this problem. We verify that the usage of the same emoji can be significantly different across platforms, with some emojis exhibiting different sentiment polarities on different platforms. We propose a solution to identify the intended emoji based on the platform-specific nature of the emoji used by the author of a social media post. We apply our solution to sentiment analysis, a task that can benefit from the emoji calibration technique we use in this work. We conduct experiments to evaluate the effectiveness of the mapping in this task. |
| Tasks | Calibration, Sentiment Analysis |
| Published | 2017-09-14 |
| URL | http://arxiv.org/abs/1709.04969v1 |
| http://arxiv.org/pdf/1709.04969v1.pdf | |
| PWC | https://paperswithcode.com/paper/cross-platform-emoji-interpretation-analysis |
| Repo | |
| Framework | |
Recent Advances in Recurrent Neural Networks
| Title | Recent Advances in Recurrent Neural Networks |
| Authors | Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak, Shahrokh Valaee |
| Abstract | Recurrent neural networks (RNNs) are capable of learning features and long term dependencies from sequential and time-series data. The RNNs have a stack of non-linear units where at least one connection between units forms a directed cycle. A well-trained RNN can model any dynamical system; however, training RNNs is mostly plagued by issues in learning long-term dependencies. In this paper, we present a survey on RNNs and several new advances for newcomers and professionals in the field. The fundamentals and recent advances are explained and the research challenges are introduced. |
| Tasks | Time Series |
| Published | 2017-12-29 |
| URL | http://arxiv.org/abs/1801.01078v3 |
| http://arxiv.org/pdf/1801.01078v3.pdf | |
| PWC | https://paperswithcode.com/paper/recent-advances-in-recurrent-neural-networks |
| Repo | |
| Framework | |
Comparison of SMT and RBMT; The Requirement of Hybridization for Marathi-Hindi MT
| Title | Comparison of SMT and RBMT; The Requirement of Hybridization for Marathi-Hindi MT |
| Authors | Sreelekha. S, Pushpak Bhattacharyya |
| Abstract | We present in this paper our work on comparison between Statistical Machine Translation (SMT) and Rule-based machine translation for translation from Marathi to Hindi. Rule Based systems although robust take lots of time to build. On the other hand statistical machine translation systems are easier to create, maintain and improve upon. We describe the development of a basic Marathi-Hindi SMT system and evaluate its performance. Through a detailed error analysis, we, point out the relative strengths and weaknesses of both systems. Effectively, we shall see that even with a small amount of training corpus a statistical machine translation system has many advantages for high quality domain specific machine translation over that of a rule-based counterpart. |
| Tasks | Machine Translation |
| Published | 2017-03-10 |
| URL | http://arxiv.org/abs/1703.03666v1 |
| http://arxiv.org/pdf/1703.03666v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparison-of-smt-and-rbmt-the-requirement-of |
| Repo | |
| Framework | |
Obstacle Avoidance through Deep Networks based Intermediate Perception
| Title | Obstacle Avoidance through Deep Networks based Intermediate Perception |
| Authors | Shichao Yang, Sandeep Konam, Chen Ma, Stephanie Rosenthal, Manuela Veloso, Sebastian Scherer |
| Abstract | Obstacle avoidance from monocular images is a challenging problem for robots. Though multi-view structure-from-motion could build 3D maps, it is not robust in textureless environments. Some learning based methods exploit human demonstration to predict a steering command directly from a single image. However, this method is usually biased towards certain tasks or demonstration scenarios and also biased by human understanding. In this paper, we propose a new method to predict a trajectory from images. We train our system on more diverse NYUv2 dataset. The ground truth trajectory is computed from the designed cost functions automatically. The Convolutional Neural Network perception is divided into two stages: first, predict depth map and surface normal from RGB images, which are two important geometric properties related to 3D obstacle representation. Second, predict the trajectory from the depth and normal. Results show that our intermediate perception increases the accuracy by 20% than the direct prediction. Our model generalizes well to other public indoor datasets and is also demonstrated for robot flights in simulation and experiments. |
| Tasks | |
| Published | 2017-04-27 |
| URL | http://arxiv.org/abs/1704.08759v1 |
| http://arxiv.org/pdf/1704.08759v1.pdf | |
| PWC | https://paperswithcode.com/paper/obstacle-avoidance-through-deep-networks |
| Repo | |
| Framework | |
A Separation-Based Design to Data-Driven Control for Large-Scale Partially Observed Systems
| Title | A Separation-Based Design to Data-Driven Control for Large-Scale Partially Observed Systems |
| Authors | Dan Yu, Mohammadhussein Rafieisakhaei, Suman Chakravorty |
| Abstract | This paper studies the partially observed stochastic optimal control problem for systems with state dynamics governed by Partial Differential Equations (PDEs) that leads to an extremely large problem. First, an open-loop deterministic trajectory optimization problem is solved using a black box simulation model of the dynamical system. Next, a Linear Quadratic Gaussian (LQG) controller is designed for the nominal trajectory-dependent linearized system, which is identified using input-output experimental data consisting of the impulse responses of the optimized nominal system. A computational nonlinear heat example is used to illustrate the performance of the approach. |
| Tasks | |
| Published | 2017-07-11 |
| URL | http://arxiv.org/abs/1707.03092v1 |
| http://arxiv.org/pdf/1707.03092v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-separation-based-design-to-data-driven |
| Repo | |
| Framework | |
Lipschitz Optimisation for Lipschitz Interpolation
| Title | Lipschitz Optimisation for Lipschitz Interpolation |
| Authors | Jan-Peter Calliess |
| Abstract | Techniques known as Nonlinear Set Membership prediction, Kinky Inference or Lipschitz Interpolation are fast and numerically robust approaches to nonparametric machine learning that have been proposed to be utilised in the context of system identification and learning-based control. They utilise presupposed Lipschitz properties in order to compute inferences over unobserved function values. Unfortunately, most of these approaches rely on exact knowledge about the input space metric as well as about the Lipschitz constant. Furthermore, existing techniques to estimate the Lipschitz constants from the data are not robust to noise or seem to be ad-hoc and typically are decoupled from the ultimate learning and prediction task. To overcome these limitations, we propose an approach for optimising parameters of the presupposed metrics by minimising validation set prediction errors. To avoid poor performance due to local minima, we propose to utilise Lipschitz properties of the optimisation objective to ensure global optimisation success. The resulting approach is a new flexible method for nonparametric black-box learning. We provide experimental evidence of the competitiveness of our approach on artificial as well as on real data. |
| Tasks | |
| Published | 2017-02-28 |
| URL | http://arxiv.org/abs/1702.08898v1 |
| http://arxiv.org/pdf/1702.08898v1.pdf | |
| PWC | https://paperswithcode.com/paper/lipschitz-optimisation-for-lipschitz |
| Repo | |
| Framework | |
Motion-Compensated Temporal Filtering for Critically-Sampled Wavelet-Encoded Images
| Title | Motion-Compensated Temporal Filtering for Critically-Sampled Wavelet-Encoded Images |
| Authors | Vildan Atalay Aydin, Hassan Foroosh |
| Abstract | We propose a novel motion estimation/compensation (ME/MC) method for wavelet-based (in-band) motion compensated temporal filtering (MCTF), with application to low-bitrate video coding. Unlike the conventional in-band MCTF algorithms, which require redundancy to overcome the shift-variance problem of critically sampled (complete) discrete wavelet transforms (DWT), we perform ME/MC steps directly on DWT coefficients by avoiding the need of shift-invariance. We omit upsampling, the inverse-DWT (IDWT), and the calculation of redundant DWT coefficients, while achieving arbitrary subpixel accuracy without interpolation, and high video quality even at very low-bitrates, by deriving the exact relationships between DWT subbands of input image sequences. Experimental results demonstrate the accuracy of the proposed method, confirming that our model for ME/MC effectively improves video coding quality. |
| Tasks | Motion Estimation |
| Published | 2017-05-13 |
| URL | http://arxiv.org/abs/1705.05741v1 |
| http://arxiv.org/pdf/1705.05741v1.pdf | |
| PWC | https://paperswithcode.com/paper/motion-compensated-temporal-filtering-for |
| Repo | |
| Framework | |
Query Complexity of Clustering with Side Information
| Title | Query Complexity of Clustering with Side Information |
| Authors | Arya Mazumdar, Barna Saha |
| Abstract | Suppose, we are given a set of $n$ elements to be clustered into $k$ (unknown) clusters, and an oracle/expert labeler that can interactively answer pair-wise queries of the form, “do two elements $u$ and $v$ belong to the same cluster?". The goal is to recover the optimum clustering by asking the minimum number of queries. In this paper, we initiate a rigorous theoretical study of this basic problem of query complexity of interactive clustering, and provide strong information theoretic lower bounds, as well as nearly matching upper bounds. Most clustering problems come with a similarity matrix, which is used by an automated process to cluster similar points together. Our main contribution in this paper is to show the dramatic power of side information aka similarity matrix on reducing the query complexity of clustering. A similarity matrix represents noisy pair-wise relationships such as one computed by some function on attributes of the elements. A natural noisy model is where similarity values are drawn independently from some arbitrary probability distribution $f_+$ when the underlying pair of elements belong to the same cluster, and from some $f_-$ otherwise. We show that given such a similarity matrix, the query complexity reduces drastically from $\Theta(nk)$ (no similarity matrix) to $O(\frac{k^2\log{n}}{\cH^2(f_+\f_-)})$ where $\cH^2$ denotes the squared Hellinger divergence. Moreover, this is also information-theoretic optimal within an $O(\log{n})$ factor. Our algorithms are all efficient, and parameter free, i.e., they work without any knowledge of $k, f_+$ and $f_-$, and only depend logarithmically with $n$. Along the way, our work also reveals intriguing connection to popular community detection models such as the {\em stochastic block model}, significantly generalizes them, and opens up many venues for interesting future research. |
| Tasks | Community Detection |
| Published | 2017-06-23 |
| URL | http://arxiv.org/abs/1706.07719v1 |
| http://arxiv.org/pdf/1706.07719v1.pdf | |
| PWC | https://paperswithcode.com/paper/query-complexity-of-clustering-with-side |
| Repo | |
| Framework | |
Artificial Neural Networks that Learn to Satisfy Logic Constraints
| Title | Artificial Neural Networks that Learn to Satisfy Logic Constraints |
| Authors | Gadi Pinkas, Shimon Cohen |
| Abstract | Logic-based problems such as planning, theorem proving, or puzzles, typically involve combinatoric search and structured knowledge representation. Artificial neural networks are very successful statistical learners, however, for many years, they have been criticized for their weaknesses in representing and in processing complex structured knowledge which is crucial for combinatoric search and symbol manipulation. Two neural architectures are presented, which can encode structured relational knowledge in neural activation, and store bounded First Order Logic constraints in connection weights. Both architectures learn to search for a solution that satisfies the constraints. Learning is done by unsupervised practicing on problem instances from the same domain, in a way that improves the network-solving speed. No teacher exists to provide answers for the problem instances of the training and test sets. However, the domain constraints are provided as prior knowledge to a loss function that measures the degree of constraint violations. Iterations of activation calculation and learning are executed until a solution that maximally satisfies the constraints emerges on the output units. As a test case, block-world planning problems are used to train networks that learn to plan in that domain, but the techniques proposed could be used more generally as in integrating prior symbolic knowledge with statistical learning |
| Tasks | Automated Theorem Proving |
| Published | 2017-12-08 |
| URL | http://arxiv.org/abs/1712.03049v1 |
| http://arxiv.org/pdf/1712.03049v1.pdf | |
| PWC | https://paperswithcode.com/paper/artificial-neural-networks-that-learn-to |
| Repo | |
| Framework | |
Integrating User and Agent Models: A Deep Task-Oriented Dialogue System
| Title | Integrating User and Agent Models: A Deep Task-Oriented Dialogue System |
| Authors | Weiyan Wang, Yuxiang WU, Yu Zhang, Zhongqi Lu, Kaixiang Mo, Qiang Yang |
| Abstract | Task-oriented dialogue systems can efficiently serve a large number of customers and relieve people from tedious works. However, existing task-oriented dialogue systems depend on handcrafted actions and states or extra semantic labels, which sometimes degrades user experience despite the intensive human intervention. Moreover, current user simulators have limited expressive ability so that deep reinforcement Seq2Seq models have to rely on selfplay and only work in some special cases. To address those problems, we propose a uSer and Agent Model IntegrAtion (SAMIA) framework inspired by an observation that the roles of the user and agent models are asymmetric. Firstly, this SAMIA framework model the user model as a Seq2Seq learning problem instead of ranking or designing rules. Then the built user model is used as a leverage to train the agent model by deep reinforcement learning. In the test phase, the output of the agent model is filtered by the user model to enhance the stability and robustness. Experiments on a real-world coffee ordering dataset verify the effectiveness of the proposed SAMIA framework. |
| Tasks | Task-Oriented Dialogue Systems |
| Published | 2017-11-10 |
| URL | http://arxiv.org/abs/1711.03697v1 |
| http://arxiv.org/pdf/1711.03697v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-user-and-agent-models-a-deep-task |
| Repo | |
| Framework | |
