January 29, 2020

3183 words 15 mins read

Paper Group ANR 574

Gaussian processes with linear operator inequality constraints. A Robust Image Watermarking System Based on Deep Neural Networks. Sentiment Analysis On Indian Indigenous Languages: A Review On Multilingual Opinion Mining. Similarity Learning Networks for Animal Individual Re-Identification – Beyond the Capabilities of a Human Observer. Fair Dimens …

Gaussian processes with linear operator inequality constraints


Title	Gaussian processes with linear operator inequality constraints
Authors	Christian Agrell
Abstract	This paper presents an approach for constrained Gaussian Process (GP) regression where we assume that a set of linear transformations of the process are bounded. It is motivated by machine learning applications for high-consequence engineering systems, where this kind of information is often made available from phenomenological knowledge. We consider a GP $f$ over functions on $\mathcal{X} \subset \mathbb{R}^{n}$ taking values in $\mathbb{R}$, where the process $\mathcal{L}f$ is still Gaussian when $\mathcal{L}$ is a linear operator. Our goal is to model $f$ under the constraint that realizations of $\mathcal{L}f$ are confined to a convex set of functions. In particular, we require that $a \leq \mathcal{L}f \leq b$, given two functions $a$ and $b$ where $a < b$ pointwise. This formulation provides a consistent way of encoding multiple linear constraints, such as shape-constraints based on e.g. boundedness, monotonicity or convexity. We adopt the approach of using a sufficiently dense set of virtual observation locations where the constraint is required to hold, and derive the exact posterior for a conjugate likelihood. The results needed for stable numerical implementation are derived, together with an efficient sampling scheme for estimating the posterior process.
Tasks	Gaussian Processes
Published	2019-01-10
URL	https://arxiv.org/abs/1901.03134v2
PDF	https://arxiv.org/pdf/1901.03134v2.pdf
PWC	https://paperswithcode.com/paper/gaussian-processes-with-linear-operator
Repo
Framework

A Robust Image Watermarking System Based on Deep Neural Networks


Title	A Robust Image Watermarking System Based on Deep Neural Networks
Authors	Xin Zhong, Frank Y. Shih
Abstract	Digital image watermarking is the process of embedding and extracting watermark covertly on a carrier image. Incorporating deep learning networks with image watermarking has attracted increasing attention during recent years. However, existing deep learning-based watermarking systems cannot achieve robustness, blindness, and automated embedding and extraction simultaneously. In this paper, a fully automated image watermarking system based on deep neural networks is proposed to generalize the image watermarking processes. An unsupervised deep learning structure and a novel loss computation are proposed to achieve high capacity and high robustness without any prior knowledge of possible attacks. Furthermore, a challenging application of watermark extraction from camera-captured images is provided to validate the practicality as well as the robustness of the proposed system. Experimental results show the superiority performance of the proposed system as comparing against several currently available techniques.
Tasks
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11331v1
PDF	https://arxiv.org/pdf/1908.11331v1.pdf
PWC	https://paperswithcode.com/paper/a-robust-image-watermarking-system-based-on
Repo
Framework

Sentiment Analysis On Indian Indigenous Languages: A Review On Multilingual Opinion Mining


Title	Sentiment Analysis On Indian Indigenous Languages: A Review On Multilingual Opinion Mining
Authors	Sonali Rajesh Shah, Abhishek Kaushik
Abstract	An increase in the use of smartphones has laid to the use of the internet and social media platforms. The most commonly used social media platforms are Twitter, Facebook, WhatsApp and Instagram. People are sharing their personal experiences, reviews, feedbacks on the web. The information which is available on the web is unstructured and enormous. Hence, there is a huge scope of research on understanding the sentiment of the data available on the web. Sentiment Analysis (SA) can be carried out on the reviews, feedbacks, discussions available on the web. There has been extensive research carried out on SA in the English language, but data on the web also contains different other languages which should be analyzed. This paper aims to analyze, review and discuss the approaches, algorithms, challenges faced by the researchers while carrying out the SA on Indigenous languages.
Tasks	Opinion Mining, Sentiment Analysis
Published	2019-11-28
URL	https://arxiv.org/abs/1911.12848v1
PDF	https://arxiv.org/pdf/1911.12848v1.pdf
PWC	https://paperswithcode.com/paper/sentiment-analysis-on-indian-indigenous
Repo
Framework

Similarity Learning Networks for Animal Individual Re-Identification – Beyond the Capabilities of a Human Observer


Title	Similarity Learning Networks for Animal Individual Re-Identification – Beyond the Capabilities of a Human Observer
Authors	Stefan Schneider, Graham W. Taylor, Stefan Linquist, Stefan C. Kremer
Abstract	The ability of a researcher to re-identify (re-ID) an animal individual upon re-encounter is fundamental for addressing a broad range of questions in the study of ecosystem function, community and population dynamics, and behavioural ecology. Tagging animals during mark and recapture studies is the most common method for reliable animal re-ID however camera traps are a desirable alternative, requiring less labour, much less intrusion, and prolonged and continuous monitoring into an environment. Despite these advantages, the analyses of camera traps and video for re-ID by humans are criticized for their biases related to human judgment and inconsistencies between analyses. Recent years have witnessed the emergence of deep learning systems which re-ID humans based on image and video data with near perfect accuracy. Despite this success, there are limited examples of this approach for animal re-ID. Here, we demonstrate the viability of novel deep similarity learning methods on five species: humans, chimpanzees, humpback whales, octopus and fruit flies. Our implementation demonstrates the generality of this framework as the same process provides accurate results beyond the capabilities of a human observer. In combination with a species object detection model, this methodology will allow ecologists with camera/video trap data to re-identify individuals that exit and re-enter the camera frame. Our expectation is that this is just the beginning of a major trend that could stand to revolutionize the analysis of camera trap data and, ultimately, our approach to animal ecology.
Tasks	Object Detection
Published	2019-02-21
URL	https://arxiv.org/abs/1902.09324v2
PDF	https://arxiv.org/pdf/1902.09324v2.pdf
PWC	https://paperswithcode.com/paper/similarity-learning-networks-for-animal
Repo
Framework

Fair Dimensionality Reduction and Iterative Rounding for SDPs


Title	Fair Dimensionality Reduction and Iterative Rounding for SDPs
Authors	Jamie Morgenstern, Samira Samadi, Mohit Singh, Uthaipon Tantipongpipat, Santosh Vempala
Abstract	Dimensionality reduction is a classical technique widely used for data analysis. One foundational instantiation is Principal Component Analysis (PCA), which minimizes the average reconstruction error. In this paper, we introduce the “multi-criteria dimensionality reduction” problem where we are given multiple objectives that need to be optimized simultaneously. As an application, our model captures several fairness criteria for dimensionality reduction such as the Fair-PCA problem introduced by Samadi, et. al. 2018 and the Nash Social Welfare (NSW) problem. In the Fair-PCA problem, the input data is divided into $k$ groups, and the goal is to find a single d-dimensional representation for all groups for which the maximum reconstruction error of any one group is minimized. In NSW the goal is to maximize the product of the individual variances of the groups achieved by the common low-dimensinal space. Our main result is an exact polynomial-time algorithm for the two-criterion dimensionality reduction problem when the two criteria are increasing concave functions. As an application of this result, we obtain a polynomial time algorithm for Fair-PCA for $k=2$ groups, resolving an open problem of Samadi, et. al. 2018, and a polynomial time algorithm for NSW objective for $k=2$ groups. We also give approximation algorithms for $k>2$. Our technical contribution in the above results is to prove new low-rank properties of extreme point solutions to semi-definite programs. We conclude with experiments indicating the effectiveness of algorithms based on extreme point solutions of semi-definite programs on several real-world datasets.
Tasks	Dimensionality Reduction
Published	2019-02-28
URL	https://arxiv.org/abs/1902.11281v2
PDF	https://arxiv.org/pdf/1902.11281v2.pdf
PWC	https://paperswithcode.com/paper/fair-dimensionality-reduction-and-iterative
Repo
Framework

Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer


Title	Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer
Authors	Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, Pascale Fung
Abstract	Highly performing deep neural networks come at the cost of computational complexity that limits their practicality for deployment on portable devices. We propose the low-rank transformer (LRT), a memory-efficient and fast neural architecture that significantly reduces the parameters and boosts the speed of training and inference for end-to-end speech recognition. Our approach reduces the number of parameters of the network by more than 50% and speeds up the inference time by around 1.35x compared to the baseline transformer model. The experiments show that our LRT model generalizes better and yields lower error rates on both validation and test sets compared to an uncompressed transformer model. The LRT model outperforms those from existing works on several datasets in an end-to-end setting without using an external language model or acoustic data.
Tasks	End-To-End Speech Recognition, Language Modelling, Speech Recognition
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13923v3
PDF	https://arxiv.org/pdf/1910.13923v3.pdf
PWC	https://paperswithcode.com/paper/lightweight-and-efficient-end-to-end-speech
Repo
Framework

Tensor Variable Elimination for Plated Factor Graphs


Title	Tensor Variable Elimination for Plated Factor Graphs
Authors	Fritz Obermeyer, Eli Bingham, Martin Jankowiak, Justin Chiu, Neeraj Pradhan, Alexander Rush, Noah Goodman
Abstract	A wide class of machine learning algorithms can be reduced to variable elimination on factor graphs. While factor graphs provide a unifying notation for these algorithms, they do not provide a compact way to express repeated structure when compared to plate diagrams for directed graphical models. To exploit efficient tensor algebra in graphs with plates of variables, we generalize undirected factor graphs to plated factor graphs and variable elimination to a tensor variable elimination algorithm that operates directly on plated factor graphs. Moreover, we generalize complexity bounds based on treewidth and characterize the class of plated factor graphs for which inference is tractable. As an application, we integrate tensor variable elimination into the Pyro probabilistic programming language to enable exact inference in discrete latent variable models with repeated structure. We validate our methods with experiments on both directed and undirected graphical models, including applications to polyphonic music modeling, animal movement modeling, and latent sentiment analysis.
Tasks	Latent Variable Models, Music Modeling, Probabilistic Programming, Sentiment Analysis
Published	2019-02-08
URL	https://arxiv.org/abs/1902.03210v2
PDF	https://arxiv.org/pdf/1902.03210v2.pdf
PWC	https://paperswithcode.com/paper/tensor-variable-elimination-for-plated-factor
Repo
Framework

Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity


Title	Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
Authors	Ethan Manilow, Gordon Wichern, Prem Seetharaman, Jonathan Le Roux
Abstract	Music source separation performance has greatly improved in recent years with the advent of approaches based on deep learning. Such methods typically require large amounts of labelled training data, which in the case of music consist of mixtures and corresponding instrument stems. However, stems are unavailable for most commercial music, and only limited datasets have so far been released to the public. It can thus be difficult to draw conclusions when comparing various source separation methods, as the difference in performance may stem as much from better data augmentation techniques or training tricks to alleviate the limited availability of training data, as from intrinsically better model architectures and objective functions. In this paper, we present the synthesized Lakh dataset (Slakh) as a new tool for music source separation research. Slakh consists of high-quality renderings of instrumental mixtures and corresponding stems generated from the Lakh MIDI dataset (LMD) using professional-grade sample-based virtual instruments. A first version, Slakh2100, focuses on 2100 songs, resulting in 145 hours of mixtures. While not fully comparable because it is purely instrumental, this dataset contains an order of magnitude more data than MUSDB18, the {\it de facto} standard dataset in the field. We show that Slakh can be used to effectively augment existing datasets for musical instrument separation, while opening the door to a wide array of data-intensive music signal analysis tasks.
Tasks	Data Augmentation, Music Source Separation
Published	2019-09-18
URL	https://arxiv.org/abs/1909.08494v1
PDF	https://arxiv.org/pdf/1909.08494v1.pdf
PWC	https://paperswithcode.com/paper/cutting-music-source-separation-some-slakh-a
Repo
Framework

Global Thread-Level Inference for Comment Classification in Community Question Answering


Title	Global Thread-Level Inference for Comment Classification in Community Question Answering
Authors	Shafiq Joty, Alberto Barrón-Cedeño, Giovanni Da San Martino, Simone Filice, Lluís Màrquez, Alessandro Moschitti, Preslav Nakov
Abstract	Community question answering, a recent evolution of question answering in the Web context, allows a user to quickly consult the opinion of a number of people on a particular topic, thus taking advantage of the wisdom of the crowd. Here we try to help the user by deciding automatically which answers are good and which are bad for a given question. In particular, we focus on exploiting the output structure at the thread level in order to make more consistent global decisions. More specifically, we exploit the relations between pairs of comments at any distance in the thread, which we incorporate in a graph-cut and in an ILP frameworks. We evaluated our approach on the benchmark dataset of SemEval-2015 Task 3. Results improved over the state of the art, confirming the importance of using thread level information.
Tasks	Community Question Answering, Question Answering
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08755v1
PDF	https://arxiv.org/pdf/1911.08755v1.pdf
PWC	https://paperswithcode.com/paper/global-thread-level-inference-for-comment-1
Repo
Framework

Neural Duplicate Question Detection without Labeled Training Data


Title	Neural Duplicate Question Detection without Labeled Training Data
Authors	Andreas Rücklé, Nafise Sadat Moosavi, Iryna Gurevych
Abstract	Supervised training of neural models to duplicate question detection in community Question Answering (cQA) requires large amounts of labeled question pairs, which can be costly to obtain. To minimize this cost, recent works thus often used alternative methods, e.g., adversarial domain adaptation. In this work, we propose two novel methods—weak supervision using the title and body of a question, and the automatic generation of duplicate questions—and show that both can achieve improved performances even though they do not require any labeled data. We provide a comparison of popular training strategies and show that our proposed approaches are more effective in many cases because they can utilize larger amounts of data from the cQA forums. Finally, we show that weak supervision with question title and body information is also an effective method to train cQA answer selection models without direct answer supervision.
Tasks	Answer Selection, Community Question Answering, Domain Adaptation, Question Answering
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05594v1
PDF	https://arxiv.org/pdf/1911.05594v1.pdf
PWC	https://paperswithcode.com/paper/neural-duplicate-question-detection-without-1
Repo
Framework

Low-rank representations with incoherent dictionary for face recognition


Title	Low-rank representations with incoherent dictionary for face recognition
Authors	Pei Xie, He-Feng Yin, Xiao-Jun Wu
Abstract	Face recognition remains a hot topic in computer vision, and it is challenging to tackle the problem that both the training and testing images are corrupted. In this paper, we propose a novel semi-supervised method based on the theory of the low-rank matrix recovery for face recognition, which can simultaneously learn discriminative low-rank and sparse representations for both training and testing images. To this end, a correlation penalty term is introduced into the formulation of our proposed method to learn an incoherent dictionary. Experimental results on several face image databases demonstrate the effectiveness of our method, i.e., the proposed method is robust to the illumination, expression and pose variations, as well as images with noises such as block occlusion or uniform noises.
Tasks	Face Recognition
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04478v1
PDF	https://arxiv.org/pdf/1912.04478v1.pdf
PWC	https://paperswithcode.com/paper/low-rank-representations-with-incoherent
Repo
Framework

Conditional Teacher-Student Learning


Title	Conditional Teacher-Student Learning
Authors	Zhong Meng, Jinyu Li, Yong Zhao, Yifan Gong
Abstract	The teacher-student (T/S) learning has been shown to be effective for a variety of problems such as domain adaptation and model compression. One shortcoming of the T/S learning is that a teacher model, not always perfect, sporadically produces wrong guidance in form of posterior probabilities that misleads the student model towards a suboptimal performance. To overcome this problem, we propose a conditional T/S learning scheme, in which a “smart” student model selectively chooses to learn from either the teacher model or the ground truth labels conditioned on whether the teacher can correctly predict the ground truth. Unlike a naive linear combination of the two knowledge sources, the conditional learning is exclusively engaged with the teacher model when the teacher model’s prediction is correct, and otherwise backs off to the ground truth. Thus, the student model is able to learn effectively from the teacher and even potentially surpass the teacher. We examine the proposed learning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker adaptation on Microsoft short message dictation dataset. The proposed method achieves 9.8% and 12.8% relative word error rate reductions, respectively, over T/S learning for environment adaptation and speaker-independent model for speaker adaptation.
Tasks	Domain Adaptation, Model Compression
Published	2019-04-28
URL	http://arxiv.org/abs/1904.12399v1
PDF	http://arxiv.org/pdf/1904.12399v1.pdf
PWC	https://paperswithcode.com/paper/conditional-teacher-student-learning
Repo
Framework

Examining the Mapping Functions of Denoising Autoencoders in Singing Voice Separation


Title	Examining the Mapping Functions of Denoising Autoencoders in Singing Voice Separation
Authors	Stylianos Ioannis Mimilakis, Konstantinos Drossos, Estefanía Cano, Gerald Schuller
Abstract	The goal of this work is to investigate what singing voice separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks based on the denoising autoencoder (DAE) model that are conditioned on the mixture magnitude spectra. To approximate the mapping functions, we propose an algorithm inspired by the knowledge distillation, denoted the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA, we examine the mapping functions of three fundamental DAE-based models in music source separation; one with single-layer encoder and decoder, one with multi-layer encoder and single-layer decoder, and one using skip-filtering connections (SF) with a single-layer encoding and decoding. We first train these models with realistic data to estimate the singing voice magnitude spectra from the corresponding mixture. We then use the optimized models and test spectral data as input to the NCA. Our experimental findings show that approaches based on the DAE model learn scalar filtering operators, exhibiting a predominant diagonal structure in their corresponding mapping functions, limiting the exploitation of inter-frequency structure of music data. In contrast, skip-filtering connections are shown to assist the DAE model in learning filtering operators that exploit richer inter-frequency structures.
Tasks	Denoising, Music Source Separation
Published	2019-04-12
URL	https://arxiv.org/abs/1904.06157v2
PDF	https://arxiv.org/pdf/1904.06157v2.pdf
PWC	https://paperswithcode.com/paper/examining-the-mapping-functions-of-denoising
Repo
Framework

Interactively shaping robot behaviour with unlabeled human instructions


Title	Interactively shaping robot behaviour with unlabeled human instructions
Authors	Anis Najar, Olivier Sigaud, Mohamed Chetouani
Abstract	In this paper, we propose a framework that enables a human teacher to shape a robot behaviour by interactively providing it with unlabeled instructions. We ground the meaning of instruction signals in the task learning process, and use them simultaneously for guiding the latter. We implement our framework as a modular architecture, named TICS (Task-Instruction-Contingency-Shaping) that combines different information sources: a predefined reward function, human evaluative feedback and unlabeled instructions. This approach provides a novel perspective for robotic task learning that lies between Reinforcement Learning and Supervised Learning paradigms. We evaluate our framework both in simulation and with a real robot. The experimental results demonstrate the effectiveness of our framework in accelerating the task learning process and in reducing the amount of required teaching signals.
Tasks
Published	2019-02-05
URL	http://arxiv.org/abs/1902.01670v1
PDF	http://arxiv.org/pdf/1902.01670v1.pdf
PWC	https://paperswithcode.com/paper/interactively-shaping-robot-behaviour-with
Repo
Framework

Quantum Natural Gradient


Title	Quantum Natural Gradient
Authors	James Stokes, Josh Izaac, Nathan Killoran, Giuseppe Carleo
Abstract	A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction with respect to the Quantum Information Geometry, corresponding to the real part of the Quantum Geometric Tensor (QGT), also known as the Fubini-Study metric tensor. An efficient algorithm is presented for computing a block-diagonal approximation to the Fubini-Study metric tensor for parametrized quantum circuits, which may be of independent interest.
Tasks
Published	2019-09-04
URL	https://arxiv.org/abs/1909.02108v2
PDF	https://arxiv.org/pdf/1909.02108v2.pdf
PWC	https://paperswithcode.com/paper/quantum-natural-gradient
Repo
Framework