October 19, 2019

3290 words 16 mins read

Paper Group ANR 383

Learning Invariances using the Marginal Likelihood. Unsupervised RGBD Video Object Segmentation Using GANs. Duluth UROP at SemEval-2018 Task 2: Multilingual Emoji Prediction with Ensemble Learning and Oversampling. Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction. Neural Compositional De …

Learning Invariances using the Marginal Likelihood


Title	Learning Invariances using the Marginal Likelihood
Authors	Mark van der Wilk, Matthias Bauer, ST John, James Hensman
Abstract	Generalising well in supervised learning tasks relies on correctly extrapolating the training data to a large region of the input space. One way to achieve this is to constrain the predictions to be invariant to transformations on the input that are known to be irrelevant (e.g. translation). Commonly, this is done through data augmentation, where the training set is enlarged by applying hand-crafted transformations to the inputs. We argue that invariances should instead be incorporated in the model structure, and learned using the marginal likelihood, which correctly rewards the reduced complexity of invariant models. We demonstrate this for Gaussian process models, due to the ease with which their marginal likelihood can be estimated. Our main contribution is a variational inference scheme for Gaussian processes containing invariances described by a sampling procedure. We learn the sampling procedure by back-propagating through it to maximise the marginal likelihood.
Tasks	Data Augmentation, Gaussian Processes
Published	2018-08-16
URL	http://arxiv.org/abs/1808.05563v1
PDF	http://arxiv.org/pdf/1808.05563v1.pdf
PWC	https://paperswithcode.com/paper/learning-invariances-using-the-marginal
Repo
Framework

Unsupervised RGBD Video Object Segmentation Using GANs


Title	Unsupervised RGBD Video Object Segmentation Using GANs
Authors	Maryam Sultana, Arif Mahmood, Sajid Javed, Soon Ki Jung
Abstract	Video object segmentation is a fundamental step in many advanced vision applications. Most existing algorithms are based on handcrafted features such as HOG, super-pixel segmentation or texture-based techniques, while recently deep features have been found to be more efficient. Existing algorithms observe performance degradation in the presence of challenges such as illumination variations, shadows, and color camouflage. To handle these challenges we propose a fusion based moving object segmentation algorithm which exploits color as well as depth information using GAN to achieve more accuracy. Our goal is to segment moving objects in the presence of challenging background scenes, in real environments. To address this problem, GAN is trained in an unsupervised manner on color and depth information independently with challenging video sequences. During testing, the trained GAN generates backgrounds similar to that in the test sample. The generated background samples are then compared with the test sample to segment moving objects. The final result is computed by fusion of object boundaries in both modalities, RGB and the depth. The comparison of our proposed algorithm with five state-of-the-art methods on publicly available dataset has shown the strength of our algorithm for moving object segmentation in videos in the presence of challenging real scenarios.
Tasks	Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation
Published	2018-11-05
URL	http://arxiv.org/abs/1811.01526v1
PDF	http://arxiv.org/pdf/1811.01526v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-rgbd-video-object-segmentation
Repo
Framework

Duluth UROP at SemEval-2018 Task 2: Multilingual Emoji Prediction with Ensemble Learning and Oversampling


Title	Duluth UROP at SemEval-2018 Task 2: Multilingual Emoji Prediction with Ensemble Learning and Oversampling
Authors	Shuning Jin, Ted Pedersen
Abstract	This paper describes the Duluth UROP systems that participated in SemEval–2018 Task 2, Multilingual Emoji Prediction. We relied on a variety of ensembles made up of classifiers using Naive Bayes, Logistic Regression, and Random Forests. We used unigram and bigram features and tried to offset the skewness of the data through the use of oversampling. Our task evaluation results place us 19th of 48 systems in the English evaluation, and 5th of 21 in the Spanish. After the evaluation we realized that some simple changes to preprocessing could significantly improve our results. After making these changes we attained results that would have placed us sixth in the English evaluation, and second in the Spanish.
Tasks
Published	2018-05-25
URL	http://arxiv.org/abs/1805.10267v1
PDF	http://arxiv.org/pdf/1805.10267v1.pdf
PWC	https://paperswithcode.com/paper/duluth-urop-at-semeval-2018-task-2
Repo
Framework

Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction


Title	Combining Long Short Term Memory and Convolutional Neural Network for Cross-Sentence n-ary Relation Extraction
Authors	Angrosh Mandya, Danushka Bollegala, Frans Coenen, Katie Atkinson
Abstract	We propose in this paper a combined model of Long Short Term Memory and Convolutional Neural Networks (LSTM-CNN) that exploits word embeddings and positional embeddings for cross-sentence n-ary relation extraction. The proposed model brings together the properties of both LSTMs and CNNs, to simultaneously exploit long-range sequential information and capture most informative features, essential for cross-sentence n-ary relation extraction. The LSTM-CNN model is evaluated on standard dataset on cross-sentence n-ary relation extraction, where it significantly outperforms baselines such as CNNs, LSTMs and also a combined CNN-LSTM model. The paper also shows that the LSTM-CNN model outperforms the current state-of-the-art methods on cross-sentence n-ary relation extraction.
Tasks	Relation Extraction, Word Embeddings
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00845v1
PDF	http://arxiv.org/pdf/1811.00845v1.pdf
PWC	https://paperswithcode.com/paper/combining-long-short-term-memory-and
Repo
Framework

Neural Compositional Denotational Semantics for Question Answering


Title	Neural Compositional Denotational Semantics for Question Answering
Authors	Nitish Gupta, Mike Lewis
Abstract	Answering compositional questions requiring multi-step reasoning is challenging. We introduce an end-to-end differentiable model for interpreting questions about a knowledge graph (KG), which is inspired by formal approaches to semantics. Each span of text is represented by a denotation in a KG and a vector that captures ungrounded aspects of meaning. Learned composition modules recursively combine constituent spans, culminating in a grounding for the complete sentence which answers the question. For example, to interpret “not green”, the model represents “green” as a set of KG entities and “not” as a trainable ungrounded vector—and then uses this vector to parameterize a composition function that performs a complement operation. For each sentence, we build a parse chart subsuming all possible parses, allowing the model to jointly learn both the composition operators and output structure by gradient descent from end-task supervision. The model learns a variety of challenging semantic operators, such as quantifiers, disjunctions and composed relations, and infers latent syntactic structure. It also generalizes well to longer questions than seen in its training data, in contrast to RNN, its tree-based variants, and semantic parsing baselines.
Tasks	Question Answering, Semantic Parsing
Published	2018-08-29
URL	http://arxiv.org/abs/1808.09942v1
PDF	http://arxiv.org/pdf/1808.09942v1.pdf
PWC	https://paperswithcode.com/paper/neural-compositional-denotational-semantics
Repo
Framework

Probabilistic Model-Agnostic Meta-Learning


Title	Probabilistic Model-Agnostic Meta-Learning
Authors	Chelsea Finn, Kelvin Xu, Sergey Levine
Abstract	Meta-learning for few-shot learning entails acquiring a prior over previous tasks and experiences, such that new tasks be learned from small amounts of data. However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small dataset for a new task can simply be too ambiguous to acquire a single model (e.g., a classifier) for that task that is accurate. In this paper, we propose a probabilistic meta-learning algorithm that can sample models for a new task from a model distribution. Our approach extends model-agnostic meta-learning, which adapts to new tasks via gradient descent, to incorporate a parameter distribution that is trained via a variational lower bound. At meta-test time, our algorithm adapts via a simple procedure that injects noise into gradient descent, and at meta-training time, the model is trained such that this stochastic adaptation procedure produces samples from the approximate model posterior. Our experimental results show that our method can sample plausible classifiers and regressors in ambiguous few-shot learning problems. We also show how reasoning about ambiguity can also be used for downstream active learning problems.
Tasks	Active Learning, Few-Shot Image Classification, Few-Shot Learning, Meta-Learning
Published	2018-06-07
URL	https://arxiv.org/abs/1806.02817v2
PDF	https://arxiv.org/pdf/1806.02817v2.pdf
PWC	https://paperswithcode.com/paper/probabilistic-model-agnostic-meta-learning
Repo
Framework

Neural Mesh: Introducing a Notion of Space and Conservation of Energy to Neural Networks


Title	Neural Mesh: Introducing a Notion of Space and Conservation of Energy to Neural Networks
Authors	Jacob Beck, Zoe Papakipos
Abstract	Neural networks are based on a simplified model of the brain. In this project, we wanted to relax the simplifying assumptions of a traditional neural network by making a model that more closely emulates the low level interactions of neurons. Like in an RNN, our model has a state that persists between time steps, so that the energies of neurons persist. However, unlike an RNN, our state consists of a 2 dimensional matrix, rather than a 1 dimensional vector, thereby introducing a concept of distance to other neurons within the state. In our model, neurons can only fire to adjacent neurons, as in the brain. Like in the brain, we only allow neurons to fire in a time step if they contain enough energy, or excitement. We also enforce a notion of conservation of energy, so that a neuron cannot excite its neighbors more than the excitement it already contained at that time step. Taken together, these two features allow signals in the form of activations to flow around in our network over time, making our neural mesh more closely model signals traveling through the brain the brain. Although our main goal is to design an architecture to more closely emulate the brain in the hope of having a correct internal representation of information by the time we know how to properly train a general intelligence, we did benchmark our neural mash on a specific task. We found that by increasing the runtime of the mesh, we were able to increase its accuracy without increasing the number of parameters.
Tasks
Published	2018-07-29
URL	http://arxiv.org/abs/1807.11121v1
PDF	http://arxiv.org/pdf/1807.11121v1.pdf
PWC	https://paperswithcode.com/paper/neural-mesh-introducing-a-notion-of-space-and
Repo
Framework

Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences


Title	Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences
Authors	Nikolas Hesse, Sergi Pujades, Michael J. Black, Michael Arens, Ulrich G. Hofmann, A. Sebastian Schroeder
Abstract	Statistical models of the human body surface are generally learned from thousands of high-quality 3D scans in predefined poses to cover the wide variety of human body shapes and articulations. Acquisition of such data requires expensive equipment, calibration procedures, and is limited to cooperative subjects who can understand and follow instructions, such as adults. We present a method for learning a statistical 3D Skinned Multi-Infant Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely moving infants. Quantitative experiments show that SMIL faithfully represents the RGB-D data and properly factorizes the shape and pose of the infants. To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants. SMIL provides a new tool for analyzing infant shape and movement and is a step towards an automated system for GMA.
Tasks	Calibration
Published	2018-10-17
URL	http://arxiv.org/abs/1810.07538v1
PDF	http://arxiv.org/pdf/1810.07538v1.pdf
PWC	https://paperswithcode.com/paper/learning-and-tracking-the-3d-body-shape-of
Repo
Framework

Unsupervised Domain Adaptation with Adversarial Residual Transform Networks


Title	Unsupervised Domain Adaptation with Adversarial Residual Transform Networks
Authors	Guanyu Cai, Yuqin Wang, Mengchu Zhou, Lianghua He
Abstract	Domain adaptation is widely used in learning problems lacking labels. Recent studies show that deep adversarial domain adaptation models can make markable improvements in performance, which include symmetric and asymmetric architectures. However, the former has poor generalization ability whereas the latter is very hard to train. In this paper, we propose a novel adversarial domain adaptation method named Adversarial Residual Transform Networks (ARTNs) to improve the generalization ability, which directly transforms the source features into the space of target features. In this model, residual connections are used to share features and adversarial loss is reconstructed, thus making the model more generalized and easier to train. Moreover, a special regularization term is added to the loss function to alleviate a vanishing gradient problem, which enables its training process stable. A series of experiments based on Amazon review dataset, digits datasets and Office-31 image datasets are conducted to show that the proposed ARTN can be comparable with the methods of the state-of-the-art.
Tasks	Domain Adaptation, Unsupervised Domain Adaptation
Published	2018-04-25
URL	https://arxiv.org/abs/1804.09578v2
PDF	https://arxiv.org/pdf/1804.09578v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-domain-adaptation-with
Repo
Framework

Multiscale Sparse Microcanonical Models


Title	Multiscale Sparse Microcanonical Models
Authors	Joan Bruna, Stephane Mallat
Abstract	We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single realization. Asymptotic properties of maximum entropy microcanonical and macrocanonical processes and their convergence to Gibbs measures are reviewed. We show that the Jacobian of the energy vector controls the entropy rate of microcanonical processes. Sampling maximum entropy processes through MCMC algorithms require too many operations when the number of constraints is large. We define microcanonical gradient descent processes by transporting a maximum entropy measure with a gradient descent algorithm which enforces the energy conditions. Convergence and symmetries are analyzed. Approximations of non-Gaussian processes with long range interactions are defined with multiscale energy vectors computed with wavelet and scattering transforms. Sparsity properties are captured with $\bf l^1$ norms. Approximations of Gaussian, Ising and point processes are studied, as well as image and audio texture synthesis.
Tasks	Gaussian Processes, Point Processes, Texture Synthesis
Published	2018-01-06
URL	https://arxiv.org/abs/1801.02013v3
PDF	https://arxiv.org/pdf/1801.02013v3.pdf
PWC	https://paperswithcode.com/paper/multiscale-sparse-microcanonical-models
Repo
Framework

The Brain on Low Power Architectures - Efficient Simulation of Cortical Slow Waves and Asynchronous States


Title	The Brain on Low Power Architectures - Efficient Simulation of Cortical Slow Waves and Asynchronous States
Authors	Roberto Ammendola, Andrea Biagioni, Fabrizio Capuani, Paolo Cretaro, Giulia De Bonis, Francesca Lo Cicero, Alessandro Lonardo, Michele Martinelli, Pier Stanislao Paolucci, Elena Pastorelli, Luca Pontisso, Francesco Simula, Piero Vicini
Abstract	Efficient brain simulation is a scientific grand challenge, a parallel/distributed coding challenge and a source of requirements and suggestions for future computing architectures. Indeed, the human brain includes about 10^15 synapses and 10^11 neurons activated at a mean rate of several Hz. Full brain simulation poses Exascale challenges even if simulated at the highest abstraction level. The WaveScalES experiment in the Human Brain Project (HBP) has the goal of matching experimental measures and simulations of slow waves during deep-sleep and anesthesia and the transition to other brain states. The focus is the development of dedicated large-scale parallel/distributed simulation technologies. The ExaNeSt project designs an ARM-based, low-power HPC architecture scalable to million of cores, developing a dedicated scalable interconnect system, and SWA/AW simulations are included among the driving benchmarks. At the joint between both projects is the INFN proprietary Distributed and Plastic Spiking Neural Networks (DPSNN) simulation engine. DPSNN can be configured to stress either the networking or the computation features available on the execution platforms. The simulation stresses the networking component when the neural net - composed by a relatively low number of neurons, each one projecting thousands of synapses - is distributed over a large number of hardware cores. When growing the number of neurons per core, the computation starts to be the dominating component for short range connections. This paper reports about preliminary performance results obtained on an ARM-based HPC prototype developed in the framework of the ExaNeSt project. Furthermore, a comparison is given of instantaneous power, total energy consumption, execution time and energetic cost per synaptic event of SWA/AW DPSNN simulations when executed on either ARM- or Intel-based server platforms.
Tasks
Published	2018-04-10
URL	http://arxiv.org/abs/1804.03441v1
PDF	http://arxiv.org/pdf/1804.03441v1.pdf
PWC	https://paperswithcode.com/paper/the-brain-on-low-power-architectures
Repo
Framework

Artificial Immune Systems Can Find Arbitrarily Good Approximations for the NP-Hard Number Partitioning Problem


Title	Artificial Immune Systems Can Find Arbitrarily Good Approximations for the NP-Hard Number Partitioning Problem
Authors	Dogan Corus, Pietro S. Oliveto, Donya Yazdani
Abstract	Typical artificial immune system (AIS) operators such as hypermutations with mutation potential and ageing allow to efficiently overcome local optima from which evolutionary algorithms (EAs) struggle to escape. Such behaviour has been shown for artificial example functions constructed especially to show difficulties that EAs may encounter during the optimisation process. {\color{black}However, no evidence is available indicating that these two operators have similar behaviour also in more realistic problems.} In this paper we perform an analysis for the standard NP-hard \partition problem from combinatorial optimisation and rigorously show that hypermutations and ageing allow AISs to efficiently escape from local optima where standard EAs require exponential time. As a result we prove that while EAs and random local search (RLS) may get trapped on 4/3 approximations, AISs find arbitrarily good approximate solutions of ratio (1+$\epsilon$) {\color{black}within $n(\epsilon ^{-(2/\epsilon)-1})(1-\epsilon)^{-2} e^{3} 2^{2/\epsilon} + 2n^3 2^{2/\epsilon} + 2n^3$ function evaluations in expectation. This expectation is polynomial in the problem size and exponential only in $1/\epsilon$}.
Tasks
Published	2018-06-01
URL	http://arxiv.org/abs/1806.00300v2
PDF	http://arxiv.org/pdf/1806.00300v2.pdf
PWC	https://paperswithcode.com/paper/artificial-immune-systems-can-find
Repo
Framework

Demonstration of Topological Data Analysis on a Quantum Processor


Title	Demonstration of Topological Data Analysis on a Quantum Processor
Authors	He-Liang Huang, Xi-Lin Wang, Peter P. Rohde, Yi-Han Luo, You-Wei Zhao, Chang Liu, Li Li, Nai-Le Liu, Chao-Yang Lu, Jian-Wei Pan
Abstract	Topological data analysis offers a robust way to extract useful information from noisy, unstructured data by identifying its underlying structure. Recently, an efficient quantum algorithm was proposed [Lloyd, Garnerone, Zanardi, Nat. Commun. 7, 10138 (2016)] for calculating Betti numbers of data points – topological features that count the number of topological holes of various dimensions in a scatterplot. Here, we implement a proof-of-principle demonstration of this quantum algorithm by employing a six-photon quantum processor to successfully analyze the topological features of Betti numbers of a network including three data points, providing new insights into data analysis in the era of quantum computing.
Tasks	Topological Data Analysis
Published	2018-01-19
URL	https://arxiv.org/abs/1801.06316v2
PDF	https://arxiv.org/pdf/1801.06316v2.pdf
PWC	https://paperswithcode.com/paper/demonstration-of-topological-data-analysis-on
Repo
Framework

Preliminary Studies on a Large Face Database


Title	Preliminary Studies on a Large Face Database
Authors	Benjamin Yip, Garrett Bingham, Katherine Kempfert, Jonathan Fabish, Troy Kling, Cuixian Chen, Yishi Wang
Abstract	We perform preliminary studies on a large longitudinal face database MORPH-II, which is a benchmark dataset in the field of computer vision and pattern recognition. First, we summarize the inconsistencies in the dataset and introduce the steps and strategy taken for cleaning. The potential implications of these inconsistencies on prior research are introduced. Next, we propose a new automatic subsetting scheme for evaluation protocol. It is intended to overcome the unbalanced racial and gender distributions of MORPH-II, while ensuring independence between training and testing sets. Finally, we contribute a novel global framework for age estimation that utilizes posterior probabilities from the race classification step to compute a racecomposite age estimate. Preliminary experimental results on MORPH-II are presented.
Tasks	Age Estimation
Published	2018-11-15
URL	http://arxiv.org/abs/1811.06446v1
PDF	http://arxiv.org/pdf/1811.06446v1.pdf
PWC	https://paperswithcode.com/paper/preliminary-studies-on-a-large-face-database
Repo
Framework

Fast and accurate computation of orthogonal moments for texture analysis


Title	Fast and accurate computation of orthogonal moments for texture analysis
Authors	C. Di Ruberto, L. Putzu, G. Rodriguez
Abstract	In this work we describe a fast and stable algorithm for the computation of the orthogonal moments of an image. Indeed, orthogonal moments are characterized by a high discriminative power, but some of their possible formulations are characterized by a large computational complexity, which limits their real-time application. This paper describes in detail an approach based on recurrence relations, and proposes an optimized Matlab implementation of the corresponding computational procedure, aiming to solve the above limitations and put at the community’s disposal an efficient and easy to use software. In our experiments we evaluate the effectiveness of the recurrence formulation, as well as its performance for the reconstruction task, in comparison to the closed form representation, often used in the literature. The results show a sensible reduction in the computational complexity, together with a greater accuracy in reconstruction. In order to assess and compare the accuracy of the computed moments in texture analysis, we perform classification experiments on six well-known databases of texture images. Again, the recurrence formulation performs better in classification than the closed form representation. More importantly, if computed from the GLCM of the image using the proposed stable procedure, the orthogonal moments outperform in some situations some of the most diffused state-of-the-art descriptors for texture classification.
Tasks	Texture Classification
Published	2018-03-01
URL	http://arxiv.org/abs/1803.00638v2
PDF	http://arxiv.org/pdf/1803.00638v2.pdf
PWC	https://paperswithcode.com/paper/fast-and-accurate-computation-of-orthogonal
Repo
Framework