July 30, 2019

2545 words 12 mins read

Paper Group AWR 44

Learning to solve inverse problems using Wasserstein loss. Boltzmann machines and energy-based models. Beetle Antennae Search without Parameter Tuning (BAS-WPT) for Multi-objective Optimization. Parametrization and generation of geological models with generative adversarial networks. A Fast and Robust TSVM for Pattern Classification. Formal Verific …

Learning to solve inverse problems using Wasserstein loss


Title	Learning to solve inverse problems using Wasserstein loss
Authors	Jonas Adler, Axel Ringh, Ozan Öktem, Johan Karlsson
Abstract	We propose using the Wasserstein loss for training in inverse problems. In particular, we consider a learned primal-dual reconstruction scheme for ill-posed inverse problems using the Wasserstein distance as loss function in the learning. This is motivated by miss-alignments in training data, which when using standard mean squared error loss could severely degrade reconstruction quality. We prove that training with the Wasserstein loss gives a reconstruction operator that correctly compensates for miss-alignments in certain cases, whereas training with the mean squared error gives a smeared reconstruction. Moreover, we demonstrate these effects by training a reconstruction algorithm using both mean squared error and optimal transport loss for a problem in computerized tomography.
Tasks
Published	2017-10-30
URL	http://arxiv.org/abs/1710.10898v1
PDF	http://arxiv.org/pdf/1710.10898v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-solve-inverse-problems-using
Repo	https://github.com/odlgroup/odl
Framework	none

Boltzmann machines and energy-based models


Title	Boltzmann machines and energy-based models
Authors	Takayuki Osogami
Abstract	We review Boltzmann machines and energy-based models. A Boltzmann machine defines a probability distribution over binary-valued patterns. One can learn parameters of a Boltzmann machine via gradient based approaches in a way that log likelihood of data is increased. The gradient and Hessian of a Boltzmann machine admit beautiful mathematical representations, although computing them is in general intractable. This intractability motivates approximate methods, including Gibbs sampler and contrastive divergence, and tractable alternatives, namely energy-based models.
Tasks
Published	2017-08-20
URL	http://arxiv.org/abs/1708.06008v2
PDF	http://arxiv.org/pdf/1708.06008v2.pdf
PWC	https://paperswithcode.com/paper/boltzmann-machines-and-energy-based-models
Repo	https://github.com/Kevin-Sean-Chen/Restriced_Boltzmann_Machine
Framework	none

Beetle Antennae Search without Parameter Tuning (BAS-WPT) for Multi-objective Optimization


Title	Beetle Antennae Search without Parameter Tuning (BAS-WPT) for Multi-objective Optimization
Authors	Xiangyuan Jiang, Shuai Li
Abstract	Beetle antennae search (BAS) is an efficient meta-heuristic algorithm inspired by foraging behaviors of beetles. This algorithm includes several parameters for tuning and the existing results are limited to solve single objective optimization. This work pushes forward the research on BAS by providing one variant that releases the tuning parameters and is able to handle multi-objective optimization. This new approach applies normalization to simplify the original algorithm and uses a penalty function to exploit infeasible solutions with low constraint violation to solve the constraint optimization problem. Extensive experimental studies are carried out and the results reveal efficacy of the proposed approach to constraint handling.
Tasks
Published	2017-11-07
URL	http://arxiv.org/abs/1711.02395v1
PDF	http://arxiv.org/pdf/1711.02395v1.pdf
PWC	https://paperswithcode.com/paper/beetle-antennae-search-without-parameter
Repo	https://github.com/jywang2016/rBAS
Framework	none

Parametrization and generation of geological models with generative adversarial networks


Title	Parametrization and generation of geological models with generative adversarial networks
Authors	Shing Chan, Ahmed H. Elsheikh
Abstract	One of the main challenges in the parametrization of geological models is the ability to capture complex geological structures often observed in the subsurface. In recent years, generative adversarial networks (GAN) were proposed as an efficient method for the generation and parametrization of complex data, showing state-of-the-art performances in challenging computer vision tasks such as reproducing natural images (handwritten digits, human faces, etc.). In this work, we study the application of Wasserstein GAN for the parametrization of geological models. The effectiveness of the method is assessed for uncertainty propagation tasks using several test cases involving different permeability patterns and subsurface flow problems. Results show that GANs are able to generate samples that preserve the multipoint statistical features of the geological models both visually and quantitatively. The generated samples reproduce both the geological structures and the flow statistics of the reference geology.
Tasks
Published	2017-08-05
URL	http://arxiv.org/abs/1708.01810v2
PDF	http://arxiv.org/pdf/1708.01810v2.pdf
PWC	https://paperswithcode.com/paper/parametrization-and-generation-of-geological
Repo	https://github.com/akshaysubr/TEGAN
Framework	tf

A Fast and Robust TSVM for Pattern Classification


Title	A Fast and Robust TSVM for Pattern Classification
Authors	Bin-Bin Gao, Jian-Jun Wang
Abstract	Twin support vector machine~(TSVM) is a powerful learning algorithm by solving a pair of smaller SVM-type problems. However, there are still some specific issues such as low efficiency and weak robustness when it is faced with some real applications. In this paper, we propose a Fast and Robust TSVM~(FR-TSVM) to deal with the above issues. In order to alleviate the effects of noisy inputs, we propose an effective fuzzy membership function and reformulate the TSVMs such that different input instances can make different contributions to the learning of the separating hyperplanes. To further speed up the training procedure, we develop an efficient coordinate descent algorithm with shirking to solve the involved a pair of quadratic programming problems (QPPs). Moreover, theoretical foundations of the proposed model are analyzed in details. The experimental results on several artificial and benchmark datasets indicate that the FR-TSVM not only obtains a fast learning speed but also shows a robust classification performance. Code has been made available at: https://github.com/gaobb/FR-TSVM.
Tasks
Published	2017-11-15
URL	https://arxiv.org/abs/1711.05406v3
PDF	https://arxiv.org/pdf/1711.05406v3.pdf
PWC	https://paperswithcode.com/paper/a-fast-and-robust-tsvm-for-pattern
Repo	https://github.com/gaobb/FR-TSVM
Framework	none

Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks


Title	Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks
Authors	Ruediger Ehlers
Abstract	We present an approach for the verification of feed-forward neural networks in which all nodes have a piece-wise linear activation function. Such networks are often used in deep learning and have been shown to be hard to verify for modern satisfiability modulo theory (SMT) and integer linear programming (ILP) solvers. The starting point of our approach is the addition of a global linear approximation of the overall network behavior to the verification problem that helps with SMT-like reasoning over the network behavior. We present a specialized verification algorithm that employs this approximation in a search process in which it infers additional node phases for the non-linear nodes in the network from partial node phase assignments, similar to unit propagation in classical SAT solving. We also show how to infer additional conflict clauses and safe node fixtures from the results of the analysis steps performed during the search. The resulting approach is evaluated on collision avoidance and handwritten digit recognition case studies.
Tasks	Handwritten Digit Recognition
Published	2017-05-03
URL	http://arxiv.org/abs/1705.01320v3
PDF	http://arxiv.org/pdf/1705.01320v3.pdf
PWC	https://paperswithcode.com/paper/formal-verification-of-piece-wise-linear-feed
Repo	https://github.com/progirep/planet
Framework	tf

Dataflow Matrix Machines as a Model of Computations with Linear Streams


Title	Dataflow Matrix Machines as a Model of Computations with Linear Streams
Authors	Michael Bukatin, Jon Anthony
Abstract	We overview dataflow matrix machines as a Turing complete generalization of recurrent neural networks and as a programming platform. We describe vector space of finite prefix trees with numerical leaves which allows us to combine expressive power of dataflow matrix machines with simplicity of traditional recurrent neural networks.
Tasks
Published	2017-05-03
URL	http://arxiv.org/abs/1706.00648v1
PDF	http://arxiv.org/pdf/1706.00648v1.pdf
PWC	https://paperswithcode.com/paper/dataflow-matrix-machines-as-a-model-of
Repo	https://github.com/jsa-aerial/DMM
Framework	none

Learning a Multi-View Stereo Machine


Title	Learning a Multi-View Stereo Machine
Authors	Abhishek Kar, Christian Häne, Jitendra Malik
Abstract	We present a learnt system for multi-view stereopsis. In contrast to recent learning based methods for 3D reconstruction, we leverage the underlying 3D geometry of the problem through feature projection and unprojection along viewing rays. By formulating these operations in a differentiable manner, we are able to learn the system end-to-end for the task of metric 3D reconstruction. End-to-end learning allows us to jointly reason about shape priors while conforming geometric constraints, enabling reconstruction from much fewer images (even a single image) than required by classical approaches as well as completion of unseen surfaces. We thoroughly evaluate our approach on the ShapeNet dataset and demonstrate the benefits over classical approaches as well as recent learning based methods.
Tasks	3D Reconstruction
Published	2017-08-17
URL	http://arxiv.org/abs/1708.05375v1
PDF	http://arxiv.org/pdf/1708.05375v1.pdf
PWC	https://paperswithcode.com/paper/learning-a-multi-view-stereo-machine
Repo	https://github.com/akar43/lsm
Framework	tf

Contrastive Principal Component Analysis


Title	Contrastive Principal Component Analysis
Authors	Abubakar Abid, Martin J. Zhang, Vivek K. Bagaria, James Zou
Abstract	We present a new technique called contrastive principal component analysis (cPCA) that is designed to discover low-dimensional structure that is unique to a dataset, or enriched in one dataset relative to other data. The technique is a generalization of standard PCA, for the setting where multiple datasets are available – e.g. a treatment and a control group, or a mixed versus a homogeneous population – and the goal is to explore patterns that are specific to one of the datasets. We conduct a wide variety of experiments in which cPCA identifies important dataset-specific patterns that are missed by PCA, demonstrating that it is useful for many applications: subgroup discovery, visualizing trends, feature selection, denoising, and data-dependent standardization. We provide geometrical interpretations of cPCA and show that it satisfies desirable theoretical guarantees. We also extend cPCA to nonlinear settings in the form of kernel cPCA. We have released our code as a python package and documentation is on Github.
Tasks	Denoising, Feature Selection
Published	2017-09-20
URL	http://arxiv.org/abs/1709.06716v2
PDF	http://arxiv.org/pdf/1709.06716v2.pdf
PWC	https://paperswithcode.com/paper/contrastive-principal-component-analysis
Repo	https://github.com/abidlabs/contrastive
Framework	none

A Mutually-Dependent Hadamard Kernel for Modelling Latent Variable Couplings


Title	A Mutually-Dependent Hadamard Kernel for Modelling Latent Variable Couplings
Authors	Sami Remes, Markus Heinonen, Samuel Kaski
Abstract	We introduce a novel kernel that models input-dependent couplings across multiple latent processes. The pairwise joint kernel measures covariance along inputs and across different latent signals in a mutually-dependent fashion. A latent correlation Gaussian process (LCGP) model combines these non-stationary latent components into multiple outputs by an input-dependent mixing matrix. Probit classification and support for multiple observation sets are derived by Variational Bayesian inference. Results on several datasets indicate that the LCGP model can recover the correlations between latent signals while simultaneously achieving state-of-the-art performance. We highlight the latent covariances with an EEG classification dataset where latent brain processes and their couplings simultaneously emerge from the model.
Tasks	Bayesian Inference, EEG
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08402v2
PDF	http://arxiv.org/pdf/1702.08402v2.pdf
PWC	https://paperswithcode.com/paper/a-mutually-dependent-hadamard-kernel-for
Repo	https://github.com/sremes/wishart-gibbs-kernel
Framework	none

Feature Generating Networks for Zero-Shot Learning


Title	Feature Generating Networks for Zero-Shot Learning
Authors	Yongqin Xian, Tobias Lorenz, Bernt Schiele, Zeynep Akata
Abstract	Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task. To circumvent the need for labeled examples of unseen classes, we propose a novel generative adversarial network (GAN) that synthesizes CNN features conditioned on class-level semantic information, offering a shortcut directly from a semantic descriptor of a class to a class-conditional feature distribution. Our proposed approach, pairing a Wasserstein GAN with a classification loss, is able to generate sufficiently discriminative CNN features to train softmax classifiers or any multimodal embedding method. Our experimental results demonstrate a significant boost in accuracy over the state of the art on five challenging datasets – CUB, FLO, SUN, AWA and ImageNet – in both the zero-shot learning and generalized zero-shot learning settings.
Tasks	Zero-Shot Learning
Published	2017-12-04
URL	http://arxiv.org/abs/1712.00981v2
PDF	http://arxiv.org/pdf/1712.00981v2.pdf
PWC	https://paperswithcode.com/paper/feature-generating-networks-for-zero-shot
Repo	https://github.com/akku1506/Feature-Generating-Networks-for-ZSL
Framework	tf

Composite Quantization


Title	Composite Quantization
Authors	Jingdong Wang, Ting Zhang
Abstract	This paper studies the compact coding approach to approximate nearest neighbor search. We introduce a composite quantization framework. It uses the composition of several ($M$) elements, each of which is selected from a different dictionary, to accurately approximate a $D$-dimensional vector, thus yielding accurate search, and represents the data vector by a short code composed of the indices of the selected elements in the corresponding dictionaries. Our key contribution lies in introducing a near-orthogonality constraint, which makes the search efficiency is guaranteed as the cost of the distance computation is reduced to $O(M)$ from $O(D)$ through a distance table lookup scheme. The resulting approach is called near-orthogonal composite quantization. We theoretically justify the equivalence between near-orthogonal composite quantization and minimizing an upper bound of a function formed by jointly considering the quantization error and the search cost according to a generalized triangle inequality. We empirically show the efficacy of the proposed approach over several benchmark datasets. In addition, we demonstrate the superior performances in other three applications: combination with inverted multi-index, quantizing the query for mobile search, and inner-product similarity search.
Tasks	Quantization
Published	2017-12-04
URL	http://arxiv.org/abs/1712.00955v1
PDF	http://arxiv.org/pdf/1712.00955v1.pdf
PWC	https://paperswithcode.com/paper/composite-quantization
Repo	https://github.com/una-dinosauria/Rayuela.jl
Framework	none

Joint Distribution Optimal Transportation for Domain Adaptation


Title	Joint Distribution Optimal Transportation for Domain Adaptation
Authors	Nicolas Courty, Rémi Flamary, Amaury Habrard, Alain Rakotomamonjy
Abstract	This paper deals with the unsupervised domain adaptation problem, where one wants to estimate a prediction function $f$ in a given target domain without any labeled sample by exploiting the knowledge available from a source domain where labels are known. Our work makes the following assumption: there exists a non-linear transformation between the joint feature/label space distributions of the two domain $\mathcal{P}_s$ and $\mathcal{P}_t$. We propose a solution of this problem with optimal transport, that allows to recover an estimated target $\mathcal{P}^f_t=(X,f(X))$ by optimizing simultaneously the optimal coupling and $f$. We show that our method corresponds to the minimization of a bound on the target error, and provide an efficient algorithmic solution, for which convergence is proved. The versatility of our approach, both in terms of class of hypothesis or loss functions is demonstrated with real world classification and regression problems, for which we reach or surpass state-of-the-art results.
Tasks	Domain Adaptation, Unsupervised Domain Adaptation
Published	2017-05-24
URL	http://arxiv.org/abs/1705.08848v2
PDF	http://arxiv.org/pdf/1705.08848v2.pdf
PWC	https://paperswithcode.com/paper/joint-distribution-optimal-transportation-for
Repo	https://github.com/rflamary/JDOT
Framework	none

A Deep Multimodal Approach for Cold-start Music Recommendation


Title	A Deep Multimodal Approach for Cold-start Music Recommendation
Authors	Sergio Oramas, Oriol Nieto, Mohamed Sordo, Xavier Serra
Abstract	An increasing amount of digital music is being published daily. Music streaming services often ingest all available music, but this poses a challenge: how to recommend new artists for which prior knowledge is scarce? In this work we aim to address this so-called cold-start problem by combining text and audio information with user feedback data using deep network architectures. Our method is divided into three steps. First, artist embeddings are learned from biographies by combining semantics, text features, and aggregated usage data. Second, track embeddings are learned from the audio signal and available feedback data. Finally, artist and track embeddings are combined in a multimodal network. Results suggest that both splitting the recommendation problem between feature levels (i.e., artist metadata and audio track), and merging feature embeddings in a multimodal approach improve the accuracy of the recommendations.
Tasks
Published	2017-06-29
URL	http://arxiv.org/abs/1706.09739v2
PDF	http://arxiv.org/pdf/1706.09739v2.pdf
PWC	https://paperswithcode.com/paper/a-deep-multimodal-approach-for-cold-start
Repo	https://github.com/sergiooramas/tartarus
Framework	tf

SkipNet: Learning Dynamic Routing in Convolutional Networks


Title	SkipNet: Learning Dynamic Routing in Convolutional Networks
Authors	Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, Joseph E. Gonzalez
Abstract	While deeper convolutional networks are needed to achieve maximum accuracy in visual perception tasks, for many inputs shallower networks are sufficient. We exploit this observation by learning to skip convolutional layers on a per-input basis. We introduce SkipNet, a modified residual network, that uses a gating network to selectively skip convolutional blocks based on the activations of the previous layer. We formulate the dynamic skipping problem in the context of sequential decision making and propose a hybrid learning algorithm that combines supervised learning and reinforcement learning to address the challenges of non-differentiable skipping decisions. We show SkipNet reduces computation by 30-90% while preserving the accuracy of the original model on four benchmark datasets and outperforms the state-of-the-art dynamic networks and static compression methods. We also qualitatively evaluate the gating policy to reveal a relationship between image scale and saliency and the number of layers skipped.
Tasks	Decision Making
Published	2017-11-26
URL	http://arxiv.org/abs/1711.09485v2
PDF	http://arxiv.org/pdf/1711.09485v2.pdf
PWC	https://paperswithcode.com/paper/skipnet-learning-dynamic-routing-in
Repo	https://github.com/geekJZY/arcticnet
Framework	pytorch