October 20, 2019

2961 words 14 mins read

Paper Group AWR 235

Learning the Reward Function for a Misspecified Model. Fix your classifier: the marginal value of training the last weight layer. Neural Best-Buddies: Sparse Cross-Domain Correspondence. Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations. Factorised spatial representation learning: application in semi-supervised …

Learning the Reward Function for a Misspecified Model


Title	Learning the Reward Function for a Misspecified Model
Authors	Erik Talvitie
Abstract	In model-based reinforcement learning it is typical to decouple the problems of learning the dynamics model and learning the reward function. However, when the dynamics model is flawed, it may generate erroneous states that would never occur in the true environment. It is not clear a priori what value the reward function should assign to such states. This paper presents a novel error bound that accounts for the reward model’s behavior in states sampled from the model. This bound is used to extend the existing Hallucinated DAgger-MC algorithm, which offers theoretical performance guarantees in deterministic MDPs that do not assume a perfect model can be learned. Empirically, this approach to reward learning can yield dramatic improvements in control performance when the dynamics model is flawed.
Tasks
Published	2018-01-29
URL	http://arxiv.org/abs/1801.09624v3
PDF	http://arxiv.org/pdf/1801.09624v3.pdf
PWC	https://paperswithcode.com/paper/learning-the-reward-function-for-a
Repo	https://github.com/etalvitie/hdaggermc
Framework	none

Fix your classifier: the marginal value of training the last weight layer


Title	Fix your classifier: the marginal value of training the last weight layer
Authors	Elad Hoffer, Itay Hubara, Daniel Soudry
Abstract	Neural networks are commonly used as models for classification for a wide variety of tasks. Typically, a learned affine transformation is placed at the end of such models, yielding a per-class value used for classification. This classifier can have a vast number of parameters, which grows linearly with the number of possible classes, thus requiring increasingly more resources. In this work we argue that this classifier can be fixed, up to a global scale constant, with little or no loss of accuracy for most tasks, allowing memory and computational benefits. Moreover, we show that by initializing the classifier with a Hadamard matrix we can speed up inference as well. We discuss the implications for current understanding of neural network models.
Tasks
Published	2018-01-14
URL	http://arxiv.org/abs/1801.04540v2
PDF	http://arxiv.org/pdf/1801.04540v2.pdf
PWC	https://paperswithcode.com/paper/fix-your-classifier-the-marginal-value-of
Repo	https://github.com/vaapopescu/gradient-pruning
Framework	pytorch

Neural Best-Buddies: Sparse Cross-Domain Correspondence


Title	Neural Best-Buddies: Sparse Cross-Domain Correspondence
Authors	Kfir Aberman, Jing Liao, Mingyi Shi, Dani Lischinski, Baoquan Chen, Daniel Cohen-Or
Abstract	Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications. This paper presents a novel method for sparse cross-domain correspondence. Our method is designed for pairs of images where the main objects of interest may belong to different semantic categories and differ drastically in shape and appearance, yet still contain semantically related or geometrically similar parts. Our approach operates on hierarchies of deep features, extracted from the input images by a pre-trained CNN. Specifically, starting from the coarsest layer in both hierarchies, we search for Neural Best Buddies (NBB): pairs of neurons that are mutual nearest neighbors. The key idea is then to percolate NBBs through the hierarchy, while narrowing down the search regions at each level and retaining only NBBs with significant activations. Furthermore, in order to overcome differences in appearance, each pair of search regions is transformed into a common appearance. We evaluate our method via a user study, in addition to comparisons with alternative correspondence approaches. The usefulness of our method is demonstrated using a variety of graphics applications, including cross-domain image alignment, creation of hybrid images, automatic image morphing, and more.
Tasks	Image Morphing
Published	2018-05-10
URL	http://arxiv.org/abs/1805.04140v2
PDF	http://arxiv.org/pdf/1805.04140v2.pdf
PWC	https://paperswithcode.com/paper/neural-best-buddies-sparse-cross-domain
Repo	https://github.com/kfiraberman/neural_best_buddies
Framework	pytorch

Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations


Title	Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations
Authors	Xingyu Wang, Diego Klabjan
Abstract	This paper considers the problem of inverse reinforcement learning in zero-sum stochastic games when expert demonstrations are known to be not optimal. Compared to previous works that decouple agents in the game by assuming optimality in expert strategies, we introduce a new objective function that directly pits experts against Nash Equilibrium strategies, and we design an algorithm to solve for the reward function in the context of inverse reinforcement learning with deep neural networks as model approximations. In our setting the model and algorithm do not decouple by agent. In order to find Nash Equilibrium in large-scale games, we also propose an adversarial training algorithm for zero-sum stochastic games, and show the theoretical appeal of non-existence of local optima in its objective function. In our numerical experiments, we demonstrate that our Nash Equilibrium and inverse reinforcement learning algorithms address games that are not amenable to previous approaches using tabular representations. Moreover, with sub-optimal expert demonstrations our algorithms recover both reward functions and strategies with good quality.
Tasks
Published	2018-01-07
URL	http://arxiv.org/abs/1801.02124v2
PDF	http://arxiv.org/pdf/1801.02124v2.pdf
PWC	https://paperswithcode.com/paper/competitive-multi-agent-inverse-reinforcement
Repo	https://github.com/mdabbah/pacman-rl-project
Framework	none

Factorised spatial representation learning: application in semi-supervised myocardial segmentation


Title	Factorised spatial representation learning: application in semi-supervised myocardial segmentation
Authors	Agisilaos Chartsias, Thomas Joyce, Giorgos Papanastasiou, Scott Semple, Michelle Williams, David Newby, Rohan Dharmakumar, Sotirios A. Tsaftaris
Abstract	The success and generalisation of deep learning algorithms heavily depend on learning good feature representations. In medical imaging this entails representing anatomical information, as well as properties related to the specific imaging setting. Anatomical information is required to perform further analysis, whereas imaging information is key to disentangle scanner variability and potential artefacts. The ability to factorise these would allow for training algorithms only on the relevant information according to the task. To date, such factorisation has not been attempted. In this paper, we propose a methodology of latent space factorisation relying on the cycle-consistency principle. As an example application, we consider cardiac MR segmentation, where we separate information related to the myocardium from other features related to imaging and surrounding substructures. We demonstrate the proposed method’s utility in a semi-supervised setting: we use very few labelled images together with many unlabelled images to train a myocardium segmentation neural network. Specifically, we achieve comparable performance to fully supervised networks using a fraction of labelled images in experiments on ACDC and a dataset from Edinburgh Imaging Facility QMRI. Code will be made available at https://github.com/agis85/spatial_factorisation.
Tasks	Medical Image Segmentation, Representation Learning
Published	2018-03-19
URL	http://arxiv.org/abs/1803.07031v2
PDF	http://arxiv.org/pdf/1803.07031v2.pdf
PWC	https://paperswithcode.com/paper/factorised-spatial-representation-learning
Repo	https://github.com/agis85/spatial_factorisation
Framework	none

A Dataset of Laryngeal Endoscopic Images with Comparative Study on Convolution Neural Network Based Semantic Segmentation


Title	A Dataset of Laryngeal Endoscopic Images with Comparative Study on Convolution Neural Network Based Semantic Segmentation
Authors	Max-Heinrich Laves, Jens Bicker, Lüder A. Kahrs, Tobias Ortmaier
Abstract	Purpose Automated segmentation of anatomical structures in medical image analysis is a prerequisite for autonomous diagnosis as well as various computer and robot aided interventions. Recent methods based on deep convolutional neural networks (CNN) have outperformed former heuristic methods. However, those methods were primarily evaluated on rigid, real-world environments. In this study, existing segmentation methods were evaluated for their use on a new dataset of transoral endoscopic exploration. Methods Four machine learning based methods SegNet, UNet, ENet and ErfNet were trained with supervision on a novel 7-class dataset of the human larynx. The dataset contains 536 manually segmented images from two patients during laser incisions. The Intersection-over-Union (IoU) evaluation metric was used to measure the accuracy of each method. Data augmentation and network ensembling were employed to increase segmentation accuracy. Stochastic inference was used to show uncertainties of the individual models. Patient-to-patient transfer was investigated using patient-specific fine-tuning. Results In this study, a weighted average ensemble network of UNet and ErfNet was best suited for the segmentation of laryngeal soft tissue with a mean IoU of 84.7 %. The highest efficiency was achieved by ENet with a mean inference time of 9.22 ms per image. It is shown that 10 additional images from a new patient are sufficient for patient-specific fine-tuning. Conclusion CNN-based methods for semantic segmentation are applicable to endoscopic images of laryngeal soft tissue. The segmentation can be used for active constraints or to monitor morphological changes and autonomously detect pathologies. Further improvements could be achieved by using a larger dataset or training the models in a self-supervised manner on additional unlabeled data.
Tasks	Data Augmentation, Medical Image Segmentation, Semantic Segmentation
Published	2018-07-16
URL	http://arxiv.org/abs/1807.06081v3
PDF	http://arxiv.org/pdf/1807.06081v3.pdf
PWC	https://paperswithcode.com/paper/a-dataset-of-laryngeal-endoscopic-images-with
Repo	https://github.com/imesluh/vocalfolds
Framework	none

Hard Non-Monotonic Attention for Character-Level Transduction


Title	Hard Non-Monotonic Attention for Character-Level Transduction
Authors	Shijie Wu, Pamela Shapiro, Ryan Cotterell
Abstract	Character-level string-to-string transduction is an important component of various NLP tasks. The goal is to map an input string to an output string, where the strings may be of different lengths and have characters taken from different alphabets. Recent approaches have used sequence-to-sequence models with an attention mechanism to learn which parts of the input string the model should focus on during the generation of the output string. Both soft attention and hard monotonic attention have been used, but hard non-monotonic attention has only been used in other sequence modeling tasks such as image captioning and has required a stochastic approximation to compute the gradient. In this work, we introduce an exact, polynomial-time algorithm for marginalizing over the exponential number of non-monotonic alignments between two strings, showing that hard attention models can be viewed as neural reparameterizations of the classical IBM Model 1. We compare soft and hard non-monotonic attention experimentally and find that the exact algorithm significantly improves performance over the stochastic approximation and outperforms soft attention.
Tasks	Image Captioning
Published	2018-08-29
URL	https://arxiv.org/abs/1808.10024v2
PDF	https://arxiv.org/pdf/1808.10024v2.pdf
PWC	https://paperswithcode.com/paper/hard-non-monotonic-attention-for-character
Repo	https://github.com/shijie-wu/neural-transducer
Framework	pytorch

TernausNetV2: Fully Convolutional Network for Instance Segmentation


Title	TernausNetV2: Fully Convolutional Network for Instance Segmentation
Authors	Vladimir I. Iglovikov, Selim Seferbekov, Alexander V. Buslaev, Alexey Shvets
Abstract	The most common approaches to instance segmentation are complex and use two-stage networks with object proposals, conditional random-fields, template matching or recurrent neural networks. In this work we present TernausNetV2 - a simple fully convolutional network that allows extracting objects from a high-resolution satellite imagery on an instance level. The network has popular encoder-decoder type of architecture with skip connections but has a few essential modifications that allows using for semantic as well as for instance segmentation tasks. This approach is universal and allows to extend any network that has been successfully applied for semantic segmentation to perform instance segmentation task. In addition, we generalize network encoder that was pre-trained for RGB images to use additional input channels. It makes possible to use transfer learning from visual to a wider spectral range. For DeepGlobe-CVPR 2018 building detection sub-challenge, based on public leaderboard score, our approach shows superior performance in comparison to other methods. The source code corresponding pre-trained weights are publicly available at https://github.com/ternaus/TernausNetV2
Tasks	Instance Segmentation, Semantic Segmentation, Transfer Learning
Published	2018-06-03
URL	http://arxiv.org/abs/1806.00844v2
PDF	http://arxiv.org/pdf/1806.00844v2.pdf
PWC	https://paperswithcode.com/paper/ternausnetv2-fully-convolutional-network-for
Repo	https://github.com/ternaus/TernausNetV2
Framework	pytorch

Reconfigurable Inverted Index


Title	Reconfigurable Inverted Index
Authors	Yusuke Matsui, Ryota Hinami, Shin’ichi Satoh
Abstract	Existing approximate nearest neighbor search systems suffer from two fundamental problems that are of practical importance but have not received sufficient attention from the research community. First, although existing systems perform well for the whole database, it is difficult to run a search over a subset of the database. Second, there has been no discussion concerning the performance decrement after many items have been newly added to a system. We develop a reconfigurable inverted index (Rii) to resolve these two issues. Based on the standard IVFADC system, we design a data layout such that items are stored linearly. This enables us to efficiently run a subset search by switching the search method to a linear PQ scan if the size of a subset is small. Owing to the linear layout, the data structure can be dynamically adjusted after new items are added, maintaining the fast speed of the system. Extensive comparisons show that Rii achieves a comparable performance with state-of-the art systems such as Faiss.
Tasks
Published	2018-08-12
URL	http://arxiv.org/abs/1808.03969v1
PDF	http://arxiv.org/pdf/1808.03969v1.pdf
PWC	https://paperswithcode.com/paper/reconfigurable-inverted-index
Repo	https://github.com/matsui528/rii
Framework	none

FIVR: Fine-grained Incident Video Retrieval


Title	FIVR: Fine-grained Incident Video Retrieval
Authors	Giorgos Kordopatis-Zilos, Symeon Papadopoulos, Ioannis Patras, Ioannis Kompatsiaris
Abstract	This paper introduces the problem of Fine-grained Incident Video Retrieval (FIVR). Given a query video, the objective is to retrieve all associated videos, considering several types of associations that range from duplicate videos to videos from the same incident. FIVR offers a single framework that contains several retrieval tasks as special cases. To address the benchmarking needs of all such tasks, we construct and present a large-scale annotated video dataset, which we call FIVR-200K, and it comprises 225,960 videos. To create the dataset, we devise a process for the collection of YouTube videos based on major news events from recent years crawled from Wikipedia and deploy a retrieval pipeline for the automatic selection of query videos based on their estimated suitability as benchmarks. We also devise a protocol for the annotation of the dataset with respect to the four types of video associations defined by FIVR. Finally, we report the results of an experimental study on the dataset comparing five state-of-the-art methods developed based on a variety of visual descriptors, highlighting the challenges of the current problem.
Tasks	Video Retrieval
Published	2018-09-11
URL	http://arxiv.org/abs/1809.04094v2
PDF	http://arxiv.org/pdf/1809.04094v2.pdf
PWC	https://paperswithcode.com/paper/fivr-fine-grained-incident-video-retrieval
Repo	https://github.com/MKLab-ITI/FIVR-200K
Framework	none

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network


Title	Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Authors	Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, Xi Zhou
Abstract	We propose a straightforward method that simultaneously reconstructs the 3D facial structure and provides dense alignment. To achieve this, we design a 2D representation called UV position map which records the 3D shape of a complete face in UV space, then train a simple Convolutional Neural Network to regress it from a single 2D image. We also integrate a weight mask into the loss function during training to improve the performance of the network. Our method does not rely on any prior face model, and can reconstruct full facial geometry along with semantic meaning. Meanwhile, our network is very light-weighted and spends only 9.8ms to process an image, which is extremely faster than previous works. Experiments on multiple challenging datasets show that our method surpasses other state-of-the-art methods on both reconstruction and alignment tasks by a large margin.
Tasks	3D Face Reconstruction, Face Alignment, Face Reconstruction
Published	2018-03-21
URL	http://arxiv.org/abs/1803.07835v1
PDF	http://arxiv.org/pdf/1803.07835v1.pdf
PWC	https://paperswithcode.com/paper/joint-3d-face-reconstruction-and-dense
Repo	https://github.com/YadiraF/PRNet
Framework	tf

Hashing with Binary Matrix Pursuit


Title	Hashing with Binary Matrix Pursuit
Authors	Fatih Cakir, Kun He, Stan Sclaroff
Abstract	We propose theoretical and empirical improvements for two-stage hashing methods. We first provide a theoretical analysis on the quality of the binary codes and show that, under mild assumptions, a residual learning scheme can construct binary codes that fit any neighborhood structure with arbitrary accuracy. Secondly, we show that with high-capacity hash functions such as CNNs, binary code inference can be greatly simplified for many standard neighborhood definitions, yielding smaller optimization problems and more robust codes. Incorporating our findings, we propose a novel two-stage hashing method that significantly outperforms previous hashing studies on widely used image retrieval benchmarks.
Tasks	Image Retrieval
Published	2018-08-06
URL	http://arxiv.org/abs/1808.01990v1
PDF	http://arxiv.org/pdf/1808.01990v1.pdf
PWC	https://paperswithcode.com/paper/hashing-with-binary-matrix-pursuit
Repo	https://github.com/fcakir/deep-mihash
Framework	none

BOCK : Bayesian Optimization with Cylindrical Kernels


Title	BOCK : Bayesian Optimization with Cylindrical Kernels
Authors	ChangYong Oh, Efstratios Gavves, Max Welling
Abstract	A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space. In this paper, we propose BOCK, Bayesian Optimization with Cylindrical Kernels, whose basic idea is to transform the ball geometry of the search space using a cylindrical transformation. Because of the transformed geometry, the Gaussian Process-based surrogate model spends less budget searching near the boundary, while concentrating its efforts relatively more near the center of the search region, where we expect the solution to be located. We evaluate BOCK extensively, showing that it is not only more accurate and efficient, but it also scales successfully to problems with a dimensionality as high as 500. We show that the better accuracy and scalability of BOCK even allows optimizing modestly sized neural network layers, as well as neural network hyperparameters.
Tasks
Published	2018-06-05
URL	https://arxiv.org/abs/1806.01619v2
PDF	https://arxiv.org/pdf/1806.01619v2.pdf
PWC	https://paperswithcode.com/paper/bock-bayesian-optimization-with-cylindrical
Repo	https://github.com/ChangYong-Oh/HyperSphere
Framework	pytorch

Watset: Local-Global Graph Clustering with Applications in Sense and Frame Induction


Title	Watset: Local-Global Graph Clustering with Applications in Sense and Frame Induction
Authors	Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
Abstract	We present a detailed theoretical and computational analysis of the Watset meta-algorithm for fuzzy graph clustering, which has been found to be widely applicable in a variety of domains. This algorithm creates an intermediate representation of the input graph that reflects the “ambiguity” of its nodes. Then, it uses hard clustering to discover clusters in this “disambiguated” intermediate graph. After outlining the approach and analyzing its computational complexity, we demonstrate that Watset shows competitive results in three applications: unsupervised synset induction from a synonymy graph, unsupervised semantic frame induction from dependency triples, and unsupervised semantic class induction from a distributional thesaurus. Our algorithm is generic and can be also applied to other networks of linguistic data.
Tasks	Graph Clustering
Published	2018-08-20
URL	https://arxiv.org/abs/1808.06696v4
PDF	https://arxiv.org/pdf/1808.06696v4.pdf
PWC	https://paperswithcode.com/paper/watset-local-global-graph-clustering-with
Repo	https://github.com/uhh-lt/triframes
Framework	pytorch

Ensemble Clustering for Graphs


Title	Ensemble Clustering for Graphs
Authors	Valérie Poulin, François Théberge
Abstract	We propose an ensemble clustering algorithm for graphs (ECG), which is based on the Louvain algorithm and the concept of consensus clustering. We validate our approach by replicating a recently published study comparing graph clustering algorithms over artificial networks, showing that ECG outperforms the leading algorithms from that study. We also illustrate how the ensemble obtained with ECG can be used to quantify the presence of community structure in the graph.
Tasks	Graph Clustering
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05578v1
PDF	http://arxiv.org/pdf/1809.05578v1.pdf
PWC	https://paperswithcode.com/paper/ensemble-clustering-for-graphs
Repo	https://github.com/ftheberge/Ensemble-Clustering-for-Graphs
Framework	none