July 29, 2019

2853 words 14 mins read

Paper Group ANR 102

Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification. Ontology Based Pivoted normalization using Vector Based Approach for information Retrieval. A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks. Multi-step Off-policy Learning Without Importance Sampling Ratios. Wha …

Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification


Title	Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification
Authors	Jonggu Kim, Jong-Hyeok Lee
Abstract	Most of neural approaches to relation classification have focused on finding short patterns that represent the semantic relation using Convolutional Neural Networks (CNNs) and those approaches have generally achieved better performances than using Recurrent Neural Networks (RNNs). In a similar intuition to the CNN models, we propose a novel RNN-based model that strongly focuses on only important parts of a sentence using multiple range-restricted bidirectional layers and attention for relation classification. Experimental results on the SemEval-2010 relation classification task show that our model is comparable to the state-of-the-art CNN-based and RNN-based models that use additional linguistic information.
Tasks	Relation Classification
Published	2017-07-05
URL	http://arxiv.org/abs/1707.01265v2
PDF	http://arxiv.org/pdf/1707.01265v2.pdf
PWC	https://paperswithcode.com/paper/multiple-range-restricted-bidirectional-gated
Repo
Framework

Ontology Based Pivoted normalization using Vector Based Approach for information Retrieval


Title	Ontology Based Pivoted normalization using Vector Based Approach for information Retrieval
Authors	Vishal Jain, Dr. Mayank Singh
Abstract	The proposed methodology is procedural i.e. it follows finite number of steps that extracts relevant documents according to users query. It is based on principles of Data Mining for analyzing web data. Data Mining first adapts integration of data to generate warehouse. Then, it extracts useful information with the help of algorithm. The task of representing extracted documents is done by using Vector Based Statistical Approach that represents each document in set of Terms.
Tasks	Information Retrieval
Published	2017-03-21
URL	http://arxiv.org/abs/1703.07384v1
PDF	http://arxiv.org/pdf/1703.07384v1.pdf
PWC	https://paperswithcode.com/paper/ontology-based-pivoted-normalization-using
Repo
Framework

A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks


Title	A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks
Authors	Xiao Li, Yao Ma, Calin Belta
Abstract	Reward engineering is an important aspect of reinforcement learning. Whether or not the user’s intentions can be correctly encapsulated in the reward function can significantly impact the learning outcome. Current methods rely on manually crafted reward functions that often require parameter tuning to obtain the desired behavior. This operation can be expensive when exploration requires systems to interact with the physical world. In this paper, we explore the use of temporal logic (TL) to specify tasks in reinforcement learning. TL formula can be translated to a real-valued function that measures its level of satisfaction against a trajectory. We take advantage of this function and propose temporal logic policy search (TLPS), a model-free learning technique that finds a policy that satisfies the TL specification. A set of simulated experiments are conducted to evaluate the proposed approach.
Tasks
Published	2017-09-27
URL	http://arxiv.org/abs/1709.09611v1
PDF	http://arxiv.org/pdf/1709.09611v1.pdf
PWC	https://paperswithcode.com/paper/a-policy-search-method-for-temporal-logic
Repo
Framework

Multi-step Off-policy Learning Without Importance Sampling Ratios


Title	Multi-step Off-policy Learning Without Importance Sampling Ratios
Authors	Ashique Rupam Mahmood, Huizhen Yu, Richard S. Sutton
Abstract	To estimate the value functions of policies from exploratory data, most model-free off-policy algorithms rely on importance sampling, where the use of importance sampling ratios often leads to estimates with severe variance. It is thus desirable to learn off-policy without using the ratios. However, such an algorithm does not exist for multi-step learning with function approximation. In this paper, we introduce the first such algorithm based on temporal-difference (TD) learning updates. We show that an explicit use of importance sampling ratios can be eliminated by varying the amount of bootstrapping in TD updates in an action-dependent manner. Our new algorithm achieves stability using a two-timescale gradient-based TD update. A prior algorithm based on lookup table representation called Tree Backup can also be retrieved using action-dependent bootstrapping, becoming a special case of our algorithm. In two challenging off-policy tasks, we demonstrate that our algorithm is stable, effectively avoids the large variance issue, and can perform substantially better than its state-of-the-art counterpart.
Tasks
Published	2017-02-09
URL	http://arxiv.org/abs/1702.03006v1
PDF	http://arxiv.org/pdf/1702.03006v1.pdf
PWC	https://paperswithcode.com/paper/multi-step-off-policy-learning-without
Repo
Framework

What Makes a Place? Building Bespoke Place Dependent Object Detectors for Robotics


Title	What Makes a Place? Building Bespoke Place Dependent Object Detectors for Robotics
Authors	Jeffrey Hawke, Alex Bewley, Ingmar Posner
Abstract	This paper is about enabling robots to improve their perceptual performance through repeated use in their operating environment, creating local expert detectors fitted to the places through which a robot moves. We leverage the concept of ‘experiences’ in visual perception for robotics, accounting for bias in the data a robot sees by fitting object detector models to a particular place. The key question we seek to answer in this paper is simply: how do we define a place? We build bespoke pedestrian detector models for autonomous driving, highlighting the necessary trade off between generalisation and model capacity as we vary the extent of the place we fit to. We demonstrate a sizeable performance gain over a current state-of-the-art detector when using computationally lightweight bespoke place-fitted detector models.
Tasks	Autonomous Driving
Published	2017-08-07
URL	http://arxiv.org/abs/1708.02330v1
PDF	http://arxiv.org/pdf/1708.02330v1.pdf
PWC	https://paperswithcode.com/paper/what-makes-a-place-building-bespoke-place
Repo
Framework

Transitive Invariance for Self-supervised Visual Representation Learning


Title	Transitive Invariance for Self-supervised Visual Representation Learning
Authors	Xiaolong Wang, Kaiming He, Abhinav Gupta
Abstract	Learning visual representations with self-supervised learning has become popular in computer vision. The idea is to design auxiliary tasks where labels are free to obtain. Most of these tasks end up providing data to learn specific kinds of invariance useful for recognition. In this paper, we propose to exploit different self-supervised approaches to learn representations invariant to (i) inter-instance variations (two objects in the same class should have similar features) and (ii) intra-instance variations (viewpoint, pose, deformations, illumination, etc). Instead of combining two approaches with multi-task learning, we argue to organize and reason the data with multiple variations. Specifically, we propose to generate a graph with millions of objects mined from hundreds of thousands of videos. The objects are connected by two types of edges which correspond to two types of invariance: “different instances but a similar viewpoint and category” and “different viewpoints of the same instance”. By applying simple transitivity on the graph with these edges, we can obtain pairs of images exhibiting richer visual invariance. We use this data to train a Triplet-Siamese network with VGG16 as the base architecture and apply the learned representations to different recognition tasks. For object detection, we achieve 63.2% mAP on PASCAL VOC 2007 using Fast R-CNN (compare to 67.3% with ImageNet pre-training). For the challenging COCO dataset, our method is surprisingly close (23.5%) to the ImageNet-supervised counterpart (24.4%) using the Faster R-CNN framework. We also show that our network can perform significantly better than the ImageNet network in the surface normal estimation task.
Tasks	Multi-Task Learning, Object Detection, Representation Learning
Published	2017-08-09
URL	http://arxiv.org/abs/1708.02901v3
PDF	http://arxiv.org/pdf/1708.02901v3.pdf
PWC	https://paperswithcode.com/paper/transitive-invariance-for-self-supervised
Repo
Framework

CANDiS: Coupled & Attention-Driven Neural Distant Supervision


Title	CANDiS: Coupled & Attention-Driven Neural Distant Supervision
Authors	Tushar Nagarajan, Sharmistha, Partha Talukdar
Abstract	Distant Supervision for Relation Extraction uses heuristically aligned text data with an existing knowledge base as training data. The unsupervised nature of this technique allows it to scale to web-scale relation extraction tasks, at the expense of noise in the training data. Previous work has explored relationships among instances of the same entity-pair to reduce this noise, but relationships among instances across entity-pairs have not been fully exploited. We explore the use of inter-instance couplings based on verb-phrase and entity type similarities. We propose a novel technique, CANDiS, which casts distant supervision using inter-instance coupling into an end-to-end neural network model. CANDiS incorporates an attention module at the instance-level to model the multi-instance nature of this problem. CANDiS outperforms existing state-of-the-art techniques on a standard benchmark dataset.
Tasks	Relation Extraction
Published	2017-10-26
URL	http://arxiv.org/abs/1710.09942v1
PDF	http://arxiv.org/pdf/1710.09942v1.pdf
PWC	https://paperswithcode.com/paper/candis-coupled-attention-driven-neural
Repo
Framework

Does robustness imply tractability? A lower bound for planted clique in the semi-random model


Title	Does robustness imply tractability? A lower bound for planted clique in the semi-random model
Authors	Jacob Steinhardt
Abstract	We consider a robust analog of the planted clique problem. In this analog, a set $S$ of vertices is chosen and all edges in $S$ are included; then, edges between $S$ and the rest of the graph are included with probability $\frac{1}{2}$, while edges not touching $S$ are allowed to vary arbitrarily. For this semi-random model, we show that the information-theoretic threshold for recovery is $\tilde{\Theta}(\sqrt{n})$, in sharp contrast to the classical information-theoretic threshold of $\Theta(\log(n))$. This matches the conjectured computational threshold for the classical planted clique problem, and thus raises the intriguing possibility that, once we require robustness, there is no computational-statistical gap for planted clique. Our lower bound involves establishing a result regarding the KL divergence of a family of perturbed Bernoulli distributions, which may be of independent interest.
Tasks
Published	2017-04-17
URL	http://arxiv.org/abs/1704.05120v2
PDF	http://arxiv.org/pdf/1704.05120v2.pdf
PWC	https://paperswithcode.com/paper/does-robustness-imply-tractability-a-lower
Repo
Framework

A Deep Learning Approach to Digitally Stain Optical Coherence Tomography Images of the Optic Nerve Head


Title	A Deep Learning Approach to Digitally Stain Optical Coherence Tomography Images of the Optic Nerve Head
Authors	Sripad Krishna Devalla, Jean-Martial Mari, Tin A. Tun, Nicholas G. Strouthidis, Tin Aung, Alexandre H. Thiery, Michael J. A. Girard
Abstract	Purpose: To develop a deep learning approach to digitally-stain optical coherence tomography (OCT) images of the optic nerve head (ONH). Methods: A horizontal B-scan was acquired through the center of the ONH using OCT (Spectralis) for 1 eye of each of 100 subjects (40 normal & 60 glaucoma). All images were enhanced using adaptive compensation. A custom deep learning network was then designed and trained with the compensated images to digitally stain (i.e. highlight) 6 tissue layers of the ONH. The accuracy of our algorithm was assessed (against manual segmentations) using the Dice coefficient, sensitivity, and specificity. We further studied how compensation and the number of training images affected the performance of our algorithm. Results: For images it had not yet assessed, our algorithm was able to digitally stain the retinal nerve fiber layer + prelamina, the retinal pigment epithelium, all other retinal layers, the choroid, and the peripapillary sclera and lamina cribrosa. For all tissues, the mean dice coefficient was $0.84 \pm 0.03$, the mean sensitivity $0.92 \pm 0.03$, and the mean specificity $0.99 \pm 0.00$. Our algorithm performed significantly better when compensated images were used for training. Increasing the number of images (from 10 to 40) to train our algorithm did not significantly improve performance, except for the RPE. Conclusion. Our deep learning algorithm can simultaneously stain neural and connective tissues in ONH images. Our approach offers a framework to automatically measure multiple key structural parameters of the ONH that may be critical to improve glaucoma management.
Tasks
Published	2017-07-24
URL	http://arxiv.org/abs/1707.07609v1
PDF	http://arxiv.org/pdf/1707.07609v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-learning-approach-to-digitally-stain
Repo
Framework

Residual Squeeze VGG16


Title	Residual Squeeze VGG16
Authors	Hussam Qassim, David Feinzimer, Abhishek Verma
Abstract	Deep learning has given way to a new era of machine learning, apart from computer vision. Convolutional neural networks have been implemented in image classification, segmentation and object detection. Despite recent advancements, we are still in the very early stages and have yet to settle on best practices for network architecture in terms of deep design, small in size and a short training time. In this work, we propose a very deep neural network comprised of 16 Convolutional layers compressed with the Fire Module adapted from the SQUEEZENET model. We also call for the addition of residual connections to help suppress degradation. This model can be implemented on almost every neural network model with fully incorporated residual learning. This proposed model Residual-Squeeze-VGG16 (ResSquVGG16) trained on the large-scale MIT Places365-Standard scene dataset. In our tests, the model performed with accuracy similar to the pre-trained VGG16 model in Top-1 and Top-5 validation accuracy while also enjoying a 23.86% reduction in training time and an 88.4% reduction in size. In our tests, this model was trained from scratch.
Tasks	Image Classification, Object Detection
Published	2017-05-05
URL	http://arxiv.org/abs/1705.03004v1
PDF	http://arxiv.org/pdf/1705.03004v1.pdf
PWC	https://paperswithcode.com/paper/residual-squeeze-vgg16
Repo
Framework

Generative-Discriminative Variational Model for Visual Recognition


Title	Generative-Discriminative Variational Model for Visual Recognition
Authors	Chih-Kuan Yeh, Yao-Hung Hubert Tsai, Yu-Chiang Frank Wang
Abstract	The paradigm shift from shallow classifiers with hand-crafted features to end-to-end trainable deep learning models has shown significant improvements on supervised learning tasks. Despite the promising power of deep neural networks (DNN), how to alleviate overfitting during training has been a research topic of interest. In this paper, we present a Generative-Discriminative Variational Model (GDVM) for visual classification, in which we introduce a latent variable inferred from inputs for exhibiting generative abilities towards prediction. In other words, our GDVM casts the supervised learning task as a generative learning process, with data discrimination to be jointly exploited for improved classification. In our experiments, we consider the tasks of multi-class classification, multi-label classification, and zero-shot learning. We show that our GDVM performs favorably against the baselines or recent generative DNN models.
Tasks	Multi-Label Classification, Zero-Shot Learning
Published	2017-06-07
URL	http://arxiv.org/abs/1706.02295v1
PDF	http://arxiv.org/pdf/1706.02295v1.pdf
PWC	https://paperswithcode.com/paper/generative-discriminative-variational-model
Repo
Framework

Neural Machine Translation with Source-Side Latent Graph Parsing


Title	Neural Machine Translation with Source-Side Latent Graph Parsing
Authors	Kazuma Hashimoto, Yoshimasa Tsuruoka
Abstract	This paper presents a novel neural machine translation model which jointly learns translation and source-side latent graph representations of sentences. Unlike existing pipelined approaches using syntactic parsers, our end-to-end model learns a latent graph parser as part of the encoder of an attention-based neural machine translation model, and thus the parser is optimized according to the translation objective. In experiments, we first show that our model compares favorably with state-of-the-art sequential and pipelined syntax-based NMT models. We also show that the performance of our model can be further improved by pre-training it with a small amount of treebank annotations. Our final ensemble model significantly outperforms the previous best models on the standard English-to-Japanese translation dataset.
Tasks	Machine Translation
Published	2017-02-08
URL	http://arxiv.org/abs/1702.02265v4
PDF	http://arxiv.org/pdf/1702.02265v4.pdf
PWC	https://paperswithcode.com/paper/neural-machine-translation-with-source-side
Repo
Framework

Relaxation heuristics for the set multicover problem with generalized upper bound constraints


Title	Relaxation heuristics for the set multicover problem with generalized upper bound constraints
Authors	Shunji Umetani, Masanao Arakawa, Mutsunori Yagiura
Abstract	We consider an extension of the set covering problem (SCP) introducing (i)~multicover and (ii)~generalized upper bound (GUB)~constraints. For the conventional SCP, the pricing method has been introduced to reduce the size of instances, and several efficient heuristic algorithms based on such reduction techniques have been developed to solve large-scale instances. However, GUB constraints often make the pricing method less effective, because they often prevent solutions from containing highly evaluated variables together. To overcome this problem, we develop heuristic algorithms to reduce the size of instances, in which new evaluation schemes of variables are introduced taking account of GUB constraints. We also develop an efficient implementation of a 2-flip neighborhood local search algorithm that reduces the number of candidates in the neighborhood without sacrificing the solution quality. In order to guide the search to visit a wide variety of good solutions, we also introduce a path relinking method that generates new solutions by combining two or more solutions obtained so far. According to computational comparison on benchmark instances, the proposed method succeeds in selecting a small number of promising variables properly and performs quite effectively even for large-scale instances having hard GUB constraints.
Tasks
Published	2017-05-14
URL	http://arxiv.org/abs/1705.04970v2
PDF	http://arxiv.org/pdf/1705.04970v2.pdf
PWC	https://paperswithcode.com/paper/relaxation-heuristics-for-the-set-multicover
Repo
Framework

Hierarchical Bayesian image analysis: from low-level modeling to robust supervised learning


Title	Hierarchical Bayesian image analysis: from low-level modeling to robust supervised learning
Authors	Adrien Lagrange, Mathieu Fauvel, Stéphane May, Nicolas Dobigeon
Abstract	Within a supervised classification framework, labeled data are used to learn classifier parameters. Prior to that, it is generally required to perform dimensionality reduction via feature extraction. These preprocessing steps have motivated numerous research works aiming at recovering latent variables in an unsupervised context. This paper proposes a unified framework to perform classification and low-level modeling jointly. The main objective is to use the estimated latent variables as features for classification and to incorporate simultaneously supervised information to help latent variable extraction. The proposed hierarchical Bayesian model is divided into three stages: a first low-level modeling stage to estimate latent variables, a second stage clustering these features into statistically homogeneous groups and a last classification stage exploiting the (possibly badly) labeled data. Performance of the model is assessed in the specific context of hyperspectral image interpretation, unifying two standard analysis techniques, namely unmixing and classification.
Tasks	Dimensionality Reduction
Published	2017-12-01
URL	http://arxiv.org/abs/1712.00368v1
PDF	http://arxiv.org/pdf/1712.00368v1.pdf
PWC	https://paperswithcode.com/paper/hierarchical-bayesian-image-analysis-from-low
Repo
Framework

A hybrid CPU-GPU parallelization scheme of variable neighborhood search for inventory optimization problems


Title	A hybrid CPU-GPU parallelization scheme of variable neighborhood search for inventory optimization problems
Authors	Nikolaos Antoniadis, Angelo Sifaleras
Abstract	In this paper, we study various parallelization schemes for the Variable Neighborhood Search (VNS) metaheuristic on a CPU-GPU system via OpenMP and OpenACC. A hybrid parallel VNS method is applied to recent benchmark problem instances for the multi-product dynamic lot sizing problem with product returns and recovery, which appears in reverse logistics and is known to be NP-hard. We report our findings regarding these parallelization approaches and present promising computational results.
Tasks
Published	2017-04-17
URL	http://arxiv.org/abs/1704.05132v1
PDF	http://arxiv.org/pdf/1704.05132v1.pdf
PWC	https://paperswithcode.com/paper/a-hybrid-cpu-gpu-parallelization-scheme-of
Repo
Framework