July 30, 2019

2725 words 13 mins read

Paper Group AWR 62

Text-Independent Speaker Verification Using 3D Convolutional Neural Networks. Hypergraph $p$-Laplacian: A Differential Geometry View. End-to-End Differentiable Proving. Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs. STag: A Stable Fiducial Marker System. Estimated Depth Map Helps Image Classification. Inferring …

Text-Independent Speaker Verification Using 3D Convolutional Neural Networks


Title	Text-Independent Speaker Verification Using 3D Convolutional Neural Networks
Authors	Amirsina Torfi, Jeremy Dawson, Nasser M. Nasrabadi
Abstract	In this paper, a novel method using 3D Convolutional Neural Network (3D-CNN) architecture has been proposed for speaker verification in the text-independent setting. One of the main challenges is the creation of the speaker models. Most of the previously-reported approaches create speaker models based on averaging the extracted features from utterances of the speaker, which is known as the d-vector system. In our paper, we propose an adaptive feature learning by utilizing the 3D-CNNs for direct speaker model creation in which, for both development and enrollment phases, an identical number of spoken utterances per speaker is fed to the network for representing the speakers’ utterances and creation of the speaker model. This leads to simultaneously capturing the speaker-related information and building a more robust system to cope with within-speaker variation. We demonstrate that the proposed method significantly outperforms the traditional d-vector verification system. Moreover, the proposed system can also be an alternative to the traditional d-vector system which is a one-shot speaker modeling system by utilizing 3D-CNNs.
Tasks	Speaker Verification, Text-Independent Speaker Verification
Published	2017-05-26
URL	http://arxiv.org/abs/1705.09422v7
PDF	http://arxiv.org/pdf/1705.09422v7.pdf
PWC	https://paperswithcode.com/paper/text-independent-speaker-verification-using
Repo	https://github.com/ArvinSiChuan/speaker-rec-keras
Framework	none

Hypergraph $p$-Laplacian: A Differential Geometry View


Title	Hypergraph $p$-Laplacian: A Differential Geometry View
Authors	Shota Saito, Danilo P Mandic, Hideyuki Suzuki
Abstract	The graph Laplacian plays key roles in information processing of relational data, and has analogies with the Laplacian in differential geometry. In this paper, we generalize the analogy between graph Laplacian and differential geometry to the hypergraph setting, and propose a novel hypergraph $p$-Laplacian. Unlike the existing two-node graph Laplacians, this generalization makes it possible to analyze hypergraphs, where the edges are allowed to connect any number of nodes. Moreover, we propose a semi-supervised learning method based on the proposed hypergraph $p$-Laplacian, and formalize them as the analogue to the Dirichlet problem, which often appears in physics. We further explore theoretical connections to normalized hypergraph cut on a hypergraph, and propose normalized cut corresponding to hypergraph $p$-Laplacian. The proposed $p$-Laplacian is shown to outperform standard hypergraph Laplacians in the experiment on a hypergraph semi-supervised learning and normalized cut setting.
Tasks
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08171v1
PDF	http://arxiv.org/pdf/1711.08171v1.pdf
PWC	https://paperswithcode.com/paper/hypergraph-p-laplacian-a-differential
Repo	https://github.com/ShotaSAITO/Hypergraph-Laplacian
Framework	none

End-to-End Differentiable Proving


Title	End-to-End Differentiable Proving
Authors	Tim Rocktäschel, Sebastian Riedel
Abstract	We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols. These neural networks are constructed recursively by taking inspiration from the backward chaining algorithm as used in Prolog. Specifically, we replace symbolic unification with a differentiable computation on vector representations of symbols using a radial basis function kernel, thereby combining symbolic reasoning with learning subsymbolic vector representations. By using gradient descent, the resulting neural network can be trained to infer facts from a given incomplete knowledge base. It learns to (i) place representations of similar symbols in close proximity in a vector space, (ii) make use of such similarities to prove queries, (iii) induce logical rules, and (iv) use provided and induced logical rules for multi-hop reasoning. We demonstrate that this architecture outperforms ComplEx, a state-of-the-art neural link prediction model, on three out of four benchmark knowledge bases while at the same time inducing interpretable function-free first-order logic rules.
Tasks	Link Prediction
Published	2017-05-31
URL	http://arxiv.org/abs/1705.11040v2
PDF	http://arxiv.org/pdf/1705.11040v2.pdf
PWC	https://paperswithcode.com/paper/end-to-end-differentiable-proving
Repo	https://github.com/uclmr/ntp
Framework	tf

Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs


Title	Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs
Authors	Bhushan Kotnis, Vivi Nastase
Abstract	Knowledge graphs are large, useful, but incomplete knowledge repositories. They encode knowledge through entities and relations which define each other through the connective structure of the graph. This has inspired methods for the joint embedding of entities and relations in continuous low-dimensional vector spaces, that can be used to induce new edges in the graph, i.e., link prediction in knowledge graphs. Learning these representations relies on contrasting positive instances with negative ones. Knowledge graphs include only positive relation instances, leaving the door open for a variety of methods for selecting negative examples. In this paper we present an empirical study on the impact of negative sampling on the learned embeddings, assessed through the task of link prediction. We use state-of-the-art knowledge graph embeddings – \rescal , TransE, DistMult and ComplEX – and evaluate on benchmark datasets – FB15k and WN18. We compare well known methods for negative sampling and additionally propose embedding based sampling methods. We note a marked difference in the impact of these sampling methods on the two datasets, with the “traditional” corrupting positives method leading to best results on WN18, while embedding based methods benefiting the task on FB15k.
Tasks	Knowledge Graph Embeddings, Knowledge Graphs, Link Prediction
Published	2017-08-22
URL	http://arxiv.org/abs/1708.06816v2
PDF	http://arxiv.org/pdf/1708.06816v2.pdf
PWC	https://paperswithcode.com/paper/analysis-of-the-impact-of-negative-sampling
Repo	https://github.com/bhushank/kge-rl
Framework	pytorch

STag: A Stable Fiducial Marker System


Title	STag: A Stable Fiducial Marker System
Authors	Burak Benligiray, Cihan Topal, Cuneyt Akinlar
Abstract	Fiducial markers provide better-defined features than the ones naturally available in the scene. For this reason, they are widely utilized in computer vision applications where reliable pose estimation is required. Factors such as imaging noise and subtle changes in illumination induce jitter on the estimated pose. Jitter impairs robustness in vision and robotics applications, and deteriorates the sense of presence and immersion in AR/VR applications. In this paper, we propose STag, a fiducial marker system that provides stable pose estimation. STag is designed to be robust against jitter factors, thus sustains pose stability better than the existing solutions. This is achieved by utilizing geometric features that can be localized more repeatably. The outer square border of the marker is used for detection and homography estimation. This is followed by a novel homography refinement step using the inner circular border. After refinement, the pose can be estimated stably and robustly across viewing conditions. These features are demonstrated with a comprehensive set of experiments, including comparisons with the state of the art fiducial marker systems.
Tasks	Homography Estimation, Pose Estimation
Published	2017-07-19
URL	https://arxiv.org/abs/1707.06292v2
PDF	https://arxiv.org/pdf/1707.06292v2.pdf
PWC	https://paperswithcode.com/paper/stag-a-stable-fiducial-marker-system
Repo	https://github.com/bbenligiray/stag
Framework	none

Estimated Depth Map Helps Image Classification


Title	Estimated Depth Map Helps Image Classification
Authors	Yihui He
Abstract	We consider image classification with estimated depth. This problem falls into the domain of transfer learning, since we are using a model trained on a set of depth images to generate depth maps (additional features) for use in another classification problem using another disjoint set of images. It’s challenging as no direct depth information is provided. Though depth estimation has been well studied, none have attempted to aid image classification with estimated depth. Therefore, we present a way of transferring domain knowledge on depth estimation to a separate image classification task over a disjoint set of train, and test data. We build a RGBD dataset based on RGB dataset and do image classification on it. Then evaluation the performance of neural networks on the RGBD dataset compared to the RGB dataset. From our experiments, the benefit is significant with shallow and deep networks. It improves ResNet-20 by 0.55% and ResNet-56 by 0.53%. Our code and dataset are available publicly.
Tasks	Depth Estimation, Image Classification, Transfer Learning
Published	2017-09-20
URL	http://arxiv.org/abs/1709.07077v1
PDF	http://arxiv.org/pdf/1709.07077v1.pdf
PWC	https://paperswithcode.com/paper/estimated-depth-map-helps-image
Repo	https://github.com/yihui-he/Estimated-Depth-Map-Helps-Image-Classification
Framework	none

Inferring and Executing Programs for Visual Reasoning


Title	Inferring and Executing Programs for Visual Reasoning
Authors	Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, Ross Girshick
Abstract	Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes. As a result, these black-box models often learn to exploit biases in the data rather than learning to perform visual reasoning. Inspired by module networks, this paper proposes a model for visual reasoning that consists of a program generator that constructs an explicit representation of the reasoning process to be performed, and an execution engine that executes the resulting program to produce an answer. Both the program generator and the execution engine are implemented by neural networks, and are trained using a combination of backpropagation and REINFORCE. Using the CLEVR benchmark for visual reasoning, we show that our model significantly outperforms strong baselines and generalizes better in a variety of settings.
Tasks	Visual Reasoning
Published	2017-05-10
URL	http://arxiv.org/abs/1705.03633v1
PDF	http://arxiv.org/pdf/1705.03633v1.pdf
PWC	https://paperswithcode.com/paper/inferring-and-executing-programs-for-visual
Repo	https://github.com/rs9000/VisualReasoning_MMnet
Framework	pytorch

Joint Syntacto-Discourse Parsing and the Syntacto-Discourse Treebank


Title	Joint Syntacto-Discourse Parsing and the Syntacto-Discourse Treebank
Authors	Kai Zhao, Liang Huang
Abstract	Discourse parsing has long been treated as a stand-alone problem independent from constituency or dependency parsing. Most attempts at this problem are pipelined rather than end-to-end, sophisticated, and not self-contained: they assume gold-standard text segmentations (Elementary Discourse Units), and use external parsers for syntactic features. In this paper we propose the first end-to-end discourse parser that jointly parses in both syntax and discourse levels, as well as the first syntacto-discourse treebank by integrating the Penn Treebank with the RST Treebank. Built upon our recent span-based constituency parser, this joint syntacto-discourse parser requires no preprocessing whatsoever (such as segmentation or feature extraction), achieves the state-of-the-art end-to-end discourse parsing accuracy.
Tasks	Dependency Parsing
Published	2017-08-28
URL	http://arxiv.org/abs/1708.08484v1
PDF	http://arxiv.org/pdf/1708.08484v1.pdf
PWC	https://paperswithcode.com/paper/joint-syntacto-discourse-parsing-and-the
Repo	https://github.com/kaayy/josydipa
Framework	none


Title	Equivariance Through Parameter-Sharing
Authors	Siamak Ravanbakhsh, Jeff Schneider, Barnabas Poczos
Abstract	We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group $\mathcal{G}$ that acts discretely on the input and output of a standard neural network layer $\phi_{W}: \Re^{M} \to \Re^{N}$, we show that $\phi_{W}$ is equivariant with respect to $\mathcal{G}$-action iff $\mathcal{G}$ explains the symmetries of the network parameters $W$. Inspired by this observation, we then propose two parameter-sharing schemes to induce the desirable symmetry on $W$. Our procedures for tying the parameters achieve $\mathcal{G}$-equivariance and, under some conditions on the action of $\mathcal{G}$, they guarantee sensitivity to all other permutation groups outside $\mathcal{G}$.
Tasks
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08389v2
PDF	http://arxiv.org/pdf/1702.08389v2.pdf
PWC	https://paperswithcode.com/paper/equivariance-through-parameter-sharing
Repo	https://github.com/raymondyeh07/chirality_nets
Framework	pytorch

Unsupervised Submodular Rank Aggregation on Score-based Permutations


Title	Unsupervised Submodular Rank Aggregation on Score-based Permutations
Authors	Jun Qi, Xu Liu, Javier Tejedor, Shunsuke Kamijo
Abstract	Unsupervised rank aggregation on score-based permutations, which is widely used in many applications, has not been deeply explored yet. This work studies the use of submodular optimization for rank aggregation on score-based permutations in an unsupervised way. Specifically, we propose an unsupervised approach based on the Lovasz Bregman divergence for setting up linear structured convex and nested structured concave objective functions. In addition, stochastic optimization methods are applied in the training process and efficient algorithms for inference can be guaranteed. The experimental results from Information Retrieval, Combining Distributed Neural Networks, Influencers in Social Networks, and Distributed Automatic Speech Recognition tasks demonstrate the effectiveness of the proposed methods.
Tasks	Information Retrieval, Speech Recognition, Stochastic Optimization
Published	2017-07-04
URL	http://arxiv.org/abs/1707.01166v3
PDF	http://arxiv.org/pdf/1707.01166v3.pdf
PWC	https://paperswithcode.com/paper/unsupervised-submodular-rank-aggregation-on
Repo	https://github.com/uwjunqi/Subrank
Framework	none

Deep Bayesian Active Learning with Image Data


Title	Deep Bayesian Active Learning with Image Data
Authors	Yarin Gal, Riashat Islam, Zoubin Ghahramani
Abstract	Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).
Tasks	Active Learning
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02910v1
PDF	http://arxiv.org/pdf/1703.02910v1.pdf
PWC	https://paperswithcode.com/paper/deep-bayesian-active-learning-with-image-data
Repo	https://github.com/tanyanair/segmentation_uncertainty
Framework	tf

Unsupervised Terminological Ontology Learning based on Hierarchical Topic Modeling


Title	Unsupervised Terminological Ontology Learning based on Hierarchical Topic Modeling
Authors	Xiaofeng Zhu, Diego Klabjan, Patrick Bless
Abstract	In this paper, we present hierarchical relationbased latent Dirichlet allocation (hrLDA), a data-driven hierarchical topic model for extracting terminological ontologies from a large number of heterogeneous documents. In contrast to traditional topic models, hrLDA relies on noun phrases instead of unigrams, considers syntax and document structures, and enriches topic hierarchies with topic relations. Through a series of experiments, we demonstrate the superiority of hrLDA over existing topic models, especially for building hierarchies. Furthermore, we illustrate the robustness of hrLDA in the settings of noisy data sets, which are likely to occur in many practical scenarios. Our ontology evaluation results show that ontologies extracted from hrLDA are very competitive with the ontologies created by domain experts.
Tasks	Topic Models
Published	2017-08-29
URL	http://arxiv.org/abs/1708.09025v1
PDF	http://arxiv.org/pdf/1708.09025v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-terminological-ontology-learning
Repo	https://github.com/XiaofengZhu/hrLDA
Framework	none

Natural Language Processing: State of The Art, Current Trends and Challenges


Title	Natural Language Processing: State of The Art, Current Trends and Challenges
Authors	Diksha Khurana, Aditya Koli, Kiran Khatter, Sukhdev Singh
Abstract	Natural language processing (NLP) has recently gained much attention for representing and analysing human language computationally. It has spread its applications in various fields such as machine translation, email spam detection, information extraction, summarization, medical, and question answering etc. The paper distinguishes four phases by discussing different levels of NLP and components of Natural Language Generation (NLG) followed by presenting the history and evolution of NLP, state of the art presenting the various applications of NLP and current trends and challenges.
Tasks	Machine Translation, Question Answering, Text Generation
Published	2017-08-17
URL	http://arxiv.org/abs/1708.05148v1
PDF	http://arxiv.org/pdf/1708.05148v1.pdf
PWC	https://paperswithcode.com/paper/natural-language-processing-state-of-the-art
Repo	https://github.com/anuragreddygv323/Important-stuff
Framework	tf

Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling


Title	Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
Authors	Diego Marcheggiani, Ivan Titov
Abstract	Semantic role labeling (SRL) is the task of identifying the predicate-argument structure of a sentence. It is typically regarded as an important step in the standard NLP pipeline. As the semantic representations are closely related to syntactic ones, we exploit syntactic information in our model. We propose a version of graph convolutional networks (GCNs), a recent class of neural networks operating on graphs, suited to model syntactic dependency graphs. GCNs over syntactic dependency trees are used as sentence encoders, producing latent feature representations of words in a sentence. We observe that GCN layers are complementary to LSTM ones: when we stack both GCN and LSTM layers, we obtain a substantial improvement over an already state-of-the-art LSTM SRL model, resulting in the best reported scores on the standard benchmark (CoNLL-2009) both for Chinese and English.
Tasks	Semantic Role Labeling
Published	2017-03-14
URL	http://arxiv.org/abs/1703.04826v4
PDF	http://arxiv.org/pdf/1703.04826v4.pdf
PWC	https://paperswithcode.com/paper/encoding-sentences-with-graph-convolutional
Repo	https://github.com/kervyRivas/Graph-convolutional-
Framework	pytorch

Optimal Auctions through Deep Learning


Title	Optimal Auctions through Deep Learning
Authors	Paul Dütting, Zhe Feng, Harikrishna Narasimhan, David C. Parkes, Sai Srivatsa Ravindranath
Abstract	Designing an incentive compatible auction that maximizes expected revenue is an intricate task. The single-item case was resolved in a seminal piece of work by Myerson in 1981. Even after 30-40 years of intense research the problem remains unsolved for settings with two or more items. In this work, we initiate the exploration of the use of tools from deep learning for the automated design of optimal auctions. We model an auction as a multi-layer neural network, frame optimal auction design as a constrained learning problem, and show how it can be solved using standard machine learning pipelines. We prove generalization bounds and present extensive experiments, recovering essentially all known analytical solutions for multi-item settings, and obtaining novel mechanisms for settings in which the optimal mechanism is unknown.
Tasks
Published	2017-06-12
URL	https://arxiv.org/abs/1706.03459v4
PDF	https://arxiv.org/pdf/1706.03459v4.pdf
PWC	https://paperswithcode.com/paper/optimal-auctions-through-deep-learning
Repo	https://github.com/saisrivatsan/deep-opt-auctions
Framework	tf