July 29, 2019

2935 words 14 mins read

Paper Group AWR 97

The low-rank hurdle model. Efficient Natural Language Response Suggestion for Smart Reply. Semantic Preserving Embeddings for Generalized Graphs. Deep Complex Networks. Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach. Efficient Parallel Translating Embedding For Knowledge Graphs. Benchmarking Single Image Dehazing and Bey …

The low-rank hurdle model


Title	The low-rank hurdle model
Authors	Christopher Dienes
Abstract	A composite loss framework is proposed for low-rank modeling of data consisting of interesting and common values, such as excess zeros or missing values. The methodology is motivated by the generalized low-rank framework and the hurdle method which is commonly used to analyze zero-inflated counts. The model is demonstrated on a manufacturing data set and applied to the problem of missing value imputation.
Tasks	Imputation
Published	2017-09-06
URL	http://arxiv.org/abs/1709.01860v1
PDF	http://arxiv.org/pdf/1709.01860v1.pdf
PWC	https://paperswithcode.com/paper/the-low-rank-hurdle-model
Repo	https://github.com/ChrisDienes/hurdle_pca
Framework	none

Efficient Natural Language Response Suggestion for Smart Reply


Title	Efficient Natural Language Response Suggestion for Smart Reply
Authors	Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-hsuan Sung, Laszlo Lukacs, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, Ray Kurzweil
Abstract	This paper presents a computationally efficient machine-learned method for natural language response suggestion. Feed-forward neural networks using n-gram embedding features encode messages into vectors which are optimized to give message-response pairs a high dot-product value. An optimized search finds response suggestions. The method is evaluated in a large-scale commercial e-mail application, Inbox by Gmail. Compared to a sequence-to-sequence approach, the new system achieves the same quality at a small fraction of the computational requirements and latency.
Tasks
Published	2017-05-01
URL	http://arxiv.org/abs/1705.00652v1
PDF	http://arxiv.org/pdf/1705.00652v1.pdf
PWC	https://paperswithcode.com/paper/efficient-natural-language-response
Repo	https://github.com/AnilRamavath/repositories
Framework	none

Semantic Preserving Embeddings for Generalized Graphs


Title	Semantic Preserving Embeddings for Generalized Graphs
Authors	Pedro Almagro-Blanco, Fernando Sancho-Caparrini
Abstract	A new approach to the study of Generalized Graphs as semantic data structures using machine learning techniques is presented. We show how vector representations maintaining semantic characteristics of the original data can be obtained from a given graph using neural encoding architectures and considering the topological properties of the graph. Semantic features of these new representations are tested by using some machine learning tasks and new directions on efficient link discovery, entitity retrieval and long distance query methodologies on large relational datasets are investigated using real datasets. —- En este trabajo se presenta un nuevo enfoque en el contexto del aprendizaje autom'atico multi-relacional para el estudio de Grafos Generalizados. Se muestra c'omo se pueden obtener representaciones vectoriales que mantienen caracter'isticas sem'anticas del grafo original utilizando codificadores neuronales y considerando las propiedades topol'ogicas del grafo. Adem'as, se eval'uan las caracter'isticas sem'anticas capturadas por estas nuevas representaciones y se investigan nuevas metodolog'ias eficientes relacionadas con Link Discovery, Entity Retrieval y consultas a larga distancia en grandes conjuntos de datos relacionales haciendo uso de bases de datos reales.
Tasks
Published	2017-09-07
URL	http://arxiv.org/abs/1709.02759v1
PDF	http://arxiv.org/pdf/1709.02759v1.pdf
PWC	https://paperswithcode.com/paper/semantic-preserving-embeddings-for
Repo	https://github.com/palmagro/gg2vec
Framework	none

Deep Complex Networks


Title	Deep Complex Networks
Authors	Chiheb Trabelsi, Olexa Bilaniuk, Ying Zhang, Dmitriy Serdyuk, Sandeep Subramanian, João Felipe Santos, Soroush Mehri, Negar Rostamzadeh, Yoshua Bengio, Christopher J Pal
Abstract	At present, the vast majority of building blocks, techniques, and architectures for deep learning are based on real-valued operations and representations. However, recent work on recurrent neural networks and older fundamental theoretical analysis suggests that complex numbers could have a richer representational capacity and could also facilitate noise-robust memory retrieval mechanisms. Despite their attractive properties and potential for opening up entirely new neural architectures, complex-valued deep neural networks have been marginalized due to the absence of the building blocks required to design such models. In this work, we provide the key atomic components for complex-valued deep neural networks and apply them to convolutional feed-forward networks and convolutional LSTMs. More precisely, we rely on complex convolutions and present algorithms for complex batch-normalization, complex weight initialization strategies for complex-valued neural nets and we use them in experiments with end-to-end training schemes. We demonstrate that such complex-valued models are competitive with their real-valued counterparts. We test deep complex models on several computer vision tasks, on music transcription using the MusicNet dataset and on Speech Spectrum Prediction using the TIMIT dataset. We achieve state-of-the-art performance on these audio-related tasks.
Tasks	Image Classification
Published	2017-05-27
URL	http://arxiv.org/abs/1705.09792v4
PDF	http://arxiv.org/pdf/1705.09792v4.pdf
PWC	https://paperswithcode.com/paper/deep-complex-networks
Repo	https://github.com/skyblueutd/Paper
Framework	none

Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach


Title	Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach
Authors	Xingyi Zhou, Qixing Huang, Xiao Sun, Xiangyang Xue, Yichen Wei
Abstract	In this paper, we study the task of 3D human pose estimation in the wild. This task is challenging due to lack of training data, as existing datasets are either in the wild images with 2D pose or in the lab images with 3D pose. We propose a weakly-supervised transfer learning method that uses mixed 2D and 3D labels in a unified deep neutral network that presents two-stage cascaded structure. Our network augments a state-of-the-art 2D pose estimation sub-network with a 3D depth regression sub-network. Unlike previous two stage approaches that train the two sub-networks sequentially and separately, our training is end-to-end and fully exploits the correlation between the 2D pose and depth estimation sub-tasks. The deep features are better learnt through shared representations. In doing so, the 3D pose labels in controlled lab environments are transferred to in the wild images. In addition, we introduce a 3D geometric constraint to regularize the 3D pose prediction, which is effective in the absence of ground truth depth labels. Our method achieves competitive results on both 2D and 3D benchmarks.
Tasks	3D Human Pose Estimation, Pose Estimation, Pose Prediction, Transfer Learning
Published	2017-04-08
URL	http://arxiv.org/abs/1704.02447v2
PDF	http://arxiv.org/pdf/1704.02447v2.pdf
PWC	https://paperswithcode.com/paper/towards-3d-human-pose-estimation-in-the-wild
Repo	https://github.com/mengyingfei/pose-3d-pytorch-ros
Framework	pytorch

Efficient Parallel Translating Embedding For Knowledge Graphs


Title	Efficient Parallel Translating Embedding For Knowledge Graphs
Authors	Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, Xueqi Cheng
Abstract	Knowledge graph embedding aims to embed entities and relations of knowledge graphs into low-dimensional vector spaces. Translating embedding methods regard relations as the translation from head entities to tail entities, which achieve the state-of-the-art results among knowledge graph embedding methods. However, a major limitation of these methods is the time consuming training process, which may take several days or even weeks for large knowledge graphs, and result in great difficulty in practical applications. In this paper, we propose an efficient parallel framework for translating embedding methods, called ParTrans-X, which enables the methods to be paralleled without locks by utilizing the distinguished structures of knowledge graphs. Experiments on two datasets with three typical translating embedding methods, i.e., TransE [3], TransH [17], and a more efficient variant TransE- AdaGrad [10] validate that ParTrans-X can speed up the training process by more than an order of magnitude.
Tasks	Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs
Published	2017-03-30
URL	http://arxiv.org/abs/1703.10316v4
PDF	http://arxiv.org/pdf/1703.10316v4.pdf
PWC	https://paperswithcode.com/paper/efficient-parallel-translating-embedding-for
Repo	https://github.com/zdh2292390/ParTrans-X
Framework	none

Benchmarking Single Image Dehazing and Beyond


Title	Benchmarking Single Image Dehazing and Beyond
Authors	Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, Zhangyang Wang
Abstract	We present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE). RESIDE highlights diverse data sources and image contents, and is divided into five subsets, each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments on RESIDE shed light on the comparisons and limitations of state-of-the-art dehazing algorithms, and suggest promising future directions.
Tasks	Image Dehazing, Single Image Dehazing
Published	2017-12-12
URL	http://arxiv.org/abs/1712.04143v4
PDF	http://arxiv.org/pdf/1712.04143v4.pdf
PWC	https://paperswithcode.com/paper/benchmarking-single-image-dehazing-and-beyond
Repo	https://github.com/Boyiliee/RESIDE-dataset-link
Framework	none

A simple yet effective baseline for 3d human pose estimation


Title	A simple yet effective baseline for 3d human pose estimation
Authors	Julieta Martinez, Rayat Hossain, Javier Romero, James J. Little
Abstract	Following the success of deep convolutional networks, state-of-the-art methods for 3d human pose estimation have focused on deep end-to-end systems that predict 3d joint locations given raw image pixels. Despite their excellent performance, it is often not easy to understand whether their remaining error stems from a limited 2d pose (visual) understanding, or from a failure to map 2d poses into 3-dimensional positions. With the goal of understanding these sources of error, we set out to build a system that given 2d joint locations predicts 3d positions. Much to our surprise, we have found that, with current technology, “lifting” ground truth 2d joint locations to 3d space is a task that can be solved with a remarkably low error rate: a relatively simple deep feed-forward network outperforms the best reported result by about 30% on Human3.6M, the largest publicly available 3d pose estimation benchmark. Furthermore, training our system on the output of an off-the-shelf state-of-the-art 2d detector (\ie, using images as input) yields state of the art results – this includes an array of systems that have been trained end-to-end specifically for this task. Our results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.
Tasks	3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation
Published	2017-05-08
URL	http://arxiv.org/abs/1705.03098v2
PDF	http://arxiv.org/pdf/1705.03098v2.pdf
PWC	https://paperswithcode.com/paper/a-simple-yet-effective-baseline-for-3d-human
Repo	https://github.com/garyzhao/SemGCN
Framework	pytorch

Personalization in Goal-Oriented Dialog


Title	Personalization in Goal-Oriented Dialog
Authors	Chaitanya K. Joshi, Fei Mi, Boi Faltings
Abstract	The main goal of modeling human conversation is to create agents which can interact with people in both open-ended and goal-oriented scenarios. End-to-end trained neural dialog systems are an important line of research for such generalized dialog models as they do not resort to any situation-specific handcrafting of rules. However, incorporating personalization into such systems is a largely unexplored topic as there are no existing corpora to facilitate such work. In this paper, we present a new dataset of goal-oriented dialogs which are influenced by speaker profiles attached to them. We analyze the shortcomings of an existing end-to-end dialog system based on Memory Networks and propose modifications to the architecture which enable personalization. We also investigate personalization in dialog as a multi-task learning problem, and show that a single model which shares features among various profiles outperforms separate models for each profile.
Tasks	Goal-Oriented Dialog, Multi-Task Learning
Published	2017-06-22
URL	http://arxiv.org/abs/1706.07503v3
PDF	http://arxiv.org/pdf/1706.07503v3.pdf
PWC	https://paperswithcode.com/paper/personalization-in-goal-oriented-dialog
Repo	https://github.com/chaitjo/personalized-dialog
Framework	tf

Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network


Title	Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network
Authors	Kim Anh Nguyen, Sabine Schulte im Walde, Ngoc Thang Vu
Abstract	Distinguishing between antonyms and synonyms is a key task to achieve high performance in NLP systems. While they are notoriously difficult to distinguish by distributional co-occurrence models, pattern-based methods have proven effective to differentiate between the relations. In this paper, we present a novel neural network model AntSynNET that exploits lexico-syntactic patterns from syntactic parse trees. In addition to the lexical and syntactic information, we successfully integrate the distance between the related words along the syntactic path as a new pattern feature. The results from classification experiments show that AntSynNET improves the performance over prior pattern-based methods.
Tasks
Published	2017-01-11
URL	http://arxiv.org/abs/1701.02962v1
PDF	http://arxiv.org/pdf/1701.02962v1.pdf
PWC	https://paperswithcode.com/paper/distinguishing-antonyms-and-synonyms-in-a
Repo	https://github.com/nguyenkh/AntSynNET
Framework	none

Regularizing Neural Networks by Penalizing Confident Output Distributions


Title	Regularizing Neural Networks by Penalizing Confident Output Distributions
Authors	Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, Geoffrey Hinton
Abstract	We systematically explore regularizing neural networks by penalizing low entropy output distributions. We show that penalizing low entropy output distributions, which has been shown to improve exploration in reinforcement learning, acts as a strong regularizer in supervised learning. Furthermore, we connect a maximum entropy based confidence penalty to label smoothing through the direction of the KL divergence. We exhaustively evaluate the proposed confidence penalty and label smoothing on 6 common benchmarks: image classification (MNIST and Cifar-10), language modeling (Penn Treebank), machine translation (WMT’14 English-to-German), and speech recognition (TIMIT and WSJ). We find that both label smoothing and the confidence penalty improve state-of-the-art models across benchmarks without modifying existing hyperparameters, suggesting the wide applicability of these regularizers.
Tasks	Image Classification, Language Modelling, Machine Translation, Speech Recognition
Published	2017-01-23
URL	http://arxiv.org/abs/1701.06548v1
PDF	http://arxiv.org/pdf/1701.06548v1.pdf
PWC	https://paperswithcode.com/paper/regularizing-neural-networks-by-penalizing
Repo	https://github.com/makeyourownmaker/mixup
Framework	pytorch

Streaming Weak Submodularity: Interpreting Neural Networks on the Fly


Title	Streaming Weak Submodularity: Interpreting Neural Networks on the Fly
Authors	Ethan R. Elenberg, Alexandros G. Dimakis, Moran Feldman, Amin Karbasi
Abstract	In many machine learning applications, it is important to explain the predictions of a black-box classifier. For example, why does a deep neural network assign an image to a particular class? We cast interpretability of black-box classifiers as a combinatorial maximization problem and propose an efficient streaming algorithm to solve it subject to cardinality constraints. By extending ideas from Badanidiyuru et al. [2014], we provide a constant factor approximation guarantee for our algorithm in the case of random stream order and a weakly submodular objective function. This is the first such theoretical guarantee for this general class of functions, and we also show that no such algorithm exists for a worst case stream order. Our algorithm obtains similar explanations of Inception V3 predictions $10$ times faster than the state-of-the-art LIME framework of Ribeiro et al. [2016].
Tasks
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02647v3
PDF	http://arxiv.org/pdf/1703.02647v3.pdf
PWC	https://paperswithcode.com/paper/streaming-weak-submodularity-interpreting
Repo	https://github.com/eelenberg/streak
Framework	tf

Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations


Title	Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations
Authors	Rik van Noord, Johan Bos
Abstract	We evaluate the character-level translation method for neural semantic parsing on a large corpus of sentences annotated with Abstract Meaning Representations (AMRs). Using a sequence-to-sequence model, and some trivial preprocessing and postprocessing of AMRs, we obtain a baseline accuracy of 53.1 (F-score on AMR-triples). We examine five different approaches to improve this baseline result: (i) reordering AMR branches to match the word order of the input sentence increases performance to 58.3; (ii) adding part-of-speech tags (automatically produced) to the input shows improvement as well (57.2); (iii) So does the introduction of super characters (conflating frequent sequences of characters to a single character), reaching 57.4; (iv) optimizing the training process by using pre-training and averaging a set of models increases performance to 58.7; (v) adding silver-standard training data obtained by an off-the-shelf parser yields the biggest improvement, resulting in an F-score of 64.0. Combining all five techniques leads to an F-score of 71.0 on holdout data, which is state-of-the-art in AMR parsing. This is remarkable because of the relative simplicity of the approach.
Tasks	Amr Parsing, Semantic Parsing
Published	2017-05-28
URL	http://arxiv.org/abs/1705.09980v2
PDF	http://arxiv.org/pdf/1705.09980v2.pdf
PWC	https://paperswithcode.com/paper/neural-semantic-parsing-by-character-based
Repo	https://github.com/RikVN/AMR
Framework	tf

Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car


Title	Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
Authors	Mariusz Bojarski, Philip Yeres, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Lawrence Jackel, Urs Muller
Abstract	As part of a complete software stack for autonomous driving, NVIDIA has created a neural-network-based system, known as PilotNet, which outputs steering angles given images of the road ahead. PilotNet is trained using road images paired with the steering angles generated by a human driving a data-collection car. It derives the necessary domain knowledge by observing human drivers. This eliminates the need for human engineers to anticipate what is important in an image and foresee all the necessary rules for safe driving. Road tests demonstrated that PilotNet can successfully perform lane keeping in a wide variety of driving conditions, regardless of whether lane markings are present or not. The goal of the work described here is to explain what PilotNet learns and how it makes its decisions. To this end we developed a method for determining which elements in the road image most influence PilotNet’s steering decision. Results show that PilotNet indeed learns to recognize relevant objects on the road. In addition to learning the obvious features such as lane markings, edges of roads, and other cars, PilotNet learns more subtle features that would be hard to anticipate and program by engineers, for example, bushes lining the edge of the road and atypical vehicle classes.
Tasks	Autonomous Driving, Self-Driving Cars
Published	2017-04-25
URL	http://arxiv.org/abs/1704.07911v1
PDF	http://arxiv.org/pdf/1704.07911v1.pdf
PWC	https://paperswithcode.com/paper/explaining-how-a-deep-neural-network-trained
Repo	https://github.com/bobbyhaliwela/AutoRcCar-Behavioral-Clonning
Framework	tf

DAGGER: A sequential algorithm for FDR control on DAGs


Title	DAGGER: A sequential algorithm for FDR control on DAGs
Authors	Aaditya Ramdas, Jianbo Chen, Martin J. Wainwright, Michael I. Jordan
Abstract	We propose a linear-time, single-pass, top-down algorithm for multiple testing on directed acyclic graphs (DAGs), where nodes represent hypotheses and edges specify a partial ordering in which hypotheses must be tested. The procedure is guaranteed to reject a sub-DAG with bounded false discovery rate (FDR) while satisfying the logical constraint that a rejected node’s parents must also be rejected. It is designed for sequential testing settings, when the DAG structure is known a priori, but the $p$-values are obtained selectively (such as in a sequence of experiments), but the algorithm is also applicable in non-sequential settings when all $p$-values can be calculated in advance (such as variable/model selection). Our DAGGER algorithm, shorthand for Greedily Evolving Rejections on DAGs, provably controls the false discovery rate under independence, positive dependence or arbitrary dependence of the $p$-values. The DAGGER procedure specializes to known algorithms in the special cases of trees and line graphs, and simplifies to the classical Benjamini-Hochberg procedure when the DAG has no edges. We explore the empirical performance of DAGGER using simulations, as well as a real dataset corresponding to a gene ontology, showing favorable performance in terms of time and power.
Tasks	Model Selection
Published	2017-09-29
URL	http://arxiv.org/abs/1709.10250v3
PDF	http://arxiv.org/pdf/1709.10250v3.pdf
PWC	https://paperswithcode.com/paper/dagger-a-sequential-algorithm-for-fdr-control
Repo	https://github.com/Jianbo-Lab/DAGGER
Framework	none