February 2, 2020

2742 words 13 mins read

Paper Group AWR 15

Paper Group AWR 15

Practical Federated Gradient Boosting Decision Trees. OpenTapioca: Lightweight Entity Linking for Wikidata. Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use. Skin Lesion Synthesis with Generative Adversarial Networks. Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot R …

Practical Federated Gradient Boosting Decision Trees

Title Practical Federated Gradient Boosting Decision Trees
Authors Qinbin Li, Zeyi Wen, Bingsheng He
Abstract Gradient Boosting Decision Trees (GBDTs) have become very successful in recent years, with many awards in machine learning and data mining competitions. There have been several recent studies on how to train GBDTs in the federated learning setting. In this paper, we focus on horizontal federated learning, where data samples with the same features are distributed among multiple parties. However, existing studies are not efficient or effective enough for practical use. They suffer either from the inefficiency due to the usage of costly data transformations such as secret sharing and homomorphic encryption, or from the low model accuracy due to differential privacy designs. In this paper, we study a practical federated environment with relaxed privacy constraints. In this environment, a dishonest party might obtain some information about the other parties’ data, but it is still impossible for the dishonest party to derive the actual raw data of other parties. Specifically, each party boosts a number of trees by exploiting similarity information based on locality-sensitive hashing. We prove that our framework is secure without exposing the original record to other parties, while the computation overhead in the training process is kept low. Our experimental studies show that, compared with normal training with the local data of each party, our approach can significantly improve the predictive accuracy, and achieve comparable accuracy to the original GBDT with the data from all parties.
Published 2019-11-11
URL https://arxiv.org/abs/1911.04206v2
PDF https://arxiv.org/pdf/1911.04206v2.pdf
PWC https://paperswithcode.com/paper/practical-federated-gradient-boosting
Repo https://github.com/Xtra-Computing/PrivML
Framework none

OpenTapioca: Lightweight Entity Linking for Wikidata

Title OpenTapioca: Lightweight Entity Linking for Wikidata
Authors Antonin Delpeuch
Abstract We propose a simple Named Entity Linking system that can be trained from Wikidata only. This demonstrates the strengths and weaknesses of this data source for this task and provides an easily reproducible baseline to compare other systems against. Our model is lightweight to train, to run and to keep synchronous with Wikidata in real time.
Tasks Entity Linking
Published 2019-04-19
URL http://arxiv.org/abs/1904.09131v1
PDF http://arxiv.org/pdf/1904.09131v1.pdf
PWC https://paperswithcode.com/paper/opentapioca-lightweight-entity-linking-for
Repo https://github.com/wetneb/opentapioca
Framework none

Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use

Title Learning End-to-End Goal-Oriented Dialog with Maximal User Task Success and Minimal Human Agent Use
Authors Janarthanan Rajendran, Jatin Ganhotra, Lazaros Polymenakos
Abstract Neural end-to-end goal-oriented dialog systems showed promise to reduce the workload of human agents for customer service, as well as reduce wait time for users. However, their inability to handle new user behavior at deployment has limited their usage in real world. In this work, we propose an end-to-end trainable method for neural goal-oriented dialog systems which handles new user behaviors at deployment by transferring the dialog to a human agent intelligently. The proposed method has three goals: 1) maximize user’s task success by transferring to human agents, 2) minimize the load on the human agents by transferring to them only when it is essential and 3) learn online from the human agent’s responses to reduce human agents load further. We evaluate our proposed method on a modified-bAbI dialog task that simulates the scenario of new user behaviors occurring at test time. Experimental results show that our proposed method is effective in achieving the desired goals.
Tasks Goal-Oriented Dialog
Published 2019-07-17
URL https://arxiv.org/abs/1907.07638v1
PDF https://arxiv.org/pdf/1907.07638v1.pdf
PWC https://paperswithcode.com/paper/learning-end-to-end-goal-oriented-dialog-with-2
Repo https://github.com/IBM/modified-bAbI-dialog-tasks
Framework none

Skin Lesion Synthesis with Generative Adversarial Networks

Title Skin Lesion Synthesis with Generative Adversarial Networks
Authors Alceu Bissoto, Fábio Perez, Eduardo Valle, Sandra Avila
Abstract Skin cancer is by far the most common type of cancer. Early detection is the key to increase the chances for successful treatment significantly. Currently, Deep Neural Networks are the state-of-the-art results on automated skin cancer classification. To push the results further, we need to address the lack of annotated data, which is expensive and require much effort from specialists. To bypass this problem, we propose using Generative Adversarial Networks for generating realistic synthetic skin lesion images. To the best of our knowledge, our results are the first to show visually-appealing synthetic images that comprise clinically-meaningful information.
Tasks Medical Image Generation, Skin Cancer Classification
Published 2019-02-08
URL http://arxiv.org/abs/1902.03253v1
PDF http://arxiv.org/pdf/1902.03253v1.pdf
PWC https://paperswithcode.com/paper/skin-lesion-synthesis-with-generative
Repo https://github.com/alceubissoto/gan-skin-lesion
Framework tf

Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations

Title Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations
Authors Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu
Abstract Multi-hop knowledge graph (KG) reasoning is an effective and explainable method for predicting the target entity via reasoning paths in query answering (QA) task. Most previous methods assume that every relation in KGs has enough training triples, regardless of those few-shot relations which cannot provide sufficient triples for training robust reasoning models. In fact, the performance of existing multi-hop reasoning methods drops significantly on few-shot relations. In this paper, we propose a meta-based multi-hop reasoning method (Meta-KGR), which adopts meta-learning to learn effective meta parameters from high-frequency relations that could quickly adapt to few-shot relations. We evaluate Meta-KGR on two public datasets sampled from Freebase and NELL, and the experimental results show that Meta-KGR outperforms the current state-of-the-art methods in few-shot scenarios. Our code and datasets can be obtained from https://github.com/ THU-KEG/MetaKGR.
Tasks Meta-Learning
Published 2019-08-30
URL https://arxiv.org/abs/1908.11513v1
PDF https://arxiv.org/pdf/1908.11513v1.pdf
PWC https://paperswithcode.com/paper/adapting-meta-knowledge-graph-information-for
Repo https://github.com/THU-KEG/MetaKGR
Framework pytorch

Better Word Embeddings by Disentangling Contextual n-Gram Information

Title Better Word Embeddings by Disentangling Contextual n-Gram Information
Authors Prakhar Gupta, Matteo Pagliardini, Martin Jaggi
Abstract Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embeddings, results in improved unigram embeddings. We claim that training word embeddings along with higher n-gram embeddings helps in the removal of the contextual information from the unigrams, resulting in better stand-alone word embeddings. We empirically show the validity of our hypothesis by outperforming other competing word representation models by a significant margin on a wide variety of tasks. We make our models publicly available.
Tasks Word Embeddings
Published 2019-04-10
URL http://arxiv.org/abs/1904.05033v1
PDF http://arxiv.org/pdf/1904.05033v1.pdf
PWC https://paperswithcode.com/paper/better-word-embeddings-by-disentangling
Repo https://github.com/epfml/sent2vec
Framework none

Self-Balanced Dropout

Title Self-Balanced Dropout
Authors Shen Li, Chenhao Su, Renfen Hu, Zhengdong Lu
Abstract Dropout is known as an effective way to reduce overfitting via preventing co-adaptations of units. In this paper, we theoretically prove that the co-adaptation problem still exists after using dropout due to the correlations among the inputs. Based on the proof, we further propose Self-Balanced Dropout, a novel dropout method which uses a trainable variable to balance the influence of the input correlation on parameter update. We evaluate Self-Balanced Dropout on a range of tasks with both simple and complex models. The experimental results show that the mechanism can effectively solve the co-adaption problem to some extent and significantly improve the performance on all tasks.
Published 2019-08-06
URL https://arxiv.org/abs/1908.01968v1
PDF https://arxiv.org/pdf/1908.01968v1.pdf
PWC https://paperswithcode.com/paper/self-balanced-dropout
Repo https://github.com/shenshen-hungry/Self-Balanced-Dropout
Framework tf

The Replica Dataset: A Digital Replica of Indoor Spaces

Title The Replica Dataset: A Digital Replica of Indoor Spaces
Authors Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gillingham, Elias Mueggler, Luis Pesqueira, Manolis Savva, Dhruv Batra, Hauke M. Strasdat, Renzo De Nardi, Michael Goesele, Steven Lovegrove, Richard Newcombe
Abstract We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor scene reconstructions at room and building scale. Each scene consists of a dense mesh, high-resolution high-dynamic-range (HDR) textures, per-primitive semantic class and instance information, and planar mirror and glass reflectors. The goal of Replica is to enable machine learning (ML) research that relies on visually, geometrically, and semantically realistic generative models of the world - for instance, egocentric computer vision, semantic segmentation in 2D and 3D, geometric inference, and the development of embodied agents (virtual robots) performing navigation, instruction following, and question answering. Due to the high level of realism of the renderings from Replica, there is hope that ML systems trained on Replica may transfer directly to real world image and video data. Together with the data, we are releasing a minimal C++ SDK as a starting point for working with the Replica dataset. In addition, Replica is `Habitat-compatible’, i.e. can be natively used with AI Habitat for training and testing embodied agents. |
Tasks 3D Scene Reconstruction, Question Answering, Semantic Segmentation
Published 2019-06-13
URL https://arxiv.org/abs/1906.05797v1
PDF https://arxiv.org/pdf/1906.05797v1.pdf
PWC https://paperswithcode.com/paper/the-replica-dataset-a-digital-replica-of
Repo https://github.com/matthewbegun/awesome-stars
Framework tf

Towards Knowledge-Based Recommender Dialog System

Title Towards Knowledge-Based Recommender Dialog System
Authors Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, Jie Tang
Abstract In this paper, we propose a novel end-to-end framework called KBRD, which stands for Knowledge-Based Recommender Dialog System. It integrates the recommender system and the dialog generation system. The dialog system can enhance the performance of the recommendation system by introducing knowledge-grounded information about users’ preferences, and the recommender system can improve that of the dialog generation system by providing recommendation-aware vocabulary bias. Experimental results demonstrate that our proposed model has significant advantages over the baselines in both the evaluation of dialog generation and recommendation. A series of analyses show that the two systems can bring mutual benefits to each other, and the introduced knowledge contributes to both their performances.
Tasks Recommendation Systems
Published 2019-08-15
URL https://arxiv.org/abs/1908.05391v2
PDF https://arxiv.org/pdf/1908.05391v2.pdf
PWC https://paperswithcode.com/paper/towards-knowledge-based-recommender-dialog
Repo https://github.com/THUDM/KBRD
Framework pytorch

An Image Clustering Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids and MMD Distance

Title An Image Clustering Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids and MMD Distance
Authors Qiuyu Zhu, Zhengyong Wang
Abstract In this paper, we propose a novel, effective and simpler end-to-end image clustering auto-encoder algorithm: ICAE. The algorithm uses PEDCC (Predefined Evenly-Distributed Class Centroids) as the clustering centers, which ensures the inter-class distance of latent features is maximal, and adds data distribution constraint, data augmentation constraint, auto-encoder reconstruction constraint and Sobel smooth constraint to improve the clustering performance. Specifically, we perform one-to-one data augmentation to learn the more effective features. The data and the augmented data are simultaneously input into the autoencoder to obtain latent features and the augmented latent features whose similarity are constrained by an augmentation loss. Then, making use of the maximum mean discrepancy distance (MMD), we combine the latent features and augmented latent features to make their distribution close to the PEDCC distribution (uniform distribution between classes, Dirac distribution within the class) to further learn clustering-oriented features. At the same time, the MSE of the original input image and reconstructed image is used as reconstruction constraint, and the Sobel smooth loss to build generalization constraint to improve the generalization ability. Finally, extensive experiments on three common datasets MNIST, Fashion-MNIST, COIL20 are conducted. The experimental results show that the algorithm has achieved the best clustering results so far. In addition, we can use the predefined PEDCC class centers, and the decoder to clearly generate the samples of each class. The code can be downloaded at https://github.com/zyWang-Power/Clustering!
Tasks Data Augmentation, Image Clustering
Published 2019-06-10
URL https://arxiv.org/abs/1906.03905v2
PDF https://arxiv.org/pdf/1906.03905v2.pdf
PWC https://paperswithcode.com/paper/an-image-clustering-auto-encoder-based-on
Repo https://github.com/Zhengyong-Wang/Clustering
Framework pytorch

Targeted Example Generation for Compilation Errors

Title Targeted Example Generation for Compilation Errors
Authors Umair Z. Ahmed, Renuka Sindhgatta, Nisheeth Srivastava, Amey Karkare
Abstract We present TEGCER, an automated feedback tool for novice programmers. TEGCER uses supervised classification to match compilation errors in new code submissions with relevant pre-existing errors, submitted by other students before. The dense neural network used to perform this classification task is trained on 15000+ error-repair code examples. The proposed model yields a test set classification Pred@3 accuracy of 97.7% across 212 error category labels. Using this model as its base, TEGCER presents students with the closest relevant examples of solutions for their specific error on demand.
Published 2019-09-02
URL https://arxiv.org/abs/1909.00769v2
PDF https://arxiv.org/pdf/1909.00769v2.pdf
PWC https://paperswithcode.com/paper/targeted-example-generation-for-compilation
Repo https://github.com/umairzahmed/tegcer
Framework none

Efficient Large-Scale Multi-Drone Delivery Using Transit Networks

Title Efficient Large-Scale Multi-Drone Delivery Using Transit Networks
Authors Shushman Choudhury, Kiril Solovey, Mykel J. Kochenderfer, Marco Pavone
Abstract We consider the problem of controlling a large fleet of drones to deliver packages simultaneously across broad urban areas. To conserve energy, drones hop between public transit vehicles (e.g., buses and trams). We design a comprehensive algorithmic framework that strives to minimize the maximum time to complete any delivery. We address the multifaceted complexity of the problem through a two-layer approach. First, the upper layer assigns drones to package delivery sequences with a near-optimal polynomial-time task allocation algorithm. Then, the lower layer executes the allocation by periodically routing the fleet over the transit network while employing efficient bounded-suboptimal multi-agent pathfinding techniques tailored to our setting. Experiments demonstrate the efficiency of our approach on settings with up to $200$ drones, $5000$ packages, and transit networks with up to $8000$ stops in San Francisco and Washington DC. Our results show that the framework computes solutions within a few seconds (up to $2$ minutes at most) on commodity hardware, and that drones travel up to $450 %$ of their flight range with public transit.
Published 2019-09-26
URL https://arxiv.org/abs/1909.11840v2
PDF https://arxiv.org/pdf/1909.11840v2.pdf
PWC https://paperswithcode.com/paper/efficient-large-scale-multi-drone-delivery
Repo https://github.com/sisl/MultiAgentAllocationTransit.jl
Framework none

LUTNet: Rethinking Inference in FPGA Soft Logic

Title LUTNet: Rethinking Inference in FPGA Soft Logic
Authors Erwei Wang, James J. Davis, Peter Y. K. Cheung, George A. Constantinides
Abstract Research has shown that deep neural networks contain significant redundancy, and that high classification accuracies can be achieved even when weights and activations are quantised down to binary values. Network binarisation on FPGAs greatly increases area efficiency by replacing resource-hungry multipliers with lightweight XNOR gates. However, an FPGA’s fundamental building block, the K-LUT, is capable of implementing far more than an XNOR: it can perform any K-input Boolean operation. Inspired by this observation, we propose LUTNet, an end-to-end hardware-software framework for the construction of area-efficient FPGA-based neural network accelerators using the native LUTs as inference operators. We demonstrate that the exploitation of LUT flexibility allows for far heavier pruning than possible in prior works, resulting in significant area savings while achieving comparable accuracy. Against the state-of-the-art binarised neural network implementation, we achieve twice the area efficiency for several standard network models when inferencing popular datasets. We also demonstrate that even greater energy efficiency improvements are obtainable.
Published 2019-04-01
URL http://arxiv.org/abs/1904.00938v1
PDF http://arxiv.org/pdf/1904.00938v1.pdf
PWC https://paperswithcode.com/paper/lutnet-rethinking-inference-in-fpga-soft
Repo https://github.com/awai54st/LUTNet
Framework tf

Magnetoresistive RAM for error resilient XNOR-Nets

Title Magnetoresistive RAM for error resilient XNOR-Nets
Authors Michail Tzoufras, Marcin Gajek, Andrew Walker
Abstract We trained three Binarized Convolutional Neural Network architectures (LeNet-4, Network-In-Network, AlexNet) on a variety of datasets (MNIST, CIFAR-10, CIFAR-100, extended SVHN, ImageNet) using error-prone activations and tested them without errors to study the resilience of the training process. With the exception of the AlexNet when trained on the ImageNet dataset, we found that Bit Error Rates of a few percent during training do not degrade the test accuracy. Furthermore, by training the AlexNet on progressively smaller subsets of ImageNet classes, we observed increasing tolerance to activation errors. The ability to operate with high BERs is critical for reducing power consumption in existing hardware and for facilitating emerging memory technologies. We discuss how operating at moderate BER can enable Magnetoresistive RAM with higher endurance, speed and density.
Published 2019-05-24
URL https://arxiv.org/abs/1905.10927v1
PDF https://arxiv.org/pdf/1905.10927v1.pdf
PWC https://paperswithcode.com/paper/magnetoresistive-ram-for-error-resilient-xnor
Repo https://github.com/michail-tzoufras/ConvNets_with_Activation_Errors
Framework pytorch

CTCModel: a Keras Model for Connectionist Temporal Classification

Title CTCModel: a Keras Model for Connectionist Temporal Classification
Authors Yann Soullard, Cyprien Ruffino, Thierry Paquet
Abstract We report an extension of a Keras Model, called CTCModel, to perform the Connectionist Temporal Classification (CTC) in a transparent way. Combined with Recurrent Neural Networks, the Connectionist Temporal Classification is the reference method for dealing with unsegmented input sequences, i.e. with data that are a couple of observation and label sequences where each label is related to a subset of observation frames. CTCModel makes use of the CTC implementation in the Tensorflow backend for training and predicting sequences of labels using Keras. It consists of three branches made of Keras models: one for training, computing the CTC loss function; one for predicting, providing sequences of labels; and one for evaluating that returns standard metrics for analyzing sequences of predictions.
Published 2019-01-23
URL http://arxiv.org/abs/1901.07957v1
PDF http://arxiv.org/pdf/1901.07957v1.pdf
PWC https://paperswithcode.com/paper/ctcmodel-a-keras-model-for-connectionist
Repo https://github.com/ysoullard/CTCModel
Framework tf
comments powered by Disqus