October 20, 2019

3089 words 15 mins read

Paper Group ANR 34

Distilling Critical Paths in Convolutional Neural Networks. Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize. Introducing a hybrid model of DEA and data mining in evaluating efficiency. Case study: Bank Branches. Objects Localisation from Motion with Constraints. Improving CNN Training using Disentanglement for L …

Distilling Critical Paths in Convolutional Neural Networks


Title	Distilling Critical Paths in Convolutional Neural Networks
Authors	Fuxun Yu, Zhuwei Qin, Xiang Chen
Abstract	Neural network compression and acceleration are widely demanded currently due to the resource constraints on most deployment targets. In this paper, through analyzing the filter activation, gradients, and visualizing the filters’ functionality in convolutional neural networks, we show that the filters in higher layers learn extremely task-specific features, which are exclusive for only a small subset of the overall tasks, or even a single class. Based on such findings, we reveal the critical paths of information flow for different classes. And by their intrinsic property of exclusiveness, we propose a critical path distillation method, which can effectively customize the convolutional neural networks to small ones with much smaller model size and less computation.
Tasks	Neural Network Compression
Published	2018-10-28
URL	http://arxiv.org/abs/1811.02643v2
PDF	http://arxiv.org/pdf/1811.02643v2.pdf
PWC	https://paperswithcode.com/paper/distilling-critical-paths-in-convolutional
Repo
Framework

Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize


Title	Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize
Authors	Chandra Khatri, Behnam Hedayatnia, Anu Venkatesh, Jeff Nunn, Yi Pan, Qing Liu, Han Song, Anna Gottardi, Sanjeev Kwatra, Sanju Pancholi, Ming Cheng, Qinglang Chen, Lauren Stubel, Karthik Gopalakrishnan, Kate Bland, Raefer Gabriel, Arindam Mandal, Dilek Hakkani-Tur, Gene Hwang, Nate Michel, Eric King, Rohit Prasad
Abstract	Building open domain conversational systems that allow users to have engaging conversations on topics of their choice is a challenging task. Alexa Prize was launched in 2016 to tackle the problem of achieving natural, sustained, coherent and engaging open-domain dialogs. In the second iteration of the competition in 2018, university teams advanced the state of the art by using context in dialog models, leveraging knowledge graphs for language understanding, handling complex utterances, building statistical and hierarchical dialog managers, and leveraging model-driven signals from user responses. The 2018 competition also included the provision of a suite of tools and models to the competitors including the CoBot (conversational bot) toolkit, topic and dialog act detection models, conversation evaluators, and a sensitive content detection model so that the competing teams could focus on building knowledge-rich, coherent and engaging multi-turn dialog systems. This paper outlines the advances developed by the university teams as well as the Alexa Prize team to achieve the common goal of advancing the science of Conversational AI. We address several key open-ended problems such as conversational speech recognition, open domain natural language understanding, commonsense reasoning, statistical dialog management, and dialog evaluation. These collaborative efforts have driven improved experiences by Alexa users to an average rating of 3.61, the median duration of 2 mins 18 seconds, and average turns to 14.6, increases of 14%, 92%, 54% respectively since the launch of the 2018 competition. For conversational speech recognition, we have improved our relative Word Error Rate by 55% and our relative Entity Error Rate by 34% since the launch of the Alexa Prize. Socialbots improved in quality significantly more rapidly in 2018, in part due to the release of the CoBot toolkit.
Tasks	Knowledge Graphs, Speech Recognition
Published	2018-12-27
URL	http://arxiv.org/abs/1812.10757v1
PDF	http://arxiv.org/pdf/1812.10757v1.pdf
PWC	https://paperswithcode.com/paper/advancing-the-state-of-the-art-in-open-domain
Repo
Framework

Introducing a hybrid model of DEA and data mining in evaluating efficiency. Case study: Bank Branches


Title	Introducing a hybrid model of DEA and data mining in evaluating efficiency. Case study: Bank Branches
Authors	Sara Hosseinzadeh Kassani, Peyman Hosseinzadeh Kassani, Seyed Esmaeel Najafi
Abstract	The banking industry is very important for an economic cycle of each country and provides some quality of services for us. With the advancement in technology and rapidly increasing of the complexity of the business environment, it has become more competitive than the past so that efficiency analysis in the banking industry attracts much attention in recent years. From many aspects, such analyses at the branch level are more desirable. Evaluating the branch performance with the purpose of eliminating deficiency can be a crucial issue for branch managers to measure branch efficiency. This work not only can lead to a better understanding of bank branch performance but also give further information to enhance managerial decisions to recognize problematic areas. To achieve this purpose, this study presents an integrated approach based on Data Envelopment Analysis (DEA), Clustering algorithms and Polynomial Pattern Classifier for constructing a classifier to identify a class of bank branches. First, the efficiency estimates of individual branches are evaluated by using the DEA approach. Next, when the range and number of classes were identified by experts, the number of clusters is identified by an agglomerative hierarchical clustering algorithm based on some statistical methods. Next, we divide our raw data into k clusters By means of self-organizing map (SOM) neural networks. Finally, all clusters are fed into the reduced multivariate polynomial model to predict the classes of data.
Tasks
Published	2018-10-10
URL	http://arxiv.org/abs/1810.05524v1
PDF	http://arxiv.org/pdf/1810.05524v1.pdf
PWC	https://paperswithcode.com/paper/introducing-a-hybrid-model-of-dea-and-data
Repo
Framework

Objects Localisation from Motion with Constraints


Title	Objects Localisation from Motion with Constraints
Authors	Paul Gay, Alessio Del Bue
Abstract	This paper presents a method to estimate the 3D object position and occupancy given a set of object detections in multiple images and calibrated cameras. This problem is modelled as the estimation of a set of quadrics given 2D conics fit to the object bounding boxes. Although a closed form solution has been recently proposed, the resulting quadrics can be inaccurate or even be non valid ellipsoids in presence of noisy and inaccurate detections. This effect is especially important in case of small baselines, resulting in dramatic failures. To cope with this problem, we propose a set of linear constraints by matching the centres of the reprojected quadrics with the centres of the observed conics. These constraints can be solved with a linear system thus providing a more computationally efficient solution with respect to a non-linear alternative. Experiments on real data show that the proposed approach improves significantly the accuracy and the validity of the ellipsoids.
Tasks
Published	2018-03-28
URL	http://arxiv.org/abs/1803.10474v2
PDF	http://arxiv.org/pdf/1803.10474v2.pdf
PWC	https://paperswithcode.com/paper/objects-localisation-from-motion-with
Repo
Framework

Improving CNN Training using Disentanglement for Liver Lesion Classification in CT


Title	Improving CNN Training using Disentanglement for Liver Lesion Classification in CT
Authors	Avi Ben-Cohen, Roey Mechrez, Noa Yedidia, Hayit Greenspan
Abstract	Training data is the key component in designing algorithms for medical image analysis and in many cases it is the main bottleneck in achieving good results. Recent progress in image generation has enabled the training of neural network based solutions using synthetic data. A key factor in the generation of new samples is controlling the important appearance features and potentially being able to generate a new sample of a specific class with different variants. In this work we suggest the synthesis of new data by mixing the class specified and unspecified representation of different factors in the training data. Our experiments on liver lesion classification in CT show an average improvement of 7.4% in accuracy over the baseline training scheme.
Tasks	Image Generation
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00501v1
PDF	http://arxiv.org/pdf/1811.00501v1.pdf
PWC	https://paperswithcode.com/paper/improving-cnn-training-using-disentanglement
Repo
Framework

A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)


Title	A Hierarchical Structured Self-Attentive Model for Extractive Document Summarization (HSSAS)
Authors	Kamal Al-Sabahi, Zhang Zuping, Mohammed Nadher
Abstract	The recent advance in neural network architecture and training algorithms have shown the effectiveness of representation learning. The neural network-based models generate better representation than the traditional ones. They have the ability to automatically learn the distributed representation for sentences and documents. To this end, we proposed a novel model that addresses several issues that are not adequately modeled by the previously proposed models, such as the memory problem and incorporating the knowledge of document structure. Our model uses a hierarchical structured self-attention mechanism to create the sentence and document embeddings. This architecture mirrors the hierarchical structure of the document and in turn enables us to obtain better feature representation. The attention mechanism provides extra source of information to guide the summary extraction. The new model treated the summarization task as a classification problem in which the model computes the respective probabilities of sentence-summary membership. The model predictions are broken up by several features such as information content, salience, novelty and positional representation. The proposed model was evaluated on two well-known datasets, the CNN / Daily Mail, and DUC 2002. The experimental results show that our model outperforms the current extractive state-of-the-art by a considerable margin.
Tasks	Document Summarization, Extractive Document Summarization, Representation Learning
Published	2018-05-20
URL	http://arxiv.org/abs/1805.07799v1
PDF	http://arxiv.org/pdf/1805.07799v1.pdf
PWC	https://paperswithcode.com/paper/a-hierarchical-structured-self-attentive
Repo
Framework

Quantifying Learning Guarantees for Convex but Inconsistent Surrogates


Title	Quantifying Learning Guarantees for Convex but Inconsistent Surrogates
Authors	Kirill Struminsky, Simon Lacoste-Julien, Anton Osokin
Abstract	We study consistency properties of machine learning methods based on minimizing convex surrogates. We extend the recent framework of Osokin et al. (2017) for the quantitative analysis of consistency properties to the case of inconsistent surrogates. Our key technical contribution consists in a new lower bound on the calibration function for the quadratic surrogate, which is non-trivial (not always zero) for inconsistent cases. The new bound allows to quantify the level of inconsistency of the setting and shows how learning with inconsistent surrogates can have guarantees on sample complexity and optimization difficulty. We apply our theory to two concrete cases: multi-class classification with the tree-structured loss and ranking with the mean average precision loss. The results show the approximation-computation trade-offs caused by inconsistent surrogates and their potential benefits.
Tasks	Calibration
Published	2018-10-26
URL	http://arxiv.org/abs/1810.11544v2
PDF	http://arxiv.org/pdf/1810.11544v2.pdf
PWC	https://paperswithcode.com/paper/quantifying-learning-guarantees-for-convex
Repo
Framework

Efficient Deep Learning on Multi-Source Private Data


Title	Efficient Deep Learning on Multi-Source Private Data
Authors	Nick Hynes, Raymond Cheng, Dawn Song
Abstract	Machine learning models benefit from large and diverse datasets. Using such datasets, however, often requires trusting a centralized data aggregator. For sensitive applications like healthcare and finance this is undesirable as it could compromise patient privacy or divulge trade secrets. Recent advances in secure and privacy-preserving computation, including trusted hardware enclaves and differential privacy, offer a way for mutually distrusting parties to efficiently train a machine learning model without revealing the training data. In this work, we introduce Myelin, a deep learning framework which combines these privacy-preservation primitives, and use it to establish a baseline level of performance for fully private machine learning.
Tasks
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06689v1
PDF	http://arxiv.org/pdf/1807.06689v1.pdf
PWC	https://paperswithcode.com/paper/efficient-deep-learning-on-multi-source
Repo
Framework

CSfM: Community-based Structure from Motion


Title	CSfM: Community-based Structure from Motion
Authors	Hainan Cui, Shuhan Shen, Xiang Gao, Zhanyi Hu
Abstract	Structure-from-Motion approaches could be broadly divided into two classes: incremental and global. While incremental manner is robust to outliers, it suffers from error accumulation and heavy computation load. The global manner has the advantage of simultaneously estimating all camera poses, but it is usually sensitive to epipolar geometry outliers. In this paper, we propose an adaptive community-based SfM (CSfM) method which takes both robustness and efficiency into consideration. First, the epipolar geometry graph is partitioned into separate communities. Then, the reconstruction problem is solved for each community in parallel. Finally, the reconstruction results are merged by a novel global similarity averaging method, which solves three convex $L1$ optimization problems. Experimental results show that our method performs better than many of the state-of-the-art global SfM approaches in terms of computational efficiency, while achieves similar or better reconstruction accuracy and robustness than many of the state-of-the-art incremental SfM approaches.
Tasks
Published	2018-03-23
URL	http://arxiv.org/abs/1803.08716v1
PDF	http://arxiv.org/pdf/1803.08716v1.pdf
PWC	https://paperswithcode.com/paper/csfm-community-based-structure-from-motion
Repo
Framework

Homophonic Quotients of Linguistic Free Groups: German, Korean, and Turkish


Title	Homophonic Quotients of Linguistic Free Groups: German, Korean, and Turkish
Authors	Herbert Gangl, Gizem Karaali, Woohyung Lee
Abstract	In 1993, the homophonic quotient groups for French and English (the quotient of the free group generated by the French (respectively English) alphabet determined by relations representing standard pronunciation rules) were explicitly characterized [5]. In this paper we apply the same methodology to three different language systems: German, Korean, and Turkish. We argue that our results point to some interesting differences between these three languages (or at least their current script systems).
Tasks
Published	2018-08-10
URL	http://arxiv.org/abs/1808.04254v1
PDF	http://arxiv.org/pdf/1808.04254v1.pdf
PWC	https://paperswithcode.com/paper/homophonic-quotients-of-linguistic-free
Repo
Framework

Keypoint Transfer for Fast Whole-Body Segmentation


Title	Keypoint Transfer for Fast Whole-Body Segmentation
Authors	Christian Wachinger, Matthew Toews, Georg Langs, William Wells, Polina Golland
Abstract	We introduce an approach for image segmentation based on sparse correspondences between keypoints in testing and training images. Keypoints represent automatically identified distinctive image locations, where each keypoint correspondence suggests a transformation between images. We use these correspondences to transfer label maps of entire organs from the training images to the test image. The keypoint transfer algorithm includes three steps: (i) keypoint matching, (ii) voting-based keypoint labeling, and (iii) keypoint-based probabilistic transfer of organ segmentations. We report segmentation results for abdominal organs in whole-body CT and MRI, as well as in contrast-enhanced CT and MRI. Our method offers a speed-up of about three orders of magnitude in comparison to common multi-atlas segmentation, while achieving an accuracy that compares favorably. Moreover, keypoint transfer does not require the registration to an atlas or a training phase. Finally, the method allows for the segmentation of scans with highly variable field-of-view.
Tasks	Semantic Segmentation
Published	2018-06-22
URL	http://arxiv.org/abs/1806.08723v1
PDF	http://arxiv.org/pdf/1806.08723v1.pdf
PWC	https://paperswithcode.com/paper/keypoint-transfer-for-fast-whole-body
Repo
Framework

Not All Attributes are Created Equal: $d_{\mathcal{X}}$-Private Mechanisms for Linear Queries


Title	Not All Attributes are Created Equal: $d_{\mathcal{X}}$-Private Mechanisms for Linear Queries
Authors	Parameswaran Kamalaruban, Victor Perrier, Hassan Jameel Asghar, Mohamed Ali Kaafar
Abstract	Differential privacy provides strong privacy guarantees simultaneously enabling useful insights from sensitive datasets. However, it provides the same level of protection for all elements (individuals and attributes) in the data. There are practical scenarios where some data attributes need more/less protection than others. In this paper, we consider $d_{\mathcal{X}}$-privacy, an instantiation of the privacy notion introduced in \cite{chatzikokolakis2013broadening}, which allows this flexibility by specifying a separate privacy budget for each pair of elements in the data domain. We describe a systematic procedure to tailor any existing differentially private mechanism that assumes a query set and a sensitivity vector as input into its $d_{\mathcal{X}}$-private variant, specifically focusing on linear queries. Our proposed meta procedure has broad applications as linear queries form the basis of a range of data analysis and machine learning algorithms, and the ability to define a more flexible privacy budget across the data domain results in improved privacy/utility tradeoff in these applications. We propose several $d_{\mathcal{X}}$-private mechanisms, and provide theoretical guarantees on the trade-off between utility and privacy. We also experimentally demonstrate the effectiveness of our procedure, by evaluating our proposed $d_{\mathcal{X}}$-private Laplace mechanism on both synthetic and real datasets using a set of randomly generated linear queries.
Tasks
Published	2018-06-06
URL	https://arxiv.org/abs/1806.02389v2
PDF	https://arxiv.org/pdf/1806.02389v2.pdf
PWC	https://paperswithcode.com/paper/d_mathcalx-private-mechanisms-for-linear
Repo
Framework

Meta-Learning Multi-task Communication


Title	Meta-Learning Multi-task Communication
Authors	Pengfei Liu, Xuanjing Huang
Abstract	In this paper, we describe a general framework: Parameters Read-Write Networks (PRaWNs) to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks. We propose to alleviate this problem by incorporating an inductive bias into the process of multi-task learning, that each task can keep informed of not only the knowledge stored in other tasks but the way how other tasks maintain their knowledge. In practice, we achieve above inductive bias by allowing different tasks to communicate by passing both hidden variables and gradients explicitly. Experimentally, we evaluate proposed methods on three groups of tasks and two types of settings (\textsc{in-task} and \textsc{out-of-task}). Quantitative and qualitative results show their effectiveness.
Tasks	Meta-Learning, Multi-Task Learning
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09988v1
PDF	http://arxiv.org/pdf/1810.09988v1.pdf
PWC	https://paperswithcode.com/paper/meta-learning-multi-task-communication
Repo
Framework

MyCaffe: A Complete C# Re-Write of Caffe with Reinforcement Learning


Title	MyCaffe: A Complete C# Re-Write of Caffe with Reinforcement Learning
Authors	David W. Brown
Abstract	Over the past few years Caffe, from Berkeley AI Research, has gained a strong following in the deep learning community with over 15K forks on the github.com/BLVC/Caffe site. With its well organized, very modular C++ design it is easy to work with and very fast. However, in the world of Windows development, C# has helped accelerate development with many of the enhancements that it offers over C++, such as garbage collection, a very rich .NET programming framework and easy database access via Entity Frameworks. So how can a C# developer use the advances of C# to take full advantage of the benefits offered by the Berkeley Caffe deep learning system? The answer is the fully open source, ‘MyCaffe’ for Windows .NET programmers. MyCaffe is an open source, complete C# language re-write of Berkeley’s Caffe. This article describes the general architecture of MyCaffe including the newly added MyCaffeTrainerRL for Reinforcement Learning. In addition, this article discusses how MyCaffe closely follows the C++ Caffe, while talking efficiently to the low level NVIDIA CUDA hardware to offer a high performance, highly programmable deep learning system for Windows .NET programmers.
Tasks
Published	2018-10-04
URL	http://arxiv.org/abs/1810.02272v1
PDF	http://arxiv.org/pdf/1810.02272v1.pdf
PWC	https://paperswithcode.com/paper/mycaffe-a-complete-c-re-write-of-caffe-with
Repo
Framework

EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer


Title	EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
Authors	Zhijie Wu, Chunjin Song, Yang Zhou, Minglun Gong, Hui Huang
Abstract	Style transfer has been an important topic both in computer vision and graphics. Since the seminal work of Gatys et al. first demonstrates the power of stylization through optimization in the deep feature space, quite a few approaches have achieved real-time arbitrary style transfer with straightforward statistic matching techniques. In this work, our key observation is that only considering features in the input style image for the global deep feature statistic matching or local patch swap may not always ensure a satisfactory style transfer; see e.g., Figure 1. Instead, we propose a novel transfer framework, EFANet, that aims to jointly analyze and better align exchangeable features extracted from content and style image pair. In this way, the style features from the style image seek for the best compatibility with the content information in the content image, leading to more structured stylization results. In addition, a new whitening loss is developed for purifying the computed content features and better fusion with styles in feature space. Qualitative and quantitative experiments demonstrate the advantages of our approach.
Tasks	Style Transfer
Published	2018-11-26
URL	https://arxiv.org/abs/1811.10352v3
PDF	https://arxiv.org/pdf/1811.10352v3.pdf
PWC	https://paperswithcode.com/paper/pair-wise-exchangeable-feature-extraction-for
Repo
Framework