January 26, 2020

3248 words 16 mins read

Paper Group ANR 1479

Customers Churn Prediction in Financial Institution Using Artificial Neural Network. Machine Learning Based Routing Congestion Prediction in FPGA High-Level Synthesis. Machine Unlearning. Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations. Semi-Supervised Neural Text Generation by Joint Learning of Natural Langu …

Customers Churn Prediction in Financial Institution Using Artificial Neural Network


Title	Customers Churn Prediction in Financial Institution Using Artificial Neural Network
Authors	Kamorudeen A. Amuda, Adesesan B. Adeyemo
Abstract	In this study, a predictive model using Multi-layer Perceptron of Artificial Neural Network architecture was developed to predict customer churn in a financial institution. Previous researches have used supervised machine learning classifiers such as Logistic Regression, Decision Tree, Support Vector Machine, K-Nearest Neighbors, and Random Forest. These classifiers require human effort to perform feature engineering which leads to over-specified and incomplete feature selection. Therefore, this research developed a model to eliminate manual feature engineering in data preprocessing stage. Fifty thousand customers? data were extracted from the database of one of the leading financial institution in Nigeria for the study. The multi-layer perceptron model was built with python programming language and used two overfitting techniques (Dropout and L2 regularization). The implementation done in python was compared with another model in Neuro solution infinity software. The results showed that the Artificial Neural Network software development (Python) had comparable performance with that obtained from the Neuro Solution Infinity software. The accuracy rates are 97.53% and 97.4% while ROC (Receiver Operating Characteristic) curve graphs are 0.89 and 0.85 respectively.
Tasks	Feature Engineering, Feature Selection, L2 Regularization
Published	2019-12-23
URL	https://arxiv.org/abs/1912.11346v1
PDF	https://arxiv.org/pdf/1912.11346v1.pdf
PWC	https://paperswithcode.com/paper/customers-churn-prediction-in-financial
Repo
Framework

Machine Learning Based Routing Congestion Prediction in FPGA High-Level Synthesis


Title	Machine Learning Based Routing Congestion Prediction in FPGA High-Level Synthesis
Authors	Jieru Zhao, Tingyuan Liang, Sharad Sinha, Wei Zhang
Abstract	High-level synthesis (HLS) shortens the development time of hardware designs and enables faster design space exploration at a higher abstraction level. Optimization of complex applications in HLS is challenging due to the effects of implementation issues such as routing congestion. Routing congestion estimation is absent or inaccurate in existing HLS design methods and tools. Early and accurate congestion estimation is of great benefit to guide the optimization in HLS and improve the efficiency of implementation. However, routability, a serious concern in FPGA designs, has been difficult to evaluate in HLS without analyzing post-implementation details after Place and Route. To this end, we propose a novel method to predict routing congestion in HLS using machine learning and map the expected congested regions in the design to the relevant high-level source code. This is greatly beneficial in early identification of routability oriented bottlenecks in the high-level source code without running time-consuming register-transfer level (RTL) implementation flow. Experiments demonstrate that our approach accurately estimates vertical and horizontal routing congestion with errors of 6.71% and 10.05% respectively. By presenting Face Detection application as a case study, we show that by discovering the bottlenecks in high-level source code, routing congestion can be easily and quickly resolved compared to the efforts involved in RTL implementation and design feedback.
Tasks	Face Detection
Published	2019-05-06
URL	https://arxiv.org/abs/1905.03852v1
PDF	https://arxiv.org/pdf/1905.03852v1.pdf
PWC	https://paperswithcode.com/paper/190503852
Repo
Framework

Machine Unlearning


Title	Machine Unlearning
Authors	Lucas Bourtoule, Varun Chandrasekaran, Christopher Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, Nicolas Papernot
Abstract	Once users have shared their data online, it is generally difficult for them to revoke access and ask for the data to be deleted. Machine learning (ML) exacerbates this problem because any model trained with said data may have memorized it, putting users at risk of a successful privacy attack exposing their information. Yet, having models unlearn is notoriously difficult. After a data point is removed from a training set, one often resorts to entirely retraining downstream models from scratch. We introduce SISA training, a framework that decreases the number of model parameters affected by an unlearning request and caches intermediate outputs of the training algorithm to limit the number of model updates that need to be computed to have these parameters unlearn. This framework reduces the computational overhead associated with unlearning, even in the worst-case setting where unlearning requests are made uniformly across the training set. In some cases, we may have a prior on the distribution of unlearning requests that will be issued by users. We may take this prior into account to partition and order data accordingly and further decrease overhead from unlearning. Our evaluation spans two datasets from different application domains, with corresponding motivations for unlearning. Under no distributional assumptions, we observe that SISA training improves unlearning for the Purchase dataset by 3.13x, and 1.658x for the SVHN dataset, over retraining from scratch. We also validate how knowledge of the unlearning distribution provides further improvements in retraining time by simulating a scenario where we model unlearning requests that come from users of a commercial product that is available in countries with varying sensitivity to privacy. Our work contributes to practical data governance in machine learning.
Tasks
Published	2019-12-09
URL	https://arxiv.org/abs/1912.03817v1
PDF	https://arxiv.org/pdf/1912.03817v1.pdf
PWC	https://paperswithcode.com/paper/machine-unlearning
Repo
Framework

Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations


Title	Semi-supervised Thai Sentence Segmentation Using Local and Distant Word Representations
Authors	Chanatip Saetia, Ekapol Chuangsuwanich, Tawunrat Chalothorn, Peerapon Vateekul
Abstract	A sentence is typically treated as the minimal syntactic unit used for extracting valuable information from a longer piece of text. However, in written Thai, there are no explicit sentence markers. We proposed a deep learning model for the task of sentence segmentation that includes three main contributions. First, we integrate n-gram embedding as a local representation to capture word groups near sentence boundaries. Second, to focus on the keywords of dependent clauses, we combine the model with a distant representation obtained from self-attention modules. Finally, due to the scarcity of labeled data, for which annotation is difficult and time-consuming, we also investigate and adapt Cross-View Training (CVT) as a semi-supervised learning technique, allowing us to utilize unlabeled data to improve the model representations. In the Thai sentence segmentation experiments, our model reduced the relative error by 7.4% and 10.5% compared with the baseline models on the Orchid and UGWC datasets, respectively. We also applied our model to the task of pronunciation recovery on the IWSLT English dataset. Our model outperformed the prior sequence tagging models, achieving a relative error reduction of 2.5%. Ablation studies revealed that utilizing n-gram presentations was the main contributing factor for Thai, while the semi-supervised training helped the most for English.
Tasks
Published	2019-08-04
URL	https://arxiv.org/abs/1908.01294v2
PDF	https://arxiv.org/pdf/1908.01294v2.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-thai-sentence-segmentation
Repo
Framework

Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models


Title	Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models
Authors	Raheel Qader, François Portet, Cyril Labbé
Abstract	In Natural Language Generation (NLG), End-to-End (E2E) systems trained through deep learning have recently gained a strong interest. Such deep models need a large amount of carefully annotated data to reach satisfactory performance. However, acquiring such datasets for every new NLG application is a tedious and time-consuming task. In this paper, we propose a semi-supervised deep learning scheme that can learn from non-annotated data and annotated data when available. It uses an NLG and a Natural Language Understanding (NLU) sequence-to-sequence models which are learned jointly to compensate for the lack of annotation. Experiments on two benchmark datasets show that, with limited amount of annotated data, the method can achieve very competitive results while not using any pre-processing or re-scoring tricks. These findings open the way to the exploitation of non-annotated datasets which is the current bottleneck for the E2E NLG system development to new applications.
Tasks	Text Generation
Published	2019-09-29
URL	https://arxiv.org/abs/1910.03484v1
PDF	https://arxiv.org/pdf/1910.03484v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-neural-text-generation-by
Repo
Framework

Semantic variation operators for multidimensional genetic programming


Title	Semantic variation operators for multidimensional genetic programming
Authors	William La Cava, Jason H. Moore
Abstract	Multidimensional genetic programming represents candidate solutions as sets of programs, and thereby provides an interesting framework for exploiting building block identification. Towards this goal, we investigate the use of machine learning as a way to bias which components of programs are promoted, and propose two semantic operators to choose where useful building blocks are placed during crossover. A forward stagewise crossover operator we propose leads to significant improvements on a set of regression problems, and produces state-of-the-art results in a large benchmark study. We discuss this architecture and others in terms of their propensity for allowing heuristic search to utilize information during the evolutionary process. Finally, we look at the collinearity and complexity of the data representations that result from these architectures, with a view towards disentangling factors of variation in application.
Tasks
Published	2019-04-18
URL	http://arxiv.org/abs/1904.08577v1
PDF	http://arxiv.org/pdf/1904.08577v1.pdf
PWC	https://paperswithcode.com/paper/semantic-variation-operators-for
Repo
Framework

Fairness in Recommendation Ranking through Pairwise Comparisons


Title	Fairness in Recommendation Ranking through Pairwise Comparisons
Authors	Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, Cristos Goodrow
Abstract	Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information. As such it is important to ask: what are the possible fairness risks, how can we quantify them, and how should we address them? In this paper we offer a set of novel metrics for evaluating algorithmic fairness concerns in recommender systems. In particular we show how measuring fairness based on pairwise comparisons from randomized experiments provides a tractable means to reason about fairness in rankings from recommender systems. Building on this metric, we offer a new regularizer to encourage improving this metric during model training and thus improve fairness in the resulting rankings. We apply this pairwise regularization to a large-scale, production recommender system and show that we are able to significantly improve the system’s pairwise fairness.
Tasks	Recommendation Systems
Published	2019-03-02
URL	http://arxiv.org/abs/1903.00780v1
PDF	http://arxiv.org/pdf/1903.00780v1.pdf
PWC	https://paperswithcode.com/paper/fairness-in-recommendation-ranking-through
Repo
Framework

An Information-theoretic Approach to Unsupervised Feature Selection for High-Dimensional Data


Title	An Information-theoretic Approach to Unsupervised Feature Selection for High-Dimensional Data
Authors	Shao-Lun Huang, Xiangxiang Xu, Lizhong Zheng
Abstract	In this paper, we propose an information-theoretic approach to design the functional representations to extract the hidden common structure shared by a set of random variables. The main idea is to measure the common information between the random variables by Watanabe’s total correlation, and then find the hidden attributes of these random variables such that the common information is reduced the most given these attributes. We show that these attributes can be characterized by an exponential family specified by the eigen-decomposition of some pairwise joint distribution matrix. Then, we adopt the log-likelihood functions for estimating these attributes as the desired functional representations of the random variables, and show that such representations are informative to describe the common structure. Moreover, we design both the multivariate alternating conditional expectation (MACE) algorithm to compute the proposed functional representations for discrete data, and a novel neural network training approach for continuous or high-dimensional data. Furthermore, we show that our approach has deep connections to existing techniques, such as Hirschfeld-Gebelein-R'{e}nyi (HGR) maximal correlation, linear principal component analysis (PCA), and consistent functional map, which establishes insightful connections between information theory and machine learning. Finally, the performances of our algorithms are validated by numerical simulations.
Tasks	Feature Selection
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03196v1
PDF	https://arxiv.org/pdf/1910.03196v1.pdf
PWC	https://paperswithcode.com/paper/an-information-theoretic-approach-to-1
Repo
Framework

The Detection of Distributional Discrepancy for Text Generation


Title	The Detection of Distributional Discrepancy for Text Generation
Authors	Xingyuan Chen, Ping Cai, Peng Jin, Haokun Du, Hongjun Wang, Xingyu Dai, Jiajun Chen
Abstract	The text generated by neural language models is not as good as the real text. This means that their distributions are different. Generative Adversarial Nets (GAN) are used to alleviate it. However, some researchers argue that GAN variants do not work at all. When both sample quality (such as Bleu) and sample diversity (such as self-Bleu) are taken into account, the GAN variants even are worse than a well-adjusted language model. But, Bleu and self-Bleu can not precisely measure this distributional discrepancy. In fact, how to measure the distributional discrepancy between real text and generated text is still an open problem. In this paper, we theoretically propose two metric functions to measure the distributional difference between real text and generated text. Besides that, a method is put forward to estimate them. First, we evaluate language model with these two functions and find the difference is huge. Then, we try several methods to use the detected discrepancy signal to improve the generator. However the difference becomes even bigger than before. Experimenting on two existing language GANs, the distributional discrepancy between real text and generated text increases with more adversarial learning rounds. It demonstrates both of these language GANs fail.
Tasks	Language Modelling, Text Generation
Published	2019-09-28
URL	https://arxiv.org/abs/1910.04859v2
PDF	https://arxiv.org/pdf/1910.04859v2.pdf
PWC	https://paperswithcode.com/paper/the-detection-of-distributional-discrepancy-1
Repo
Framework

Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation


Title	Read, Attend and Comment: A Deep Architecture for Automatic News Comment Generation
Authors	Ze Yang, Can Xu, Wei Wu, Zhoujun Li
Abstract	Automatic news comment generation is a new testbed for techniques of natural language generation. In this paper, we propose a “read-attend-comment” procedure for news comment generation and formalize the procedure with a reading network and a generation network. The reading network comprehends a news article and distills some important points from it, then the generation network creates a comment by attending to the extracted discrete points and the news title. We optimize the model in an end-to-end manner by maximizing a variational lower bound of the true objective using the back-propagation algorithm. Experimental results on two datasets indicate that our model can significantly outperform existing methods in terms of both automatic evaluation and human judgment.
Tasks	Text Generation
Published	2019-09-26
URL	https://arxiv.org/abs/1909.11974v3
PDF	https://arxiv.org/pdf/1909.11974v3.pdf
PWC	https://paperswithcode.com/paper/read-attend-and-comment-a-deep-architecture
Repo
Framework

Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods


Title	Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods
Authors	Oana-Maria Camburu, Eleonora Giunchiglia, Jakob Foerster, Thomas Lukasiewicz, Phil Blunsom
Abstract	For AI systems to garner widespread public acceptance, we must develop methods capable of explaining the decisions of black-box models such as neural networks. In this work, we identify two issues of current explanatory methods. First, we show that two prevalent perspectives on explanations — feature-additivity and feature-selection — lead to fundamentally different instance-wise explanations. In the literature, explainers from different perspectives are currently being directly compared, despite their distinct explanation goals. The second issue is that current post-hoc explainers are either validated under simplistic scenarios (on simple models such as linear regression, or on models trained on syntactic datasets), or, when applied to real-world neural networks, explainers are commonly validated under the assumption that the learned models behave reasonably. However, neural networks often rely on unreasonable correlations, even when producing correct decisions. We introduce a verification framework for explanatory methods under the feature-selection perspective. Our framework is based on a non-trivial neural network architecture trained on a real-world task, and for which we are able to provide guarantees on its inner workings. We validate the efficacy of our evaluation by showing the failure modes of current explainers. We aim for this framework to provide a publicly available, off-the-shelf evaluation when the feature-selection perspective on explanations is needed.
Tasks	Feature Selection
Published	2019-10-04
URL	https://arxiv.org/abs/1910.02065v3
PDF	https://arxiv.org/pdf/1910.02065v3.pdf
PWC	https://paperswithcode.com/paper/can-i-trust-the-explainer-verifying-post-hoc-1
Repo
Framework

Semi-supervised GANs to Infer Travel Modes in GPS Trajectories


Title	Semi-supervised GANs to Infer Travel Modes in GPS Trajectories
Authors	Ali Yazdizadeh, Zachary Patterson, Bilal Farooq
Abstract	Semi-supervised Generative Adversarial Networks (GANs) are developed in the context of travel mode inference with uni-dimensional smartphone trajectory data. We use data from a large-scale smartphone travel survey in Montreal, Canada. We convert GPS trajectories into fixed-sized segments with five channels (variables). We develop different GANs architectures and compare their prediction results with Convolutional Neural Networks (CNNs). The best semi-supervised GANs model led to a prediction accuracy of 83.4%, while the best CNN model was able to achieve the prediction accuracy of 81.3%. The results compare favorably with previous studies, especially when taking the large-scale real-world nature of the dataset into account.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10768v1
PDF	http://arxiv.org/pdf/1902.10768v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-gans-to-infer-travel-modes-in
Repo
Framework

Deep Learning based Wearable Assistive System for Visually Impaired People


Title	Deep Learning based Wearable Assistive System for Visually Impaired People
Authors	Yimin Lin, Kai Wang, Wanxin Yi, Shiguo Lian
Abstract	In this paper, we propose a deep learning based assistive system to improve the environment perception experience of visually impaired (VI). The system is composed of a wearable terminal equipped with an RGBD camera and an earphone, a powerful processor mainly for deep learning inferences and a smart phone for touch-based interaction. A data-driven learning approach is proposed to predict safe and reliable walkable instructions using RGBD data and the established semantic map. This map is also used to help VI understand their 3D surrounding objects and layout through well-designed touchscreen interactions. The quantitative and qualitative experimental results show that our learning based obstacle avoidance approach achieves excellent results in both indoor and outdoor datasets with low-lying obstacles. Meanwhile, user studies have also been carried out in various scenarios and showed the improvement of VI’s environment perception experience with our system.
Tasks
Published	2019-08-09
URL	https://arxiv.org/abs/1908.03364v1
PDF	https://arxiv.org/pdf/1908.03364v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-based-wearable-assistive-system
Repo
Framework

Progressive Focus Search for the Static and Stochastic VRPTW with both Random Customers and Reveal Times


Title	Progressive Focus Search for the Static and Stochastic VRPTW with both Random Customers and Reveal Times
Authors	Michael Saint-Guillain, Christine Solnon, Yves Deville
Abstract	Static stochastic VRPs aim at modeling real-life VRPs by considering uncertainty on data. In particular, the SS-VRPTW-CR considers stochastic customers with time windows and does not make any assumption on their reveal times, which are stochastic as well. Based on customer request probabilities, we look for an a priori solution composed preventive vehicle routes, minimizing the expected number of unsatisfied customer requests at the end of the day. A route describes a sequence of strategic vehicle relocations, from which nearby requests can be rapidly reached. Instead of reoptimizing online, a so-called recourse strategy defines the way the requests are handled, whenever they appear. In this paper, we describe a new recourse strategy for the SS-VRPTW-CR, improving vehicle routes by skipping useless parts. We show how to compute the expected cost of a priori solutions, in pseudo-polynomial time, for this recourse strategy. We introduce a new meta-heuristic, called Progressive Focus Search (PFS), which may be combined with any local-search based algorithm for solving static stochastic optimization problems. PFS accelerates the search by using approximation factors: from an initial rough simplified problem, the search progressively focuses to the actual problem description. We evaluate our contributions on a new, real-world based, public benchmark.
Tasks	Stochastic Optimization
Published	2019-02-08
URL	http://arxiv.org/abs/1902.03930v1
PDF	http://arxiv.org/pdf/1902.03930v1.pdf
PWC	https://paperswithcode.com/paper/progressive-focus-search-for-the-static-and
Repo
Framework

Reconstruction of r-Regular Objects from Trinary Images


Title	Reconstruction of r-Regular Objects from Trinary Images
Authors	Helene Svane, Andrew du Plessis
Abstract	We study digital images of r-regular objects where a pixel is black if it is completely inside the object, white if it is completely inside the complement of the object, and grey otherwise. We call such images trinary. We discuss possible configurations of pixels in trinary images of r-regular objects at certain resolutions and propose a method for reconstructing objects from such images. We show that the reconstructed object is close to the original object in Hausdorff norm, and that there is a homeomorphism of the plane taking the reconstructed set to the original.
Tasks
Published	2019-03-26
URL	http://arxiv.org/abs/1903.10942v1
PDF	http://arxiv.org/pdf/1903.10942v1.pdf
PWC	https://paperswithcode.com/paper/reconstruction-of-r-regular-objects-from
Repo
Framework