July 29, 2019

3230 words 16 mins read

Paper Group ANR 54

Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations. Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient?. Discriminatory Transfer. On the Complexity of Semantic Integration of OWL Ontologies. Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image- …

Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations


Title	Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations
Authors	Weiming Xiang, Hoang-Dung Tran, Taylor T. Johnson
Abstract	Neural networks have been widely used to solve complex real-world problems. Due to the complicate, nonlinear, non-convex nature of neural networks, formal safety guarantees for the output behaviors of neural networks will be crucial for their applications in safety-critical systems.In this paper, the output reachable set computation and safety verification problems for a class of neural networks consisting of Rectified Linear Unit (ReLU) activation functions are addressed. A layer-by-layer approach is developed to compute output reachable set. The computation is formulated in the form of a set of manipulations for a union of polyhedra, which can be efficiently applied with the aid of polyhedron computation tools. Based on the output reachable set computation results, the safety verification for a ReLU neural network can be performed by checking the intersections of unsafe regions and output reachable set described by a union of polyhedra. A numerical example of a randomly generated ReLU neural network is provided to show the effectiveness of the approach developed in this paper.
Tasks
Published	2017-12-21
URL	http://arxiv.org/abs/1712.08163v1
PDF	http://arxiv.org/pdf/1712.08163v1.pdf
PWC	https://paperswithcode.com/paper/reachable-set-computation-and-safety
Repo
Framework

Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient?


Title	Expanding Abbreviations in a Strongly Inflected Language: Are Morphosyntactic Tags Sufficient?
Authors	Piotr Żelasko
Abstract	In this paper, the problem of recovery of morphological information lost in abbreviated forms is addressed with a focus on highly inflected languages. Evidence is presented that the correct inflected form of an expanded abbreviation can in many cases be deduced solely from the morphosyntactic tags of the context. The prediction model is a deep bidirectional LSTM network with tag embedding. The training and evaluation data are gathered by finding the words which could have been abbreviated and using their corresponding morphosyntactic tags as the labels, while the tags of the context words are used as the input features for classification. The network is trained on over 10 million words from the Polish Sejm Corpus and achieves 74.2% prediction accuracy on a smaller, but more general National Corpus of Polish. The analysis of errors suggests that performance in this task may improve if some prior knowledge about the abbreviated word is incorporated into the model.
Tasks
Published	2017-08-20
URL	http://arxiv.org/abs/1708.05992v2
PDF	http://arxiv.org/pdf/1708.05992v2.pdf
PWC	https://paperswithcode.com/paper/expanding-abbreviations-in-a-strongly-1
Repo
Framework

Discriminatory Transfer


Title	Discriminatory Transfer
Authors	Chao Lan, Jun Huan
Abstract	We observe standard transfer learning can improve prediction accuracies of target tasks at the cost of lowering their prediction fairness – a phenomenon we named discriminatory transfer. We examine prediction fairness of a standard hypothesis transfer algorithm and a standard multi-task learning algorithm, and show they both suffer discriminatory transfer on the real-world Communities and Crime data set. The presented case study introduces an interaction between fairness and transfer learning, as an extension of existing fairness studies that focus on single task learning.
Tasks	Multi-Task Learning, Transfer Learning
Published	2017-07-03
URL	http://arxiv.org/abs/1707.00780v4
PDF	http://arxiv.org/pdf/1707.00780v4.pdf
PWC	https://paperswithcode.com/paper/discriminatory-transfer
Repo
Framework

On the Complexity of Semantic Integration of OWL Ontologies


Title	On the Complexity of Semantic Integration of OWL Ontologies
Authors	Yevgeny Kazakov, Denis Ponomaryov
Abstract	We propose a new mechanism for integration of OWL ontologies using semantic import relations. In contrast to the standard OWL importing, we do not require all axioms of the imported ontologies to be taken into account for reasoning tasks, but only their logical implications over a chosen signature. This property comes natural in many ontology integration scenarios, especially when the number of ontologies is large. In this paper, we study the complexity of reasoning over ontologies with semantic import relations and establish a range of tight complexity bounds for various fragments of OWL.
Tasks
Published	2017-05-12
URL	http://arxiv.org/abs/1705.04719v1
PDF	http://arxiv.org/pdf/1705.04719v1.pdf
PWC	https://paperswithcode.com/paper/on-the-complexity-of-semantic-integration-of
Repo
Framework

Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision


Title	Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision
Authors	Hsiao-Yu Fish Tung, Adam W. Harley, William Seto, Katerina Fragkiadaki
Abstract	Researchers have developed excellent feed-forward models that learn to map images to desired outputs, such as to the images’ latent factors, or to other images, using supervised learning. Learning such mappings from unlabelled data, or improving upon supervised models by exploiting unlabelled data, remains elusive. We argue that there are two important parts to learning without annotations: (i) matching the predictions to the input observations, and (ii) matching the predictions to known priors. We propose Adversarial Inverse Graphics networks (AIGNs): weakly supervised neural network models that combine feedback from rendering their predictions, with distribution matching between their predictions and a collection of ground-truth factors. We apply AIGNs to 3D human pose estimation and 3D structure and egomotion estimation, and outperform models supervised by only paired annotations. We further apply AIGNs to facial image transformation using super-resolution and inpainting renderers, while deliberately adding biases in the ground-truth datasets. Our model seamlessly incorporates such biases, rendering input faces towards young, old, feminine, masculine or Tom Cruise-like equivalents (depending on the chosen bias), or adding lip and nose augmentations while inpainting concealed lips and noses.
Tasks	3D Human Pose Estimation, Image-to-Image Translation, Pose Estimation, Super-Resolution
Published	2017-05-31
URL	http://arxiv.org/abs/1705.11166v3
PDF	http://arxiv.org/pdf/1705.11166v3.pdf
PWC	https://paperswithcode.com/paper/adversarial-inverse-graphics-networks
Repo
Framework

Utilizing artificial neural networks to predict demand for weather-sensitive products at retail stores


Title	Utilizing artificial neural networks to predict demand for weather-sensitive products at retail stores
Authors	Elham Taghizadeh
Abstract	One key requirement for effective supply chain management is the quality of its inventory management. Various inventory management methods are typically employed for different types of products based on their demand patterns, product attributes, and supply network. In this paper, our goal is to develop robust demand prediction methods for weather sensitive products at retail stores. We employ historical datasets from Walmart, whose customers and markets are often exposed to extreme weather events which can have a huge impact on sales regarding the affected stores and products. We want to accurately predict the sales of 111 potentially weather-sensitive products around the time of major weather events at 45 of Walmart retails locations in the U.S. Intuitively, we may expect an uptick in the sales of umbrellas before a big thunderstorm, but it is difficult for replenishment managers to predict the level of inventory needed to avoid being out-of-stock or overstock during and after that storm. While they rely on a variety of vendor tools to predict sales around extreme weather events, they mostly employ a time-consuming process that lacks a systematic measure of effectiveness. We employ all the methods critical to any analytics project and start with data exploration. Critical features are extracted from the raw historical dataset for demand forecasting accuracy and robustness. In particular, we employ Artificial Neural Network for forecasting demand for each product sold around the time of major weather events. Finally, we evaluate our model to evaluate their accuracy and robustness.
Tasks
Published	2017-11-20
URL	http://arxiv.org/abs/1711.08325v1
PDF	http://arxiv.org/pdf/1711.08325v1.pdf
PWC	https://paperswithcode.com/paper/utilizing-artificial-neural-networks-to
Repo
Framework

Provably Minimally-Distorted Adversarial Examples


Title	Provably Minimally-Distorted Adversarial Examples
Authors	Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill
Abstract	The ability to deploy neural networks in real-world, safety-critical systems is severely limited by the presence of adversarial examples: slightly perturbed inputs that are misclassified by the network. In recent years, several techniques have been proposed for increasing robustness to adversarial examples — and yet most of these have been quickly shown to be vulnerable to future attacks. For example, over half of the defenses proposed by papers accepted at ICLR 2018 have already been broken. We propose to address this difficulty through formal verification techniques. We show how to construct provably minimally distorted adversarial examples: given an arbitrary neural network and input sample, we can construct adversarial examples which we prove are of minimal distortion. Using this approach, we demonstrate that one of the recent ICLR defense proposals, adversarial retraining, provably succeeds at increasing the distortion required to construct adversarial examples by a factor of 4.2.
Tasks
Published	2017-09-29
URL	http://arxiv.org/abs/1709.10207v2
PDF	http://arxiv.org/pdf/1709.10207v2.pdf
PWC	https://paperswithcode.com/paper/provably-minimally-distorted-adversarial
Repo
Framework

On Wasserstein Reinforcement Learning and the Fokker-Planck equation


Title	On Wasserstein Reinforcement Learning and the Fokker-Planck equation
Authors	Pierre H. Richemond, Brendan Maginnis
Abstract	Policy gradients methods often achieve better performance when the change in policy is limited to a small Kullback-Leibler divergence. We derive policy gradients where the change in policy is limited to a small Wasserstein distance (or trust region). This is done in the discrete and continuous multi-armed bandit settings with entropy regularisation. We show that in the small steps limit with respect to the Wasserstein distance $W_2$, policy dynamics are governed by the Fokker-Planck (heat) equation, following the Jordan-Kinderlehrer-Otto result. This means that policies undergo diffusion and advection, concentrating near actions with high reward. This helps elucidate the nature of convergence in the probability matching setup, and provides justification for empirical practices such as Gaussian policy priors and additive gradient noise.
Tasks
Published	2017-12-19
URL	http://arxiv.org/abs/1712.07185v1
PDF	http://arxiv.org/pdf/1712.07185v1.pdf
PWC	https://paperswithcode.com/paper/on-wasserstein-reinforcement-learning-and-the
Repo
Framework

A Comparative Study on Different Types of Approaches to Bengali document Categorization


Title	A Comparative Study on Different Types of Approaches to Bengali document Categorization
Authors	Md. Saiful Islam, Fazla Elahi Md Jubayer, Syed Ikhtiar Ahmed
Abstract	Document categorization is a technique where the category of a document is determined. In this paper three well-known supervised learning techniques which are Support Vector Machine(SVM), Na"ive Bayes(NB) and Stochastic Gradient Descent(SGD) compared for Bengali document categorization. Besides classifier, classification also depends on how feature is selected from dataset. For analyzing those classifier performances on predicting a document against twelve categories several feature selection techniques are also applied in this article namely Chi square distribution, normalized TFIDF (term frequency-inverse document frequency) with word analyzer. So, we attempt to explore the efficiency of those three-classification algorithms by using two different feature selection techniques in this article.
Tasks	Feature Selection
Published	2017-01-27
URL	http://arxiv.org/abs/1701.08694v1
PDF	http://arxiv.org/pdf/1701.08694v1.pdf
PWC	https://paperswithcode.com/paper/a-comparative-study-on-different-types-of
Repo
Framework

An Invariant Model of the Significance of Different Body Parts in Recognizing Different Actions


Title	An Invariant Model of the Significance of Different Body Parts in Recognizing Different Actions
Authors	Yuping Shen, Hassan Foroosh
Abstract	In this paper, we show that different body parts do not play equally important roles in recognizing a human action in video data. We investigate to what extent a body part plays a role in recognition of different actions and hence propose a generic method of assigning weights to different body points. The approach is inspired by the strong evidence in the applied perception community that humans perform recognition in a foveated manner, that is they recognize events or objects by only focusing on visually significant aspects. An important contribution of our method is that the computation of the weights assigned to body parts is invariant to viewing directions and camera parameters in the input data. We have performed extensive experiments to validate the proposed approach and demonstrate its significance. In particular, results show that considerable improvement in performance is gained by taking into account the relative importance of different body parts as defined by our approach.
Tasks
Published	2017-05-22
URL	http://arxiv.org/abs/1705.08293v1
PDF	http://arxiv.org/pdf/1705.08293v1.pdf
PWC	https://paperswithcode.com/paper/an-invariant-model-of-the-significance-of
Repo
Framework

Adaptive Multiple-Arm Identification


Title	Adaptive Multiple-Arm Identification
Authors	Jiecao Chen, Xi Chen, Qin Zhang, Yuan Zhou
Abstract	We study the problem of selecting $K$ arms with the highest expected rewards in a stochastic $n$-armed bandit game. This problem has a wide range of applications, e.g., A/B testing, crowdsourcing, simulation optimization. Our goal is to develop a PAC algorithm, which, with probability at least $1-\delta$, identifies a set of $K$ arms with the aggregate regret at most $\epsilon$. The notion of aggregate regret for multiple-arm identification was first introduced in \cite{Zhou:14} , which is defined as the difference of the averaged expected rewards between the selected set of arms and the best $K$ arms. In contrast to \cite{Zhou:14} that only provides instance-independent sample complexity, we introduce a new hardness parameter for characterizing the difficulty of any given instance. We further develop two algorithms and establish the corresponding sample complexity in terms of this hardness parameter. The derived sample complexity can be significantly smaller than state-of-the-art results for a large class of instances and matches the instance-independent lower bound upto a $\log(\epsilon^{-1})$ factor in the worst case. We also prove a lower bound result showing that the extra $\log(\epsilon^{-1})$ is necessary for instance-dependent algorithms using the introduced hardness parameter.
Tasks
Published	2017-06-04
URL	http://arxiv.org/abs/1706.01026v1
PDF	http://arxiv.org/pdf/1706.01026v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-multiple-arm-identification
Repo
Framework

Machine Learning Molecular Dynamics for the Simulation of Infrared Spectra


Title	Machine Learning Molecular Dynamics for the Simulation of Infrared Spectra
Authors	Michael Gastegger, Jörg Behler, Philipp Marquetand
Abstract	Machine learning has emerged as an invaluable tool in many research areas. In the present work, we harness this power to predict highly accurate molecular infrared spectra with unprecedented computational efficiency. To account for vibrational anharmonic and dynamical effects – typically neglected by conventional quantum chemistry approaches – we base our machine learning strategy on ab initio molecular dynamics simulations. While these simulations are usually extremely time consuming even for small molecules, we overcome these limitations by leveraging the power of a variety of machine learning techniques, not only accelerating simulations by several orders of magnitude, but also greatly extending the size of systems that can be treated. To this end, we develop a molecular dipole moment model based on environment dependent neural network charges and combine it with the neural network potentials of Behler and Parrinello. Contrary to the prevalent big data philosophy, we are able to obtain very accurate machine learning models for the prediction of infrared spectra based on only a few hundreds of electronic structure reference points. This is made possible through the introduction of a fully automated sampling scheme and the use of molecular forces during neural network potential training. We demonstrate the power of our machine learning approach by applying it to model the infrared spectra of a methanol molecule, n-alkanes containing up to 200 atoms and the protonated alanine tripeptide, which at the same time represents the first application of machine learning techniques to simulate the dynamics of a peptide. In all these case studies we find excellent agreement between the infrared spectra predicted via machine learning models and the respective theoretical and experimental spectra.
Tasks
Published	2017-05-16
URL	http://arxiv.org/abs/1705.05907v1
PDF	http://arxiv.org/pdf/1705.05907v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-molecular-dynamics-for-the
Repo
Framework

Email Babel: Does Language Affect Criminal Activity in Compromised Webmail Accounts?


Title	Email Babel: Does Language Affect Criminal Activity in Compromised Webmail Accounts?
Authors	Emeric Bernard-Jones, Jeremiah Onaolapo, Gianluca Stringhini
Abstract	We set out to understand the effects of differing language on the ability of cybercriminals to navigate webmail accounts and locate sensitive information in them. To this end, we configured thirty Gmail honeypot accounts with English, Romanian, and Greek language settings. We populated the accounts with email messages in those languages by subscribing them to selected online newsletters. We hid email messages about fake bank accounts in fifteen of the accounts to mimic real-world webmail users that sometimes store sensitive information in their accounts. We then leaked credentials to the honey accounts via paste sites on the Surface Web and the Dark Web, and collected data for fifteen days. Our statistical analyses on the data show that cybercriminals are more likely to discover sensitive information (bank account information) in the Greek accounts than the remaining accounts, contrary to the expectation that Greek ought to constitute a barrier to the understanding of non-Greek visitors to the Greek accounts. We also extracted the important words among the emails that cybercriminals accessed (as an approximation of the keywords that they searched for within the honey accounts), and found that financial terms featured among the top words. In summary, we show that language plays a significant role in the ability of cybercriminals to access sensitive information hidden in compromised webmail accounts.
Tasks
Published	2017-04-25
URL	http://arxiv.org/abs/1704.07759v1
PDF	http://arxiv.org/pdf/1704.07759v1.pdf
PWC	https://paperswithcode.com/paper/email-babel-does-language-affect-criminal
Repo
Framework

Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy?


Title	Weakly Submodular Maximization Beyond Cardinality Constraints: Does Randomization Help Greedy?
Authors	Lin Chen, Moran Feldman, Amin Karbasi
Abstract	Submodular functions are a broad class of set functions, which naturally arise in diverse areas. Many algorithms have been suggested for the maximization of these functions. Unfortunately, once the function deviates from submodularity, the known algorithms may perform arbitrarily poorly. Amending this issue, by obtaining approximation results for set functions generalizing submodular functions, has been the focus of recent works. One such class, known as weakly submodular functions, has received a lot of attention. A key result proved by Das and Kempe (2011) showed that the approximation ratio of the greedy algorithm for weakly submodular maximization subject to a cardinality constraint degrades smoothly with the distance from submodularity. However, no results have been obtained for maximization subject to constraints beyond cardinality. In particular, it is not known whether the greedy algorithm achieves any non-trivial approximation ratio for such constraints. In this paper, we prove that a randomized version of the greedy algorithm (previously used by Buchbinder et al. (2014) for a different problem) achieves an approximation ratio of $(1 + 1/\gamma)^{-2}$ for the maximization of a weakly submodular function subject to a general matroid constraint, where $\gamma$ is a parameter measuring the distance of the function from submodularity. Moreover, we also experimentally compare the performance of this version of the greedy algorithm on real world problems against natural benchmarks, and show that the algorithm we study performs well also in practice. To the best of our knowledge, this is the first algorithm with a non-trivial approximation guarantee for maximizing a weakly submodular function subject to a constraint other than the simple cardinality constraint. In particular, it is the first algorithm with such a guarantee for the important and broad class of matroid constraints.
Tasks
Published	2017-07-13
URL	http://arxiv.org/abs/1707.04347v1
PDF	http://arxiv.org/pdf/1707.04347v1.pdf
PWC	https://paperswithcode.com/paper/weakly-submodular-maximization-beyond
Repo
Framework

Deep Regression Forests for Age Estimation


Title	Deep Regression Forests for Age Estimation
Authors	Wei Shen, Yilu Guo, Yan Wang, Kai Zhao, Bo Wang, Alan Yuille
Abstract	Age estimation from facial images is typically cast as a nonlinear regression problem. The main challenge of this problem is the facial feature space w.r.t. ages is heterogeneous, due to the large variation in facial appearance across different persons of the same age and the non-stationary property of aging patterns. In this paper, we propose Deep Regression Forests (DRFs), an end-to-end model, for age estimation. DRFs connect the split nodes to a fully connected layer of a convolutional neural network (CNN) and deal with heterogeneous data by jointly learning input-dependant data partitions at the split nodes and data abstractions at the leaf nodes. This joint learning follows an alternating strategy: First, by fixing the leaf nodes, the split nodes as well as the CNN parameters are optimized by Back-propagation; Then, by fixing the split nodes, the leaf nodes are optimized by iterating a step-size free and fast-converging update rule derived from Variational Bounding. We verify the proposed DRFs on three standard age estimation benchmarks and achieve state-of-the-art results on all of them.
Tasks	Age Estimation
Published	2017-12-19
URL	http://arxiv.org/abs/1712.07195v1
PDF	http://arxiv.org/pdf/1712.07195v1.pdf
PWC	https://paperswithcode.com/paper/deep-regression-forests-for-age-estimation
Repo
Framework