October 18, 2019

3050 words 15 mins read

Paper Group ANR 460

Feature-Wise Bias Amplification. Interpretable Discovery in Large Image Data Sets. Bi-Directional Neural Machine Translation with Synthetic Parallel Data. How Much Are You Willing to Share? A “Poker-Styled” Selective Privacy Preserving Framework for Recommender Systems. Examining Performance of Sketch-to-Image Translation Models with Multiclass Aut …

Feature-Wise Bias Amplification


Title	Feature-Wise Bias Amplification
Authors	Klas Leino, Emily Black, Matt Fredrikson, Shayak Sen, Anupam Datta
Abstract	We study the phenomenon of bias amplification in classifiers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth. We demonstrate that bias amplification can arise via an inductive bias in gradient descent methods that results in the overestimation of the importance of moderately-predictive “weak” features if insufficient training data is available. This overestimation gives rise to feature-wise bias amplification – a previously unreported form of bias that can be traced back to the features of a trained model. Through analysis and experiments, we show that while some bias cannot be mitigated without sacrificing accuracy, feature-wise bias amplification can be mitigated through targeted feature selection. We present two new feature selection algorithms for mitigating bias amplification in linear models, and show how they can be adapted to convolutional neural networks efficiently. Our experiments on synthetic and real data demonstrate that these algorithms consistently lead to reduced bias without harming accuracy, in some cases eliminating predictive bias altogether while providing modest gains in accuracy.
Tasks	Feature Selection
Published	2018-12-21
URL	https://arxiv.org/abs/1812.08999v2
PDF	https://arxiv.org/pdf/1812.08999v2.pdf
PWC	https://paperswithcode.com/paper/feature-wise-bias-amplification
Repo
Framework

Interpretable Discovery in Large Image Data Sets


Title	Interpretable Discovery in Large Image Data Sets
Authors	Kiri L. Wagstaff, Jake Lee
Abstract	Automated detection of new, interesting, unusual, or anomalous images within large data sets has great value for applications from surveillance (e.g., airport security) to science (observations that don’t fit a given theory can lead to new discoveries). Many image data analysis systems are turning to convolutional neural networks (CNNs) to represent image content due to their success in achieving high classification accuracy rates. However, CNN representations are notoriously difficult for humans to interpret. We describe a new strategy that combines novelty detection with CNN image features to achieve rapid discovery with interpretable explanations of novel image content. We applied this technique to familiar images from ImageNet as well as to a scientific image collection from planetary science.
Tasks
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08340v1
PDF	http://arxiv.org/pdf/1806.08340v1.pdf
PWC	https://paperswithcode.com/paper/interpretable-discovery-in-large-image-data
Repo
Framework

Bi-Directional Neural Machine Translation with Synthetic Parallel Data


Title	Bi-Directional Neural Machine Translation with Synthetic Parallel Data
Authors	Xing Niu, Michael Denkowski, Marine Carpuat
Abstract	Despite impressive progress in high-resource settings, Neural Machine Translation (NMT) still struggles in low-resource and out-of-domain scenarios, often failing to match the quality of phrase-based translation. We propose a novel technique that combines back-translation and multilingual NMT to improve performance in these difficult cases. Our technique trains a single model for both directions of a language pair, allowing us to back-translate source or target monolingual data without requiring an auxiliary model. We then continue training on the augmented parallel data, enabling a cycle of improvement for a single model that can incorporate any source, target, or parallel data to improve both translation directions. As a byproduct, these models can reduce training and deployment costs significantly compared to uni-directional models. Extensive experiments show that our technique outperforms standard back-translation in low-resource scenarios, improves quality on cross-domain tasks, and effectively reduces costs across the board.
Tasks	Machine Translation
Published	2018-05-29
URL	http://arxiv.org/abs/1805.11213v2
PDF	http://arxiv.org/pdf/1805.11213v2.pdf
PWC	https://paperswithcode.com/paper/bi-directional-neural-machine-translation
Repo
Framework


Title	How Much Are You Willing to Share? A “Poker-Styled” Selective Privacy Preserving Framework for Recommender Systems
Authors	Manoj Reddy Dareddy, Ariyam Das, Junghoo Cho, Carlo Zaniolo
Abstract	Most industrial recommender systems rely on the popular collaborative filtering (CF) technique for providing personalized recommendations to its users. However, the very nature of CF is adversarial to the idea of user privacy, because users need to share their preferences with others in order to be grouped with like-minded people and receive accurate recommendations. While previous privacy preserving approaches have been successful inasmuch as they concealed user preference information to some extent from a centralized recommender system, they have also, nevertheless, incurred significant trade-offs in terms of privacy, scalability, and accuracy. They are also vulnerable to privacy breaches by malicious actors. In light of these observations, we propose a novel selective privacy preserving (SP2) paradigm that allows users to custom define the scope and extent of their individual privacies, by marking their personal ratings as either public (which can be shared) or private (which are never shared and stored only on the user device). Our SP2 framework works in two steps: (i) First, it builds an initial recommendation model based on the sum of all public ratings that have been shared by users and (ii) then, this public model is fine-tuned on each user’s device based on the user private ratings, thus eventually learning a more accurate model. Furthermore, in this work, we introduce three different algorithms for implementing an end-to-end SP2 framework that can scale effectively from thousands to hundreds of millions of items. Our user survey shows that an overwhelming fraction of users are likely to rate much more items to improve the overall recommendations when they can control what ratings will be publicly shared with others.
Tasks	Recommendation Systems
Published	2018-06-04
URL	http://arxiv.org/abs/1806.00914v1
PDF	http://arxiv.org/pdf/1806.00914v1.pdf
PWC	https://paperswithcode.com/paper/how-much-are-you-willing-to-share-a-poker
Repo
Framework

Examining Performance of Sketch-to-Image Translation Models with Multiclass Automatically Generated Paired Training Data


Title	Examining Performance of Sketch-to-Image Translation Models with Multiclass Automatically Generated Paired Training Data
Authors	Dichao Hu
Abstract	Image translation is a computer vision task that involves translating one representation of the scene into another. Various approaches have been proposed and achieved highly desirable results. Nevertheless, its accomplishment requires abundant paired training data which are expensive to acquire. Therefore, models for translation are usually trained on a set of paired training data which are carefully and laboriously designed. Our work is focused on learning through automatically generated paired data. We propose a method to generate fake sketches from images using an adversarial network and then pair the images with corresponding fake sketches to form large-scale multi-class paired training data for training a sketch-to-image translation model. Our model is an encoder-decoder architecture where the encoder generates fake sketches from images and the decoder performs sketch-to-image translation. Qualitative results show that the encoder can be used for generating large-scale multi-class paired data under low supervision. Our current dataset now contains 61255 image and (fake) sketch pairs from 256 different categories. These figures can be greatly increased in the future thanks to our weak reliance on manually labeled data.
Tasks
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00249v1
PDF	http://arxiv.org/pdf/1811.00249v1.pdf
PWC	https://paperswithcode.com/paper/examining-performance-of-sketch-to-image
Repo
Framework

Mobility Offer Allocations for Corporate Mobility as a Service


Title	Mobility Offer Allocations for Corporate Mobility as a Service
Authors	Sebastian Knopp, Benjamin Biesinger, Matthias Prandtstetter
Abstract	Corporate mobility is often based on a fixed assignment of vehicles to employees. Relaxing this fixation while including alternatives such as public transportation or taxis for business and private trips could increase fleet utilization and foster the use of battery electric vehicles. Along this idea we propose a flexible booking system, leading to the introduction of the NP-hard mobility offer allocation problem which is closely related to multi-interval scheduling problems. We describe problem specific conflict graphs for representing and exploring the structure of feasible solutions. A characterization of all maximum cliques in these conflict graphs reveals symmetries which allow to formulate stronger integer linear programming models. We also present an adaptive large neighborhood search based approach which makes use of conflict graphs as well. In a computational study, the approaches are evaluated and it is demonstrated that, depending on instances and run-time requirements, either a solver for the integer linear programming model, fast greedy heuristics, or the adaptive large neighborhood search outperforms the others.
Tasks
Published	2018-10-11
URL	http://arxiv.org/abs/1810.05659v2
PDF	http://arxiv.org/pdf/1810.05659v2.pdf
PWC	https://paperswithcode.com/paper/a-resource-allocation-based-approach-for
Repo
Framework

Embedding Text in Hyperbolic Spaces


Title	Embedding Text in Hyperbolic Spaces
Authors	Bhuwan Dhingra, Christopher J. Shallue, Mohammad Norouzi, Andrew M. Dai, George E. Dahl
Abstract	Natural language text exhibits hierarchical structure in a variety of respects. Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised learning algorithms that work on text data. Recent work by Nickel & Kiela (2017) proposed using hyperbolic instead of Euclidean embedding spaces to represent hierarchical data and demonstrated encouraging results when embedding graphs. In this work, we extend their method with a re-parameterization technique that allows us to learn hyperbolic embeddings of arbitrarily parameterized objects. We apply this framework to learn word and sentence embeddings in hyperbolic space in an unsupervised manner from text corpora. The resulting embeddings seem to encode certain intuitive notions of hierarchy, such as word-context frequency and phrase constituency. However, the implicit continuous hierarchy in the learned hyperbolic space makes interrogating the model’s learned hierarchies more difficult than for models that learn explicit edges between items. The learned hyperbolic embeddings show improvements over Euclidean embeddings in some – but not all – downstream tasks, suggesting that hierarchical organization is more useful for some tasks than others.
Tasks	Sentence Embeddings
Published	2018-06-12
URL	http://arxiv.org/abs/1806.04313v1
PDF	http://arxiv.org/pdf/1806.04313v1.pdf
PWC	https://paperswithcode.com/paper/embedding-text-in-hyperbolic-spaces
Repo
Framework

Machine Learning for Predictive Analytics of Compute Cluster Jobs


Title	Machine Learning for Predictive Analytics of Compute Cluster Jobs
Authors	Dan Andresen, William Hsu, Huichen Yang, Adedolapo Okanlawon
Abstract	We address the problem of predicting whether sufficient memory and CPU resources have been requested for jobs at submission time. For this purpose, we examine the task of training a supervised machine learning system to predict the outcome - whether the job will fail specifically due to insufficient resources - as a classification task. Sufficiently high accuracy, precision, and recall at this task facilitates more anticipatory decision support applications in the domain of HPC resource allocation. Our preliminary results using a new test bed show that the probability of failed jobs is associated with information freely available at job submission time and may thus be usable by a learning system for user modeling that gives personalized feedback to users.
Tasks
Published	2018-05-20
URL	http://arxiv.org/abs/1806.01116v1
PDF	http://arxiv.org/pdf/1806.01116v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-for-predictive-analytics-of
Repo
Framework

Leveraging Disease Progression Learning for Medical Image Recognition


Title	Leveraging Disease Progression Learning for Medical Image Recognition
Authors	Qicheng Lao, Thomas Fevens, Boyu Wang
Abstract	Unlike natural images, medical images often have intrinsic characteristics that can be leveraged for neural network learning. For example, images that belong to different stages of a disease may continuously follow a certain progression pattern. In this paper, we propose a novel method that leverages disease progression learning for medical image recognition. In our method, sequences of images ordered by disease stages are learned by a neural network that consists of a shared vision model for feature extraction and a long short-term memory network for the learning of stage sequences. Auxiliary vision outputs are also included to capture stage features that tend to be discrete along the disease progression. Our proposed method is evaluated on a public diabetic retinopathy dataset, and achieves about 3.3% improvement in disease staging accuracy, compared to the baseline method that does not use disease progression learning.
Tasks
Published	2018-06-26
URL	http://arxiv.org/abs/1806.10128v2
PDF	http://arxiv.org/pdf/1806.10128v2.pdf
PWC	https://paperswithcode.com/paper/leveraging-disease-progression-learning-for
Repo
Framework

Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification


Title	Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
Authors	Rohit Tripathy, Ilias Bilionis
Abstract	State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to deal with the curse of dimensionality - the exponential increase in the volume of the input space, as the number of parameters increases linearly. In this work, we demonstrate the use of deep neural networks (DNN) to construct surrogate models for numerical simulators. We parameterize the structure of the DNN in a manner that lends the DNN surrogate the interpretation of recovering a low dimensional nonlinear manifold. The model response is a parameterized nonlinear function of the low dimensional projections of the input. We think of this low dimensional manifold as a nonlinear generalization of the notion of the active subspace. Our approach is demonstrated with a problem on uncertainty propagation in a stochastic elliptic partial differential equation (SPDE) with uncertain diffusion coefficient. We deviate from traditional formulations of the SPDE problem by not imposing a specific covariance structure on the random diffusion coefficient. Instead, we attempt to solve a more challenging problem of learning a map between an arbitrary snapshot of the diffusion field and the response.
Tasks
Published	2018-02-02
URL	http://arxiv.org/abs/1802.00850v1
PDF	http://arxiv.org/pdf/1802.00850v1.pdf
PWC	https://paperswithcode.com/paper/deep-uq-learning-deep-neural-network
Repo
Framework

Boundary-based Image Forgery Detection by Fast Shallow CNN


Title	Boundary-based Image Forgery Detection by Fast Shallow CNN
Authors	Zhongping Zhang, Yixuan Zhang, Zheng Zhou, Jiebo Luo
Abstract	Image forgery detection is the task of detecting and localizing forged parts in tampered images. Previous works mostly focus on high resolution images using traces of resampling features, demosaicing features or sharpness of edges. However, a good detection method should also be applicable to low resolution images because compressed or resized images are common these days. To this end, we propose a Shallow Convolutional Neural Network(SCNN), capable of distinguishing the boundaries of forged regions from original edges in low resolution images. SCNN is designed to utilize the information of chroma and saturation. Based on SCNN, two approaches that are named Sliding Windows Detection (SWD) and Fast SCNN, respectively, are developed to detect and localize image forgery region. In this paper, we substantiate that Fast SCNN can detect drastic change of chroma and saturation. In image forgery detection experiments Our model is evaluated on the CASIA 2.0 dataset. The results show that Fast SCNN performs well on low resolution images and achieves significant improvements over the state-of-the-art.
Tasks	Demosaicking
Published	2018-01-20
URL	http://arxiv.org/abs/1801.06732v2
PDF	http://arxiv.org/pdf/1801.06732v2.pdf
PWC	https://paperswithcode.com/paper/boundary-based-image-forgery-detection-by
Repo
Framework

Applied Federated Learning: Improving Google Keyboard Query Suggestions


Title	Applied Federated Learning: Improving Google Keyboard Query Suggestions
Authors	Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, Françoise Beaufays
Abstract	Federated learning is a distributed form of machine learning where both the training data and model training are decentralized. In this paper, we use federated learning in a commercial, global-scale setting to train, evaluate and deploy a model to improve virtual keyboard search suggestion quality without direct access to the underlying user data. We describe our observations in federated training, compare metrics to live deployments, and present resulting quality increases. In whole, we demonstrate how federated learning can be applied end-to-end to both improve user experiences and enhance user privacy.
Tasks
Published	2018-12-07
URL	http://arxiv.org/abs/1812.02903v1
PDF	http://arxiv.org/pdf/1812.02903v1.pdf
PWC	https://paperswithcode.com/paper/applied-federated-learning-improving-google
Repo
Framework

Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing


Title	Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing
Authors	Pegah Karimi, Nicholas Davis, Kazjon Grace, Mary Lou Maher
Abstract	We present a system for identifying conceptual shifts between visual categories, which will form the basis for a co-creative drawing system to help users draw more creative sketches. The system recognizes human sketches and matches them to structurally similar sketches from categories to which they do not belong. This would allow a co-creative drawing system to produce an ambiguous sketch that blends features from both categories.
Tasks
Published	2018-01-02
URL	http://arxiv.org/abs/1801.00723v1
PDF	http://arxiv.org/pdf/1801.00723v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-identifying-potential
Repo
Framework

Automated Performance Assessment in Transoesophageal Echocardiography with Convolutional Neural Networks


Title	Automated Performance Assessment in Transoesophageal Echocardiography with Convolutional Neural Networks
Authors	Evangelos B. Mazomenos, Kamakshi Bansal, Bruce Martin, Andrew Smith, Susan Wright, Danail Stoyanov
Abstract	Transoesophageal echocardiography (TEE) is a valuable diagnostic and monitoring imaging modality. Proper image acquisition is essential for diagnosis, yet current assessment techniques are solely based on manual expert review. This paper presents a supervised deep learn ing framework for automatically evaluating and grading the quality of TEE images. To obtain the necessary dataset, 38 participants of varied experience performed TEE exams with a high-fidelity virtual reality (VR) platform. Two Convolutional Neural Network (CNN) architectures, AlexNet and VGG, structured to perform regression, were finetuned and validated on manually graded images from three evaluators. Two different scoring strategies, a criteria-based percentage and an overall general impression, were used. The developed CNN models estimate the average score with a root mean square accuracy ranging between 84%-93%, indicating the ability to replicate expert valuation. Proposed strategies for automated TEE assessment can have a significant impact on the training process of new TEE operators, providing direct feedback and facilitating the development of the necessary dexterous skills.
Tasks
Published	2018-06-13
URL	http://arxiv.org/abs/1806.05154v1
PDF	http://arxiv.org/pdf/1806.05154v1.pdf
PWC	https://paperswithcode.com/paper/automated-performance-assessment-in
Repo
Framework

Stealing Neural Networks via Timing Side Channels


Title	Stealing Neural Networks via Timing Side Channels
Authors	Vasisht Duddu, Debasis Samanta, D Vijay Rao, Valentina E. Balas
Abstract	Deep learning is gaining importance in many applications. However, Neural Networks face several security and privacy threats. This is particularly significant in the scenario where Cloud infrastructures deploy a service with Neural Network model at the back end. Here, an adversary can extract the Neural Network parameters, infer the regularization hyperparameter, identify if a data point was part of the training data, and generate effective transferable adversarial examples to evade classifiers. This paper shows how a Neural Network model is susceptible to timing side channel attack. In this paper, a black box Neural Network extraction attack is proposed by exploiting the timing side channels to infer the depth of the network. Although, constructing an equivalent architecture is a complex search problem, it is shown how Reinforcement Learning with knowledge distillation can effectively reduce the search space to infer a target model. The proposed approach has been tested with VGG architectures on CIFAR10 data set. It is observed that it is possible to reconstruct substitute models with test accuracy close to the target models and the proposed approach is scalable and independent of type of Neural Network architectures.
Tasks
Published	2018-12-31
URL	https://arxiv.org/abs/1812.11720v4
PDF	https://arxiv.org/pdf/1812.11720v4.pdf
PWC	https://paperswithcode.com/paper/stealing-neural-networks-via-timing-side
Repo
Framework