July 27, 2019

3166 words 15 mins read

Paper Group ANR 692

Estimation Considerations in Contextual Bandits. Speaker Identification in each of the Neutral and Shouted Talking Environments based on Gender-Dependent Approach Using SPHMMs. Identifying Best Interventions through Online Importance Sampling. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods. An Integrated and Scalable …

Estimation Considerations in Contextual Bandits


Title	Estimation Considerations in Contextual Bandits
Authors	Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, Guido Imbens
Abstract	Contextual bandit algorithms are sensitive to the estimation method of the outcome model as well as the exploration method used, particularly in the presence of rich heterogeneity or complex outcome models, which can lead to difficult estimation problems along the path of learning. We study a consideration for the exploration vs. exploitation framework that does not arise in multi-armed bandits but is crucial in contextual bandits; the way exploration and exploitation is conducted in the present affects the bias and variance in the potential outcome model estimation in subsequent stages of learning. We develop parametric and non-parametric contextual bandits that integrate balancing methods from the causal inference literature in their estimation to make it less prone to problems of estimation bias. We provide the first regret bound analyses for contextual bandits with balancing in the domain of linear contextual bandits that match the state of the art regret bounds. We demonstrate the strong practical advantage of balanced contextual bandits on a large number of supervised learning datasets and on a synthetic example that simulates model mis-specification and prejudice in the initial training data. Additionally, we develop contextual bandits with simpler assignment policies by leveraging sparse model estimation methods from the econometrics literature and demonstrate empirically that in the early stages they can improve the rate of learning and decrease regret.
Tasks	Causal Inference, Multi-Armed Bandits
Published	2017-11-19
URL	http://arxiv.org/abs/1711.07077v4
PDF	http://arxiv.org/pdf/1711.07077v4.pdf
PWC	https://paperswithcode.com/paper/estimation-considerations-in-contextual
Repo
Framework

Speaker Identification in each of the Neutral and Shouted Talking Environments based on Gender-Dependent Approach Using SPHMMs


Title	Speaker Identification in each of the Neutral and Shouted Talking Environments based on Gender-Dependent Approach Using SPHMMs
Authors	Ismail Shahin
Abstract	It is well known that speaker identification performs extremely well in the neutral talking environments; however, the identification performance is declined sharply in the shouted talking environments. This work aims at proposing, implementing and testing a new approach to enhance the declined performance in the shouted talking environments. The new proposed approach is based on gender-dependent speaker identification using Suprasegmental Hidden Markov Models (SPHMMs) as classifiers. This proposed approach has been tested on two different and separate speech databases: our collected database and the Speech Under Simulated and Actual Stress (SUSAS) database. The results of this work show that gender-dependent speaker identification based on SPHMMs outperforms gender-independent speaker identification based on the same models and gender-dependent speaker identification based on Hidden Markov Models (HMMs) by about 6% and 8%, respectively. The results obtained based on the proposed approach are close to those obtained in subjective evaluation by human judges.
Tasks	Speaker Identification
Published	2017-06-29
URL	http://arxiv.org/abs/1706.09767v1
PDF	http://arxiv.org/pdf/1706.09767v1.pdf
PWC	https://paperswithcode.com/paper/speaker-identification-in-each-of-the-neutral
Repo
Framework

Identifying Best Interventions through Online Importance Sampling


Title	Identifying Best Interventions through Online Importance Sampling
Authors	Rajat Sen, Karthikeyan Shanmugam, Alexandros G. Dimakis, Sanjay Shakkottai
Abstract	Motivated by applications in computational advertising and systems biology, we consider the problem of identifying the best out of several possible soft interventions at a source node $V$ in an acyclic causal directed graph, to maximize the expected value of a target node $Y$ (located downstream of $V$). Our setting imposes a fixed total budget for sampling under various interventions, along with cost constraints on different types of interventions. We pose this as a best arm identification bandit problem with $K$ arms where each arm is a soft intervention at $V,$ and leverage the information leakage among the arms to provide the first gap dependent error and simple regret bounds for this problem. Our results are a significant improvement over the traditional best arm identification results. We empirically show that our algorithms outperform the state of the art in the Flow Cytometry data-set, and also apply our algorithm for model interpretation of the Inception-v3 deep net that classifies images.
Tasks
Published	2017-01-10
URL	http://arxiv.org/abs/1701.02789v3
PDF	http://arxiv.org/pdf/1701.02789v3.pdf
PWC	https://paperswithcode.com/paper/identifying-best-interventions-through-online
Repo
Framework

Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods


Title	Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
Authors	Nicholas Carlini, David Wagner
Abstract	Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.
Tasks
Published	2017-05-20
URL	http://arxiv.org/abs/1705.07263v2
PDF	http://arxiv.org/pdf/1705.07263v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-examples-are-not-easily-detected
Repo
Framework

An Integrated and Scalable Platform for Proactive Event-Driven Traffic Management


Title	An Integrated and Scalable Platform for Proactive Event-Driven Traffic Management
Authors	Alain Kibangou, Alexander Artikis, Evangelos Michelioudakis, Georgios Paliouras, Marius Schmitt, John Lygeros, Chris Baber, Natan Morar, Fabiana Fournier, Inna Skarbovsky
Abstract	Traffic on freeways can be managed by means of ramp meters from Road Traffic Control rooms. Human operators cannot efficiently manage a network of ramp meters. To support them, we present an intelligent platform for traffic management which includes a new ramp metering coordination scheme in the decision making module, an efficient dashboard for interacting with human operators, machine learning tools for learning event definitions and Complex Event Processing tools able to deal with uncertainties inherent to the traffic use case. Unlike the usual approach, the devised event-driven platform is able to predict a congestion up to 4 minutes before it really happens. Proactive decision making can then be established leading to significant improvement of traffic conditions.
Tasks	Decision Making
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02810v1
PDF	http://arxiv.org/pdf/1703.02810v1.pdf
PWC	https://paperswithcode.com/paper/an-integrated-and-scalable-platform-for
Repo
Framework

Self-Adaptation of Activity Recognition Systems to New Sensors


Title	Self-Adaptation of Activity Recognition Systems to New Sensors
Authors	David Bannach, Martin Jänicke, Vitor F. Rey, Sven Tomforde, Bernhard Sick, Paul Lukowicz
Abstract	Traditional activity recognition systems work on the basis of training, taking a fixed set of sensors into account. In this article, we focus on the question how pattern recognition can leverage new information sources without any, or with minimal user input. Thus, we present an approach for opportunistic activity recognition, where ubiquitous sensors lead to dynamically changing input spaces. Our method is a variation of well-established principles of machine learning, relying on unsupervised clustering to discover structure in data and inferring cluster labels from a small number of labeled dates in a semi-supervised manner. Elaborating the challenges, evaluations of over 3000 sensor combinations from three multi-user experiments are presented in detail and show the potential benefit of our approach.
Tasks	Activity Recognition
Published	2017-01-30
URL	http://arxiv.org/abs/1701.08528v1
PDF	http://arxiv.org/pdf/1701.08528v1.pdf
PWC	https://paperswithcode.com/paper/self-adaptation-of-activity-recognition
Repo
Framework

Improved Search in Hamming Space using Deep Multi-Index Hashing


Title	Improved Search in Hamming Space using Deep Multi-Index Hashing
Authors	Hanjiang Lai, Yan Pan
Abstract	Similarity-preserving hashing is a widely-used method for nearest neighbour search in large-scale image retrieval tasks. There has been considerable research on generating efficient image representation via the deep-network-based hashing methods. However, the issue of efficient searching in the deep representation space remains largely unsolved. To this end, we propose a simple yet efficient deep-network-based multi-index hashing method for simultaneously learning the powerful image representation and the efficient searching. To achieve these two goals, we introduce the multi-index hashing (MIH) mechanism into the proposed deep architecture, which divides the binary codes into multiple substrings. Due to the non-uniformly distributed codes will result in inefficiency searching, we add the two balanced constraints at feature-level and instance-level, respectively. Extensive evaluations on several benchmark image retrieval datasets show that the learned balanced binary codes bring dramatic speedups and achieve comparable performance over the existing baselines.
Tasks	Image Retrieval
Published	2017-10-19
URL	http://arxiv.org/abs/1710.06993v1
PDF	http://arxiv.org/pdf/1710.06993v1.pdf
PWC	https://paperswithcode.com/paper/improved-search-in-hamming-space-using-deep
Repo
Framework

Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records


Title	Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records
Authors	Zhengping Che, Yu Cheng, Shuangfei Zhai, Zhaonan Sun, Yan Liu
Abstract	The rapid growth of Electronic Health Records (EHRs), as well as the accompanied opportunities in Data-Driven Healthcare (DDH), has been attracting widespread interests and attentions. Recent progress in the design and applications of deep learning methods has shown promising results and is forcing massive changes in healthcare academia and industry, but most of these methods rely on massive labeled data. In this work, we propose a general deep learning framework which is able to boost risk prediction performance with limited EHR data. Our model takes a modified generative adversarial network namely ehrGAN, which can provide plausible labeled EHR data by mimicking real patient records, to augment the training dataset in a semi-supervised learning manner. We use this generative model together with a convolutional neural network (CNN) based prediction model to improve the onset prediction performance. Experiments on two real healthcare datasets demonstrate that our proposed framework produces realistic data samples and achieves significant improvements on classification tasks with the generated data over several stat-of-the-art baselines.
Tasks
Published	2017-09-06
URL	http://arxiv.org/abs/1709.01648v1
PDF	http://arxiv.org/pdf/1709.01648v1.pdf
PWC	https://paperswithcode.com/paper/boosting-deep-learning-risk-prediction-with
Repo
Framework

Combining Multiple Views for Visual Speech Recognition


Title	Combining Multiple Views for Visual Speech Recognition
Authors	Marina Zimmermann, Mostafa Mehdipour Ghazi, Hazım Kemal Ekenel, Jean-Philippe Thiran
Abstract	Visual speech recognition is a challenging research problem with a particular practical application of aiding audio speech recognition in noisy scenarios. Multiple camera setups can be beneficial for the visual speech recognition systems in terms of improved performance and robustness. In this paper, we explore this aspect and provide a comprehensive study on combining multiple views for visual speech recognition. The thorough analysis covers fusion of all possible view angle combinations both at feature level and decision level. The employed visual speech recognition system in this study extracts features through a PCA-based convolutional neural network, followed by an LSTM network. Finally, these features are processed in a tandem system, being fed into a GMM-HMM scheme. The decision fusion acts after this point by combining the Viterbi path log-likelihoods. The results show that the complementary information contained in recordings from different view angles improves the results significantly. For example, the sentence correctness on the test set is increased from 76% for the highest performing single view ($30^\circ$) to up to 83% when combining this view with the frontal and $60^\circ$ view angles.
Tasks	Speech Recognition, Visual Speech Recognition
Published	2017-10-19
URL	http://arxiv.org/abs/1710.07168v2
PDF	http://arxiv.org/pdf/1710.07168v2.pdf
PWC	https://paperswithcode.com/paper/combining-multiple-views-for-visual-speech
Repo
Framework

Extending and Improving Wordnet via Unsupervised Word Embeddings


Title	Extending and Improving Wordnet via Unsupervised Word Embeddings
Authors	Mikhail Khodak, Andrej Risteski, Christiane Fellbaum, Sanjeev Arora
Abstract	This work presents an unsupervised approach for improving WordNet that builds upon recent advances in document and sense representation via distributional semantics. We apply our methods to construct Wordnets in French and Russian, languages which both lack good manual constructions.1 These are evaluated on two new 600-word test sets for word-to-synset matching and found to improve greatly upon synset recall, outperforming the best automated Wordnets in F-score. Our methods require very few linguistic resources, thus being applicable for Wordnet construction in low-resources languages, and may further be applied to sense clustering and other Wordnet improvements.
Tasks	Word Embeddings
Published	2017-04-29
URL	http://arxiv.org/abs/1705.00217v1
PDF	http://arxiv.org/pdf/1705.00217v1.pdf
PWC	https://paperswithcode.com/paper/extending-and-improving-wordnet-via
Repo
Framework

Joint Gaussian Processes for Biophysical Parameter Retrieval


Title	Joint Gaussian Processes for Biophysical Parameter Retrieval
Authors	Daniel Heestermans Svendsen, Luca Martino, Manuel Campos-Taberner, Francisco Javier García-Haro, Gustau Camps-Valls
Abstract	Solving inverse problems is central to geosciences and remote sensing. Radiative transfer models (RTMs) represent mathematically the physical laws which govern the phenomena in remote sensing applications (forward models). The numerical inversion of the RTM equations is a challenging and computationally demanding problem, and for this reason, often the application of a nonlinear statistical regression is preferred. In general, regression models predict the biophysical parameter of interest from the corresponding received radiance. However, this approach does not employ the physical information encoded in the RTMs. An alternative strategy, which attempts to include the physical knowledge, consists in learning a regression model trained using data simulated by an RTM code. In this work, we introduce a nonlinear nonparametric regression model which combines the benefits of the two aforementioned approaches. The inversion is performed taking into account jointly both real observations and RTM-simulated data. The proposed Joint Gaussian Process (JGP) provides a solid framework for exploiting the regularities between the two types of data. The JGP automatically detects the relative quality of the simulated and real data, and combines them accordingly. This occurs by learning an additional hyper-parameter w.r.t. a standard GP model, and fitting parameters through maximizing the pseudo-likelihood of the real observations. The resulting scheme is both simple and robust, i.e., capable of adapting to different scenarios. The advantages of the JGP method compared to benchmark strategies are shown considering RTM-simulated and real observations in different experiments. Specifically, we consider leaf area index (LAI) retrieval from Landsat data combined with simulated data generated by the PROSAIL model.
Tasks	Gaussian Processes
Published	2017-11-14
URL	http://arxiv.org/abs/1711.05197v1
PDF	http://arxiv.org/pdf/1711.05197v1.pdf
PWC	https://paperswithcode.com/paper/joint-gaussian-processes-for-biophysical
Repo
Framework

Are Saddles Good Enough for Deep Learning?


Title	Are Saddles Good Enough for Deep Learning?
Authors	Adepu Ravi Sankar, Vineeth N Balasubramanian
Abstract	Recent years have seen a growing interest in understanding deep neural networks from an optimization perspective. It is understood now that converging to low-cost local minima is sufficient for such models to become effective in practice. However, in this work, we propose a new hypothesis based on recent theoretical findings and empirical studies that deep neural network models actually converge to saddle points with high degeneracy. Our findings from this work are new, and can have a significant impact on the development of gradient descent based methods for training deep networks. We validated our hypotheses using an extensive experimental evaluation on standard datasets such as MNIST and CIFAR-10, and also showed that recent efforts that attempt to escape saddles finally converge to saddles with high degeneracy, which we define as `good saddles’. We also verified the famous Wigner’s Semicircle Law in our experimental results. \|
Tasks
Published	2017-06-07
URL	http://arxiv.org/abs/1706.02052v1
PDF	http://arxiv.org/pdf/1706.02052v1.pdf
PWC	https://paperswithcode.com/paper/are-saddles-good-enough-for-deep-learning
Repo
Framework

Kernel Two-Sample Hypothesis Testing Using Kernel Set Classification


Title	Kernel Two-Sample Hypothesis Testing Using Kernel Set Classification
Authors	Hamed Masnadi-Shirazi
Abstract	The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data points that are assumed to have a common class. We prove that the average probability of error given a set is less than or equal to the Bayes error and decreases as a power of $n$ number of sample data points in the set. We use the positive definite Set Kernel for directly mapping sets of data to an associated Reproducing Kernel Hilbert Space, without the need to learn a probability distribution. We specifically solve the two-sample hypothesis testing problem using a one-class SVM in conjunction with the proposed Set Kernel. We compare the proposed method with the Maximum Mean Discrepancy, F-Test and T-Test methods on a number of challenging simulated high dimensional and small sample size data. We also perform two-sample hypothesis testing experiments on six cancer gene expression data sets and achieve zero type-I and type-II error results on all data sets.
Tasks
Published	2017-06-18
URL	http://arxiv.org/abs/1706.05612v2
PDF	http://arxiv.org/pdf/1706.05612v2.pdf
PWC	https://paperswithcode.com/paper/kernel-two-sample-hypothesis-testing-using
Repo
Framework

Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction


Title	Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Prediction
Authors	Wen Sun, Arun Venkatraman, Geoffrey J. Gordon, Byron Boots, J. Andrew Bagnell
Abstract	Researchers have demonstrated state-of-the-art performance in sequential decision making problems (e.g., robotics control, sequential prediction) with deep neural network models. One often has access to near-optimal oracles that achieve good performance on the task during training. We demonstrate that AggreVaTeD — a policy gradient extension of the Imitation Learning (IL) approach of (Ross & Bagnell, 2014) — can leverage such an oracle to achieve faster and better solutions with less training data than a less-informed Reinforcement Learning (RL) technique. Using both feedforward and recurrent neural network predictors, we present stochastic gradient procedures on a sequential prediction task, dependency-parsing from raw image data, as well as on various high dimensional robotics control problems. We also provide a comprehensive theoretical study of IL that demonstrates we can expect up to exponentially lower sample complexity for learning with AggreVaTeD than with RL algorithms, which backs our empirical findings. Our results and theory indicate that the proposed approach can achieve superior performance with respect to the oracle when the demonstrator is sub-optimal.
Tasks	Decision Making, Dependency Parsing, Imitation Learning
Published	2017-03-03
URL	http://arxiv.org/abs/1703.01030v1
PDF	http://arxiv.org/pdf/1703.01030v1.pdf
PWC	https://paperswithcode.com/paper/deeply-aggrevated-differentiable-imitation
Repo
Framework

Comparison of ontology alignment systems across single matching task via the McNemar’s test


Title	Comparison of ontology alignment systems across single matching task via the McNemar’s test
Authors	Majid Mohammadi, Amir Ahooye Atashin, Wout Hofman, Yao-Hua Tan
Abstract	Ontology alignment is widely-used to find the correspondences between different ontologies in diverse fields.After discovering the alignments,several performance scores are available to evaluate them.The scores typically require the identified alignment and a reference containing the underlying actual correspondences of the given ontologies.The current trend in the alignment evaluation is to put forward a new score(e.g., precision, weighted precision, etc.)and to compare various alignments by juxtaposing the obtained scores. However,it is substantially provocative to select one measure among others for comparison.On top of that, claiming if one system has a better performance than one another cannot be substantiated solely by comparing two scalars.In this paper,we propose the statistical procedures which enable us to theoretically favor one system over one another.The McNemar’s test is the statistical means by which the comparison of two ontology alignment systems over one matching task is drawn.The test applies to a 2x2 contingency table which can be constructed in two different ways based on the alignments,each of which has their own merits/pitfalls.The ways of the contingency table construction and various apposite statistics from the McNemar’s test are elaborated in minute detail.In the case of having more than two alignment systems for comparison, the family-wise error rate is expected to happen. Thus, the ways of preventing such an error are also discussed.A directed graph visualizes the outcome of the McNemar’s test in the presence of multiple alignment systems.From this graph, it is readily understood if one system is better than one another or if their differences are imperceptible.The proposed statistical methodologies are applied to the systems participated in the OAEI 2016 anatomy track, and also compares several well-known similarity metrics for the same matching problem.
Tasks
Published	2017-03-29
URL	http://arxiv.org/abs/1704.00045v2
PDF	http://arxiv.org/pdf/1704.00045v2.pdf
PWC	https://paperswithcode.com/paper/comparison-of-ontology-alignment-systems
Repo
Framework