October 17, 2019

2753 words 13 mins read

Paper Group ANR 835

Open Problems in Engineering and Quality Assurance of Safety Critical Machine Learning Systems. An Explicit Convergence Rate for Nesterov’s Method from SDP. Selected Qualitative Spatio-temporal Calculi Developed for Constraint Reasoning: A Review. A database linking piano and orchestral MIDI scores with application to automatic projective orchestra …

Open Problems in Engineering and Quality Assurance of Safety Critical Machine Learning Systems


Title	Open Problems in Engineering and Quality Assurance of Safety Critical Machine Learning Systems
Authors	Hiroshi Kuwajima, Hirotoshi Yasuoka, Toshihiro Nakae
Abstract	Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems using machine-learning and deep-learning models, such as automated-driving vehicles. Quality assurance frameworks are required for such machine learning systems, but there are no widely accepted and established quality-assurance concepts and techniques. At the same time, open problems and the relevant technical fields are not organized. To establish standard quality assurance frameworks, it is necessary to visualize and organize these open problems in an interdisciplinary way, so that the experts from many different technical fields may discuss these problems in depth and develop solutions. In the present study, we identify, classify, and explore the open problems in quality assurance of safety-critical machine-learning systems, and their relevant corresponding industry and technological trends, using automated-driving vehicles as an example. Our results show that addressing these open problems requires incorporating knowledge from several different technological and industrial fields, including the automobile industry, statistics, software engineering, and machine learning.
Tasks
Published	2018-12-07
URL	http://arxiv.org/abs/1812.03057v1
PDF	http://arxiv.org/pdf/1812.03057v1.pdf
PWC	https://paperswithcode.com/paper/open-problems-in-engineering-and-quality
Repo
Framework

An Explicit Convergence Rate for Nesterov’s Method from SDP


Title	An Explicit Convergence Rate for Nesterov’s Method from SDP
Authors	Sam Safavi, Bikash Joshi, Guilherme França, José Bento
Abstract	The framework of Integral Quadratic Constraints (IQC) introduced by Lessard et al. (2014) reduces the computation of upper bounds on the convergence rate of several optimization algorithms to semi-definite programming (SDP). In particular, this technique was applied to Nesterov’s accelerated method (NAM). For quadratic functions, this SDP was explicitly solved leading to a new bound on the convergence rate of NAM, and for arbitrary strongly convex functions it was shown numerically that IQC can improve bounds from Nesterov (2004). Unfortunately, an explicit analytic solution to the SDP was not provided. In this paper, we provide such an analytical solution, obtaining a new general and explicit upper bound on the convergence rate of NAM, which we further optimize over its parameters. To the best of our knowledge, this is the best, and explicit, upper bound on the convergence rate of NAM for strongly convex functions.
Tasks
Published	2018-01-13
URL	http://arxiv.org/abs/1801.04492v1
PDF	http://arxiv.org/pdf/1801.04492v1.pdf
PWC	https://paperswithcode.com/paper/an-explicit-convergence-rate-for-nesterovs
Repo
Framework

Selected Qualitative Spatio-temporal Calculi Developed for Constraint Reasoning: A Review


Title	Selected Qualitative Spatio-temporal Calculi Developed for Constraint Reasoning: A Review
Authors	Debasis Mitra
Abstract	In this article a few of the qualitative spatio-temporal knowledge representation techniques developed by the constraint reasoning community within artificial intelligence are reviewed. The objective is to provide a broad exposure to any other interested group who may utilize these representations. The author has a particular interest in applying these calculi (in a broad sense) in topological data analysis, as these schemes are highly qualitative in nature.
Tasks	Topological Data Analysis
Published	2018-12-03
URL	http://arxiv.org/abs/1812.02580v1
PDF	http://arxiv.org/pdf/1812.02580v1.pdf
PWC	https://paperswithcode.com/paper/selected-qualitative-spatio-temporal-calculi
Repo
Framework

A database linking piano and orchestral MIDI scores with application to automatic projective orchestration


Title	A database linking piano and orchestral MIDI scores with application to automatic projective orchestration
Authors	Léopold Crestel, Philippe Esling, Lena Heng, Stephen McAdams
Abstract	This article introduces the Projective Orchestral Database (POD), a collection of MIDI scores composed of pairs linking piano scores to their corresponding orchestrations. To the best of our knowledge, this is the first database of its kind, which performs piano or orchestral prediction, but more importantly which tries to learn the correlations between piano and orchestral scores. Hence, we also introduce the projective orchestration task, which consists in learning how to perform the automatic orchestration of a piano score. We show how this task can be addressed using learning methods and also provide methodological guidelines in order to properly use this database.
Tasks
Published	2018-10-19
URL	http://arxiv.org/abs/1810.08611v1
PDF	http://arxiv.org/pdf/1810.08611v1.pdf
PWC	https://paperswithcode.com/paper/a-database-linking-piano-and-orchestral-midi
Repo
Framework

Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective


Title	Mean-field approximation, convex hierarchies, and the optimality of correlation rounding: a unified perspective
Authors	Vishesh Jain, Frederic Koehler, Andrej Risteski
Abstract	The free energy is a key quantity of interest in Ising models, but unfortunately, computing it in general is computationally intractable. Two popular (variational) approximation schemes for estimating the free energy of general Ising models (in particular, even in regimes where correlation decay does not hold) are: (i) the mean-field approximation with roots in statistical physics, which estimates the free energy from below, and (ii) hierarchies of convex relaxations with roots in theoretical computer science, which estimate the free energy from above. We show, surprisingly, that the tight regime for both methods to compute the free energy to leading order is identical. More precisely, we show that the mean-field approximation is within $O((n\J_{F})^{2/3})$ of the free energy, where $\J_F$ denotes the Frobenius norm of the interaction matrix of the Ising model. This simultaneously subsumes both the breakthrough work of Basak and Mukherjee, who showed the tight result that the mean-field approximation is within $o(n)$ whenever $\J_{F} = o(\sqrt{n})$, as well as the work of Jain, Koehler, and Mossel, who gave the previously best known non-asymptotic bound of $O((n\J_{F})^{2/3}\log^{1/3}(n\J_{F}))$. We give a simple, algorithmic proof of this result using a convex relaxation proposed by Risteski based on the Sherali-Adams hierarchy, automatically giving sub-exponential time approximation schemes for the free energy in this entire regime. Our algorithmic result is tight under Gap-ETH. We furthermore combine our techniques with spin glass theory to prove (in a strong sense) the optimality of correlation rounding, refuting a recent conjecture of Allen, O’Donnell, and Zhou. Finally, we give the tight generalization of all of these results to $k$-MRFs, capturing as a special case previous work on approximating MAX-$k$-CSP.
Tasks
Published	2018-08-22
URL	http://arxiv.org/abs/1808.07226v2
PDF	http://arxiv.org/pdf/1808.07226v2.pdf
PWC	https://paperswithcode.com/paper/mean-field-approximation-convex-hierarchies
Repo
Framework

A New Lower Bound for Agnostic Learning with Sample Compression Schemes


Title	A New Lower Bound for Agnostic Learning with Sample Compression Schemes
Authors	Steve Hanneke, Aryeh Kontorovich
Abstract	We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size-$k$ agnostic sample compression schemes are of the form $\sqrt{\frac{k \log(n/k)}{n}}$, which contrasts with agnostic learning with classes of VC dimension $k$, where the optimal rates are of the form $\sqrt{\frac{k}{n}}$.
Tasks
Published	2018-05-21
URL	http://arxiv.org/abs/1805.08140v1
PDF	http://arxiv.org/pdf/1805.08140v1.pdf
PWC	https://paperswithcode.com/paper/a-new-lower-bound-for-agnostic-learning-with
Repo
Framework

On Periodic Functions as Regularizers for Quantization of Neural Networks


Title	On Periodic Functions as Regularizers for Quantization of Neural Networks
Authors	Maxim Naumov, Utku Diril, Jongsoo Park, Benjamin Ray, Jedrzej Jablonski, Andrew Tulloch
Abstract	Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters. In contrast with popular quantization schemes based on thresholds, we use a novel technique based on periodic functions, such as continuous trigonometric sine or cosine as well as non-continuous hat functions. We apply these functions component-wise and add the sum over the model parameters as a regularizer to the model loss during training. The frequency and amplitude hyper-parameters of these functions can be adjusted during training. The regularization pushes the weights into discrete points that can be encoded as integers. We show that using this technique the resulting quantized models exhibit the same accuracy as the original ones on CIFAR-10 and ImageNet datasets.
Tasks	Quantization
Published	2018-11-24
URL	http://arxiv.org/abs/1811.09862v1
PDF	http://arxiv.org/pdf/1811.09862v1.pdf
PWC	https://paperswithcode.com/paper/on-periodic-functions-as-regularizers-for
Repo
Framework

Sex-Classification from Cell-Phones Periocular Iris Images


Title	Sex-Classification from Cell-Phones Periocular Iris Images
Authors	Juan Tapia, Claudia Arellano, Ignacio Viedma
Abstract	Selfie soft biometrics has great potential for various applications ranging from marketing, security and online banking. However, it faces many challenges since there is limited control in data acquisition conditions. This chapter presents a Super-Resolution-Convolutional Neural Networks (SRCNNs) approach that increases the resolution of low quality periocular iris images cropped from selfie images of subject’s faces. This work shows that increasing image resolution (2x and 3x) can improve the sex-classification rate when using a Random Forest classifier. The best sex-classification rate was 90.15% for the right and 87.15% for the left eye. This was achieved when images were upscaled from 150x150 to 450x450 pixels. These results compare well with the state of the art and show that when improving image resolution with the SRCNN the sex-classification rate increases. Additionally, a novel selfie database captured from 150 subjects with an iPhone X was created (available upon request).
Tasks	Super-Resolution
Published	2018-12-31
URL	http://arxiv.org/abs/1812.11702v1
PDF	http://arxiv.org/pdf/1812.11702v1.pdf
PWC	https://paperswithcode.com/paper/sex-classification-from-cell-phones
Repo
Framework

The Fact Extraction and VERification (FEVER) Shared Task


Title	The Fact Extraction and VERification (FEVER) Shared Task
Authors	James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, Arpit Mittal
Abstract	We present the results of the first Fact Extraction and VERification (FEVER) Shared Task. The task challenged participants to classify whether human-written factoid claims could be Supported or Refuted using evidence retrieved from Wikipedia. We received entries from 23 competing teams, 19 of which scored higher than the previously published baseline. The best performing system achieved a FEVER score of 64.21%. In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.
Tasks
Published	2018-11-27
URL	http://arxiv.org/abs/1811.10971v2
PDF	http://arxiv.org/pdf/1811.10971v2.pdf
PWC	https://paperswithcode.com/paper/the-fact-extraction-and-verification-fever
Repo
Framework

Transfer Learning with Human Corneal Tissues: An Analysis of Optimal Cut-Off Layer


Title	Transfer Learning with Human Corneal Tissues: An Analysis of Optimal Cut-Off Layer
Authors	Nadezhda Prodanova, Johannes Stegmaier, Stephan Allgeier, Sebastian Bohn, Oliver Stachs, Bernd Köhler, Ralf Mikut, Andreas Bartschat
Abstract	Transfer learning is a powerful tool to adapt trained neural networks to new tasks. Depending on the similarity of the original task to the new task, the selection of the cut-off layer is critical. For medical applications like tissue classification, the last layers of an object classification network might not be optimal. We found that on real data of human corneal tissues the best feature representation can be found in the middle layers of the Inception-v3 and in the rear layers of the VGG-19 architecture.
Tasks	Object Classification, Transfer Learning
Published	2018-06-19
URL	http://arxiv.org/abs/1806.07073v2
PDF	http://arxiv.org/pdf/1806.07073v2.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-with-human-corneal-tissues
Repo
Framework

Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System


Title	Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System
Authors	Weicheng Cai, Jinkun Chen, Ming Li
Abstract	In this paper, we explore the encoding/pooling layer and loss function in the end-to-end speaker and language recognition system. First, a unified and interpretable end-to-end system for both speaker and language recognition is developed. It accepts variable-length input and produces an utterance level result. In the end-to-end system, the encoding layer plays a role in aggregating the variable-length input sequence into an utterance level representation. Besides the basic temporal average pooling, we introduce a self-attentive pooling layer and a learnable dictionary encoding layer to get the utterance level representation. In terms of loss function for open-set speaker verification, to get more discriminative speaker embedding, center loss and angular softmax loss is introduced in the end-to-end system. Experimental results on Voxceleb and NIST LRE 07 datasets show that the performance of end-to-end learning system could be significantly improved by the proposed encoding layer and loss function.
Tasks	Speaker Verification
Published	2018-04-14
URL	http://arxiv.org/abs/1804.05160v1
PDF	http://arxiv.org/pdf/1804.05160v1.pdf
PWC	https://paperswithcode.com/paper/exploring-the-encoding-layer-and-loss
Repo
Framework

Principal component-guided sparse regression


Title	Principal component-guided sparse regression
Authors	J. Kenneth Tay, Jerome Friedman, Robert Tibshirani
Abstract	We propose a new method for supervised learning, especially suited to wide data where the number of features is much greater than the number of observations. The method combines the lasso ($\ell_1$) sparsity penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of the feature matrix. We call the proposed method the “principal components lasso” (“pcLasso”). The method can be especially powerful if the features are pre-assigned to groups (such as cell-pathways, assays or protein interaction networks). In that case, pcLasso shrinks each group-wise component of the solution toward the leading principal components of that group. In the process, it also carries out selection of the feature groups. We provide some theory for this method and illustrate it on a number of simulated and real data examples.
Tasks
Published	2018-10-10
URL	http://arxiv.org/abs/1810.04651v3
PDF	http://arxiv.org/pdf/1810.04651v3.pdf
PWC	https://paperswithcode.com/paper/principal-component-guided-sparse-regression
Repo
Framework

A Combined CNN and LSTM Model for Arabic Sentiment Analysis


Title	A Combined CNN and LSTM Model for Arabic Sentiment Analysis
Authors	Abdulaziz M. Alayba, Vasile Palade, Matthew England, Rahat Iqbal
Abstract	Deep neural networks have shown good data modelling capabilities when dealing with challenging and large datasets from a wide range of application areas. Convolutional Neural Networks (CNNs) offer advantages in selecting good features and Long Short-Term Memory (LSTM) networks have proven good abilities of learning sequential data. Both approaches have been reported to provide improved results in areas such image processing, voice recognition, language translation and other Natural Language Processing (NLP) tasks. Sentiment classification for short text messages from Twitter is a challenging task, and the complexity increases for Arabic language sentiment classification tasks because Arabic is a rich language in morphology. In addition, the availability of accurate pre-processing tools for Arabic is another current limitation, along with limited research available in this area. In this paper, we investigate the benefits of integrating CNNs and LSTMs and report obtained improved accuracy for Arabic sentiment analysis on different datasets. Additionally, we seek to consider the morphological diversity of particular Arabic words by using different sentiment classification levels.
Tasks	Arabic Sentiment Analysis, Sentiment Analysis
Published	2018-07-09
URL	http://arxiv.org/abs/1807.02911v3
PDF	http://arxiv.org/pdf/1807.02911v3.pdf
PWC	https://paperswithcode.com/paper/a-combined-cnn-and-lstm-model-for-arabic
Repo
Framework

A Capsule Network-based Embedding Model for Search Personalization


Title	A Capsule Network-based Embedding Model for Search Personalization
Authors	Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dinh Phung
Abstract	Search personalization aims to tailor search results to each specific user based on the user’s personal interests and preferences (i.e., the user profile). Recent research approaches to search personalization by modelling the potential 3-way relationship between the submitted query, the user and the search results (i.e., documents). That relationship is then used to personalize the search results to that user. In this paper, we introduce a novel embedding model based on capsule network, which recently is a breakthrough in deep learning, to model the 3-way relationships for search personalization. In the model, each user (submitted query or returned document) is embedded by a vector in the same vector space. The 3-way relationship is described as a triple of (query, user, document) which is then modeled as a 3-column matrix containing the three embedding vectors. After that, the 3-column matrix is fed into a deep learning architecture to re-rank the search results returned by a basis ranker. Experimental results on query logs from a commercial web search engine show that our model achieves better performances than the basis ranker as well as strong search personalization baselines.
Tasks
Published	2018-04-12
URL	http://arxiv.org/abs/1804.04266v2
PDF	http://arxiv.org/pdf/1804.04266v2.pdf
PWC	https://paperswithcode.com/paper/a-capsule-network-based-embedding-model-for
Repo
Framework

Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributions


Title	Sample-Efficient Learning of Nonprehensile Manipulation Policies via Physics-Based Informed State Distributions
Authors	Lerrel Pinto, Aditya Mandalika, Brian Hou, Siddhartha Srinivasa
Abstract	This paper proposes a sample-efficient yet simple approach to learning closed-loop policies for nonprehensile manipulation. Although reinforcement learning (RL) can learn closed-loop policies without requiring access to underlying physics models, it suffers from poor sample complexity on challenging tasks. To overcome this problem, we leverage rearrangement planning to provide an informative physics-based prior on the environment’s optimal state-visitation distribution. Specifically, we present a new technique, Learning with Planned Episodic Resets (LeaPER), that resets the environment’s state to one informed by the prior during the learning phase. We experimentally show that LeaPER significantly outperforms traditional RL approaches by a factor of up to 5X on simulated rearrangement. Further, we relax dynamics from quasi-static to welded contacts to illustrate that LeaPER is robust to the use of simpler physics models. Finally, LeaPER’s closed-loop policies significantly improve task success rates relative to both open-loop controls with a planned path or simple feedback controllers that track open-loop trajectories. We demonstrate the performance and behavior of LeaPER on a physical 7-DOF manipulator in https://youtu.be/feS-zFq6J1c.
Tasks
Published	2018-10-24
URL	http://arxiv.org/abs/1810.10654v1
PDF	http://arxiv.org/pdf/1810.10654v1.pdf
PWC	https://paperswithcode.com/paper/sample-efficient-learning-of-nonprehensile
Repo
Framework