October 16, 2019

3330 words 16 mins read

Paper Group ANR 1112

SciSports: Learning football kinematics through two-dimensional tracking data. Adversarial Noise Attacks of Deep Learning Architectures – Stability Analysis via Sparse Modeled Signals. On the Randomized Complexity of Minimizing a Convex Quadratic Function. Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions. Conducting …

SciSports: Learning football kinematics through two-dimensional tracking data


Title	SciSports: Learning football kinematics through two-dimensional tracking data
Authors	Anatoliy Babic, Harshit Bansal, Gianluca Finocchio, Julian Golak, Mark Peletier, Jim Portegies, Clara Stegehuis, Anuj Tyagi, Roland Vincze, William Weimin Yoo
Abstract	SciSports is a Dutch startup company specializing in football analytics. This paper describes a joint research effort with SciSports, during the Study Group Mathematics with Industry 2018 at Eindhoven, the Netherlands. The main challenge that we addressed was to automatically process empirical football players’ trajectories, in order to extract useful information from them. The data provided to us was two-dimensional positional data during entire matches. We developed methods based on Newtonian mechanics and the Kalman filter, Generative Adversarial Nets and Variational Autoencoders. In addition, we trained a discriminator network to recognize and discern different movement patterns of players. The Kalman-filter approach yields an interpretable model, in which a small number of player-dependent parameters can be fit; in theory this could be used to distinguish among players. The Generative-Adversarial-Nets approach appears promising in theory, and some initial tests showed an improvement with respect to the baseline, but the limits in time and computational power meant that we could not fully explore it. We also trained a Discriminator network to distinguish between two players based on their trajectories; after training, the network managed to distinguish between some pairs of players, but not between others. After training, the Variational Autoencoders generated trajectories that are difficult to distinguish, visually, from the data. These experiments provide an indication that deep generative models can learn the underlying structure and statistics of football players’ trajectories. This can serve as a starting point for determining player qualities based on such trajectory data.
Tasks
Published	2018-08-14
URL	http://arxiv.org/abs/1808.04550v1
PDF	http://arxiv.org/pdf/1808.04550v1.pdf
PWC	https://paperswithcode.com/paper/scisports-learning-football-kinematics
Repo
Framework

Adversarial Noise Attacks of Deep Learning Architectures – Stability Analysis via Sparse Modeled Signals


Title	Adversarial Noise Attacks of Deep Learning Architectures – Stability Analysis via Sparse Modeled Signals
Authors	Yaniv Romano, Aviad Aberdam, Jeremias Sulam, Michael Elad
Abstract	Despite their impressive performance, deep convolutional neural networks (CNNs) have been shown to be sensitive to small adversarial perturbations. These nuisances, which one can barely notice, are powerful enough to fool sophisticated and well performing classifiers, leading to ridiculous misclassification results. In this paper we analyze the stability of state-of-the-art deep-learning classification machines to adversarial perturbations, where we assume that the signals belong to the (possibly multi-layer) sparse representation model. We start with convolutional sparsity and then proceed to its multi-layered version, which is tightly connected to CNNs. Our analysis links between the stability of the classification to noise and the underlying structure of the signal, quantified by the sparsity of its representation under a fixed dictionary. In addition, we offer similar stability theorems for two practical pursuit algorithms, which are posed as two different deep-learning architectures - the layered Thresholding and the layered Basis Pursuit. Our analysis establishes the better robustness of the later to adversarial attacks. We corroborate these theoretical results by numerical experiments on three datasets: MNIST, CIFAR-10 and CIFAR-100.
Tasks
Published	2018-05-29
URL	https://arxiv.org/abs/1805.11596v3
PDF	https://arxiv.org/pdf/1805.11596v3.pdf
PWC	https://paperswithcode.com/paper/adversarial-noise-attacks-of-deep-learning
Repo
Framework

On the Randomized Complexity of Minimizing a Convex Quadratic Function


Title	On the Randomized Complexity of Minimizing a Convex Quadratic Function
Authors	Max Simchowitz
Abstract	Minimizing a convex, quadratic objective of the form $f_{\mathbf{A},\mathbf{b}}(x) := \frac{1}{2}x^\top \mathbf{A} x - \langle \mathbf{b}, x \rangle$ for $\mathbf{A} \succ 0 $ is a fundamental problem in machine learning and optimization. In this work, we prove gradient-query complexity lower bounds for minimizing convex quadratic functions which apply to both deterministic and \emph{randomized} algorithms. Specifically, for $\kappa > 1$, we exhibit a distribution over $(\mathbf{A},\mathbf{b})$ with condition number $\mathrm{cond}(\mathbf{A}) \le \kappa$, such that any \emph{randomized} algorithm requires $\Omega(\sqrt{\kappa})$ gradient queries to find a solution $\hat x$ for which $\hat x - \mathbf x_\star\ \le \epsilon_0\mathbf{x}_{\star}$, where $\mathbf x_{\star} = \mathbf{A}^{-1}\mathbf{b}$ is the optimal solution, and $\epsilon_0$ a small constant. Setting $\kappa =1/\epsilon$, this lower bound implies the minimax rate of $T = \Omega(\lambda_1(\mathbf{A})\mathbf x_\star^2/\sqrt{\epsilon})$ queries required to minimize an arbitrary convex quadratic function up to error $f(\hat{x}) - f(\mathbf x_\star) \le \epsilon$. Our lower bound holds for a distribution derived from classical ensembles in random matrix theory, and relies on a careful reduction from adaptively estimating a planted vector $\mathbf u$ in a deformed Wigner model. A key step in deriving sharp lower bounds is demonstrating that the optimization error $\mathbf x_\star - \hat x$ cannot align too closely with $\mathbf{u}$. To this end, we prove an upper bound on the cosine between $\mathbf x_\star - \hat x$ and $\mathbf u$ in terms of the MMSE of estimating the plant $\mathbf u$ in a deformed Wigner model. We then bound the MMSE by carefully modifying a result due to Lelarge and Miolane 2016, which rigorously establishes a general replica-symmetric formula for planted matrix models.
Tasks
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09386v7
PDF	http://arxiv.org/pdf/1807.09386v7.pdf
PWC	https://paperswithcode.com/paper/on-the-randomized-complexity-of-minimizing-a
Repo
Framework

Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions


Title	Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions
Authors	Feiyun Zhu, Jun Guo, Ruoyu Li, Junzhou Huang
Abstract	We consider the actor-critic contextual bandit for the mobile health (mHealth) intervention. State-of-the-art decision-making algorithms generally ignore the outliers in the dataset. In this paper, we propose a novel robust contextual bandit method for the mHealth. It can achieve the conflicting goal of reducing the influence of outliers while seeking for a similar solution compared with the state-of-the-art contextual bandit methods on the datasets without outliers. Such performance relies on two technologies: (1) the capped-$\ell_{2}$ norm; (2) a reliable method to set the thresholding hyper-parameter, which is inspired by one of the most fundamental techniques in the statistics. Although the model is non-convex and non-differentiable, we propose an effective reweighted algorithm and provide solid theoretical analyses. We prove that the proposed algorithm can find sufficiently decreasing points after each iteration and finally converges after a finite number of iterations. Extensive experiment results on two datasets demonstrate that our method can achieve almost identical results compared with state-of-the-art contextual bandit methods on the dataset without outliers, and significantly outperform those state-of-the-art methods on the badly noised dataset with outliers in a variety of parameter settings.
Tasks	Decision Making
Published	2018-02-27
URL	http://arxiv.org/abs/1802.09714v1
PDF	http://arxiv.org/pdf/1802.09714v1.pdf
PWC	https://paperswithcode.com/paper/robust-actor-critic-contextual-bandit-for
Repo
Framework

Conducting Feasibility Studies for Knowledge Based Systems


Title	Conducting Feasibility Studies for Knowledge Based Systems
Authors	John Kingston
Abstract	This paper describes how to carry out a feasibility study for a potential knowledge based system application. It discusses factors to be considered under three headings: the business case, the technical feasibility, and stakeholder issues. It concludes with a case study of a feasibility study for a KBS to guide surgeons in diagnosis and treatment of thyroid conditions.
Tasks
Published	2018-09-21
URL	http://arxiv.org/abs/1809.08059v1
PDF	http://arxiv.org/pdf/1809.08059v1.pdf
PWC	https://paperswithcode.com/paper/conducting-feasibility-studies-for-knowledge
Repo
Framework

Learning to Navigate the Web


Title	Learning to Navigate the Web
Authors	Izzeddin Gur, Ulrich Rueckert, Aleksandra Faust, Dilek Hakkani-Tur
Abstract	Learning in environments with large state and action spaces, and sparse rewards, can hinder a Reinforcement Learning (RL) agent’s learning through trial-and-error. For instance, following natural language instructions on the Web (such as booking a flight ticket) leads to RL settings where input vocabulary and number of actionable elements on a page can grow very large. Even though recent approaches improve the success rate on relatively simple environments with the help of human demonstrations to guide the exploration, they still fail in environments where the set of possible instructions can reach millions. We approach the aforementioned problems from a different perspective and propose guided RL approaches that can generate unbounded amount of experience for an agent to learn from. Instead of learning from a complicated instruction with a large vocabulary, we decompose it into multiple sub-instructions and schedule a curriculum in which an agent is tasked with a gradually increasing subset of these relatively easier sub-instructions. In addition, when the expert demonstrations are not available, we propose a novel meta-learning framework that generates new instruction following tasks and trains the agent more effectively. We train DQN, deep reinforcement learning agent, with Q-value function approximated with a novel QWeb neural network architecture on these smaller, synthetic instructions. We evaluate the ability of our agent to generalize to new instructions on World of Bits benchmark, on forms with up to 100 elements, supporting 14 million possible instructions. The QWeb agent outperforms the baseline without using any human demonstration achieving 100% success rate on several difficult environments.
Tasks	Meta-Learning
Published	2018-12-21
URL	http://arxiv.org/abs/1812.09195v1
PDF	http://arxiv.org/pdf/1812.09195v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-navigate-the-web
Repo
Framework

Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events


Title	Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events
Authors	Sanjeel Parekh, Slim Essid, Alexey Ozerov, Ngoc Q. K. Duong, Patrick Pérez, Gaël Richard
Abstract	Audio-visual representation learning is an important task from the perspective of designing machines with the ability to understand complex events. To this end, we propose a novel multimodal framework that instantiates multiple instance learning. We show that the learnt representations are useful for classifying events and localizing their characteristic audio-visual elements. The system is trained using only video-level event labels without any timing information. An important feature of our method is its capacity to learn from unsynchronized audio-visual events. We achieve state-of-the-art results on a large-scale dataset of weakly-labeled audio event videos. Visualizations of localized visual regions and audio segments substantiate our system’s efficacy, especially when dealing with noisy situations where modality-specific cues appear asynchronously.
Tasks	Multiple Instance Learning, Representation Learning
Published	2018-04-19
URL	http://arxiv.org/abs/1804.07345v2
PDF	http://arxiv.org/pdf/1804.07345v2.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-representation-learning-for
Repo
Framework

Effects of Word Embeddings on Neural Network-based Pitch Accent Detection


Title	Effects of Word Embeddings on Neural Network-based Pitch Accent Detection
Authors	Sabrina Stehwien, Ngoc Thang Vu, Antje Schweitzer
Abstract	Pitch accent detection often makes use of both acoustic and lexical features based on the fact that pitch accents tend to correlate with certain words. In this paper, we extend a pitch accent detector that involves a convolutional neural network to include word embeddings, which are state-of-the-art vector representations of words. We examine the effect these features have on within-corpus and cross-corpus experiments on three English datasets. The results show that while word embeddings can improve the performance in corpus-dependent experiments, they also have the potential to make generalization to unseen data more challenging.
Tasks	Word Embeddings
Published	2018-05-14
URL	http://arxiv.org/abs/1805.05237v2
PDF	http://arxiv.org/pdf/1805.05237v2.pdf
PWC	https://paperswithcode.com/paper/effects-of-word-embeddings-on-neural-network
Repo
Framework

Thermal Features for Presentation Attack Detection in Hand Biometrics


Title	Thermal Features for Presentation Attack Detection in Hand Biometrics
Authors	Ewelina Bartuzi, Mateusz Trokielewicz
Abstract	This paper proposes a method for utilizing thermal features of the hand for the purpose of presentation attack detection (PAD) that can be employed in a hand biometrics system’s pipeline. By envisaging two different operational modes of our system, and by employing a DCNN-based classifiers fine-tuned with a dataset of real and fake hand representations captured in both visible and ther- mal spectrum, we were able to bring two important deliverables. First, a PAD method operating in an open-set mode, capable of correctly discerning 100% of fake thermal samples, achieving Attack Presentation Classification Error Rate (APCER) and Bona-Fide Presentation Classification Error Rate (BPCER) equal to 0%, which can be easily implemented into any existing system as a separate component. Second, a hand biometrics system operating in a closed-set mode, that has PAD built right into the recognition pipeline, and operating simultaneously with the user-wise classification, achieving rank-1 recognition accuracy of up to 99.75%. We also show that thermal images of the human hand, in addition to liveness features they carry, can also improve classification accuracy of a biometric system, when coupled with visible light images. To follow the reproducibility guidelines and to stimulate further research in this area, we share the trained model weights, source codes, and a newly created dataset of fake hand representations with interested researchers.
Tasks
Published	2018-09-12
URL	http://arxiv.org/abs/1809.04364v1
PDF	http://arxiv.org/pdf/1809.04364v1.pdf
PWC	https://paperswithcode.com/paper/thermal-features-for-presentation-attack
Repo
Framework

Style Obfuscation by Invariance


Title	Style Obfuscation by Invariance
Authors	Chris Emmery, Enrique Manjavacas, Grzegorz Chrupała
Abstract	The task of obfuscating writing style using sequence models has previously been investigated under the framework of obfuscation-by-transfer, where the input text is explicitly rewritten in another style. These approaches also often lead to major alterations to the semantic content of the input. In this work, we propose obfuscation-by-invariance, and investigate to what extent models trained to be explicitly style-invariant preserve semantics. We evaluate our architectures on parallel and non-parallel corpora, and compare automatic and human evaluations on the obfuscated sentences. Our experiments show that style classifier performance can be reduced to chance level, whilst the automatic evaluation of the output is seemingly equal to models applying style-transfer. However, based on human evaluation we demonstrate a trade-off between the level of obfuscation and the observed quality of the output in terms of meaning preservation and grammaticality.
Tasks	Style Transfer
Published	2018-05-18
URL	http://arxiv.org/abs/1805.07143v1
PDF	http://arxiv.org/pdf/1805.07143v1.pdf
PWC	https://paperswithcode.com/paper/style-obfuscation-by-invariance
Repo
Framework

3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training


Title	3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training
Authors	Yingda Xia, Fengze Liu, Dong Yang, Jinzheng Cai, Lequan Yu, Zhuotun Zhu, Daguang Xu, Alan Yuille, Holger Roth
Abstract	While making a tremendous impact in various fields, deep neural networks usually require large amounts of labeled data for training which are expensive to collect in many applications, especially in the medical domain. Unlabeled data, on the other hand, is much more abundant. Semi-supervised learning techniques, such as co-training, could provide a powerful tool to leverage unlabeled data. In this paper, we propose a novel framework, uncertainty-aware multi-view co-training (UMCT), to address semi-supervised learning on 3D data, such as volumetric data from medical imaging. In our work, co-training is achieved by exploiting multi-viewpoint consistency of 3D data. We generate different views by rotating or permuting the 3D data and utilize asymmetrical 3D kernels to encourage diversified features in different sub-networks. In addition, we propose an uncertainty-weighted label fusion mechanism to estimate the reliability of each view’s prediction with Bayesian deep learning. As one view requires the supervision from other views in co-training, our self-adaptive approach computes a confidence score for the prediction of each unlabeled sample in order to assign a reliable pseudo label. Thus, our approach can take advantage of unlabeled data during training. We show the effectiveness of our proposed semi-supervised method on several public datasets from medical image segmentation tasks (NIH pancreas & LiTS liver tumor dataset). Meanwhile, a fully-supervised method based on our approach achieved state-of-the-art performances on both the LiTS liver tumor segmentation and the Medical Segmentation Decathlon (MSD) challenge, demonstrating the robustness and value of our framework, even when fully supervised training is feasible.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2018-11-29
URL	https://arxiv.org/abs/1811.12506v2
PDF	https://arxiv.org/pdf/1811.12506v2.pdf
PWC	https://paperswithcode.com/paper/3d-semi-supervised-learning-with-uncertainty
Repo
Framework

A note on belief structures and S-approximation spaces


Title	A note on belief structures and S-approximation spaces
Authors	Ali Shakiba, Amir Kafshdar Goharshady, MohammadReza Hooshmandasl, Mohsen Alambardar Meybodi
Abstract	We study relations between evidence theory and S-approximation spaces. Both theories have their roots in the analysis of Dempster’s multivalued mappings and lower and upper probabilities and have close relations to rough sets. We show that an S-approximation space, satisfying a monotonicity condition, can induce a natural belief structure which is a fundamental block in evidence theory. We also demonstrate that one can induce a natural belief structure on one set, given a belief structure on another set if those sets are related by a partial monotone S-approximation space.
Tasks
Published	2018-05-27
URL	https://arxiv.org/abs/1805.10672v4
PDF	https://arxiv.org/pdf/1805.10672v4.pdf
PWC	https://paperswithcode.com/paper/a-note-on-belief-structures-and-s
Repo
Framework

Online Illumination Invariant Moving Object Detection by Generative Neural Network


Title	Online Illumination Invariant Moving Object Detection by Generative Neural Network
Authors	Fateme Bahri, Moein Shakeri, Nilanjan Ray
Abstract	Moving object detection (MOD) is a significant problem in computer vision that has many real world applications. Different categories of methods have been proposed to solve MOD. One of the challenges is to separate moving objects from illumination changes and shadows that are present in most real world videos. State-of-the-art methods that can handle illumination changes and shadows work in a batch mode; thus, these methods are not suitable for long video sequences or real-time applications. In this paper, we propose an extension of a state-of-the-art batch MOD method (ILISD) to an online/incremental MOD using unsupervised and generative neural networks, which use illumination invariant image representations. For each image in a sequence, we use a low-dimensional representation of a background image by a neural network and then based on the illumination invariant representation, decompose the foreground image into: illumination change and moving objects. Optimization is performed by stochastic gradient descent in an end-to-end and unsupervised fashion. Our algorithm can work in both batch and online modes. In the batch mode, like other batch methods, optimizer uses all the images. In online mode, images can be incrementally fed into the optimizer. Based on our experimental evaluation on benchmark image sequences, both the online and the batch modes of our algorithm achieve state-of-the-art accuracy on most data sets.
Tasks	Object Detection
Published	2018-08-03
URL	http://arxiv.org/abs/1808.01066v1
PDF	http://arxiv.org/pdf/1808.01066v1.pdf
PWC	https://paperswithcode.com/paper/online-illumination-invariant-moving-object
Repo
Framework

Policy Transfer with Strategy Optimization


Title	Policy Transfer with Strategy Optimization
Authors	Wenhao Yu, C. Karen Liu, Greg Turk
Abstract	Computer simulation provides an automatic and safe way for training robotic control policies to achieve complex tasks such as locomotion. However, a policy trained in simulation usually does not transfer directly to the real hardware due to the differences between the two environments. Transfer learning using domain randomization is a promising approach, but it usually assumes that the target environment is close to the distribution of the training environments, thus relying heavily on accurate system identification. In this paper, we present a different approach that leverages domain randomization for transferring control policies to unknown environments. The key idea that, instead of learning a single policy in the simulation, we simultaneously learn a family of policies that exhibit different behaviors. When tested in the target environment, we directly search for the best policy in the family based on the task performance, without the need to identify the dynamic parameters. We evaluate our method on five simulated robotic control problems with different discrepancies in the training and testing environment and demonstrate that our method can overcome larger modeling errors compared to training a robust policy or an adaptive policy.
Tasks	Transfer Learning
Published	2018-10-12
URL	http://arxiv.org/abs/1810.05751v2
PDF	http://arxiv.org/pdf/1810.05751v2.pdf
PWC	https://paperswithcode.com/paper/policy-transfer-with-strategy-optimization
Repo
Framework

Backdooring Convolutional Neural Networks via Targeted Weight Perturbations


Title	Backdooring Convolutional Neural Networks via Targeted Weight Perturbations
Authors	Jacob Dumford, Walter Scheirer
Abstract	We present a new type of backdoor attack that exploits a vulnerability of convolutional neural networks (CNNs) that has been previously unstudied. In particular, we examine the application of facial recognition. Deep learning techniques are at the top of the game for facial recognition, which means they have now been implemented in many production-level systems. Alarmingly, unlike other commercial technologies such as operating systems and network devices, deep learning-based facial recognition algorithms are not presently designed with security requirements or audited for security vulnerabilities before deployment. Given how young the technology is and how abstract many of the internal workings of these algorithms are, neural network-based facial recognition systems are prime targets for security breaches. As more and more of our personal information begins to be guarded by facial recognition (e.g., the iPhone X), exploring the security vulnerabilities of these systems from a penetration testing standpoint is crucial. Along these lines, we describe a general methodology for backdooring CNNs via targeted weight perturbations. Using a five-layer CNN and ResNet-50 as case studies, we show that an attacker is able to significantly increase the chance that inputs they supply will be falsely accepted by a CNN while simultaneously preserving the error rates for legitimate enrolled classes.
Tasks
Published	2018-12-07
URL	http://arxiv.org/abs/1812.03128v1
PDF	http://arxiv.org/pdf/1812.03128v1.pdf
PWC	https://paperswithcode.com/paper/backdooring-convolutional-neural-networks-via
Repo
Framework