July 27, 2019

3711 words 18 mins read

Paper Group ANR 625

Machine Learning as Statistical Data Assimilation. Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks. Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation. Variational models for joint subsampling and reconstruction of turbulence-degraded images. P …

Machine Learning as Statistical Data Assimilation


Title	Machine Learning as Statistical Data Assimilation
Authors	H. D. I. Abarbanel, P. J. Rozdeba, S. Shirman
Abstract	We identify a strong equivalence between neural network based machine learning (ML) methods and the formulation of statistical data assimilation (DA), known to be a problem in statistical physics. DA, as used widely in physical and biological sciences, systematically transfers information in observations to a model of the processes producing the observations. The correspondence is that layer label in the ML setting is the analog of time in the data assimilation setting. Utilizing aspects of this equivalence we discuss how to establish the global minimum of the cost functions in the ML context, using a variational annealing method from DA. This provides a design method for optimal networks for ML applications and may serve as the basis for understanding the success of “deep learning”. Results from an ML example are presented. When the layer label is taken to be continuous, the Euler-Lagrange equation for the ML optimization problem is an ordinary differential equation, and we see that the problem being solved is a two point boundary value problem. The use of continuous layers is denoted “deepest learning”. The Hamiltonian version provides a direct rationale for back propagation as a solution method for the canonical momentum; however, it suggests other solution methods are to be preferred.
Tasks
Published	2017-10-19
URL	http://arxiv.org/abs/1710.07276v1
PDF	http://arxiv.org/pdf/1710.07276v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-as-statistical-data
Repo
Framework

Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks


Title	Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks
Authors	Brandon Victor, Zhen He, Stuart Morgan, Dino Miniutti
Abstract	In many sports, it is useful to analyse video of an athlete in competition for training purposes. In swimming, stroke rate is a common metric used by coaches; requiring a laborious labelling of each individual stroke. We show that using a Convolutional Neural Network (CNN) we can automatically detect discrete events in continuous video (in this case, swimming strokes). We create a CNN that learns a mapping from a window of frames to a point on a smooth 1D target signal, with peaks denoting the location of a stroke, evaluated as a sliding window. To our knowledge this process of training and utilizing a CNN has not been investigated before; either in sports or fundamental computer vision research. Most research has been focused on action recognition and using it to classify many clips in continuous video for action localisation. In this paper we demonstrate our process works well on the task of detecting swimming strokes in the wild. However, without modifying the model architecture or training method, the process is also shown to work equally well on detecting tennis strokes, implying that this is a general process. The outputs of our system are surprisingly smooth signals that predict an arbitrary event at least as accurately as humans (manually evaluated from a sample of negative results). A number of different architectures are evaluated, pertaining to slightly different problem formulations and signal targets.
Tasks	Temporal Action Localization
Published	2017-05-28
URL	http://arxiv.org/abs/1705.09894v1
PDF	http://arxiv.org/pdf/1705.09894v1.pdf
PWC	https://paperswithcode.com/paper/continuous-video-to-simple-signals-for
Repo
Framework

Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation


Title	Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation
Authors	Jinzhuo Wang, Wenmin Wang, Ronggang Wang, Wen Gao
Abstract	Monte Carlo tree search (MCTS) is extremely popular in computer Go which determines each action by enormous simulations in a broad and deep search tree. However, human experts select most actions by pattern analysis and careful evaluation rather than brute search of millions of future nteractions. In this paper, we propose a computer Go system that follows experts way of thinking and playing. Our system consists of two parts. The first part is a novel deep alternative neural network (DANN) used to generate candidates of next move. Compared with existing deep convolutional neural network (DCNN), DANN inserts recurrent layer after each convolutional layer and stacks them in an alternative manner. We show such setting can preserve more contexts of local features and its evolutions which are beneficial for move prediction. The second part is a long-term evaluation (LTE) module used to provide a reliable evaluation of candidates rather than a single probability from move predictor. This is consistent with human experts nature of playing since they can foresee tens of steps to give an accurate estimation of candidates. In our system, for each candidate, LTE calculates a cumulative reward after several future interactions when local variations are settled. Combining criteria from the two parts, our system determines the optimal choice of next move. For more comprehensive experiments, we introduce a new professional Go dataset (PGD), consisting of 253233 professional records. Experiments on GoGoD and PGD datasets show the DANN can substantially improve performance of move prediction over pure DCNN. When combining LTE, our system outperforms most relevant approaches and open engines based on MCTS.
Tasks
Published	2017-06-13
URL	http://arxiv.org/abs/1706.04052v1
PDF	http://arxiv.org/pdf/1706.04052v1.pdf
PWC	https://paperswithcode.com/paper/beyond-monte-carlo-tree-search-playing-go
Repo
Framework

Variational models for joint subsampling and reconstruction of turbulence-degraded images


Title	Variational models for joint subsampling and reconstruction of turbulence-degraded images
Authors	Chun Pong Lau, Yu Hin Lai, Lok Ming Lui
Abstract	Turbulence-degraded image frames are distorted by both turbulent deformations and space-time-varying blurs. To suppress these effects, we propose a multi-frame reconstruction scheme to recover a latent image from the observed image sequence. Recent approaches are commonly based on registering each frame to a reference image, by which geometric turbulent deformations can be estimated and a sharp image can be restored. A major challenge is that a fine reference image is usually unavailable, as every turbulence-degraded frame is distorted. A high-quality reference image is crucial for the accurate estimation of geometric deformations and fusion of frames. Besides, it is unlikely that all frames from the image sequence are useful, and thus frame selection is necessary and highly beneficial. In this work, we propose a variational model for joint subsampling of frames and extraction of a clear image. A fine image and a suitable choice of subsample are simultaneously obtained by iteratively reducing an energy functional. The energy consists of a fidelity term measuring the discrepancy between the extracted image and the subsampled frames, as well as regularization terms on the extracted image and the subsample. Different choices of fidelity and regularization terms are explored. By carefully selecting suitable frames and extracting the image, the quality of the reconstructed image can be significantly improved. Extensive experiments have been carried out, which demonstrate the efficacy of our proposed model. In addition, the extracted subsamples and images can be put in existing algorithms to produce improved results.
Tasks
Published	2017-12-08
URL	http://arxiv.org/abs/1712.03825v1
PDF	http://arxiv.org/pdf/1712.03825v1.pdf
PWC	https://paperswithcode.com/paper/variational-models-for-joint-subsampling-and
Repo
Framework

Patch-based Carcinoma Detection on Confocal Laser Endomicroscopy Images – A Cross-Site Robustness Assessment


Title	Patch-based Carcinoma Detection on Confocal Laser Endomicroscopy Images – A Cross-Site Robustness Assessment
Authors	Marc Aubreville, Miguel Goncalves, Christian Knipfer, Nicolai Oetter, Tobias Wuerfl, Helmut Neumann, Florian Stelzle, Christopher Bohr, Andreas Maier
Abstract	Deep learning technologies such as convolutional neural networks (CNN) provide powerful methods for image recognition and have recently been employed in the field of automated carcinoma detection in confocal laser endomicroscopy (CLE) images. CLE is a (sub-)surface microscopic imaging technique that reaches magnifications of up to 1000x and is thus suitable for in vivo structural tissue analysis. In this work, we aim to evaluate the prospects of a priorly developed deep learning-based algorithm targeted at the identification of oral squamous cell carcinoma with regard to its generalization to further anatomic locations of squamous cell carcinomas in the area of head and neck. We applied the algorithm on images acquired from the vocal fold area of five patients with histologically verified squamous cell carcinoma and presumably healthy control images of the clinically normal contra-lateral vocal cord. We find that the network trained on the oral cavity data reaches an accuracy of 89.45% and an area-under-the-curve (AUC) value of 0.955, when applied on the vocal cords data. Compared to the state of the art, we achieve very similar results, yet with an algorithm that was trained on a completely disjunct data set. Concatenating both data sets yielded further improvements in cross-validation with an accuracy of 90.81% and AUC of 0.970. In this study, for the first time to our knowledge, a deep learning mechanism for the identification of oral carcinomas using CLE Images could be applied to other disciplines in the area of head and neck. This study shows the prospect of the algorithmic approach to generalize well on other malignant entities of the head and neck, regardless of the anatomical location and furthermore in an examiner-independent manner.
Tasks
Published	2017-07-25
URL	https://arxiv.org/abs/1707.08149v2
PDF	https://arxiv.org/pdf/1707.08149v2.pdf
PWC	https://paperswithcode.com/paper/patch-based-carcinoma-detection-on-confocal
Repo
Framework

Information Processing by Networks of Quantum Decision Makers


Title	Information Processing by Networks of Quantum Decision Makers
Authors	V. I. Yukalov, E. P. Yukalova, D. Sornette
Abstract	We suggest a model of a multi-agent society of decision makers taking decisions being based on two criteria, one is the utility of the prospects and the other is the attractiveness of the considered prospects. The model is the generalization of quantum decision theory, developed earlier for single decision makers realizing one-step decisions, in two principal aspects. First, several decision makers are considered simultaneously, who interact with each other through information exchange. Second, a multistep procedure is treated, when the agents exchange information many times. Several decision makers exchanging information and forming their judgement, using quantum rules, form a kind of a quantum information network, where collective decisions develop in time as a result of information exchange. In addition to characterizing collective decisions that arise in human societies, such networks can describe dynamical processes occurring in artificial quantum intelligence composed of several parts or in a cluster of quantum computers. The practical usage of the theory is illustrated on the dynamic disjunction effect for which three quantitative predictions are made: (i) the probabilistic behavior of decision makers at the initial stage of the process is described; (ii) the decrease of the difference between the initial prospect probabilities and the related utility factors is proved; (iii) the existence of a common consensus after multiple exchange of information is predicted. The predicted numerical values are in very good agreement with empirical data.
Tasks
Published	2017-12-15
URL	http://arxiv.org/abs/1712.05734v1
PDF	http://arxiv.org/pdf/1712.05734v1.pdf
PWC	https://paperswithcode.com/paper/information-processing-by-networks-of-quantum
Repo
Framework

Strategic Classification from Revealed Preferences


Title	Strategic Classification from Revealed Preferences
Authors	Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, Zhiwei Steven Wu
Abstract	We study an online linear classification problem, in which the data is generated by strategic agents who manipulate their features in an effort to change the classification outcome. In rounds, the learner deploys a classifier, and an adversarially chosen agent arrives, possibly manipulating her features to optimally respond to the learner. The learner has no knowledge of the agents’ utility functions or “real” features, which may vary widely across agents. Instead, the learner is only able to observe their “revealed preferences” — i.e. the actual manipulated feature vectors they provide. For a broad family of agent cost functions, we give a computationally efficient learning algorithm that is able to obtain diminishing “Stackelberg regret” — a form of policy regret that guarantees that the learner is obtaining loss nearly as small as that of the best classifier in hindsight, even allowing for the fact that agents will best-respond differently to the optimal classifier.
Tasks
Published	2017-10-22
URL	http://arxiv.org/abs/1710.07887v1
PDF	http://arxiv.org/pdf/1710.07887v1.pdf
PWC	https://paperswithcode.com/paper/strategic-classification-from-revealed
Repo
Framework

Learning Neural Markers of Schizophrenia Disorder Using Recurrent Neural Networks


Title	Learning Neural Markers of Schizophrenia Disorder Using Recurrent Neural Networks
Authors	Jumana Dakka, Pouya Bashivan, Mina Gheiratmand, Irina Rish, Shantenu Jha, Russell Greiner
Abstract	Smart systems that can accurately diagnose patients with mental disorders and identify effective treatments based on brain functional imaging data are of great applicability and are gaining much attention. Most previous machine learning studies use hand-designed features, such as functional connectivity, which does not maintain the potential useful information in the spatial relationship between brain regions and the temporal profile of the signal in each region. Here we propose a new method based on recurrent-convolutional neural networks to automatically learn useful representations from segments of 4-D fMRI recordings. Our goal is to exploit both spatial and temporal information in the functional MRI movie (at the whole-brain voxel level) for identifying patients with schizophrenia.
Tasks
Published	2017-12-01
URL	http://arxiv.org/abs/1712.00512v1
PDF	http://arxiv.org/pdf/1712.00512v1.pdf
PWC	https://paperswithcode.com/paper/learning-neural-markers-of-schizophrenia
Repo
Framework

Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization


Title	Multi-View Spectral Clustering via Structured Low-Rank Matrix Factorization
Authors	Yang Wang, Lin Wu
Abstract	Multi-view data clustering attracts more attention than their single view counterparts due to the fact that leveraging multiple independent and complementary information from multi-view feature spaces outperforms the single one. Multi-view Spectral Clustering aims at yielding the data partition agreement over their local manifold structures by seeking eigenvalue-eigenvector decompositions. However, as we observed, such classical paradigm still suffers from (1) overlooking the flexible local manifold structure, caused by (2) enforcing the low-rank data correlation agreement among all views; worse still, (3) LRR is not intuitively flexible to capture the latent data clustering structures. In this paper, we present the structured LRR by factorizing into the latent low-dimensional data-cluster representations, which characterize the data clustering structure for each view. Upon such representation, (b) the laplacian regularizer is imposed to be capable of preserving the flexible local manifold structure for each view. (c) We present an iterative multi-view agreement strategy by minimizing the divergence objective among all factorized latent data-cluster representations during each iteration of optimization process, where such latent representation from each view serves to regulate those from other views, such intuitive process iteratively coordinates all views to be agreeable. (d) We remark that such data-cluster representation can flexibly encode the data clustering structure from any view with adaptive input cluster number. To this end, (e) a novel non-convex objective function is proposed via the efficient alternating minimization strategy. The complexity analysis are also presented. The extensive experiments conducted against the real-world multi-view datasets demonstrate the superiority over state-of-the-arts.
Tasks
Published	2017-09-05
URL	http://arxiv.org/abs/1709.01212v3
PDF	http://arxiv.org/pdf/1709.01212v3.pdf
PWC	https://paperswithcode.com/paper/multi-view-spectral-clustering-via-structured
Repo
Framework

Machine Learning Approach to RF Transmitter Identification


Title	Machine Learning Approach to RF Transmitter Identification
Authors	K. Youssef, Louis-S. Bouchard, K. Z. Haigh, H. Krovi, J. Silovsky, C. P. Vander Valk
Abstract	With the development and widespread use of wireless devices in recent years (mobile phones, Internet of Things, Wi-Fi), the electromagnetic spectrum has become extremely crowded. In order to counter security threats posed by rogue or unknown transmitters, it is important to identify RF transmitters not by the data content of the transmissions but based on the intrinsic physical characteristics of the transmitters. RF waveforms represent a particular challenge because of the extremely high data rates involved and the potentially large number of transmitters present in a given location. These factors outline the need for rapid fingerprinting and identification methods that go beyond the traditional hand-engineered approaches. In this study, we investigate the use of machine learning (ML) strategies to the classification and identification problems, and the use of wavelets to reduce the amount of data required. Four different ML strategies are evaluated: deep neural nets (DNN), convolutional neural nets (CNN), support vector machines (SVM), and multi-stage training (MST) using accelerated Levenberg-Marquardt (A-LM) updates. The A-LM MST method preconditioned by wavelets was by far the most accurate, achieving 100% classification accuracy of transmitters, as tested using data originating from 12 different transmitters. We discuss strategies for extension of MST to a much larger number of transmitters.
Tasks
Published	2017-11-05
URL	http://arxiv.org/abs/1711.01559v2
PDF	http://arxiv.org/pdf/1711.01559v2.pdf
PWC	https://paperswithcode.com/paper/machine-learning-approach-to-rf-transmitter
Repo
Framework

Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks


Title	Evolution in Groups: A deeper look at synaptic cluster driven evolution of deep neural networks
Authors	Mohammad Javad Shafiee, Elnaz Barshan, Alexander Wong
Abstract	A promising paradigm for achieving highly efficient deep neural networks is the idea of evolutionary deep intelligence, which mimics biological evolution processes to progressively synthesize more efficient networks. A crucial design factor in evolutionary deep intelligence is the genetic encoding scheme used to simulate heredity and determine the architectures of offspring networks. In this study, we take a deeper look at the notion of synaptic cluster-driven evolution of deep neural networks which guides the evolution process towards the formation of a highly sparse set of synaptic clusters in offspring networks. Utilizing a synaptic cluster-driven genetic encoding, the probabilistic encoding of synaptic traits considers not only individual synaptic properties but also inter-synaptic relationships within a deep neural network. This process results in highly sparse offspring networks which are particularly tailored for parallel computational devices such as GPUs and deep neural network accelerator chips. Comprehensive experimental results using four well-known deep neural network architectures (LeNet-5, AlexNet, ResNet-56, and DetectNet) on two different tasks (object categorization and object detection) demonstrate the efficiency of the proposed method. Cluster-driven genetic encoding scheme synthesizes networks that can achieve state-of-the-art performance with significantly smaller number of synapses than that of the original ancestor network. ($\sim$125-fold decrease in synapses for MNIST). Furthermore, the improved cluster efficiency in the generated offspring networks ($\sim$9.71-fold decrease in clusters for MNIST and a $\sim$8.16-fold decrease in clusters for KITTI) is particularly useful for accelerated performance on parallel computing hardware architectures such as those in GPUs and deep neural network accelerator chips.
Tasks	Object Detection
Published	2017-04-07
URL	http://arxiv.org/abs/1704.02081v1
PDF	http://arxiv.org/pdf/1704.02081v1.pdf
PWC	https://paperswithcode.com/paper/evolution-in-groups-a-deeper-look-at-synaptic
Repo
Framework

Recovery of Sparse and Low Rank Components of Matrices Using Iterative Method with Adaptive Thresholding


Title	Recovery of Sparse and Low Rank Components of Matrices Using Iterative Method with Adaptive Thresholding
Authors	Nematollah Zarmehi, Farokh Marvasti
Abstract	In this letter, we propose an algorithm for recovery of sparse and low rank components of matrices using an iterative method with adaptive thresholding. In each iteration, the low rank and sparse components are obtained using a thresholding operator. This algorithm is fast and can be implemented easily. We compare it with one of the most common fast methods in which the rank and sparsity are approximated by $\ell_1$ norm. We also apply it to some real applications where the noise is not so sparse. The simulation results show that it has a suitable performance with low run-time.
Tasks
Published	2017-03-09
URL	http://arxiv.org/abs/1703.03722v2
PDF	http://arxiv.org/pdf/1703.03722v2.pdf
PWC	https://paperswithcode.com/paper/recovery-of-sparse-and-low-rank-components-of
Repo
Framework

Optimizing Kernel Machines using Deep Learning


Title	Optimizing Kernel Machines using Deep Learning
Authors	Huan Song, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Andreas Spanias
Abstract	Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrom kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.
Tasks
Published	2017-11-15
URL	http://arxiv.org/abs/1711.05374v1
PDF	http://arxiv.org/pdf/1711.05374v1.pdf
PWC	https://paperswithcode.com/paper/optimizing-kernel-machines-using-deep
Repo
Framework

Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project


Title	Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Project
Authors	Zhimeng Zhang, Jianan Wu, Xuan Zhang, Chi Zhang
Abstract	Although many methods perform well in single camera tracking, multi-camera tracking remains a challenging problem with less attention. DukeMTMC is a large-scale, well-annotated multi-camera tracking benchmark which makes great progress in this field. This report is dedicated to briefly introduce our method on DukeMTMC and show that simple hierarchical clustering with well-trained person re-identification features can get good results on this dataset.
Tasks	Person Re-Identification
Published	2017-12-27
URL	http://arxiv.org/abs/1712.09531v1
PDF	http://arxiv.org/pdf/1712.09531v1.pdf
PWC	https://paperswithcode.com/paper/multi-target-multi-camera-tracking-by
Repo
Framework

Adversarial Generation of Real-time Feedback with Neural Networks for Simulation-based Training


Title	Adversarial Generation of Real-time Feedback with Neural Networks for Simulation-based Training
Authors	Xingjun Ma, Sudanthi Wijewickrema, Shuo Zhou, Yun Zhou, Zakaria Mhammedi, Stephen O’Leary, James Bailey
Abstract	Simulation-based training (SBT) is gaining popularity as a low-cost and convenient training technique in a vast range of applications. However, for a SBT platform to be fully utilized as an effective training tool, it is essential that feedback on performance is provided automatically in real-time during training. It is the aim of this paper to develop an efficient and effective feedback generation method for the provision of real-time feedback in SBT. Existing methods either have low effectiveness in improving novice skills or suffer from low efficiency, resulting in their inability to be used in real-time. In this paper, we propose a neural network based method to generate feedback using the adversarial technique. The proposed method utilizes a bounded adversarial update to minimize a L1 regularized loss via back-propagation. We empirically show that the proposed method can be used to generate simple, yet effective feedback. Also, it was observed to have high effectiveness and efficiency when compared to existing methods, thus making it a promising option for real-time feedback generation in SBT.
Tasks
Published	2017-03-04
URL	http://arxiv.org/abs/1703.01460v3
PDF	http://arxiv.org/pdf/1703.01460v3.pdf
PWC	https://paperswithcode.com/paper/adversarial-generation-of-real-time-feedback
Repo
Framework