January 27, 2020

3229 words 16 mins read

Paper Group ANR 1278

4D Seismic History Matching Incorporating Unsupervised Learning. Comparison-Based Indexing From First Principles. Automated classification of plasma regions using 3D particle energy distribution. Merlin: Enabling Machine Learning-Ready HPC Ensembles. Human Vocal Sentiment Analysis. Dual Grid Net: hand mesh vertex regression from single depth maps. …

4D Seismic History Matching Incorporating Unsupervised Learning


Title	4D Seismic History Matching Incorporating Unsupervised Learning
Authors	Clement Etienam
Abstract	The work discussed and presented in this paper focuses on the history matching of reservoirs by integrating 4D seismic data into the inversion process using machine learning techniques. A new integrated scheme for the reconstruction of petrophysical properties with a modified Ensemble Smoother with Multiple Data Assimilation (ES-MDA) in a synthetic reservoir is proposed. The permeability field inside the reservoir is parametrised with an unsupervised learning approach, namely K-means with Singular Value Decomposition (K-SVD). This is combined with the Orthogonal Matching Pursuit (OMP) technique which is very typical for sparsity promoting regularisation schemes. Moreover, seismic attributes, in particular, acoustic impedance, are parametrised with the Discrete Cosine Transform (DCT). This novel combination of techniques from machine learning, sparsity regularisation, seismic imaging and history matching aims to address the ill-posedness of the inversion of historical production data efficiently using ES-MDA. In the numerical experiments provided, I demonstrate that these sparse representations of the petrophysical properties and the seismic attributes enables to obtain better production data matches to the true production data and to quantify the propagating waterfront better compared to more traditional methods that do not use comparable parametrisation techniques.
Tasks
Published	2019-05-16
URL	https://arxiv.org/abs/1905.07469v1
PDF	https://arxiv.org/pdf/1905.07469v1.pdf
PWC	https://paperswithcode.com/paper/4d-seismic-history-matching-incorporating
Repo
Framework

Comparison-Based Indexing From First Principles


Title	Comparison-Based Indexing From First Principles
Authors	Magnus Lie Hetland
Abstract	Basic assumptions about comparison-based indexing are laid down and a general design space is derived from these. An index structure spanning this design space (the sprawl) is described, along with an associated family of partitioning predicates, or regions (the ambits), as well as algorithms for search and, to some extent, construction. The sprawl of ambits forms a unification and generalization of current indexing methods, and a jumping-off point for future designs.
Tasks
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06318v1
PDF	https://arxiv.org/pdf/1908.06318v1.pdf
PWC	https://paperswithcode.com/paper/comparison-based-indexing-from-first
Repo
Framework

Automated classification of plasma regions using 3D particle energy distribution


Title	Automated classification of plasma regions using 3D particle energy distribution
Authors	Vyacheslav Olshevsky, Yuri V. Khotyaintsev, Andrey Divin, Gian Luca Delzanno, Sven Anderzen, Pawel Herman, Steven W. D. Chien, Levon Avanov, Stefano Markidis
Abstract	Even though automatic classification and interpretation would be highly desired features for the Magnetospheric Multiscale mission (MMS), the gold rush era in machine learning has yet to reach the science done with observations collected by MMS. We investigate the properties of the ion sky maps produced by the Dual Ion Spectrometers (DIS) from the Fast Plasma Investigation (FPI). Running the Principal Component Analysis (PCA) on a mixed subset of the data suggests that more than 500 components are needed to cover 80% of the variance. Hence, simple machine learning techniques might not deal with classification of plasma regions. Use of a three-dimensional (3D) convolutional autoencoder (3D-CAE) allows to reduce the data dimensionality by 128 times while still maintaining a decent quality energy distribution. However, k-means clustering computed over the compressed data is not capable of separating measurements according to the physical properties of the plasma. A three-dimensional convolutional neural network (3D-CNN), trained on a rather small amount of human labelled training examples is able to predict plasma regions with 99% accuracy. The low probability predictions of the 3D-CNN reveal the mixed state regions, such as the magnetopause or bow shock, which are of key interest to researchers of the MMS mission. The 3D-CNN and data processing software could easily be deployed on ground-based computers and provide classification for the whole MMS database. Data processing through the trained 3D-CNN is fast and efficient, opening up the possibility for deployment in data-centers or in situ operation onboard the spacecraft.
Tasks
Published	2019-08-15
URL	https://arxiv.org/abs/1908.05715v2
PDF	https://arxiv.org/pdf/1908.05715v2.pdf
PWC	https://paperswithcode.com/paper/automated-classification-of-plasma-regions
Repo
Framework

Merlin: Enabling Machine Learning-Ready HPC Ensembles


Title	Merlin: Enabling Machine Learning-Ready HPC Ensembles
Authors	J. Luc Peterson, Rushil Anirudh, Kevin Athey, Benjamin Bay, Peer-Timo Bremer, Vic Castillo, Francesco Di Natale, David Fox, Jim A. Gaffney, David Hysom, Sam Ade Jacobs, Bhavya Kailkhura, Joe Koning, Bogdan Kustowski, Steven Langer, Peter Robinson, Jessica Semler, Brian Spears, Jayaraman Thiagarajan, Brian Van Essen, Jae-Seung Yeom
Abstract	With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. In this paper, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. In addition to its design and some examples, we describe how Merlin was deployed on the Sierra Supercomputer at Lawrence Livermore National Laboratory to create an unprecedented benchmark inertial confinement fusion dataset of approximately 100 million individual simulations and over 24 terabytes of multi-modal physics-based scalar, vector and hyperspectral image data.
Tasks
Published	2019-12-05
URL	https://arxiv.org/abs/1912.02892v1
PDF	https://arxiv.org/pdf/1912.02892v1.pdf
PWC	https://paperswithcode.com/paper/merlin-enabling-machine-learning-ready-hpc
Repo
Framework

Human Vocal Sentiment Analysis


Title	Human Vocal Sentiment Analysis
Authors	Andrew Huang, Puwei Bao
Abstract	In this paper, we use several techniques with conventional vocal feature extraction (MFCC, STFT), along with deep-learning approaches such as CNN, and also context-level analysis, by providing the textual data, and combining different approaches for improved emotion-level classification. We explore models that have not been tested to gauge the difference in performance and accuracy. We apply hyperparameter sweeps and data augmentation to improve performance. Finally, we see if a real-time approach is feasible, and can be readily integrated into existing systems.
Tasks	Data Augmentation, Sentiment Analysis
Published	2019-05-19
URL	https://arxiv.org/abs/1905.08632v1
PDF	https://arxiv.org/pdf/1905.08632v1.pdf
PWC	https://paperswithcode.com/paper/human-vocal-sentiment-analysis
Repo
Framework

Dual Grid Net: hand mesh vertex regression from single depth maps


Title	Dual Grid Net: hand mesh vertex regression from single depth maps
Authors	Chengde Wan, Thomas Probst, Luc Van Gool, Angela Yao
Abstract	We present a method for recovering the dense 3D surface of the hand by regressing the vertex coordinates of a mesh model from a single depth map. To this end, we use a two-stage 2D fully convolutional network architecture. In the first stage, the network estimates a dense correspondence field for every pixel on the depth map or image grid to the mesh grid. In the second stage, we design a differentiable operator to map features learned from the previous stage and regress a 3D coordinate map on the mesh grid. Finally, we sample from the mesh grid to recover the mesh vertices, and fit it an articulated template mesh in closed form. During inference, the network can predict all the mesh vertices, transformation matrices for every joint and the joint coordinates in a single forward pass. When given supervision on the sparse key-point coordinates, our method achieves state-of-the-art accuracy on NYU dataset for key point localization while recovering mesh vertices and a dense correspondence map. Our framework can also be learned through self-supervision by minimizing a set of data fitting and kinematic prior terms. With multi-camera rig during training to resolve self-occlusion, it can perform competitively with strongly supervised methods Without any human annotation.
Tasks
Published	2019-07-24
URL	https://arxiv.org/abs/1907.10695v1
PDF	https://arxiv.org/pdf/1907.10695v1.pdf
PWC	https://paperswithcode.com/paper/dual-grid-net-hand-mesh-vertex-regression
Repo
Framework

Applying Knowledge Transfer for Water Body Segmentation in Peru


Title	Applying Knowledge Transfer for Water Body Segmentation in Peru
Authors	Jessenia Gonzalez, Debjani Bhowmick, Cesar Beltran, Kris Sankaran, Yoshua Bengio
Abstract	In this work, we present the application of convolutional neural networks for segmenting water bodies in satellite images. We first use a variant of the U-Net model to segment rivers and lakes from very high-resolution images from Peru. To circumvent the issue of scarce labelled data, we investigate the applicability of a knowledge transfer-based model that learns the mapping from high-resolution labelled images and combines it with the very high-resolution mapping so that better segmentation can be achieved. We train this model in a single process, end-to-end. Our preliminary results show that adding the information from the available high-resolution images does not help out-of-the-box, and in fact worsen results. This leads us to infer that the high-resolution data could be from a different distribution, and its addition leads to increased variance in our results.
Tasks	Transfer Learning
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00957v1
PDF	https://arxiv.org/pdf/1912.00957v1.pdf
PWC	https://paperswithcode.com/paper/applying-knowledge-transfer-for-water-body
Repo
Framework

Data-driven preference learning methods for value-driven multiple criteria sorting with interacting criteria


Title	Data-driven preference learning methods for value-driven multiple criteria sorting with interacting criteria
Authors	Jiapeng Liu, Milosz Kadzinski, Xiuwu Liao, Xiaoxin Mao
Abstract	The learning of predictive models for data-driven decision support has been a prevalent topic in many fields. However, construction of models that would capture interactions among input variables is a challenging task. In this paper, we present a new preference learning approach for multiple criteria sorting with potentially interacting criteria. It employs an additive piecewise-linear value function as the basic preference model, which is augmented with components for handling the interactions. To construct such a model from a given set of assignment examples concerning reference alternatives, we develop a convex quadratic programming model. Since its complexity does not depend on the number of training samples, the proposed approach is capable for dealing with data-intensive tasks. To improve the generalization of the constructed model on new instances and to overcome the problem of over-fitting, we employ the regularization techniques. We also propose a few novel methods for classifying non-reference alternatives in order to enhance the applicability of our approach to different datasets. The practical usefulness of the proposed method is demonstrated on a problem of parametric evaluation of research units, whereas its predictive performance is studied on several monotone learning datasets. The experimental results indicate that our approach compares favourably with the classical UTADIS method and the Choquet integral-based sorting model.
Tasks
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08506v1
PDF	https://arxiv.org/pdf/1905.08506v1.pdf
PWC	https://paperswithcode.com/paper/data-driven-preference-learning-methods-for
Repo
Framework

An Interactive Insight Identification and Annotation Framework for Power Grid Pixel Maps using DenseU-Hierarchical VAE


Title	An Interactive Insight Identification and Annotation Framework for Power Grid Pixel Maps using DenseU-Hierarchical VAE
Authors	Tianye Zhang, Haozhe Feng, Zexian Chen, Can Wang, Yanhao Huang, Yong Tang, Wei Chen
Abstract	Insights in power grid pixel maps (PGPMs) refer to important facility operating states and unexpected changes in the power grid. Identifying insights helps analysts understand the collaboration of various parts of the grid so that preventive and correct operations can be taken to avoid potential accidents. Existing solutions for identifying insights in PGPMs are performed manually, which may be laborious and expertise-dependent. In this paper, we propose an interactive insight identification and annotation framework by leveraging an enhanced variational autoencoder (VAE). In particular, a new architecture, DenseU-Hierarchical VAE (DUHiV), is designed to learn representations from large-sized PGPMs, which achieves a significantly tighter evidence lower bound (ELBO) than existing Hierarchical VAEs with a Multilayer Perceptron architecture. Our approach supports modulating the derived representations in an interactive visual interface, discover potential insights and create multi-label annotations. Evaluations using real-world PGPMs datasets show that our framework outperforms the baseline models in identifying and annotating insights.
Tasks
Published	2019-05-22
URL	https://arxiv.org/abs/1905.12164v1
PDF	https://arxiv.org/pdf/1905.12164v1.pdf
PWC	https://paperswithcode.com/paper/190512164
Repo
Framework

Predicting Performance using Approximate State Space Model for Liquid State Machines


Title	Predicting Performance using Approximate State Space Model for Liquid State Machines
Authors	Ajinkya Gorad, Vivek Saraswat, Udayan Ganguly
Abstract	Liquid State Machine (LSM) is a brain-inspired architecture used for solving problems like speech recognition and time series prediction. LSM comprises of a randomly connected recurrent network of spiking neurons. This network propagates the non-linear neuronal and synaptic dynamics. Maass et al. have argued that the non-linear dynamics of LSMs is essential for its performance as a universal computer. Lyapunov exponent (mu), used to characterize the “non-linearity” of the network, correlates well with LSM performance. We propose a complementary approach of approximating the LSM dynamics with a linear state space representation. The spike rates from this model are well correlated to the spike rates from LSM. Such equivalence allows the extraction of a “memory” metric (tau_M) from the state transition matrix. tau_M displays high correlation with performance. Further, high tau_M system require lesser epochs to achieve a given accuracy. Being computationally cheap (1800x time efficient compared to LSM), the tau_M metric enables exploration of the vast parameter design space. We observe that the performance correlation of the tau_M surpasses the Lyapunov exponent (mu), (2-4x improvement) in the high-performance regime over multiple datasets. In fact, while mu increases monotonically with network activity, the performance reaches a maxima at a specific activity described in literature as the “edge of chaos”. On the other hand, tau_M remains correlated with LSM performance even as mu increases monotonically. Hence, tau_M captures the useful memory of network activity that enables LSM performance. It also enables rapid design space exploration and fine-tuning of LSM parameters for high performance.
Tasks	Speech Recognition, Time Series, Time Series Prediction
Published	2019-01-18
URL	http://arxiv.org/abs/1901.06240v1
PDF	http://arxiv.org/pdf/1901.06240v1.pdf
PWC	https://paperswithcode.com/paper/predicting-performance-using-approximate
Repo
Framework

Automatic Repair and Type Binding of Undeclared Variables using Neural Networks


Title	Automatic Repair and Type Binding of Undeclared Variables using Neural Networks
Authors	Venkatesh Theru Mohan, Ali Jannesari
Abstract	Deep learning had been used in program analysis for the prediction of hidden software defects using software defect datasets, security vulnerabilities using generative adversarial networks as well as identifying syntax errors by learning a trained neural machine translation on program codes. However, all these approaches either require defect datasets or bug-free source codes that are executable for training the deep learning model. Our neural network model is neither trained with any defect datasets nor bug-free programming source codes, instead it is trained using structural semantic details of Abstract Syntax Tree (AST) where each node represents a construct appearing in the source code. This model is implemented to fix one of the most common semantic errors, such as undeclared variable errors as well as infer their type information before program compilation. By this approach, the model has achieved in correctly locating and identifying 81% of the programs on prutor dataset of 1059 programs with only undeclared variable errors and also inferring their types correctly in 80% of the programs.
Tasks	Machine Translation
Published	2019-07-14
URL	https://arxiv.org/abs/1907.06205v1
PDF	https://arxiv.org/pdf/1907.06205v1.pdf
PWC	https://paperswithcode.com/paper/automatic-repair-and-type-binding-of
Repo
Framework

Parameter Constrained Transfer Learning for Low Dose PET Image Denoising


Title	Parameter Constrained Transfer Learning for Low Dose PET Image Denoising
Authors	Yu Gong, Yueyang Teng, Hongming Shan, Taohui Xiao, Ming Li, Guodong Liang, Ge Wang, Shanshan Wang
Abstract	Positron emission tomography (PET) is widely used in clinical practice. However, the potential risk of PET-associated radiation dose to patients needs to be minimized. With reduction of the radiation dose, the resultant images may suffer from noise and artifacts which compromises the diagnostic performance. In this paper, we propose a parameter-constrained generative adversarial network with Wasserstein distance and perceptual loss (PC-WGAN) for low-dose PET image denoising. This method makes two main contributions: 1) a PC-WGAN framework is designed to denoise low-dose PET images without compromising structural details; and 2) a transfer learning strategy is developed to train PC-WGAN with parameters being constrained, which has major merits; namely, making the training process of PC-WGAN efficient and improving the quality of denoised images. The experimental results on clinical data show that the proposed network can suppress image noise more effectively while preserving better image fidelity than three selected state-of-the-art methods.
Tasks	Denoising, Image Denoising, Transfer Learning
Published	2019-10-13
URL	https://arxiv.org/abs/1910.06749v1
PDF	https://arxiv.org/pdf/1910.06749v1.pdf
PWC	https://paperswithcode.com/paper/parameter-constrained-transfer-learning-for
Repo
Framework

Leveraging Trust and Distrust in Recommender Systems via Deep Learning


Title	Leveraging Trust and Distrust in Recommender Systems via Deep Learning
Authors	Dimitrios Rafailidis
Abstract	The data scarcity of user preferences and the cold-start problem often appear in real-world applications and limit the recommendation accuracy of collaborative filtering strategies. Leveraging the selections of social friends and foes can efficiently face both problems. In this study, we propose a strategy that performs social deep pairwise learning. Firstly, we design a ranking loss function incorporating multiple ranking criteria based on the choice in users, and the choice in their friends and foes to improve the accuracy in the top-k recommendation task. We capture the nonlinear correlations between user preferences and the social information of trust and distrust relationships via a deep learning strategy. In each backpropagation step, we follow a social negative sampling strategy to meet the multiple ranking criteria of our ranking loss function. We conduct comprehensive experiments on a benchmark dataset from Epinions, among the largest publicly available that has been reported in the relevant literature. The experimental results demonstrate that the proposed model beats other state-of-the art methods, attaining an 11.49% average improvement over the most competitive model. We show that our deep learning strategy plays an important role in capturing the nonlinear correlations between user preferences and the social information of trust and distrust relationships, and demonstrate the importance of our social negative sampling strategy on the proposed model.
Tasks	Recommendation Systems
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13612v1
PDF	https://arxiv.org/pdf/1905.13612v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-trust-and-distrust-in-recommender
Repo
Framework

Multi-Stage Prediction Networks for Data Harmonization


Title	Multi-Stage Prediction Networks for Data Harmonization
Authors	Stefano B. Blumberg, Marco Palombo, Can Son Khoo, Chantal M. W. Tax, Ryutaro Tanno, Daniel C. Alexander
Abstract	In this paper, we introduce multi-task learning (MTL) to data harmonization (DH); where we aim to harmonize images across different acquisition platforms and sites. This allows us to integrate information from multiple acquisitions and improve the predictive performance and learning efficiency of the harmonization model. Specifically, we introduce the Multi Stage Prediction (MSP) Network, a MTL framework that incorporates neural networks of potentially disparate architectures, trained for different individual acquisition platforms, into a larger architecture that is refined in unison. The MSP utilizes high-level features of single networks for individual tasks, as inputs of additional neural networks to inform the final prediction, therefore exploiting redundancy across tasks to make the most of limited training data. We validate our methods on a dMRI harmonization challenge dataset, where we predict three modern platform types, from one obtained from an old scanner. We show how MTL architectures, such as the MSP, produce around 20% improvement of patch-based mean-squared error over current state-of-the-art methods and that our MSP outperforms off-the-shelf MTL networks. Our code is available https://github.com/sbb-gh/ .
Tasks	Multi-Task Learning
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11629v1
PDF	https://arxiv.org/pdf/1907.11629v1.pdf
PWC	https://paperswithcode.com/paper/multi-stage-prediction-networks-for-data
Repo
Framework

Lifelong Learning Starting From Zero


Title	Lifelong Learning Starting From Zero
Authors	Claes Strannegård, Herman Carlström, Niklas Engsner, Fredrik Mäkeläinen, Filip Slottner Seholm, Morteza Haghir Chehreghani
Abstract	We present a deep neural-network model for lifelong learning inspired by several forms of neuroplasticity. The neural network develops continuously in response to signals from the environment. In the beginning, the network is a blank slate with no nodes at all. It develops according to four rules: (i) expansion, which adds new nodes to memorize new input combinations; (ii) generalization, which adds new nodes that generalize from existing ones; (iii) forgetting, which removes nodes that are of relatively little use; and (iv) backpropagation, which fine-tunes the network parameters. We analyze the model from the perspective of accuracy, energy efficiency, and versatility and compare it to other network models, finding better performance in several cases.
Tasks
Published	2019-06-24
URL	https://arxiv.org/abs/1906.09852v1
PDF	https://arxiv.org/pdf/1906.09852v1.pdf
PWC	https://paperswithcode.com/paper/lifelong-learning-starting-from-zero
Repo
Framework