January 31, 2020

3249 words 16 mins read

Paper Group ANR 195

Malware Detection using Machine Learning and Deep Learning. Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning. Adaptive Deep Kernel Learning. Towards Interpretable Sparse Graph Representation Learning with Laplacian Pooling. Leveraging binding-site structure for drug discovery w …

Malware Detection using Machine Learning and Deep Learning


Title	Malware Detection using Machine Learning and Deep Learning
Authors	Hemant Rathore, Swati Agarwal, Sanjay K. Sahay, Mohit Sewak
Abstract	Research shows that over the last decade, malware has been growing exponentially, causing substantial financial losses to various organizations. Different anti-malware companies have been proposing solutions to defend attacks from these malware. The velocity, volume, and the complexity of malware are posing new challenges to the anti-malware community. Current state-of-the-art research shows that recently, researchers and anti-virus organizations started applying machine learning and deep learning methods for malware analysis and detection. We have used opcode frequency as a feature vector and applied unsupervised learning in addition to supervised learning for malware classification. The focus of this tutorial is to present our work on detecting malware with 1) various machine learning algorithms and 2) deep learning models. Our results show that the Random Forest outperforms Deep Neural Network with opcode frequency as a feature. Also in feature reduction, Deep Auto-Encoders are overkill for the dataset, and elementary function like Variance Threshold perform better than others. In addition to the proposed methodologies, we will also discuss the additional issues and the unique challenges in the domain, open research problems, limitations, and future directions.
Tasks	Malware Classification, Malware Detection
Published	2019-04-04
URL	http://arxiv.org/abs/1904.02441v1
PDF	http://arxiv.org/pdf/1904.02441v1.pdf
PWC	https://paperswithcode.com/paper/malware-detection-using-machine-learning-and
Repo
Framework

Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning


Title	Reconstructing Perceived Images from Brain Activity by Visually-guided Cognitive Representation and Adversarial Learning
Authors	Ziqi Ren, Jie Li, Xuetong Xue, Xin Li, Fan Yang, Zhicheng Jiao, Xinbo Gao
Abstract	Reconstructing visual stimulus (image) only from human brain activity measured with functional Magnetic Resonance Imaging (fMRI) is a significant and meaningful task in Human-AI collaboration. However, the inconsistent distribution and representation between fMRI signals and visual images cause the heterogeneity gap. Moreover, the fMRI data is often extremely high-dimensional and contains a lot of visually-irrelevant information. Existing methods generally suffer from these issues so that a satisfactory reconstruction is still challenging. In this paper, we show that it is possible to overcome these challenges by learning visually-guided cognitive latent representations from the fMRI signals, and inversely decoding them to the image stimuli. The resulting framework is called Dual-Variational Autoencoder/ Generative Adversarial Network (D-VAE/GAN), which combines the advantages of adversarial representation learning with knowledge distillation. In addition, we introduce a novel three-stage learning approach which enables the (cognitive) encoder to gradually distill useful knowledge from the paired (visual) encoder during the learning process. Extensive experimental results on both artificial and natural images have demonstrated that our method could achieve surprisingly good results and outperform all other alternatives.
Tasks	Image Reconstruction, Representation Learning
Published	2019-06-27
URL	https://arxiv.org/abs/1906.12181v2
PDF	https://arxiv.org/pdf/1906.12181v2.pdf
PWC	https://paperswithcode.com/paper/reconstructing-perceived-images-from-brain
Repo
Framework

Adaptive Deep Kernel Learning


Title	Adaptive Deep Kernel Learning
Authors	Prudencio Tossou, Basile Dura, Francois Laviolette, Mario Marchand, Alexandre Lacoste
Abstract	Deep kernel learning provides an elegant and principled framework for combining the structural properties of deep learning algorithms with the flexibility of kernel methods. By means of a deep neural network, it consists of learning a kernel operator which is combined with a differentiable kernel algorithm for inference. While previous work within this framework has mostly explored learning a single kernel for large datasets, we focus herein on learning a kernel family for a variety of tasks in few-shot regression settings. Compared to single deep kernel learning, our novel algorithm permits finding the appropriate kernel for each task during inference, rather than using the same for all tasks. As such, our algorithm performs more effectively with complex task distributions in few-shot learning, which we demonstrate by benchmarking against existing state-of-the-art algorithms using real-world, few-shot regression tasks related to drug discovery.
Tasks	Drug Discovery, Few-Shot Learning, few-shot regression
Published	2019-05-28
URL	https://arxiv.org/abs/1905.12131v1
PDF	https://arxiv.org/pdf/1905.12131v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-deep-kernel-learning
Repo
Framework

Towards Interpretable Sparse Graph Representation Learning with Laplacian Pooling


Title	Towards Interpretable Sparse Graph Representation Learning with Laplacian Pooling
Authors	Emmanuel Noutahi, Dominique Beani, Julien Horwood, Sébastien Giguère, Prudencio Tossou
Abstract	Recent work in graph neural networks (GNNs) has led to improvements in molecular activity and property prediction tasks. Unfortunately, GNNs often fail to capture the relative importance of interactions between molecular substructures, in part due to the absence of efficient intermediate pooling steps. To address these issues, we propose LaPool (Laplacian Pooling), a novel, data-driven, and interpretable hierarchical graph pooling method that takes into account both node features and graph structure to improve molecular representation. We benchmark LaPool on molecular graph prediction and understanding tasks and show that it outperforms recent GNNs. Interestingly, LaPool also remains competitive on non-molecular tasks. Both quantitative and qualitative assessments are done to demonstrate LaPool’s improved interpretability and highlight its potential benefits in drug design. Finally, we demonstrate LaPool’s utility for the generation of valid and novel molecules by incorporating it into an adversarial autoencoder.
Tasks	Drug Discovery, Graph Representation Learning, Representation Learning
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11577v3
PDF	https://arxiv.org/pdf/1905.11577v3.pdf
PWC	https://paperswithcode.com/paper/towards-interpretable-sparse-graph
Repo
Framework

Leveraging binding-site structure for drug discovery with point-cloud methods


Title	Leveraging binding-site structure for drug discovery with point-cloud methods
Authors	Vincent Mallet, Carlos G. Oliver, Nicolas Moitessier, Jerome Waldispuhl
Abstract	Computational drug discovery strategies can be broadly placed in two categories: ligand-based methods which identify novel molecules by similarity with known ligands, and structure-based methods which predict molecules with high-affinity to a given 3D structure (e.g. a protein). However, ligand-based methods do not leverage information about the binding site, and structure-based approaches rely on the knowledge of a finite set of ligands binding the target. In this work, we introduce TarLig, a novel approach that aims to bridge the gap between ligand and structure-based approaches. We use the 3D structure of the binding site as input to a model which predicts the ligand preferences of the binding site. The resulting predictions could then offer promising seeds and constraints in the chemical space search, based on the binding site structure. TarLig outperforms standard models by introducing a data-alignment and augmentation technique. The recent popularity of Volumetric 3DCNN pipelines in structural bioinformatics suggests that this extra step could help a wide range of methods to improve their results with minimal modifications.
Tasks	Drug Discovery
Published	2019-05-28
URL	https://arxiv.org/abs/1905.12033v1
PDF	https://arxiv.org/pdf/1905.12033v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-binding-site-structure-for-drug
Repo
Framework

Joint Interaction and Trajectory Prediction for Autonomous Driving using Graph Neural Networks


Title	Joint Interaction and Trajectory Prediction for Autonomous Driving using Graph Neural Networks
Authors	Donsuk Lee, Yiming Gu, Jerrick Hoang, Micol Marchetti-Bowick
Abstract	In this work, we aim to predict the future motion of vehicles in a traffic scene by explicitly modeling their pairwise interactions. Specifically, we propose a graph neural network that jointly predicts the discrete interaction modes and 5-second future trajectories for all agents in the scene. Our model infers an interaction graph whose nodes are agents and whose edges capture the long-term interaction intents among the agents. In order to train the model to recognize known modes of interaction, we introduce an auto-labeling function to generate ground truth interaction labels. Using a large-scale real-world driving dataset, we demonstrate that jointly predicting the trajectories along with the explicit interaction types leads to significantly lower trajectory error than baseline methods. Finally, we show through simulation studies that the learned interaction modes are semantically meaningful.
Tasks	Autonomous Driving, Trajectory Prediction
Published	2019-12-17
URL	https://arxiv.org/abs/1912.07882v1
PDF	https://arxiv.org/pdf/1912.07882v1.pdf
PWC	https://paperswithcode.com/paper/joint-interaction-and-trajectory-prediction
Repo
Framework

LORAKI: Autocalibrated Recurrent Neural Networks for Autoregressive MRI Reconstruction in k-Space


Title	LORAKI: Autocalibrated Recurrent Neural Networks for Autoregressive MRI Reconstruction in k-Space
Authors	Tae Hyung Kim, Pratyush Garg, Justin P. Haldar
Abstract	We propose and evaluate a new MRI reconstruction method named LORAKI that trains an autocalibrated scan-specific recurrent neural network (RNN) to recover missing k-space data. Methods like GRAPPA, SPIRiT, and AC-LORAKS assume that k-space data has shift-invariant autoregressive structure, and that the scan-specific autoregression relationships needed to recover missing samples can be learned from fully-sampled autocalibration (ACS) data. Recently, the structure of the linear GRAPPA method has been translated into a nonlinear deep learning method named RAKI. RAKI uses ACS data to train an artificial neural network to interpolate missing k-space samples, and often outperforms GRAPPA. In this work, we apply a similar principle to translate the linear AC-LORAKS method (simultaneously incorporating support, phase, and parallel imaging constraints) into a nonlinear deep learning method named LORAKI. Since AC-LORAKS is iterative and convolutional, LORAKI takes the form of a convolutional RNN. This new architecture admits a wide range of sampling patterns, and even calibrationless patterns are possible if synthetic ACS data is generated. The performance of LORAKI was evaluated with retrospectively undersampled brain datasets, with comparisons against other related reconstruction methods. Results suggest that LORAKI can provide improved reconstruction compared to other scan-specific autocalibrated reconstruction methods like GRAPPA, RAKI, and AC-LORAKS. LORAKI offers a new deep-learning approach to MRI reconstruction based on RNNs in k-space, and enables improved image quality and enhanced sampling flexibility.
Tasks
Published	2019-04-20
URL	http://arxiv.org/abs/1904.09390v2
PDF	http://arxiv.org/pdf/1904.09390v2.pdf
PWC	https://paperswithcode.com/paper/190409390
Repo
Framework

Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset


Title	Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset
Authors	Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Daniel Duckworth, Semih Yavuz, Ben Goodrich, Amit Dubey, Andy Cedilnik, Kyu-Young Kim
Abstract	A significant barrier to progress in data-driven approaches to building dialog systems is the lack of high quality, goal-oriented conversational data. To help satisfy this elementary requirement, we introduce the initial release of the Taskmaster-1 dataset which includes 13,215 task-based dialogs comprising six domains. Two procedures were used to create this collection, each with unique advantages. The first involves a two-person, spoken “Wizard of Oz” (WOz) approach in which trained agents and crowdsourced workers interact to complete the task while the second is “self-dialog” in which crowdsourced workers write the entire dialog themselves. We do not restrict the workers to detailed scripts or to a small knowledge base and hence we observe that our dataset contains more realistic and diverse conversations in comparison to existing datasets. We offer several baseline models including state of the art neural seq2seq architectures with benchmark performance as well as qualitative human evaluations. Dialogs are labeled with API calls and arguments, a simple and cost effective approach which avoids the requirement of complex annotation schema. The layer of abstraction between the dialog model and the service provider API allows for a given model to interact with multiple services that provide similar functionally. Finally, the dataset will evoke interest in written vs. spoken language, discourse patterns, error handling and other linguistic phenomena related to dialog system research, development and design.
Tasks
Published	2019-09-01
URL	https://arxiv.org/abs/1909.05358v1
PDF	https://arxiv.org/pdf/1909.05358v1.pdf
PWC	https://paperswithcode.com/paper/taskmaster-1-toward-a-realistic-and-diverse
Repo
Framework

Project Thyia: A Forever Gameplayer


Title	Project Thyia: A Forever Gameplayer
Authors	Raluca D. Gaina, Simon M. Lucas, Diego Perez-Liebana
Abstract	The space of Artificial Intelligence entities is dominated by conversational bots. Some of them fit in our pockets and we take them everywhere we go, or allow them to be a part of human homes. Siri, Alexa, they are recognised as present in our world. But a lot of games research is restricted to existing in the separate realm of software. We enter different worlds when playing games, but those worlds cease to exist once we quit. Similarly, AI game-players are run once on a game (or maybe for longer periods of time, in the case of learning algorithms which need some, still limited, period for training), and they cease to exist once the game ends. But what if they didn’t? What if there existed artificial game-players that continuously played games, learned from their experiences and kept getting better? What if they interacted with the real world and us, humans: live-streaming games, chatting with viewers, accepting suggestions for strategies or games to play, forming opinions on popular game titles? In this paper, we introduce the vision behind a new project called Thyia, which focuses around creating a present, continuous, `always-on’, interactive game-player. \|
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04023v1
PDF	https://arxiv.org/pdf/1906.04023v1.pdf
PWC	https://paperswithcode.com/paper/project-thyia-a-forever-gameplayer
Repo
Framework

Biphasic Learning of GANs for High-Resolution Image-to-Image Translation


Title	Biphasic Learning of GANs for High-Resolution Image-to-Image Translation
Authors	Jie Cao, Huaibo Huang, Yi Li, Jingtuo Liu, Ran He, Zhenan Sun
Abstract	Despite that the performance of image-to-image translation has been significantly improved by recent progress in generative models, current methods still suffer from severe degradation in training stability and sample quality when applied to the high-resolution situation. In this work, we present a novel training framework for GANs, namely biphasic learning, to achieve image-to-image translation in multiple visual domains at $1024^2$ resolution. Our core idea is to design an adjustable objective function that varies across training phases. Within the biphasic learning framework, we propose a novel inherited adversarial loss to achieve the enhancement of model capacity and stabilize the training phase transition. Furthermore, we introduce a perceptual-level consistency loss through mutual information estimation and maximization. To verify the superiority of the proposed method, we apply it to a wide range of face-related synthesis tasks and conduct experiments on multiple large-scale datasets. Through comprehensive quantitative analyses, we demonstrate that our method significantly outperforms existing methods.
Tasks	Image-to-Image Translation
Published	2019-04-14
URL	http://arxiv.org/abs/1904.06624v1
PDF	http://arxiv.org/pdf/1904.06624v1.pdf
PWC	https://paperswithcode.com/paper/biphasic-learning-of-gans-for-high-resolution
Repo
Framework

Quantifying Long Range Dependence in Language and User Behavior to improve RNNs


Title	Quantifying Long Range Dependence in Language and User Behavior to improve RNNs
Authors	Francois Belletti, Minmin Chen, Ed H. Chi
Abstract	Characterizing temporal dependence patterns is a critical step in understanding the statistical properties of sequential data. Long Range Dependence (LRD) — referring to long-range correlations decaying as a power law rather than exponentially w.r.t. distance — demands a different set of tools for modeling the underlying dynamics of the sequential data. While it has been widely conjectured that LRD is present in language modeling and sequential recommendation, the amount of LRD in the corresponding sequential datasets has not yet been quantified in a scalable and model-independent manner. We propose a principled estimation procedure of LRD in sequential datasets based on established LRD theory for real-valued time series and apply it to sequences of symbols with million-item-scale dictionaries. In our measurements, the procedure estimates reliably the LRD in the behavior of users as they write Wikipedia articles and as they interact with YouTube. We further show that measuring LRD better informs modeling decisions in particular for RNNs whose ability to capture LRD is still an active area of research. The quantitative measure informs new Evolutive Recurrent Neural Networks (EvolutiveRNNs) designs, leading to state-of-the-art results on language understanding and sequential recommendation tasks at a fraction of the computational cost.
Tasks	Language Modelling, Time Series
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09414v1
PDF	https://arxiv.org/pdf/1905.09414v1.pdf
PWC	https://paperswithcode.com/paper/quantifying-long-range-dependence-in-language
Repo
Framework

Likelihood-Free Inference and Generation of Molecular Graphs


Title	Likelihood-Free Inference and Generation of Molecular Graphs
Authors	Sebastian Pölsterl, Christian Wachinger
Abstract	Recent methods for generating novel molecules use graph representations of molecules and employ various forms of graph convolutional neural networks for inference. However, training requires solving an expensive graph isomorphism problem, which previous approaches do not address or solve only approximately. In this work, we propose LF-MolGAN, a likelihood-free approach for de novo molecule generation that avoids explicitly computing a reconstruction loss. Our approach extends generative adversarial networks by including an adversarial cycle-consistency loss to implicitly enforce the reconstruction property. To capture properties unique to molecules, such as valence, we extend Graph Isomorphism Network to multi-graphs. To quantify the performance of models, we propose to compute the distance between distributions of physicochemical properties with the 1-Wasserstein distance. We demonstrate that LF-MolGAN more accurately learns the distribution over the space of molecules than all baselines. Moreover, it can be utilized for drug discovery by efficiently searching the space of molecules using molecules’ continuous latent representation.
Tasks	Drug Discovery
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10310v1
PDF	https://arxiv.org/pdf/1905.10310v1.pdf
PWC	https://paperswithcode.com/paper/likelihood-free-inference-and-generation-of
Repo
Framework

An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity


Title	An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity
Authors	Lijun Ding, Alp Yurtsever, Volkan Cevher, Joel A. Tropp, Madeleine Udell
Abstract	This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form. This method is particularly effective for weakly constrained SDPs. The key idea is to formulate an approximate complementarity principle: Given an approximate solution to the dual SDP, the primal SDP has an approximate solution whose range is contained in the eigenspace with small eigenvalues of the dual slack matrix. For weakly constrained SDPs, this eigenspace has very low dimension, so this observation significantly reduces the search space for the primal solution. This result suggests an algorithmic strategy that can be implemented with minimal storage: (1) Solve the dual SDP approximately; (2) compress the primal SDP to the eigenspace with small eigenvalues of the dual slack matrix; (3) solve the compressed primal SDP. The paper also provides numerical experiments showing that this approach is successful for a range of interesting large-scale SDPs.
Tasks
Published	2019-02-09
URL	http://arxiv.org/abs/1902.03373v1
PDF	http://arxiv.org/pdf/1902.03373v1.pdf
PWC	https://paperswithcode.com/paper/an-optimal-storage-approach-to-semidefinite
Repo
Framework

Hashing based Answer Selection


Title	Hashing based Answer Selection
Authors	Dong Xu, Wu-Jun Li
Abstract	Answer selection is an important subtask of question answering (QA), where deep models usually achieve better performance. Most deep models adopt question-answer interaction mechanisms, such as attention, to get vector representations for answers. When these interaction based deep models are deployed for online prediction, the representations of all answers need to be recalculated for each question. This procedure is time-consuming for deep models with complex encoders like BERT which usually have better accuracy than simple encoders. One possible solution is to store the matrix representation (encoder output) of each answer in memory to avoid recalculation. But this will bring large memory cost. In this paper, we propose a novel method, called hashing based answer selection (HAS), to tackle this problem. HAS adopts a hashing strategy to learn a binary matrix representation for each answer, which can dramatically reduce the memory cost for storing the matrix representations of answers. Hence, HAS can adopt complex encoders like BERT in the model, but the online prediction of HAS is still fast with a low memory cost. Experimental results on three popular answer selection datasets show that HAS can outperform existing models to achieve state-of-the-art performance.
Tasks	Answer Selection, Question Answering
Published	2019-05-26
URL	https://arxiv.org/abs/1905.10718v1
PDF	https://arxiv.org/pdf/1905.10718v1.pdf
PWC	https://paperswithcode.com/paper/hashing-based-answer-selection
Repo
Framework

Adaptive Task Allocation for Asynchronous Federated Mobile Edge Learning


Title	Adaptive Task Allocation for Asynchronous Federated Mobile Edge Learning
Authors	Umair Mohammad, Sameh Sorour
Abstract	This paper proposes a scheme to efficiently execute distributed learning tasks in an asynchronous manner while minimizing the gradient staleness on wireless edge nodes with heterogeneous computing and communication capacities. The designed approach considered in this paper ensures that all devices work for a certain duration that covers the time for data/model distribution, learning iterations, model collection and global aggregation. The resulting problem is an integer non-convex program with quadratic equality constraints as well as linear equality and inequality constraints. Because the problem is NP-hard, we relax the integer constraints in order to solve it efficiently with available solvers. Analytical bounds are derived using the KKT conditions and Lagrangian analysis in conjunction with the suggest-and-improve approach. Results show that our approach reduces the gradient staleness and can offer better accuracy than the synchronous scheme and the asynchronous scheme with equal task allocation.
Tasks
Published	2019-05-05
URL	https://arxiv.org/abs/1905.01656v1
PDF	https://arxiv.org/pdf/1905.01656v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-task-allocation-for-asynchronous
Repo
Framework