Paper Group ANR 474
Deep Markov Spatio-Temporal Factorization. Machine learning on DNA-encoded libraries: A new paradigm for hit-finding. CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting. Scaling MAP-Elites to Deep Neuroevolution. Convolutional Neural Network Pruning Using Filter Attenuation. Multi …
Deep Markov Spatio-Temporal Factorization
| Title | Deep Markov Spatio-Temporal Factorization |
| Authors | Amirreza Farnoosh, Behnaz Rezaei, Eli Zachary Sennesh, Zulqarnain Khan, Jennifer Dy, Ajay Satpute, J Benjamin Hutchinson, Jan-Willem van de Meent, Sarah Ostadabbas |
| Abstract | We introduce deep Markov spatio-temporal factorization (DMSTF), a deep generative model for spatio-temporal data. Like other factor analysis methods, DMSTF approximates high-dimensional data by a product between time-dependent weights and spatially dependent factors. These weights and factors are in turn represented in terms of lower-dimensional latent variables that we infer using stochastic variational inference. The innovation in DMSTF is that we parameterize weights in terms of a deep Markovian prior, which is able to characterize nonlinear temporal dynamics. We parameterize the corresponding variational distribution using a bidirectional recurrent network. This results in a flexible family of hierarchical deep generative factor analysis models that can be extended to perform time series clustering, or perform factor analysis in the presence of a control signal. Our experiments, which consider simulated data, fMRI data, and traffic data, demonstrate that DMSTF outperforms related methods in terms of reconstruction accuracy and can perform forecasting in a variety domains with nonlinear temporal transitions. |
| Tasks | Time Series, Time Series Clustering |
| Published | 2020-03-22 |
| URL | https://arxiv.org/abs/2003.09779v1 |
| https://arxiv.org/pdf/2003.09779v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-markov-spatio-temporal-factorization |
| Repo | |
| Framework | |
Machine learning on DNA-encoded libraries: A new paradigm for hit-finding
| Title | Machine learning on DNA-encoded libraries: A new paradigm for hit-finding |
| Authors | Kevin McCloskey, Eric A. Sigel, Steven Kearnes, Ling Xue, Xia Tian, Dennis Moccia, Diana Gikunju, Sana Bazzaz, Betty Chan, Matthew A. Clark, John W. Cuozzo, Marie-Aude Guié, John P. Guilinger, Christelle Huguet, Christopher D. Hupp, Anthony D. Keefe, Christopher J. Mulhern, Ying Zhang, Patrick Riley |
| Abstract | DNA-encoded small molecule libraries (DELs) have enabled discovery of novel inhibitors for many distinct protein targets of therapeutic value through screening of libraries with up to billions of unique small molecules. We demonstrate a new approach applying machine learning to DEL selection data by identifying active molecules from a large commercial collection and a virtual library of easily synthesizable compounds. We train models using only DEL selection data and apply automated or automatable filters with chemist review restricted to the removal of molecules with potential for instability or reactivity. We validate this approach with a large prospective study (nearly 2000 compounds tested) across three diverse protein targets: sEH (a hydrolase), ER{\alpha} (a nuclear receptor), and c-KIT (a kinase). The approach is effective, with an overall hit rate of {\sim}30% at 30 {\textmu}M and discovery of potent compounds (IC50 <10 nM) for every target. The model makes useful predictions even for molecules dissimilar to the original DEL and the compounds identified are diverse, predominantly drug-like, and different from known ligands. Collectively, the quality and quantity of DEL selection data; the power of modern machine learning methods; and access to large, inexpensive, commercially-available libraries creates a powerful new approach for hit finding. |
| Tasks | |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2002.02530v1 |
| https://arxiv.org/pdf/2002.02530v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-on-dna-encoded-libraries-a |
| Repo | |
| Framework | |
CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting
| Title | CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting |
| Authors | Pranav Rajpurkar, Anirudh Joshi, Anuj Pareek, Phil Chen, Amirhossein Kiani, Jeremy Irvin, Andrew Y. Ng, Matthew P. Lungren |
| Abstract | Although there have been several recent advances in the application of deep learning algorithms to chest x-ray interpretation, we identify three major challenges for the translation of chest x-ray algorithms to the clinical setting. We examine the performance of the top 10 performing models on the CheXpert challenge leaderboard on three tasks: (1) TB detection, (2) pathology detection on photos of chest x-rays, and (3) pathology detection on data from an external institution. First, we find that the top 10 chest x-ray models on the CheXpert competition achieve an average AUC of 0.851 on the task of detecting TB on two public TB datasets without fine-tuning or including the TB labels in training data. Second, we find that the average performance of the models on photos of x-rays (AUC = 0.916) is similar to their performance on the original chest x-ray images (AUC = 0.924). Third, we find that the models tested on an external dataset either perform comparably to or exceed the average performance of radiologists. We believe that our investigation will inform rapid translation of deep learning algorithms to safe and effective clinical decision support tools that can be validated prospectively with large impact studies and clinical trials. |
| Tasks | |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11379v2 |
| https://arxiv.org/pdf/2002.11379v2.pdf | |
| PWC | https://paperswithcode.com/paper/chexpedition-investigating-generalization |
| Repo | |
| Framework | |
Scaling MAP-Elites to Deep Neuroevolution
| Title | Scaling MAP-Elites to Deep Neuroevolution |
| Authors | Cédric Colas, Joost Huizinga, Vashisht Madhavan, Jeff Clune |
| Abstract | Quality-Diversity (QD) algorithms, and MAP-Elites (ME) in particular, have proven very useful for a broad range of applications including enabling real robots to recover quickly from joint damage, solving strongly deceptive maze tasks or evolving robot morphologies to discover new gaits. However, present implementations of MAP-Elites and other QD algorithms seem to be limited to low-dimensional controllers with far fewer parameters than modern deep neural network models. In this paper, we propose to leverage the efficiency of Evolution Strategies (ES) to scale MAP-Elites to high-dimensional controllers parameterized by large neural networks. We design and evaluate a new hybrid algorithm called MAP-Elites with Evolution Strategies (ME-ES) for post-damage recovery in a difficult high-dimensional control task where traditional ME fails. Additionally,we show that ME-ES performs efficient exploration, on par with state-of-the-art exploration algorithms in high-dimensional control tasks with strongly deceptive rewards. |
| Tasks | Efficient Exploration |
| Published | 2020-03-03 |
| URL | https://arxiv.org/abs/2003.01825v1 |
| https://arxiv.org/pdf/2003.01825v1.pdf | |
| PWC | https://paperswithcode.com/paper/scaling-map-elites-to-deep-neuroevolution |
| Repo | |
| Framework | |
Convolutional Neural Network Pruning Using Filter Attenuation
| Title | Convolutional Neural Network Pruning Using Filter Attenuation |
| Authors | Morteza Mousa-Pasandi, Mohsen Hajabdollahi, Nader Karimi, Shadrokh Samavi, Shahram Shirani |
| Abstract | Filters are the essential elements in convolutional neural networks (CNNs). Filters are corresponded to the feature maps and form the main part of the computational and memory requirement for the CNN processing. In filter pruning methods, a filter with all of its components, including channels and connections, are removed. The removal of a filter can cause a drastic change in the network’s performance. Also, the removed filters cannot come back to the network structure. We want to address these problems in this paper. We propose a CNN pruning method based on filter attenuation in which weak filters are not directly removed. Instead, weak filters are attenuated and gradually removed. In the proposed attenuation approach, weak filters are not abruptly removed, and there is a chance for these filters to return to the network. The filter attenuation method is assessed using the VGG model for the Cifar10 image classification task. Simulation results show that the filter attenuation works with different pruning criteria, and better results are obtained in comparison with the conventional pruning methods. |
| Tasks | Image Classification, Network Pruning |
| Published | 2020-02-09 |
| URL | https://arxiv.org/abs/2002.03299v1 |
| https://arxiv.org/pdf/2002.03299v1.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-neural-network-pruning-using |
| Repo | |
| Framework | |
Multi-Robot Formation Control Using Reinforcement Learning
| Title | Multi-Robot Formation Control Using Reinforcement Learning |
| Authors | Abhay Rawat, Kamalakar Karlapalem |
| Abstract | In this paper, we present a machine learning approach to move a group of robots in a formation. We model the problem as a multi-agent reinforcement learning problem. Our aim is to design a control policy for maintaining a desired formation among a number of agents (robots) while moving towards a desired goal. This is achieved by training our agents to track two agents of the group and maintain the formation with respect to those agents. We consider all agents to be homogeneous and model them as unicycle [1]. In contrast to the leader-follower approach, where each agent has an independent goal, our approach aims to train the agents to be cooperative and work towards the common goal. Our motivation to use this method is to make a fully decentralized multi-agent formation system and scalable for a number of agents. |
| Tasks | Multi-agent Reinforcement Learning |
| Published | 2020-01-13 |
| URL | https://arxiv.org/abs/2001.04527v1 |
| https://arxiv.org/pdf/2001.04527v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-robot-formation-control-using |
| Repo | |
| Framework | |
Latency-Aware Differentiable Neural Architecture Search
| Title | Latency-Aware Differentiable Neural Architecture Search |
| Authors | Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Bowen Shi, Qi Tian, Hongkai Xiong |
| Abstract | Differentiable neural architecture search methods became popular in recent years, mainly due to their low search costs and flexibility in designing the search space. However, these methods suffer the difficulty in optimizing network, so that the searched network is often unfriendly to hardware. This paper deals with this problem by adding a differentiable latency loss term into optimization, so that the search process can tradeoff between accuracy and latency with a balancing coefficient. The core of latency prediction is to encode each network architecture and feed it into a multi-layer regressor, with the training data which can be easily collected from randomly sampling a number of architectures and evaluating them on the hardware. We evaluate our approach on NVIDIA Tesla-P100 GPUs. With 100K sampled architectures (requiring a few hours), the latency prediction module arrives at a relative error of lower than 10%. Equipped with this module, the search method can reduce the latency by 20% meanwhile preserving the accuracy. Our approach also enjoys the ability of being transplanted to a wide range of hardware platforms with very few efforts, or being used to optimizing other non-differentiable factors such as power consumption. |
| Tasks | Neural Architecture Search |
| Published | 2020-01-17 |
| URL | https://arxiv.org/abs/2001.06392v2 |
| https://arxiv.org/pdf/2001.06392v2.pdf | |
| PWC | https://paperswithcode.com/paper/latency-aware-differentiable-neural |
| Repo | |
| Framework | |
Merge-split Markov chain Monte Carlo for community detection
| Title | Merge-split Markov chain Monte Carlo for community detection |
| Authors | Tiago P. Peixoto |
| Abstract | We present a Markov chain Monte Carlo scheme based on merges and splits of groups that is capable of efficiently sampling from the posterior distribution of network partitions, defined according to the stochastic block model (SBM). We demonstrate how schemes based on the move of single nodes between groups systematically fail at correctly sampling from the posterior distribution even on small networks, and how our merge-split approach behaves significantly better, and improves the mixing time of the Markov chain by several orders of magnitude in typical cases. We also show how the scheme can be straightforwardly extended to nested versions of the SBM, yielding asymptotically exact samples of hierarchical network partitions. |
| Tasks | Community Detection |
| Published | 2020-03-16 |
| URL | https://arxiv.org/abs/2003.07070v2 |
| https://arxiv.org/pdf/2003.07070v2.pdf | |
| PWC | https://paperswithcode.com/paper/merge-split-markov-chain-monte-carlo-for |
| Repo | |
| Framework | |
Deep synthesis regularization of inverse problems
| Title | Deep synthesis regularization of inverse problems |
| Authors | Daniel Obmann, Johannes Schwab, Markus Haltmeier |
| Abstract | Recently, a large number of efficient deep learning methods for solving inverse problems have been developed and show outstanding numerical performance. For these deep learning methods, however, a solid theoretical foundation in the form of reconstruction guarantees is missing. In contrast, for classical reconstruction methods, such as convex variational and frame-based regularization, theoretical convergence and convergence rate results are well established. In this paper, we introduce deep synthesis regularization (DESYRE) using neural networks as nonlinear synthesis operator bridging the gap between these two worlds. The proposed method allows to exploit the deep learning benefits of being well adjustable to available training data and on the other hand comes with a solid mathematical foundation. We present a complete convergence analysis with convergence rates for the proposed deep synthesis regularization. We present a strategy for constructing a synthesis network as part of an analysis-synthesis sequence together with an appropriate training strategy. Numerical results show the plausibility of our approach. |
| Tasks | |
| Published | 2020-02-01 |
| URL | https://arxiv.org/abs/2002.00155v1 |
| https://arxiv.org/pdf/2002.00155v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-synthesis-regularization-of-inverse |
| Repo | |
| Framework | |
Analysis of ResearchGate, A Community Detection Approach
| Title | Analysis of ResearchGate, A Community Detection Approach |
| Authors | Mohammad Heydari, Babak Teimourpour |
| Abstract | We are living in the data age. Communications over scientific networks creates new opportunities for researchers who aim to discover the hidden pattern in these huge repositories. This study utilizes network science to create collaboration network of Iranian Scientific Institutions. A modularity-based approach applied to find network communities. To reach a big picture of science production flow, analysis of the collaboration network is crucial. Our results demonstrated that geographic location closeness and ethnic attributes has important roles in academic collaboration network establishment. Besides, it shows that famous scientific centers in the capital city of Iran, Tehran has strong influence on the production flow of scientific activities. These academic papers are mostly viewed and downloaded from the United State of America, China, India, and Iran. The motivation of this research is that by discovering hidden communities in the network and finding the structure of intuitions communications, we can identify each scientific center research potential separately and clear mutual scientific fields. Therefore, an efficient strategic program can be designed, developed and tested to keep scientific centers in progress way and navigate their research goals into a straight useful roadmap to identify and fill the unknown gaps. |
| Tasks | Community Detection |
| Published | 2020-03-12 |
| URL | https://arxiv.org/abs/2003.05591v2 |
| https://arxiv.org/pdf/2003.05591v2.pdf | |
| PWC | https://paperswithcode.com/paper/analysis-of-researchgate-a-community |
| Repo | |
| Framework | |
Struct-MMSB: Mixed Membership Stochastic Blockmodels with Interpretable Structured Priors
| Title | Struct-MMSB: Mixed Membership Stochastic Blockmodels with Interpretable Structured Priors |
| Authors | Yue Zhang, Arti Ramesh |
| Abstract | The mixed membership stochastic blockmodel (MMSB) is a popular framework for community detection and network generation. It learns a low-rank mixed membership representation for each node across communities by exploiting the underlying graph structure. MMSB assumes that the membership distributions of the nodes are independently drawn from a Dirichlet distribution, which limits its capability to model highly correlated graph structures that exist in real-world networks. In this paper, we present a flexible richly structured MMSB model, \textit{Struct-MMSB}, that uses a recently developed statistical relational learning model, hinge-loss Markov random fields (HL-MRFs), as a structured prior to model complex dependencies among node attributes, multi-relational links, and their relationship with mixed-membership distributions. Our model is specified using a probabilistic programming templating language that uses weighted first-order logic rules, which enhances the model’s interpretability. Further, our model is capable of learning latent characteristics in real-world networks via meaningful latent variables encoded as a complex combination of observed features and membership distributions. We present an expectation-maximization based inference algorithm that learns latent variables and parameters iteratively, a scalable stochastic variation of the inference algorithm, and a method to learn the weights of HL-MRF structured priors. We evaluate our model on six datasets across three different types of networks and corresponding modeling scenarios and demonstrate that our models are able to achieve an improvement of 15% on average in test log-likelihood and faster convergence when compared to state-of-the-art network models. |
| Tasks | Community Detection, Probabilistic Programming, Relational Reasoning |
| Published | 2020-02-21 |
| URL | https://arxiv.org/abs/2002.09523v1 |
| https://arxiv.org/pdf/2002.09523v1.pdf | |
| PWC | https://paperswithcode.com/paper/struct-mmsb-mixed-membership-stochastic |
| Repo | |
| Framework | |
Network Clustering Via Kernel-ARMA Modeling and the Grassmannian The Brain-Network Case
| Title | Network Clustering Via Kernel-ARMA Modeling and the Grassmannian The Brain-Network Case |
| Authors | Cong Ye, Konstantinos Slavakis, Pratik V. Patil, Johan Nakuci, Sarah F. Muldoon, John Medaglia |
| Abstract | This paper introduces a clustering framework for networks with nodes annotated with time-series data. The framework addresses all types of network-clustering problems: State clustering, node clustering within states (a.k.a. topology identification or community detection), and even subnetwork-state-sequence identification/tracking. Via a bottom-up approach, features are first extracted from the raw nodal time-series data by kernel autoregressive-moving-average modeling to reveal non-linear dependencies and low-rank representations, and then mapped onto the Grassmann manifold (Grassmannian). All clustering tasks are performed by leveraging the underlying Riemannian geometry of the Grassmannian in a novel way. To validate the proposed framework, brain-network clustering is considered, where extensive numerical tests on synthetic and real functional magnetic resonance imaging (fMRI) data demonstrate that the advocated learning framework compares favorably versus several state-of-the-art clustering schemes. |
| Tasks | Community Detection, Time Series |
| Published | 2020-02-18 |
| URL | https://arxiv.org/abs/2002.09943v1 |
| https://arxiv.org/pdf/2002.09943v1.pdf | |
| PWC | https://paperswithcode.com/paper/network-clustering-via-kernel-arma-modeling |
| Repo | |
| Framework | |
Community Detection on Mixture Multi-layer Networks via Regularized Tensor Decomposition
| Title | Community Detection on Mixture Multi-layer Networks via Regularized Tensor Decomposition |
| Authors | Bing-Yi Jing, Ting Li, Zhongyuan Lyu, Dong Xia |
| Abstract | We study the problem of community detection in multi-layer networks, where pairs of nodes can be related in multiple modalities. We introduce a general framework, i.e., mixture multi-layer stochastic block model (MMSBM), which includes many earlier models as special cases. We propose a tensor-based algorithm (TWIST) to reveal both global/local memberships of nodes, and memberships of layers. We show that the TWIST procedure can accurately detect the communities with small misclassification error as the number of nodes and/or the number of layers increases. Numerical studies confirm our theoretical findings. To our best knowledge, this is the first systematic study on the mixture multi-layer networks using tensor decomposition. The method is applied to two real datasets: worldwide trading networks and malaria parasite genes networks, yielding new and interesting findings. |
| Tasks | Community Detection |
| Published | 2020-02-10 |
| URL | https://arxiv.org/abs/2002.04457v1 |
| https://arxiv.org/pdf/2002.04457v1.pdf | |
| PWC | https://paperswithcode.com/paper/community-detection-on-mixture-multi-layer |
| Repo | |
| Framework | |
Learning Style-Aware Symbolic Music Representations by Adversarial Autoencoders
| Title | Learning Style-Aware Symbolic Music Representations by Adversarial Autoencoders |
| Authors | Andrea Valenti, Antonio Carta, Davide Bacciu |
| Abstract | We address the challenging open problem of learning an effective latent space for symbolic music data in generative music modeling. We focus on leveraging adversarial regularization as a flexible and natural mean to imbue variational autoencoders with context information concerning music genre and style. Through the paper, we show how Gaussian mixtures taking into account music metadata information can be used as an effective prior for the autoencoder latent space, introducing the first Music Adversarial Autoencoder (MusAE). The empirical analysis on a large scale benchmark shows that our model has a higher reconstruction accuracy than state-of-the-art models based on standard variational autoencoders. It is also able to create realistic interpolations between two musical sequences, smoothly changing the dynamics of the different tracks. Experiments show that the model can organise its latent space accordingly to low-level properties of the musical pieces, as well as to embed into the latent variables the high-level genre information injected from the prior distribution to increase its overall performance. This allows us to perform changes to the generated pieces in a principled way. |
| Tasks | Music Modeling |
| Published | 2020-01-15 |
| URL | https://arxiv.org/abs/2001.05494v2 |
| https://arxiv.org/pdf/2001.05494v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-a-latent-space-of-style-aware |
| Repo | |
| Framework | |
Semantic Holism and Word Representations in Artificial Neural Networks
| Title | Semantic Holism and Word Representations in Artificial Neural Networks |
| Authors | Tomáš Musil |
| Abstract | Artificial neural networks are a state-of-the-art solution for many problems in natural language processing. What can we learn about language and meaning from the way artificial neural networks represent it? Word representations obtained from the Skip-gram variant of the word2vec model exhibit interesting semantic properties. This is usually explained by referring to the general distributional hypothesis, which states that the meaning of the word is given by the contexts where it occurs. We propose a more specific approach based on Frege’s holistic and functional approach to meaning. Taking Tugendhat’s formal reinterpretation of Frege’s work as a starting point, we demonstrate that it is analogical to the process of training the Skip-gram model and offers a possible explanation of its semantic properties. |
| Tasks | |
| Published | 2020-03-11 |
| URL | https://arxiv.org/abs/2003.05522v1 |
| https://arxiv.org/pdf/2003.05522v1.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-holism-and-word-representations-in |
| Repo | |
| Framework | |
