Paper Group AWR 2
Destructiveness of Lexicographic Parsimony Pressure and Alleviation by a Concatenation Crossover in Genetic Programming. Matrix Factorization on GPUs with Memory Optimization and Approximate Computing. Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Data. Cadre Modeling: Simultaneously Discovering Subpopula …
Destructiveness of Lexicographic Parsimony Pressure and Alleviation by a Concatenation Crossover in Genetic Programming
| Title | Destructiveness of Lexicographic Parsimony Pressure and Alleviation by a Concatenation Crossover in Genetic Programming |
| Authors | Timo Kötzing, J. A. Gregor Lagodzinski, Johannes Lengler, Anna Melnichenko |
| Abstract | For theoretical analyses there are two specifics distinguishing GP from many other areas of evolutionary computation. First, the variable size representations, in particular yielding a possible bloat (i.e. the growth of individuals with redundant parts). Second, the role and realization of crossover, which is particularly central in GP due to the tree-based representation. Whereas some theoretical work on GP has studied the effects of bloat, crossover had a surprisingly little share in this work. We analyze a simple crossover operator in combination with local search, where a preference for small solutions minimizes bloat (lexicographic parsimony pressure); the resulting algorithm is denoted Concatenation Crossover GP. For this purpose three variants of the well-studied MAJORITY test function with large plateaus are considered. We show that the Concatenation Crossover GP can efficiently optimize these test functions, while local search cannot be efficient for all three variants independent of employing bloat control. |
| Tasks | |
| Published | 2018-05-25 |
| URL | http://arxiv.org/abs/1805.10169v1 |
| http://arxiv.org/pdf/1805.10169v1.pdf | |
| PWC | https://paperswithcode.com/paper/destructiveness-of-lexicographic-parsimony |
| Repo | https://github.com/melnan/1-1-GP |
| Framework | none |
Matrix Factorization on GPUs with Memory Optimization and Approximate Computing
| Title | Matrix Factorization on GPUs with Memory Optimization and Approximate Computing |
| Authors | Wei Tan, Shiyu Chang, Liana Fong, Cheng Li, Zijun Wang, Liangliang Cao |
| Abstract | Matrix factorization (MF) discovers latent features from observations, which has shown great promises in the fields of collaborative filtering, data compression, feature extraction, word embedding, etc. While many problem-specific optimization techniques have been proposed, alternating least square (ALS) remains popular due to its general applicability e.g. easy to handle positive-unlabeled inputs, fast convergence and parallelization capability. Current MF implementations are either optimized for a single machine or with a need of a large computer cluster but still are insufficient. This is because a single machine provides limited compute power for large-scale data while multiple machines suffer from the network communication bottleneck. To address the aforementioned challenge, accelerating ALS on graphics processing units (GPUs) is a promising direction. We propose the novel approach in enhancing the MF efficiency via both memory optimization and approximate computing. The former exploits GPU memory hierarchy to increase data reuse, while the later reduces unnecessary computing without hurting the convergence of learning algorithms. Extensive experiments on large-scale datasets show that our solution not only outperforms the competing CPU solutions by a large margin but also has a 2x-4x performance gain compared to the state-of-the-art GPU solutions. Our implementations are open-sourced and publicly available. |
| Tasks | |
| Published | 2018-08-11 |
| URL | http://arxiv.org/abs/1808.03843v1 |
| http://arxiv.org/pdf/1808.03843v1.pdf | |
| PWC | https://paperswithcode.com/paper/matrix-factorization-on-gpus-with-memory |
| Repo | https://github.com/cuMF/cumf_als |
| Framework | none |
Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Data
| Title | Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Data |
| Authors | Seyed Mostafa Kia, Andre F. Marquand |
| Abstract | Normative modeling has recently been introduced as a promising approach for modeling variation of neuroimaging measures across individuals in order to derive biomarkers of psychiatric disorders. Current implementations rely on Gaussian process regression, which provides coherent estimates of uncertainty needed for the method but also suffers from drawbacks including poor scaling to large datasets and a reliance on fixed parametric kernels. In this paper, we propose a deep normative modeling framework based on neural processes (NPs) to solve these problems. To achieve this, we define a stochastic process formulation for mixed-effect models and show how NPs can be adopted for spatially structured mixed-effect modeling of neuroimaging data. This enables us to learn optimal feature representations and covariance structure for the random-effect and noise via global latent variables. In this scheme, predictive uncertainty can be approximated by sampling from the distribution of these global latent variables. On a publicly available clinical fMRI dataset, we compare the novelty detection performance of multivariate normative models estimated by the proposed NP approach to a baseline multi-task Gaussian process regression approach and show substantial improvements for certain diagnostic problems. |
| Tasks | |
| Published | 2018-12-12 |
| URL | http://arxiv.org/abs/1812.04998v2 |
| http://arxiv.org/pdf/1812.04998v2.pdf | |
| PWC | https://paperswithcode.com/paper/neural-processes-mixed-effect-models-for-deep |
| Repo | https://github.com/smkia/DNM |
| Framework | pytorch |
Cadre Modeling: Simultaneously Discovering Subpopulations and Predictive Models
| Title | Cadre Modeling: Simultaneously Discovering Subpopulations and Predictive Models |
| Authors | Alexander New, Curt Breneman, Kristin P. Bennett |
| Abstract | We consider the problem in regression analysis of identifying subpopulations that exhibit different patterns of response, where each subpopulation requires a different underlying model. Unlike statistical cohorts, these subpopulations are not known a priori; thus, we refer to them as cadres. When the cadres and their associated models are interpretable, modeling leads to insights about the subpopulations and their associations with the regression target. We introduce a discriminative model that simultaneously learns cadre assignment and target-prediction rules. Sparsity-inducing priors are placed on the model parameters, under which independent feature selection is performed for both the cadre assignment and target-prediction processes. We learn models using adaptive step size stochastic gradient descent, and we assess cadre quality with bootstrapped sample analysis. We present simulated results showing that, when the true clustering rule does not depend on the entire set of features, our method significantly outperforms methods that learn subpopulation-discovery and target-prediction rules separately. In a materials-by-design case study, our model provides state-of-the-art prediction of polymer glass transition temperature. Importantly, the method identifies cadres of polymers that respond differently to structural perturbations, thus providing design insight for targeting or avoiding specific transition temperature ranges. It identifies chemically meaningful cadres, each with interpretable models. Further experimental results show that cadre methods have generalization that is competitive with linear and nonlinear regression models and can identify robust subpopulations. |
| Tasks | Feature Selection |
| Published | 2018-02-07 |
| URL | http://arxiv.org/abs/1802.02500v2 |
| http://arxiv.org/pdf/1802.02500v2.pdf | |
| PWC | https://paperswithcode.com/paper/cadre-modeling-simultaneously-discovering |
| Repo | https://github.com/newalexander/supervised-cadres |
| Framework | tf |
Automated Vulnerability Detection in Source Code Using Deep Representation Learning
| Title | Automated Vulnerability Detection in Source Code Using Deep Representation Learning |
| Authors | Rebecca L. Russell, Louis Kim, Lei H. Hamilton, Tomo Lazovich, Jacob A. Harer, Onur Ozdemir, Paul M. Ellingwood, Marc W. McConley |
| Abstract | Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ open-source code available to develop a large-scale function-level vulnerability detection system using machine learning. To supplement existing labeled vulnerability datasets, we compiled a vast dataset of millions of open-source functions and labeled it with carefully-selected findings from three different static analyzers that indicate potential exploits. The labeled dataset is available at: https://osf.io/d45bw/. Using these datasets, we developed a fast and scalable vulnerability detection tool based on deep feature representation learning that directly interprets lexed source code. We evaluated our tool on code from both real software packages and the NIST SATE IV benchmark dataset. Our results demonstrate that deep feature representation learning on source code is a promising approach for automated software vulnerability detection. |
| Tasks | Representation Learning, Vulnerability Detection |
| Published | 2018-07-11 |
| URL | http://arxiv.org/abs/1807.04320v2 |
| http://arxiv.org/pdf/1807.04320v2.pdf | |
| PWC | https://paperswithcode.com/paper/automated-vulnerability-detection-in-source |
| Repo | https://github.com/hazimhanif/svd_exp3 |
| Framework | pytorch |
Comixify: Transform video into a comics
| Title | Comixify: Transform video into a comics |
| Authors | Maciej Pęśko, Adam Svystun, Paweł Andruszkiewicz, Przemysław Rokita, Tomasz Trzciński |
| Abstract | In this paper, we propose a solution to transform a video into a comics. We approach this task using a neural style algorithm based on Generative Adversarial Networks (GANs). Several recent works in the field of Neural Style Transfer showed that producing an image in the style of another image is feasible. In this paper, we build up on these works and extend the existing set of style transfer use cases with a working application of video comixification. To that end, we train an end-to-end solution that transforms input video into a comics in two stages. In the first stage, we propose a state-of-the-art keyframes extraction algorithm that selects a subset of frames from the video to provide the most comprehensive video context and we filter those frames using image aesthetic estimation engine. In the second stage, the style of selected keyframes is transferred into a comics. To provide the most aesthetically compelling results, we selected the most state-of-the art style transfer solution and based on that implement our own ComixGAN framework. The final contribution of our work is a Web-based working application of video comixification available at http://comixify.ii.pw.edu.pl. |
| Tasks | Style Transfer, Transform A Video Into A Comics |
| Published | 2018-12-09 |
| URL | http://arxiv.org/abs/1812.03473v1 |
| http://arxiv.org/pdf/1812.03473v1.pdf | |
| PWC | https://paperswithcode.com/paper/comixify-transform-video-into-a-comics |
| Repo | https://github.com/maciej3031/comixify |
| Framework | none |
xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
| Title | xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems |
| Authors | Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, Guangzhong Sun |
| Abstract | Combinatorial features are essential for the success of many commercial models. Manually crafting these features usually comes with high cost due to the variety, volume and velocity of raw data in web-scale systems. Factorization based models, which measure interactions in terms of vector product, can learn patterns of combinatorial features automatically and generalize to unseen features as well. With the great success of deep neural networks (DNNs) in various fields, recently researchers have proposed several DNN-based factorization model to learn both low- and high-order feature interactions. Despite the powerful ability of learning an arbitrary function from data, plain DNNs generate feature interactions implicitly and at the bit-wise level. In this paper, we propose a novel Compressed Interaction Network (CIN), which aims to generate feature interactions in an explicit fashion and at the vector-wise level. We show that the CIN share some functionalities with convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We further combine a CIN and a classical DNN into one unified model, and named this new model eXtreme Deep Factorization Machine (xDeepFM). On one hand, the xDeepFM is able to learn certain bounded-degree feature interactions explicitly; on the other hand, it can learn arbitrary low- and high-order feature interactions implicitly. We conduct comprehensive experiments on three real-world datasets. Our results demonstrate that xDeepFM outperforms state-of-the-art models. We have released the source code of xDeepFM at \url{https://github.com/Leavingseason/xDeepFM}. |
| Tasks | Click-Through Rate Prediction, Recommendation Systems |
| Published | 2018-03-14 |
| URL | http://arxiv.org/abs/1803.05170v3 |
| http://arxiv.org/pdf/1803.05170v3.pdf | |
| PWC | https://paperswithcode.com/paper/xdeepfm-combining-explicit-and-implicit |
| Repo | https://github.com/bettenW/Tencent2019_Finals_Rank1st |
| Framework | tf |
CEM-RL: Combining evolutionary and gradient-based methods for policy search
| Title | CEM-RL: Combining evolutionary and gradient-based methods for policy search |
| Authors | Aloïs Pourchot, Olivier Sigaud |
| Abstract | Deep neuroevolution and deep reinforcement learning (deep RL) algorithms are two popular approaches to policy search. The former is widely applicable and rather stable, but suffers from low sample efficiency. By contrast, the latter is more sample efficient, but the most sample efficient variants are also rather unstable and highly sensitive to hyper-parameter setting. So far, these families of methods have mostly been compared as competing tools. However, an emerging approach consists in combining them so as to get the best of both worlds. Two previously existing combinations use either an ad hoc evolutionary algorithm or a goal exploration process together with the Deep Deterministic Policy Gradient (DDPG) algorithm, a sample efficient off-policy deep RL algorithm. In this paper, we propose a different combination scheme using the simple cross-entropy method (CEM) and Twin Delayed Deep Deterministic policy gradient (td3), another off-policy deep RL algorithm which improves over ddpg. We evaluate the resulting method, cem-rl, on a set of benchmarks classically used in deep RL. We show that cem-rl benefits from several advantages over its competitors and offers a satisfactory trade-off between performance and sample efficiency. |
| Tasks | |
| Published | 2018-10-02 |
| URL | http://arxiv.org/abs/1810.01222v3 |
| http://arxiv.org/pdf/1810.01222v3.pdf | |
| PWC | https://paperswithcode.com/paper/cem-rl-combining-evolutionary-and-gradient-1 |
| Repo | https://github.com/apourchot/EvoLearn |
| Framework | pytorch |
Generative Adversarial Network in Medical Imaging: A Review
| Title | Generative Adversarial Network in Medical Imaging: A Review |
| Authors | Xin Yi, Ekta Walia, Paul Babyn |
| Abstract | Generative adversarial networks have gained a lot of attention in the computer vision community due to their capability of data generation without explicitly modelling the probability density function. The adversarial loss brought by the discriminator provides a clever way of incorporating unlabeled samples into training and imposing higher order consistency. This has proven to be useful in many cases, such as domain adaptation, data augmentation, and image-to-image translation. These properties have attracted researchers in the medical imaging community, and we have seen rapid adoption in many traditional and novel applications, such as image reconstruction, segmentation, detection, classification, and cross-modality synthesis. Based on our observations, this trend will continue and we therefore conducted a review of recent advances in medical imaging using the adversarial training scheme with the hope of benefiting researchers interested in this technique. |
| Tasks | Data Augmentation, Domain Adaptation, Image Reconstruction, Image-to-Image Translation, Medical Image Generation |
| Published | 2018-09-19 |
| URL | https://arxiv.org/abs/1809.07294v4 |
| https://arxiv.org/pdf/1809.07294v4.pdf | |
| PWC | https://paperswithcode.com/paper/generative-adversarial-network-in-medical |
| Repo | https://github.com/xinario/awesome-gan-for-medical-imaging |
| Framework | none |
GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose
| Title | GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose |
| Authors | Zhichao Yin, Jianping Shi |
| Abstract | We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and ego-motion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in an end-to-end manner. Specifically, geometric relationships are extracted over the predictions of individual modules and then combined as an image reconstruction loss, reasoning about static and dynamic scene parts separately. Furthermore, we propose an adaptive geometric consistency loss to increase robustness towards outliers and non-Lambertian regions, which resolves occlusions and texture ambiguities effectively. Experimentation on the KITTI driving dataset reveals that our scheme achieves state-of-the-art results in all of the three tasks, performing better than previously unsupervised methods and comparably with supervised ones. |
| Tasks | Image Reconstruction, Motion Estimation, Optical Flow Estimation |
| Published | 2018-03-06 |
| URL | http://arxiv.org/abs/1803.02276v2 |
| http://arxiv.org/pdf/1803.02276v2.pdf | |
| PWC | https://paperswithcode.com/paper/geonet-unsupervised-learning-of-dense-depth |
| Repo | https://github.com/yijie0710/GeoNet_pytorch |
| Framework | pytorch |
CR-GAN: Learning Complete Representations for Multi-view Generation
| Title | CR-GAN: Learning Complete Representations for Multi-view Generation |
| Authors | Yu Tian, Xi Peng, Long Zhao, Shaoting Zhang, Dimitris N. Metaxas |
| Abstract | Generating multi-view images from a single-view input is an essential yet challenging problem. It has broad applications in vision, graphics, and robotics. Our study indicates that the widely-used generative adversarial network (GAN) may learn “incomplete” representations due to the single-pathway framework: an encoder-decoder network followed by a discriminator network. We propose CR-GAN to address this problem. In addition to the single reconstruction path, we introduce a generation sideway to maintain the completeness of the learned embedding space. The two learning pathways collaborate and compete in a parameter-sharing manner, yielding considerably improved generalization ability to “unseen” dataset. More importantly, the two-pathway framework makes it possible to combine both labeled and unlabeled data for self-supervised learning, which further enriches the embedding space for realistic generations. The experimental results prove that CR-GAN significantly outperforms state-of-the-art methods, especially when generating from “unseen” inputs in wild conditions. |
| Tasks | |
| Published | 2018-06-28 |
| URL | http://arxiv.org/abs/1806.11191v1 |
| http://arxiv.org/pdf/1806.11191v1.pdf | |
| PWC | https://paperswithcode.com/paper/cr-gan-learning-complete-representations-for |
| Repo | https://github.com/bluer555/CR-GAN |
| Framework | pytorch |
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
| Title | Talking Face Generation by Adversarially Disentangled Audio-Visual Representation |
| Authors | Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, Xiaogang Wang |
| Abstract | Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech. This is a challenging task because face appearance variation and semantics of speech are coupled together in the subtle movements of the talking face regions. Existing works either construct specific face appearance model on specific subjects or model the transformation between lip motion and speech. In this work, we integrate both aspects and enable arbitrary-subject talking face generation by learning disentangled audio-visual representation. We find that the talking face sequence is actually a composition of both subject-related information and speech-related information. These two spaces are then explicitly disentangled through a novel associative-and-adversarial training process. This disentangled representation has an advantage where both audio and video can serve as inputs for generation. Extensive experiments show that the proposed approach generates realistic talking face sequences on arbitrary subjects with much clearer lip motion patterns than previous work. We also demonstrate the learned audio-visual representation is extremely useful for the tasks of automatic lip reading and audio-video retrieval. |
| Tasks | Face Generation, Talking Face Generation, Video Retrieval |
| Published | 2018-07-20 |
| URL | http://arxiv.org/abs/1807.07860v2 |
| http://arxiv.org/pdf/1807.07860v2.pdf | |
| PWC | https://paperswithcode.com/paper/talking-face-generation-by-adversarially |
| Repo | https://github.com/Hangz-nju-cuhk/Talking-Face-Generation-DAVS |
| Framework | pytorch |
Faster Training of Mask R-CNN by Focusing on Instance Boundaries
| Title | Faster Training of Mask R-CNN by Focusing on Instance Boundaries |
| Authors | Roland S. Zimmermann, Julien N. Siems |
| Abstract | We present an auxiliary task to Mask R-CNN, an instance segmentation network, which leads to faster training of the mask head. Our addition to Mask R-CNN is a new prediction head, the Edge Agreement Head, which is inspired by the way human annotators perform instance segmentation. Human annotators copy the contour of an object instance and only indirectly the occupied instance area. Hence, the edges of instance masks are particularly useful as they characterize the instance well. The Edge Agreement Head therefore encourages predicted masks to have similar image gradients to the ground-truth mask using edge detection filters. We provide a detailed survey of loss combinations and show improvements on the MS COCO Mask metrics compared to using no additional loss. Our approach marginally increases the model size and adds no additional trainable model variables. While the computational costs are increased slightly, the increment is negligible considering the high computational cost of the Mask R-CNN architecture. As the additional network head is only relevant during training, inference speed remains unchanged compared to Mask R-CNN. In a default Mask R-CNN setup, we achieve a training speed-up and a relative overall improvement of 8.1% on the MS COCO metrics compared to the baseline. |
| Tasks | Edge Detection, Instance Segmentation, Semantic Segmentation |
| Published | 2018-09-19 |
| URL | https://arxiv.org/abs/1809.07069v4 |
| https://arxiv.org/pdf/1809.07069v4.pdf | |
| PWC | https://paperswithcode.com/paper/faster-training-of-mask-r-cnn-by-focusing-on |
| Repo | https://github.com/FlashTek/mask-rcnn-edge-agreement-loss |
| Framework | tf |
An overview of deep learning in medical imaging focusing on MRI
| Title | An overview of deep learning in medical imaging focusing on MRI |
| Authors | Alexander Selvikvåg Lundervold, Arvid Lundervold |
| Abstract | What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of machine learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging. |
| Tasks | Disease Prediction, Image Retrieval |
| Published | 2018-11-25 |
| URL | http://arxiv.org/abs/1811.10052v2 |
| http://arxiv.org/pdf/1811.10052v2.pdf | |
| PWC | https://paperswithcode.com/paper/an-overview-of-deep-learning-in-medical |
| Repo | https://github.com/MMIV-ML/DLMI2018 |
| Framework | none |
Talakat: Bullet Hell Generation through Constrained Map-Elites
| Title | Talakat: Bullet Hell Generation through Constrained Map-Elites |
| Authors | Ahmed Khalifa, Scott Lee, Andy Nealen, Julian Togelius |
| Abstract | We describe a search-based approach to generating new levels for bullet hell games, which are action games characterized by and requiring avoidance of a very large amount of projectiles. Levels are represented using a domain-specific description language, and search in the space defined by this language is performed by a novel variant of the Map-Elites algorithm which incorporates a feasible- infeasible approach to constraint satisfaction. Simulation-based evaluation is used to gauge the fitness of levels, using an agent based on best-first search. The performance of the agent can be tuned according to the two dimensions of strategy and dexterity, making it possible to search for level configurations that require a specific combination of both. As far as we know, this paper describes the first generator for this game genre, and includes several algorithmic innovations. |
| Tasks | |
| Published | 2018-06-12 |
| URL | http://arxiv.org/abs/1806.04718v2 |
| http://arxiv.org/pdf/1806.04718v2.pdf | |
| PWC | https://paperswithcode.com/paper/talakat-bullet-hell-generation-through |
| Repo | https://github.com/DanieleGravina/divergence-and-quality-diversity |
| Framework | none |
