January 29, 2020

3228 words 16 mins read

Paper Group ANR 738

Beyond Photo Realism for Domain Adaptation from Synthetic Data. Fast, Small, and Simple Document Listing on Repetitive Text Collections. Online Regularization by Denoising with Applications to Phase Retrieval. A Streaming Analytics Language for Processing Cyber Data. LIA: Latently Invertible Autoencoder with Adversarial Learning. On the Anomalous G …

Beyond Photo Realism for Domain Adaptation from Synthetic Data


Title	Beyond Photo Realism for Domain Adaptation from Synthetic Data
Authors	Kristofer Schlachter, Connor DeFanti, Sebastian Herscher, Ken Perlin, Jonathan Tompson
Abstract	As synthetic imagery is used more frequently in training deep models, it is important to understand how different synthesis techniques impact the performance of such models. In this work, we perform a thorough evaluation of the effectiveness of several different synthesis techniques and their impact on the complexity of classifier domain adaptation to the “real” underlying data distribution that they seek to replicate. In addition, we propose a novel learned synthesis technique to better train classifier models than state-of-the-art offline graphical methods, while using significantly less computational resources. We accomplish this by learning a generative model to perform shading of synthetic geometry conditioned on a “g-buffer” representation of the scene to render, as well as a low sample Monte Carlo rendered image. The major contributions are (i) a dataset that allows comparison of real and synthetic versions of the same scene, (ii) an augmented data representation that boosts the stability of learning and improves the datasets accuracy, (iii) three different partially differentiable rendering techniques where lighting, denoising and shading are learned, and (iv) we improve a state of the art generative adversarial network (GAN) approach by using an ensemble of trained models to generate datasets that approach the performance of training on real data and surpass the performance of the full global illumination rendering.
Tasks	Denoising, Domain Adaptation
Published	2019-09-04
URL	https://arxiv.org/abs/1909.01960v1
PDF	https://arxiv.org/pdf/1909.01960v1.pdf
PWC	https://paperswithcode.com/paper/beyond-photo-realism-for-domain-adaptation
Repo
Framework

Fast, Small, and Simple Document Listing on Repetitive Text Collections


Title	Fast, Small, and Simple Document Listing on Repetitive Text Collections
Authors	Dustin Cobas, Gonzalo Navarro
Abstract	Document listing on string collections is the task of finding all documents where a pattern appears. It is regarded as the most fundamental document retrieval problem, and is useful in various applications. Many of the fastest-growing string collections are composed of very similar documents, such as versioned code and document collections, genome repositories, etc. Plain pattern-matching indexes designed for repetitive text collections achieve orders-of-magnitude reductions in space. Instead, there are not many analogous indexes for document retrieval. In this paper we present a simple document listing index for repetitive string collections of total length $n$ that lists the $ndoc$ distinct documents where a pattern of length $m$ appears in time $\mathcal{O}(m+ndoc \cdot \log n)$. We exploit the repetitiveness of the document array (i.e., the suffix array coarsened to document identifiers) to grammar-compress it while precomputing the answers to nonterminals, and store them in grammar-compressed form as well. Our experimental results show that our index sharply outperforms existing alternatives in the space/time tradeoff map.
Tasks
Published	2019-02-20
URL	http://arxiv.org/abs/1902.07599v1
PDF	http://arxiv.org/pdf/1902.07599v1.pdf
PWC	https://paperswithcode.com/paper/fast-small-and-simple-document-listing-on
Repo
Framework

Online Regularization by Denoising with Applications to Phase Retrieval


Title	Online Regularization by Denoising with Applications to Phase Retrieval
Authors	Zihui Wu, Yu Sun, Jiaming Liu, Ulugbek S. Kamilov
Abstract	Regularization by denoising (RED) is a powerful framework for solving imaging inverse problems. Most RED algorithms are iterative batch procedures, which limits their applicability to very large datasets. In this paper, we address this limitation by introducing a novel online RED (On-RED) algorithm, which processes a small subset of the data at a time. We establish the theoretical convergence of On-RED in convex settings and empirically discuss its effectiveness in non-convex ones by illustrating its applicability to phase retrieval. Our results suggest that On-RED is an effective alternative to the traditional RED algorithms when dealing with large datasets.
Tasks	Denoising
Published	2019-09-04
URL	https://arxiv.org/abs/1909.02040v1
PDF	https://arxiv.org/pdf/1909.02040v1.pdf
PWC	https://paperswithcode.com/paper/online-regularization-by-denoising-with
Repo
Framework

A Streaming Analytics Language for Processing Cyber Data


Title	A Streaming Analytics Language for Processing Cyber Data
Authors	Eric L. Goodman, Dirk Grunwald
Abstract	We present a domain-specific language called SAL(the Streaming Analytics Language) for processing data in a semi-streaming model. In particular we examine the use case of processing netflow data in order to identify malicious actors within a network. Because of the large volume of data generated from networks, it is often only feasible to process the data with a single pass, utilizing a streaming (O(polylog n) space requirements) or semi-streaming computing model ( O(n polylog n) space requirements). Despite these constraints, we are able to achieve an average of 0.87 for the AUC of the ROC curve for a set of situations dealing with botnet detection. The implementation of an interpreter for SAL, which we call SAM (Streaming Analytics Machine), achieves scaling results that show improved throughput to 61 nodes (976 cores), with an overall rate of 373,000 netflows per second or 32.2 billion per day. SAL provides a succinct way to describe common analyses that allow cyber analysts to find data of interest, and SAM is a scalable interpreter of the language.
Tasks
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00815v1
PDF	https://arxiv.org/pdf/1911.00815v1.pdf
PWC	https://paperswithcode.com/paper/a-streaming-analytics-language-for-processing
Repo
Framework

LIA: Latently Invertible Autoencoder with Adversarial Learning


Title	LIA: Latently Invertible Autoencoder with Adversarial Learning
Authors	Jiapeng Zhu, Deli Zhao, Bolei Zhou, Bo Zhang
Abstract	Deep generative models such as Variational AutoEncoder (VAE) and Generative Adversarial Network (GAN) play an increasingly important role in machine learning and computer vision. However, there are two fundamental issues hindering their real-world applications: the difficulty of conducting variational inference in VAE and the functional absence of encoding real-world samples in GAN. In this paper, we propose a novel algorithm named Latently Invertible Autoencoder (LIA) to address the above two issues in one framework. An invertible network and its inverse mapping are symmetrically embedded in the latent space of VAE. Thus the partial encoder first transforms the input into feature vectors and then the distribution of these feature vectors is reshaped to fit a prior by the invertible network. The decoder proceeds in the reverse order of the encoder’s composite mappings. A two-stage stochasticity-free training scheme is designed to train LIA via adversarial learning, in the sense that the decoder of LIA is first trained as a standard GAN with the invertible network and then the partial encoder is learned from an autoencoder by detaching the invertible network from LIA. Experiments conducted on the FFHQ face dataset and three LSUN datasets validate the effectiveness of LIA for inference and generation.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08090v2
PDF	https://arxiv.org/pdf/1906.08090v2.pdf
PWC	https://paperswithcode.com/paper/lia-latently-invertible-autoencoder-with
Repo
Framework

On the Anomalous Generalization of GANs


Title	On the Anomalous Generalization of GANs
Authors	Jinchen Xuan, Yunchang Yang, Ze Yang, Di He, Liwei Wang
Abstract	Generative models, especially Generative Adversarial Networks (GANs), have received significant attention recently. However, it has been observed that in terms of some attributes, e.g. the number of simple geometric primitives in an image, GANs are not able to learn the target distribution in practice. Motivated by this observation, we discover two specific problems of GANs leading to anomalous generalization behaviour, which we refer to as the sample insufficiency and the pixel-wise combination. For the first problem of sample insufficiency, we show theoretically and empirically that the batchsize of the training samples in practice may be insufficient for the discriminator to learn an accurate discrimination function. It could result in unstable training dynamics for the generator, leading to anomalous generalization. For the second problem of pixel-wise combination, we find that besides recognizing the positive training samples as real, under certain circumstances, the discriminator could be fooled to recognize the pixel-wise combinations (e.g. pixel-wise average) of the positive training samples as real. However, those combinations could be visually different from the real samples in the target distribution. With the fooled discriminator as reference, the generator would obtain biased supervision further, leading to the anomalous generalization behaviour. Additionally, in this paper, we propose methods to mitigate the anomalous generalization of GANs. Extensive experiments on benchmark show our proposed methods improve the FID score up to 30% on natural image dataset.
Tasks
Published	2019-09-27
URL	https://arxiv.org/abs/1909.12638v2
PDF	https://arxiv.org/pdf/1909.12638v2.pdf
PWC	https://paperswithcode.com/paper/on-the-anomalous-generalization-of-gans-1
Repo
Framework

EnSyth: A Pruning Approach to Synthesis of Deep Learning Ensembles


Title	EnSyth: A Pruning Approach to Synthesis of Deep Learning Ensembles
Authors	Besher Alhalabi, Mohamed Medhat Gaber, Shadi Basurra
Abstract	Deep neural networks have achieved state-of-art performance in many domains including computer vision, natural language processing and self-driving cars. However, they are very computationally expensive and memory intensive which raises significant challenges when it comes to deploy or train them on strict latency applications or resource-limited environments. As a result, many attempts have been introduced to accelerate and compress deep learning models, however the majority were not able to maintain the same accuracy of the baseline models. In this paper, we describe EnSyth, a deep learning ensemble approach to enhance the predictability of compact neural network’s models. First, we generate a set of diverse compressed deep learning models using different hyperparameters for a pruning method, after that we utilise ensemble learning to synthesise the outputs of the compressed models to compose a new pool of classifiers. Finally, we apply backward elimination on the generated pool to explore the best performing combinations of models. On CIFAR-10, CIFAR-5 data-sets with LeNet-5, EnSyth outperforms the predictability of the baseline model.
Tasks	Self-Driving Cars
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09286v1
PDF	https://arxiv.org/pdf/1907.09286v1.pdf
PWC	https://paperswithcode.com/paper/ensyth-a-pruning-approach-to-synthesis-of
Repo
Framework

Bipolar in Temporal Argumentation Framework


Title	Bipolar in Temporal Argumentation Framework
Authors	Maximiliano C. D. Budán, Maria Laura Cobo, Diego C. Martinez, Guillermo R. Simari
Abstract	A Timed Argumentation Framework (TAF) is a formalism where arguments are only valid for consideration in a given period of time, called availability intervals, which are defined for every individual argument. The original proposal is based on a single, abstract notion of attack between arguments that remains static and permanent in time. Thus, in general, when identifying the set of acceptable arguments, the outcome associated with a TAF will vary over time. In this work we introduce an extension of TAF adding the capability of modeling a support relation between arguments. In this sense, the resulting framework provides a suitable model for different time-dependent issues. Thus, the main contribution here is to provide an enhanced framework for modeling a positive (support) and negative (attack) interaction varying over time, which are relevant in many real-world situations. This leads to a Timed Bipolar Argumentation Framework (T-BAF), where classical argument extensions can be defined. The proposal aims at advancing in the integration of temporal argumentation in different application domain.
Tasks
Published	2019-03-05
URL	http://arxiv.org/abs/1903.01874v1
PDF	http://arxiv.org/pdf/1903.01874v1.pdf
PWC	https://paperswithcode.com/paper/bipolar-in-temporal-argumentation-framework
Repo
Framework

Dynamic Cell Imaging in PET with Optimal Transport Regularization


Title	Dynamic Cell Imaging in PET with Optimal Transport Regularization
Authors	Bernhard Schmitzer, Klaus P. Schäfers, Benedikt Wirth
Abstract	We propose a novel dynamic image reconstruction method from PET listmode data that could be particularly suited to tracking single or small numbers of cells. In contrast to conventional PET reconstruction our method combines the information from all detected events not only to reconstruct the dynamic evolution of the radionuclide distribution, but also to improve the reconstruction at each single time point by enforcing temporal consistency. This is achieved via optimal transport regularization where in principle, among all possible temporally evolving radionuclide distributions consistent with the PET measurement, the one is chosen with least kinetic motion energy. The reconstruction is found by convex optimization so that there is no dependence on the initialization of the method. We study its behaviour on simulated data of a human PET system and demonstrate its robustness even in settings with very low radioactivity. In contrast to previously reported cell tracking algorithms, our technique is oblivious to the number of tracked cells. Without any additional complexity one or multiple cells can be reconstructed, and the model automatically determines the number of particles. For instance, four radiolabelled cells moving at a velocity of 3.1 mm/s and a PET recorded count rate of 1.1 cps (for each cell) could be simultaneously tracked with a tracking accuracy of 5.3 mm inside a simulated human body.
Tasks	Image Reconstruction
Published	2019-02-20
URL	https://arxiv.org/abs/1902.07521v2
PDF	https://arxiv.org/pdf/1902.07521v2.pdf
PWC	https://paperswithcode.com/paper/dynamic-cell-imaging-in-pet-with-optimal
Repo
Framework

Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems


Title	Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems
Authors	Jaynta Mandi, Emir Demirović, Peter. J Stuckey, Tias Guns
Abstract	Combinatorial optimization assumes that all parameters of the optimization problem, e.g. the weights in the objective function is fixed. Often, these weights are mere estimates and increasingly machine learning techniques are used to for their estimation. Recently, Smart Predict and Optimize (SPO) has been proposed for problems with a linear objective function over the predictions, more specifically linear programming problems. It takes the regret of the predictions on the linear problem into account, by repeatedly solving it during learning. We investigate the use of SPO to solve more realistic discrete optimization problems. The main challenge is the repeated solving of the optimization problem. To this end, we investigate ways to relax the problem as well as warmstarting the learning and the solving. Our results show that even for discrete problems it often suffices to train by solving the relaxation in the SPO loss. Furthermore, this approach outperforms, for most instances, the state-of-the-art approach of Wilder, Dilkina, and Tambe. We experiment with weighted knapsack problems as well as complex scheduling problems and show for the first time that a predict-and-optimize approach can successfully be used on large-scale combinatorial optimization problems.
Tasks	Combinatorial Optimization
Published	2019-11-22
URL	https://arxiv.org/abs/1911.10092v1
PDF	https://arxiv.org/pdf/1911.10092v1.pdf
PWC	https://paperswithcode.com/paper/smart-predict-and-optimize-for-hard
Repo
Framework

Intelligent Processing in Vehicular Ad hoc Networks: a Survey


Title	Intelligent Processing in Vehicular Ad hoc Networks: a Survey
Authors	Yang Liu
Abstract	The intelligent Processing technique is more and more attractive to researchers due to its ability to deal with key problems in Vehicular Ad hoc networks. However, several problems in applying intelligent processing technologies in VANETs remain open. The existing applications are comprehensively reviewed and discussed, and classified into different categories in this paper. Their strategies, advantages/disadvantages, and performances are elaborated. By generalizing different tactics in various applications related to different scenarios of VANETs and evaluating their performances, several promising directions for future research have been suggested.
Tasks
Published	2019-03-28
URL	http://arxiv.org/abs/1903.11916v1
PDF	http://arxiv.org/pdf/1903.11916v1.pdf
PWC	https://paperswithcode.com/paper/intelligent-processing-in-vehicular-ad-hoc
Repo
Framework

Weakly Supervised Deep Learning Approach in Streaming Environments


Title	Weakly Supervised Deep Learning Approach in Streaming Environments
Authors	Mahardhika Pratama, Andri Ashfahani, Mohamad Abdul Hady
Abstract	The feasibility of existing data stream algorithms is often hindered by the weakly supervised condition of data streams. A self-evolving deep neural network, namely Parsimonious Network (ParsNet), is proposed as a solution to various weakly-supervised data stream problems. A self-labelling strategy with hedge (SLASH) is proposed in which its auto-correction mechanism copes with \textit{the accumulation of mistakes} significantly affecting the model’s generalization. ParsNet is developed from a closed-loop configuration of the self-evolving generative and discriminative training processes exploiting shared parameters in which its structure flexibly grows and shrinks to overcome the issue of concept drift with/without labels. The numerical evaluation has been performed under two challenging problems, namely sporadic access to ground truth and infinitely delayed access to the ground truth. Our numerical study shows the advantage of ParsNet with a substantial margin from its counterparts in the high-dimensional data streams and infinite delay simulation protocol. To support the reproducible research initiative, the source code of ParsNet along with supplementary materials are made available at https://bit.ly/2qNW7p4.
Tasks
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00847v3
PDF	https://arxiv.org/pdf/1911.00847v3.pdf
PWC	https://paperswithcode.com/paper/weakly-supervised-deep-learning-approach-in
Repo
Framework

An End-to-End HW/SW Co-Design Methodology to Design Efficient Deep Neural Network Systems using Virtual Models


Title	An End-to-End HW/SW Co-Design Methodology to Design Efficient Deep Neural Network Systems using Virtual Models
Authors	Michael J. Klaiber, Sebastian Vogel, Axel Acosta, Robert Korn, Leonardo Ecco, Kristine Back, Andre Guntoro, Ingo Feldner
Abstract	End-to-end performance estimation and measurement of deep neural network (DNN) systems become more important with increasing complexity of DNN systems consisting of hardware and software components. The methodology proposed in this paper aims at a reduced turn-around time for evaluating different design choices of hardware and software components of DNN systems. This reduction is achieved by moving the performance estimation from the implementation phase to the concept phase by employing virtual hardware models instead of gathering measurement results from physical prototypes. Deep learning compilers introduce hardware-specific transformations and are, therefore, considered a part of the design flow of virtual system models to extract end-to-end performance estimations. To validate the run-time accuracy of the proposed methodology, a system processing the DilatedVGG DNN is realized both as virtual system model and as hardware implementation. The results show that up to 92 % accuracy can be reached in predicting the processing time of the DNN inference.
Tasks
Published	2019-10-25
URL	https://arxiv.org/abs/1910.11632v2
PDF	https://arxiv.org/pdf/1910.11632v2.pdf
PWC	https://paperswithcode.com/paper/an-end-to-end-hwsw-co-design-methodology-to
Repo
Framework

Feature Engineering Combined with 1 D Convolutional Neural Network for Improved Mortality Prediction


Title	Feature Engineering Combined with 1 D Convolutional Neural Network for Improved Mortality Prediction
Authors	Saumil Maheshwari, Rohit Verma, Anupam Shukla, Ritu Tiwari, Rishu Garg
Abstract	The intensive care units (ICUs) are responsible for generating a wealth of useful data in the form of Electronic Health Record (EHR). This data allows for the development of a prediction tool with perfect knowledge backing. We aimed to build a mortality prediction model on 2012 Physionet Challenge mortality prediction database of 4000 patients admitted in ICU. The challenges in the dataset, such as high dimensionality, imbalanced distribution, and missing values were tackled with analytical methods and tools via feature engineering and new variable construction. The objective of the research is to utilize the relations among the clinical variables and construct new variables which would establish the effectiveness of 1-Dimensional Convolutional Neural Network (1- D CNN) with constructed features. Its performance with the traditional machine learning algorithms like XGBoost classifier, Support Vector Machine (SVM), K-Neighbours Classifier (K-NN), and Random Forest Classifier (RF) is compared for Area Under Curve (AUC). The investigation reveals the best AUC of 0.848 using 1-D CNN model.
Tasks	Feature Engineering, Mortality Prediction
Published	2019-12-08
URL	https://arxiv.org/abs/1912.03789v1
PDF	https://arxiv.org/pdf/1912.03789v1.pdf
PWC	https://paperswithcode.com/paper/feature-engineering-combined-with-1-d
Repo
Framework

A Hybrid Retrieval-Generation Neural Conversation Model


Title	A Hybrid Retrieval-Generation Neural Conversation Model
Authors	Liu Yang, Junjie Hu, Minghui Qiu, Chen Qu, Jianfeng Gao, W. Bruce Croft, Xiaodong Liu, Yelong Shen, Jingjing Liu
Abstract	Intelligent personal assistant systems that are able to have multi-turn conversations with human users are becoming increasingly popular. Most previous research has been focused on using either retrieval-based or generation-based methods to develop such systems. Retrieval-based methods have the advantage of returning fluent and informative responses with great diversity. However, the performance of the methods is limited by the size of the response repository. On the other hand, generation-based methods can produce highly coherent responses on any topics. But the generated responses are often generic and not informative due to the lack of grounding knowledge. In this paper, we propose a hybrid neural conversation model that combines the merits of both response retrieval and generation methods. Experimental results on Twitter and Foursquare data show that the proposed model outperforms both retrieval-based methods and generation-based methods (including a recently proposed knowledge-grounded neural conversation model) under both automatic evaluation metrics and human evaluation. We hope that the findings in this study provide new insights on how to integrate text retrieval and text generation models for building conversation systems.
Tasks	Text Generation
Published	2019-04-19
URL	https://arxiv.org/abs/1904.09068v2
PDF	https://arxiv.org/pdf/1904.09068v2.pdf
PWC	https://paperswithcode.com/paper/a-hybrid-retrieval-generation-neural
Repo
Framework