January 31, 2020

3538 words 17 mins read

Paper Group ANR 152

Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning. Light-weighted Saliency Detection with Distinctively Lower Memory Cost and Model Size. Modeling and Optimization with Gaussian Processes in Reduced Eigenbases – Extended Version. embComp: Visual Interactive Comparison of Vector Embeddings. IEDM, an Ontology fo …

Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning


Title	Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning
Authors	Tailai Wen, Roy Keyes
Abstract	Time series anomaly detection plays a critical role in automated monitoring systems. Most previous deep learning efforts related to time series anomaly detection were based on recurrent neural networks (RNN). In this paper, we propose a time series segmentation approach based on convolutional neural networks (CNN) for anomaly detection. Moreover, we propose a transfer learning framework that pretrains a model on a large-scale synthetic univariate time series data set and then fine-tunes its weights on small-scale, univariate or multivariate data sets with previously unseen classes of anomalies. For the multivariate case, we introduce a novel network architecture. The approach was tested on multiple synthetic and real data sets successfully.
Tasks	Anomaly Detection, Time Series, Transfer Learning
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13628v1
PDF	https://arxiv.org/pdf/1905.13628v1.pdf
PWC	https://paperswithcode.com/paper/time-series-anomaly-detection-using
Repo
Framework

Light-weighted Saliency Detection with Distinctively Lower Memory Cost and Model Size


Title	Light-weighted Saliency Detection with Distinctively Lower Memory Cost and Model Size
Authors	Shanghua Xiao
Abstract	Deep neural networks (DNNs) based saliency detection approaches have succeed in recent years, and improved the performance by a great margin via increasingly sophisticated network architecture. Despite the performance improvement, the computational cost is excessively high for such low level visual task. In this work, we propose a light-weighted saliency detection approach with distinctively lower runtime memory cost and model size. We evaluated the performance of our approach on multiple benchmark datasets, and achieved competitive results comparing with state-of-the-art methods on multiple metrics. We also evaluated the computational cost of our approach with multiple measurements. The runtime memory cost of our approach is 42 to 99 times fewer comparing with the previous DNNs based methods. The model size of our approach is 63 to 129 times smaller, and takes less than 1 Megabytes storage space with out any deep compression technique.
Tasks	Saliency Detection
Published	2019-01-15
URL	http://arxiv.org/abs/1901.05002v1
PDF	http://arxiv.org/pdf/1901.05002v1.pdf
PWC	https://paperswithcode.com/paper/light-weighted-saliency-detection-with
Repo
Framework

Modeling and Optimization with Gaussian Processes in Reduced Eigenbases – Extended Version


Title	Modeling and Optimization with Gaussian Processes in Reduced Eigenbases – Extended Version
Authors	David Gaudrie, Rodolphe Le Riche, Victor Picheny, Benoit Enaux, Vincent Herbert
Abstract	Parametric shape optimization aims at minimizing an objective function f(x) where x are CAD parameters. This task is difficult when f is the output of an expensive-to-evaluate numerical simulator and the number of CAD parameters is large. Most often, the set of all considered CAD shapes resides in a manifold of lower effective dimension in which it is preferable to build the surrogate model and perform the optimization. In this work, we uncover the manifold through a high-dimensional shape mapping and build a new coordinate system made of eigenshapes. The surrogate model is learned in the space of eigenshapes: a regularized likelihood maximization provides the most relevant dimensions for the output. The final surrogate model is detailed (anisotropic) with respect to the most sensitive eigenshapes and rough (isotropic) in the remaining dimensions. Last, the optimization is carried out with a focus on the critical dimensions, the remaining ones being coarsely optimized through a random embedding and the manifold being accounted for through a replication strategy. At low budgets, the methodology leads to a more accurate model and a faster optimization than the classical approach of directly working with the CAD parameters.
Tasks	Gaussian Processes
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11272v1
PDF	https://arxiv.org/pdf/1908.11272v1.pdf
PWC	https://paperswithcode.com/paper/modeling-and-optimization-with-gaussian
Repo
Framework

embComp: Visual Interactive Comparison of Vector Embeddings


Title	embComp: Visual Interactive Comparison of Vector Embeddings
Authors	Florian Heimerl, Christoph Kralj, Torsten Möller, Michael Gleicher
Abstract	This work introduces embComp, a novel approach for comparing two embeddings that capture the similarity between objects, such as word and document embeddings. We survey scenarios where comparing these embedding spaces is useful. From those scenarios, we derive common tasks, introduce visual analysis methods that support these tasks, and combine them into a comprehensive system. One of embComp’s central features are overview visualizations that are based on metrics for measuring differences in local structure around objects. Summarizing these local metrics over the embeddings provides global overviews of similarities and differences. These global views enable a user to identify sets of interesting objects whose relationships in the embeddings can be compared. Detail views allow comparison of the local structure around selected objects and relating this local information to the global views. Integrating and connecting all of these components, \sysname supports a range of analysis workflows that help understand similarities and differences between embedding spaces. We assess our approach by applying it in several use cases, including understanding corpora differences via word vector embeddings, and understanding algorithmic differences in generating embeddings.
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01542v1
PDF	https://arxiv.org/pdf/1911.01542v1.pdf
PWC	https://paperswithcode.com/paper/embcomp-visual-interactive-comparison-of
Repo
Framework

IEDM, an Ontology for Irradiation Experiment Data Management


Title	IEDM, an Ontology for Irradiation Experiment Data Management
Authors	Blerina Gkotse, Pierre Jouvelot, Federico Ravotti
Abstract	Irradiation experiments (IE) are an essential step in the development of High-Energy Physics (HEP) particle accelerators and detectors. They assess the radiation hardness of materials used in HEP experimental devices by simulating, in a short time, the common long-term degradation effects due to their bombardment by high-energy particles. IEs are also used in other scientific and industrial fields such as medicine (e.g., for cancer treatment, medical imaging, etc.), space/avionics (e.g., for radiation testing of payload equipment) as well as in industry (e.g., for food sterilization). Usually carried out with ionizing radiation, these complex processes require highly specialized infrastructures: the irradiation facilities. Currently, hundreds of such facilities exist worldwide. To help develop best practices and promote computer-assisted handling and management of IEs, we introduce IEDM, a new OWL-based Irradiation Experiment Data Management ontology. This paper provides an overview of the classes and properties of IEDM. Since one of the key design choices for IEDM was to maximize the reuse of existing foundational ontologies such as the Ontology of Scientific Experiments (EXPO), the Ontology of Units of Measure (OM) and the Friend-of-a-Friend Ontology (FOAF), we discuss the methodological issues of the integration of IEDM with these imported ontologies. We illustrate the use of IEDM via an actual IE recently performed at IRRAD, the CERN proton irradiation facility. Finally, we discuss other motivations for this work, including the use of IEDM for the generation of user interfaces for IE management, and their impact on our methodology.
Tasks
Published	2019-01-16
URL	http://arxiv.org/abs/1901.05233v1
PDF	http://arxiv.org/pdf/1901.05233v1.pdf
PWC	https://paperswithcode.com/paper/iedm-an-ontology-for-irradiation-experiment
Repo
Framework

Negative sampling in semi-supervised learning


Title	Negative sampling in semi-supervised learning
Authors	John Chen, Vatsal Shah, Anastasios Kyrillidis
Abstract	We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS3L loss to MixMatch, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets.
Tasks
Published	2019-11-12
URL	https://arxiv.org/abs/1911.05166v1
PDF	https://arxiv.org/pdf/1911.05166v1.pdf
PWC	https://paperswithcode.com/paper/negative-sampling-in-semi-supervised-learning
Repo
Framework

A Two Stage GAN for High Resolution Retinal Image Generation and Segmentation


Title	A Two Stage GAN for High Resolution Retinal Image Generation and Segmentation
Authors	Paolo Andreini, Simone Bonechi, Monica Bianchini, Alessandro Mecocci, Franco Scarselli, Andrea Sodi
Abstract	In recent years, the use of deep learning is becoming increasingly popular in computer vision. However, the effective training of deep architectures usually relies on huge sets of annotated data. This is critical in the medical field where it is difficult and expensive to obtain annotated images. In this paper, we use Generative Adversarial Networks (GANs) for synthesizing high quality retinal images, along with the corresponding semantic label-maps, to be used instead of real images during the training process. Differently from other previous proposals, we suggest a two step approach: first, a progressively growing GAN is trained to generate the semantic label-maps, which describe the blood vessel structure (i.e. vasculature); second, an image-to-image translation approach is used to obtain realistic retinal images from the generated vasculature. By using only a handful of training samples, our approach generates realistic high resolution images, that can be effectively used to enlarge small available datasets. Comparable results have been obtained employing the generated images in place of real data during training. The practical viability of the proposed approach has been demonstrated by applying it on two well established benchmark sets for retinal vessel segmentation, both containing a very small number of training samples. Our method obtained better performances with respect to state-of-the-art techniques.
Tasks	Image Generation, Image-to-Image Translation, Retinal Vessel Segmentation
Published	2019-07-29
URL	https://arxiv.org/abs/1907.12296v1
PDF	https://arxiv.org/pdf/1907.12296v1.pdf
PWC	https://paperswithcode.com/paper/a-two-stage-gan-for-high-resolution-retinal
Repo
Framework

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures


Title	Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures
Authors	Johannes Günther, Nadia M. Ady, Alex Kearney, Michael R. Dawson, Patrick M. Pilarski
Abstract	Predictions and predictive knowledge have seen recent success in improving not only robot control but also other applications ranging from industrial process control to rehabilitation. A property that makes these predictive approaches well suited for robotics is that they can be learned online and incrementally through interaction with the environment. However, a remaining challenge for many prediction-learning approaches is an appropriate choice of prediction-learning parameters, especially parameters that control the magnitude of a learning machine’s updates to its predictions (the learning rate or step size). To begin to address this challenge, we examine the use of online step-size adaptation using a sensor-rich robotic arm. Our method of choice, Temporal-Difference Incremental Delta-Bar-Delta (TIDBD), learns and adapts step sizes on a feature level; importantly, TIDBD allows step-size tuning and representation learning to occur at the same time. We show that TIDBD is a practical alternative for classic Temporal-Difference (TD) learning via an extensive parameter search. Both approaches perform comparably in terms of predicting future aspects of a robotic data stream. Furthermore, the use of a step-size adaptation method like TIDBD appears to allow a system to automatically detect and characterize common sensor failures in a robotic application. Together, these results promise to improve the ability of robotic devices to learn from interactions with their environments in a robust way, providing key capabilities for autonomous agents and robots.
Tasks	Representation Learning
Published	2019-08-15
URL	https://arxiv.org/abs/1908.05751v2
PDF	https://arxiv.org/pdf/1908.05751v2.pdf
PWC	https://paperswithcode.com/paper/examining-the-use-of-temporal-difference
Repo
Framework

Almost Optimal Tensor Sketch


Title	Almost Optimal Tensor Sketch
Authors	Thomas D. Ahle, Jakob B. T. Knudsen
Abstract	We construct a matrix $M\in R^{m\otimes d^c}$ with just $m=O(c,\lambda,\varepsilon^{-2}\text{poly}\log1/\varepsilon\delta)$ rows, which preserves the norm $\Mx_2=(1\pm\varepsilon)\x_2$ of all $x$ in any given $\lambda$ dimensional subspace of $ R^d$ with probability at least $1-\delta$. This matrix can be applied to tensors $x^{(1)}\otimes\dots\otimes x^{(c)}\in R^{d^c}$ in $O(c, m \min{d,m})$ time – hence the name “Tensor Sketch”. (Here $x\otimes y = \text{asvec}(xy^T) = [x_1y_1, x_1y_2,\dots,x_1y_m,x_2y_1,\dots,x_ny_m]\in R^{nm}$.) This improves upon earlier Tensor Sketch constructions by Pagh and Pham~[TOCT 2013, SIGKDD 2013] and Avron et al.~[NIPS 2014] which require $m=\Omega(3^c\lambda^2\delta^{-1})$ rows for the same guarantees. The factors of $\lambda$, $\varepsilon^{-2}$ and $\log1/\delta$ can all be shown to be necessary making our sketch optimal up to log factors. With another construction we get $\lambda$ times more rows $m=\tilde O(c,\lambda^2,\varepsilon^{-2}(\log1/\delta)^3)$, but the matrix can be applied to any vector $x^{(1)}\otimes\dots\otimes x^{(c)}\in R^{d^c}$ in just $\tilde O(c, (d+m))$ time. This matches the application time of Tensor Sketch while still improving the exponential dependencies in $c$ and $\log1/\delta$. Technically, we show two main lemmas: (1) For many Johnson Lindenstrauss (JL) constructions, if $Q,Q’\in R^{m\times d}$ are independent JL matrices, the element-wise product $Qx \circ Q’y$ equals $M(x\otimes y)$ for some $M\in R^{m\times d^2}$ which is itself a JL matrix. (2) If $M^{(i)}\in R^{m\times md}$ are independent JL matrices, then $M^{(1)}(x \otimes (M^{(2)}y \otimes \dots)) = M(x\otimes y\otimes \dots)$ for some $M\in R^{m\times d^c}$ which is itself a JL matrix. Combining these two results give an efficient sketch for tensors of any size.
Tasks
Published	2019-09-03
URL	https://arxiv.org/abs/1909.01821v1
PDF	https://arxiv.org/pdf/1909.01821v1.pdf
PWC	https://paperswithcode.com/paper/almost-optimal-tensor-sketch
Repo
Framework

Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space


Title	Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
Authors	Taiji Suzuki, Atsushi Nitanda
Abstract	Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on {\it anisotropic Besov spaces}. The anisotropic Besov space is characterized by direction-dependent smoothness and includes several function classes that have been investigated thus far. We demonstrate that the approximation error and estimation error of deep learning only depend on the average value of the smoothness parameters in all directions. Consequently, the curse of dimensionality can be avoided if the smoothness of the target function is highly anisotropic. Unlike existing studies, our analysis does not require a low-dimensional structure of the input data. We also investigate the minimax optimality of deep learning and compare its performance with that of the kernel method (more generally, linear estimators). The results show that deep learning has better dependence on the input dimensionality if the target function possesses anisotropic smoothness, and it achieves an adaptive rate for functions with spatially inhomogeneous smoothness.
Tasks
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12799v1
PDF	https://arxiv.org/pdf/1910.12799v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-is-adaptive-to-intrinsic
Repo
Framework

Conservation AI: Live Stream Analysis for the Detection of Endangered Species Using Convolutional Neural Networks and Drone Technology


Title	Conservation AI: Live Stream Analysis for the Detection of Endangered Species Using Convolutional Neural Networks and Drone Technology
Authors	C. Chalmers, P. Fergus, Serge Wich, Aday Curbelo Montanez
Abstract	Many different species are adversely affected by poaching. In response to this escalating crisis, efforts to stop poaching using hidden cameras, drones and DNA tracking have been implemented with varying degrees of success. Limited resources, costs and logistical limitations are often the cause of most unsuccessful poaching interventions. The study presented in this paper outlines a flexible and interoperable framework for the automatic detection of animals and poaching activity to facilitate early intervention practices. Using a robust deep learning pipeline, a convolutional neural network is trained and implemented to detect rhinos and cars (considered an important tool in poaching for fast access and artefact transportation in natural habitats) in the study, that are found within live video streamed from drones Transfer learning with the Faster RCNN Resnet 101 is performed to train a custom model with 350 images of rhinos and 350 images of cars. Inference is performed using a frame sampling technique to address the required trade-off control precision and processing speed and maintain synchronisation with the live feed. Inference models are hosted on a web platform using flask web serving, OpenCV and TensorFlow 1.13. Video streams are transmitted from a DJI Mavic Pro 2 drone using the Real-Time Messaging Protocol (RMTP). The best trained Faster RCNN model achieved a mAP of 0.83 @IOU 0.50 and 0.69 @IOU 0.75 respectively. In comparison an SSD-mobilenetmodel trained under the same experimental conditions achieved a mAP of 0.55 @IOU .50 and 0.27 @IOU 0.75.The results demonstrate that using a FRCNN and off-the-shelf drones is a promising and scalable option for a range of conservation projects.
Tasks	Transfer Learning
Published	2019-10-16
URL	https://arxiv.org/abs/1910.07360v1
PDF	https://arxiv.org/pdf/1910.07360v1.pdf
PWC	https://paperswithcode.com/paper/conservation-ai-live-stream-analysis-for-the
Repo
Framework

Deep Single Image Deraining Via Estimating Transmission and Atmospheric Light in rainy Scenes


Title	Deep Single Image Deraining Via Estimating Transmission and Atmospheric Light in rainy Scenes
Authors	Yinglong Wang, Qinfeng Shi, Ehsan Abbasnejad, Chao Ma, Xiaoping Ma, Bing Zeng
Abstract	Rain removal in images/videos is still an important task in computer vision field and attracting attentions of more and more people. Traditional methods always utilize some incomplete priors or filters (e.g. guided filter) to remove rain effect. Deep learning gives more probabilities to better solve this task. However, they remove rain either by evaluating background from rainy image directly or learning a rain residual first then subtracting the residual to obtain a clear background. No other models are used in deep learning based de-raining methods to remove rain and obtain other information about rainy scenes. In this paper, we utilize an extensively-used image degradation model which is derived from atmospheric scattering principles to model the formation of rainy images and try to learn the transmission, atmospheric light in rainy scenes and remove rain further. To reach this goal, we propose a robust evaluation method of global atmospheric light in a rainy scene. Instead of using the estimated atmospheric light directly to learn a network to calculate transmission, we utilize it as ground truth and design a simple but novel triangle-shaped network structure to learn atmospheric light for every rainy image, then fine-tune the network to obtain a better estimation of atmospheric light during the training of transmission network. Furthermore, more efficient ShuffleNet Units are utilized in transmission network to learn transmission map and the de-raining image is then obtained by the image degradation model. By subjective and objective comparisons, our method outperforms the selected state-of-the-art works.
Tasks	Rain Removal, Single Image Deraining
Published	2019-06-22
URL	https://arxiv.org/abs/1906.09433v1
PDF	https://arxiv.org/pdf/1906.09433v1.pdf
PWC	https://paperswithcode.com/paper/deep-single-image-deraining-via-estimating
Repo
Framework

Semi-supervised Stochastic Multi-Domain Learning using Variational Inference


Title	Semi-supervised Stochastic Multi-Domain Learning using Variational Inference
Authors	Yitong Li, Timothy Baldwin, Trevor Cohn
Abstract	Supervised models of NLP rely on large collections of text which closely resemble the intended testing setting. Unfortunately matching text is often not available in sufficient quantity, and moreover, within any domain of text, data is often highly heterogenous. In this paper we propose a method to distill the important domain signal as part of a multi-domain learning system, using a latent variable model in which parts of a neural model are stochastically gated based on the inferred domain. We compare the use of discrete versus continuous latent variables, operating in a domain-supervised or a domain semi-supervised setting, where the domain is known only for a subset of training inputs. We show that our model leads to substantial performance improvements over competitive benchmark domain adaptation methods, including methods using adversarial learning.
Tasks	Domain Adaptation
Published	2019-06-07
URL	https://arxiv.org/abs/1906.02897v1
PDF	https://arxiv.org/pdf/1906.02897v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-stochastic-multi-domain
Repo
Framework

Global forensic geolocation with deep neural networks


Title	Global forensic geolocation with deep neural networks
Authors	Neal S. Grantham, Brian J. Reich, Eric B. Laber, Krishna Pacifici, Robert R. Dunn, Noah Fierer, Matthew Gebert, Julia S. Allwood, Seth A. Faith
Abstract	An important problem in forensic analyses is identifying the provenance of materials at a crime scene, such as biological material on a piece of clothing. This procedure, known as geolocation, is conventionally guided by expert knowledge of the biological evidence and therefore tends to be application-specific, labor-intensive, and subjective. Purely data-driven methods have yet to be fully realized due in part to the lack of a sufficiently rich data source. However, high-throughput sequencing technologies are able to identify tens of thousands of microbial taxa using DNA recovered from a single swab collected from nearly any object or surface. We present a new algorithm for geolocation that aggregates over an ensemble of deep neural network classifiers trained on randomly-generated Voronoi partitions of a spatial domain. We apply the algorithm to fungi present in each of 1300 dust samples collected across the continental United States and then to a global dataset of dust samples from 28 countries. Our algorithm makes remarkably good point predictions with more than half of the geolocation errors under 100 kilometers for the continental analysis and nearly 90% classification accuracy of a sample’s country of origin for the global analysis. We suggest that the effectiveness of this model sets the stage for a new, quantitative approach to forensic geolocation.
Tasks
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11765v1
PDF	https://arxiv.org/pdf/1905.11765v1.pdf
PWC	https://paperswithcode.com/paper/global-forensic-geolocation-with-deep-neural
Repo
Framework

CycleGAN with a Blur Kernel for Deconvolution Microscopy: Optimal Transport Geometry


Title	CycleGAN with a Blur Kernel for Deconvolution Microscopy: Optimal Transport Geometry
Authors	Sungjun Lim, Sang-Eun Lee, Sunghoe Chang, Jong Chul Ye
Abstract	Deconvolution microscopy has been extensively used to improve the resolution of the widefield fluorescent microscopy. However, classical deconvolution approaches require the measurement or estimation of the point spread function (PSF), and are usually computationally expensive. Recently, convolutional neural network (CNN) approaches have been extensively studied as fast and high performance alternatives. Unfortunately, the CNN approaches usually require matched high resolution images for supervised training. In this paper, we present a novel unsupervised cycle-consistent generative adversarial network (cycleGAN) with a linear blur kernel, which can be used for both blind- and non-blind image deconvolution. In contrast to the conventional cycleGAN approaches that require two generators, the proposed cycleGAN approach needs only a single generator, which significantly improves the robustness of network training. We show that the proposed architecture is indeed a dual formulation of an optimal transport problem that uses a special form of penalized least squares as transport cost. Experimental results using simulated and real experimental data confirm the efficacy of the algorithm.
Tasks	Image Deconvolution
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09414v2
PDF	https://arxiv.org/pdf/1908.09414v2.pdf
PWC	https://paperswithcode.com/paper/cyclegan-with-a-blur-kernel-for-deconvolution
Repo
Framework