July 29, 2019

2916 words 14 mins read

Paper Group AWR 202

Lossy Image Compression with Compressive Autoencoders. Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning. Static Gesture Recognition using Leap Motion. Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation. Investigation of Different Skeleton Features for CNN-based 3D Action Recogni …

Lossy Image Compression with Compressive Autoencoders


Title	Lossy Image Compression with Compressive Autoencoders
Authors	Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár
Abstract	We propose a new approach to the problem of optimizing autoencoders for lossy image compression. New media formats, changing hardware technology, as well as diverse requirements and content types create a need for compression algorithms which are more flexible than existing codecs. Autoencoders have the potential to address this need, but are difficult to optimize directly due to the inherent non-differentiabilty of the compression loss. We here show that minimal changes to the loss are sufficient to train deep autoencoders competitive with JPEG 2000 and outperforming recently proposed approaches based on RNNs. Our network is furthermore computationally efficient thanks to a sub-pixel architecture, which makes it suitable for high-resolution images. This is in contrast to previous work on autoencoders for compression using coarser approximations, shallower architectures, computationally expensive methods, or focusing on small images.
Tasks	Image Compression
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00395v1
PDF	http://arxiv.org/pdf/1703.00395v1.pdf
PWC	https://paperswithcode.com/paper/lossy-image-compression-with-compressive
Repo	https://github.com/alexandru-dinu/cae
Framework	pytorch

Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning


Title	Deep Packet: A Novel Approach For Encrypted Traffic Classification Using Deep Learning
Authors	Mohammad Lotfollahi, Ramin Shirali Hossein Zade, Mahdi Jafari Siavoshani, Mohammdsadegh Saberian
Abstract	Internet traffic classification has become more important with rapid growth of current Internet network and online applications. There have been numerous studies on this topic which have led to many different approaches. Most of these approaches use predefined features extracted by an expert in order to classify network traffic. In contrast, in this study, we propose a \emph{deep learning} based approach which integrates both feature extraction and classification phases into one system. Our proposed scheme, called “Deep Packet,” can handle both \emph{traffic characterization} in which the network traffic is categorized into major classes (\eg, FTP and P2P) and application identification in which end-user applications (\eg, BitTorrent and Skype) identification is desired. Contrary to most of the current methods, Deep Packet can identify encrypted traffic and also distinguishes between VPN and non-VPN network traffic. After an initial pre-processing phase on data, packets are fed into Deep Packet framework that embeds stacked autoencoder and convolution neural network in order to classify network traffic. Deep packet with CNN as its classification model achieved recall of $0.98$ in application identification task and $0.94$ in traffic categorization task. To the best of our knowledge, Deep Packet outperforms all of the proposed classification methods on UNB ISCX VPN-nonVPN dataset.
Tasks
Published	2017-09-08
URL	http://arxiv.org/abs/1709.02656v3
PDF	http://arxiv.org/pdf/1709.02656v3.pdf
PWC	https://paperswithcode.com/paper/deep-packet-a-novel-approach-for-encrypted
Repo	https://github.com/YuriBogdanov/DeepPacket
Framework	pytorch

Static Gesture Recognition using Leap Motion


Title	Static Gesture Recognition using Leap Motion
Authors	Babak Toghiani-Rizi, Christofer Lind, Maria Svensson, Marcus Windmark
Abstract	In this report, an automated bartender system was developed for making orders in a bar using hand gestures. The gesture recognition of the system was developed using Machine Learning techniques, where the model was trained to classify gestures using collected data. The final model used in the system reached an average accuracy of 95%. The system raised ethical concerns both in terms of user interaction and having such a system in a real world scenario, but it could initially work as a complement to a real bartender.
Tasks	Gesture Recognition
Published	2017-05-16
URL	http://arxiv.org/abs/1705.05884v1
PDF	http://arxiv.org/pdf/1705.05884v1.pdf
PWC	https://paperswithcode.com/paper/static-gesture-recognition-using-leap-motion
Repo	https://github.com/windmark/static-gesture-recognition
Framework	none

Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation


Title	Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation
Authors	Sarah Tan, Rich Caruana, Giles Hooker, Yin Lou
Abstract	Black-box risk scoring models permeate our lives, yet are typically proprietary or opaque. We propose Distill-and-Compare, a model distillation and comparison approach to audit such models. To gain insight into black-box models, we treat them as teachers, training transparent student models to mimic the risk scores assigned by black-box models. We compare the student model trained with distillation to a second un-distilled transparent model trained on ground-truth outcomes, and use differences between the two models to gain insight into the black-box model. Our approach can be applied in a realistic setting, without probing the black-box model API. We demonstrate the approach on four public data sets: COMPAS, Stop-and-Frisk, Chicago Police, and Lending Club. We also propose a statistical test to determine if a data set is missing key features used to train the black-box model. Our test finds that the ProPublica data is likely missing key feature(s) used in COMPAS.
Tasks
Published	2017-10-17
URL	http://arxiv.org/abs/1710.06169v4
PDF	http://arxiv.org/pdf/1710.06169v4.pdf
PWC	https://paperswithcode.com/paper/distill-and-compare-auditing-black-box-models
Repo	https://github.com/shftan/auditblackbox
Framework	none

Investigation of Different Skeleton Features for CNN-based 3D Action Recognition


Title	Investigation of Different Skeleton Features for CNN-based 3D Action Recognition
Authors	Zewei Ding, Pichao Wang, Philip O. Ogunbona, Wanqing Li
Abstract	Deep learning techniques are being used in skeleton based action recognition tasks and outstanding performance has been reported. Compared with RNN based methods which tend to overemphasize temporal information, CNN-based approaches can jointly capture spatio-temporal information from texture color images encoded from skeleton sequences. There are several skeleton-based features that have proven effective in RNN-based and handcrafted-feature-based methods. However, it remains unknown whether they are suitable for CNN-based approaches. This paper proposes to encode five spatial skeleton features into images with different encoding methods. In addition, the performance implication of different joints used for feature extraction is studied. The proposed method achieved state-of-the-art performance on NTU RGB+D dataset for 3D human action analysis. An accuracy of 75.32% was achieved in Large Scale 3D Human Activity Analysis Challenge in Depth Videos.
Tasks	3D Human Action Recognition, Skeleton Based Action Recognition, Temporal Action Localization
Published	2017-05-02
URL	http://arxiv.org/abs/1705.00835v1
PDF	http://arxiv.org/pdf/1705.00835v1.pdf
PWC	https://paperswithcode.com/paper/investigation-of-different-skeleton-features
Repo	https://github.com/dzwallkilled/IEforAR
Framework	none

Unrolled Optimization with Deep Priors


Title	Unrolled Optimization with Deep Priors
Authors	Steven Diamond, Vincent Sitzmann, Felix Heide, Gordon Wetzstein
Abstract	A broad class of problems at the core of computational imaging, sensing, and low-level computer vision reduces to the inverse problem of extracting latent images that follow a prior distribution, from measurements taken under a known physical image formation model. Traditionally, hand-crafted priors along with iterative optimization methods have been used to solve such problems. In this paper we present unrolled optimization with deep priors, a principled framework for infusing knowledge of the image formation into deep networks that solve inverse problems in imaging, inspired by classical iterative methods. We show that instances of the framework outperform the state-of-the-art by a substantial margin for a wide variety of imaging problems, such as denoising, deblurring, and compressed sensing magnetic resonance imaging (MRI). Moreover, we conduct experiments that explain how the framework is best used and why it outperforms previous methods.
Tasks	Deblurring, Denoising
Published	2017-05-22
URL	http://arxiv.org/abs/1705.08041v2
PDF	http://arxiv.org/pdf/1705.08041v2.pdf
PWC	https://paperswithcode.com/paper/unrolled-optimization-with-deep-priors
Repo	https://github.com/Zhengqi-Wu/Unrolled-optimization-with-deep-priors
Framework	none

Countering Adversarial Images using Input Transformations


Title	Countering Adversarial Images using Input Transformations
Authors	Chuan Guo, Mayank Rana, Moustapha Cisse, Laurens van der Maaten
Abstract	This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier. Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images. The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methods
Tasks	Adversarial Defense, Image Classification
Published	2017-10-31
URL	http://arxiv.org/abs/1711.00117v3
PDF	http://arxiv.org/pdf/1711.00117v3.pdf
PWC	https://paperswithcode.com/paper/countering-adversarial-images-using-input
Repo	https://github.com/facebookresearch/adversarial_image_defenses
Framework	pytorch

Continual Learning Through Synaptic Intelligence


Title	Continual Learning Through Synaptic Intelligence
Authors	Friedemann Zenke, Ben Poole, Surya Ganguli
Abstract	While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many tasks simultaneously. In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. Each synapse accumulates task relevant information over time, and exploits this information to rapidly store new memories without forgetting old ones. We evaluate our approach on continual learning of classification tasks, and show that it dramatically reduces forgetting while maintaining computational efficiency.
Tasks	Continual Learning
Published	2017-03-13
URL	http://arxiv.org/abs/1703.04200v3
PDF	http://arxiv.org/pdf/1703.04200v3.pdf
PWC	https://paperswithcode.com/paper/continual-learning-through-synaptic
Repo	https://github.com/chrhenning/hypercl
Framework	pytorch

Active Learning Using Uncertainty Information


Title	Active Learning Using Uncertainty Information
Authors	Yazhou Yang, Marco Loog
Abstract	Many active learning methods belong to the retraining-based approaches, which select one unlabeled instance, add it to the training set with its possible labels, retrain the classification model, and evaluate the criteria that we base our selection on. However, since the true label of the selected instance is unknown, these methods resort to calculating the average-case or worse-case performance with respect to the unknown label. In this paper, we propose a different method to solve this problem. In particular, our method aims to make use of the uncertainty information to enhance the performance of retraining-based models. We apply our method to two state-of-the-art algorithms and carry out extensive experiments on a wide variety of real-world datasets. The results clearly demonstrate the effectiveness of the proposed method and indicate it can reduce human labeling efforts in many real-life applications.
Tasks	Active Learning
Published	2017-02-27
URL	http://arxiv.org/abs/1702.08540v1
PDF	http://arxiv.org/pdf/1702.08540v1.pdf
PWC	https://paperswithcode.com/paper/active-learning-using-uncertainty-information
Repo	https://github.com/agusdmb/Active-Learning-Framework
Framework	none

Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding


Title	Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding
Authors	George C. Linderman, Manas Rachh, Jeremy G. Hoskins, Stefan Steinerberger, Yuval Kluger
Abstract	t-distributed Stochastic Neighborhood Embedding (t-SNE) is a method for dimensionality reduction and visualization that has become widely popular in recent years. Efficient implementations of t-SNE are available, but they scale poorly to datasets with hundreds of thousands to millions of high dimensional data-points. We present Fast Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE), which dramatically accelerates the computation of t-SNE. The most time-consuming step of t-SNE is a convolution that we accelerate by interpolating onto an equispaced grid and subsequently using the fast Fourier transform to perform the convolution. We also optimize the computation of input similarities in high dimensions using multi-threaded approximate nearest neighbors. We further present a modification to t-SNE called “late exaggeration,” which allows for easier identification of clusters in t-SNE embeddings. Finally, for datasets that cannot be loaded into the memory, we present out-of-core randomized principal component analysis (oocPCA), so that the top principal components of a dataset can be computed without ever fully loading the matrix, hence allowing for t-SNE of large datasets to be computed on resource-limited machines.
Tasks	Dimensionality Reduction
Published	2017-12-25
URL	http://arxiv.org/abs/1712.09005v1
PDF	http://arxiv.org/pdf/1712.09005v1.pdf
PWC	https://paperswithcode.com/paper/efficient-algorithms-for-t-distributed
Repo	https://github.com/KlugerLab/t-SNE-Heatmaps
Framework	none

Decision Stream: Cultivating Deep Decision Trees


Title	Decision Stream: Cultivating Deep Decision Trees
Authors	Dmitry Ignatov, Andrey Ignatov
Abstract	Various modifications of decision trees have been extensively used during the past years due to their high efficiency and interpretability. Tree node splitting based on relevant feature selection is a key step of decision tree learning, at the same time being their major shortcoming: the recursive nodes partitioning leads to geometric reduction of data quantity in the leaf nodes, which causes an excessive model complexity and data overfitting. In this paper, we present a novel architecture - a Decision Stream, - aimed to overcome this problem. Instead of building a tree structure during the learning process, we propose merging nodes from different branches based on their similarity that is estimated with two-sample test statistics, which leads to generation of a deep directed acyclic graph of decision rules that can consist of hundreds of levels. To evaluate the proposed solution, we test it on several common machine learning problems - credit scoring, twitter sentiment analysis, aircraft flight control, MNIST and CIFAR image classification, synthetic data classification and regression. Our experimental results reveal that the proposed approach significantly outperforms the standard decision tree learning methods on both regression and classification tasks, yielding a prediction error decrease up to 35%.
Tasks	Feature Selection, Image Classification, Sentiment Analysis, Twitter Sentiment Analysis
Published	2017-04-25
URL	http://arxiv.org/abs/1704.07657v3
PDF	http://arxiv.org/pdf/1704.07657v3.pdf
PWC	https://paperswithcode.com/paper/decision-stream-cultivating-deep-decision
Repo	https://github.com/aiff22/Decision-Stream
Framework	none

ArtGAN: Artwork Synthesis with Conditional Categorical GANs


Title	ArtGAN: Artwork Synthesis with Conditional Categorical GANs
Authors	Wei Ren Tan, Chee Seng Chan, Hernan Aguirre, Kiyoshi Tanaka
Abstract	This paper proposes an extension to the Generative Adversarial Networks (GANs), namely as ARTGAN to synthetically generate more challenging and complex images such as artwork that have abstract characteristics. This is in contrast to most of the current solutions that focused on generating natural images such as room interiors, birds, flowers and faces. The key innovation of our work is to allow back-propagation of the loss function w.r.t. the labels (randomly assigned to each generated images) to the generator from the discriminator. With the feedback from the label information, the generator is able to learn faster and achieve better generated image quality. Empirically, we show that the proposed ARTGAN is capable to create realistic artwork, as well as generate compelling real world images that globally look natural with clear shape on CIFAR-10.
Tasks	Art Analysis, Conditional Image Generation, Image Generation
Published	2017-02-11
URL	http://arxiv.org/abs/1702.03410v2
PDF	http://arxiv.org/pdf/1702.03410v2.pdf
PWC	https://paperswithcode.com/paper/artgan-artwork-synthesis-with-conditional
Repo	https://github.com/cs-chan/Artwork-Synthesis
Framework	tf

SurfNet: Generating 3D shape surfaces using deep residual networks


Title	SurfNet: Generating 3D shape surfaces using deep residual networks
Authors	Ayan Sinha, Asim Unmesh, Qixing Huang, Karthik Ramani
Abstract	3D shape models are naturally parameterized using vertices and faces, \ie, composed of polygons forming a surface. However, current 3D learning paradigms for predictive and generative tasks using convolutional neural networks focus on a voxelized representation of the object. Lifting convolution operators from the traditional 2D to 3D results in high computational overhead with little additional benefit as most of the geometry information is contained on the surface boundary. Here we study the problem of directly generating the 3D shape surface of rigid and non-rigid shapes using deep convolutional neural networks. We develop a procedure to create consistent `geometry images’ representing the shape surface of a category of 3D objects. We then use this consistent representation for category-specific shape surface generation from a parametric representation or an image by developing novel extensions of deep residual networks for the task of geometry image generation. Our experiments indicate that our network learns a meaningful representation of shape surfaces allowing it to interpolate between shape orientations and poses, invent new shape surfaces and reconstruct 3D shape surfaces from previously unseen images. \|
Tasks	3D Shape Generation, Image Generation
Published	2017-03-12
URL	http://arxiv.org/abs/1703.04079v1
PDF	http://arxiv.org/pdf/1703.04079v1.pdf
PWC	https://paperswithcode.com/paper/surfnet-generating-3d-shape-surfaces-using
Repo	https://github.com/sinhayan/surfnet
Framework	none

Is Big Data Sufficient for a Reliable Detection of Non-Technical Losses?


Title	Is Big Data Sufficient for a Reliable Detection of Non-Technical Losses?
Authors	Patrick Glauner, Angelo Migliosi, Jorge Meira, Petko Valtchev, Radu State, Franck Bettinger
Abstract	Non-technical losses (NTL) occur during the distribution of electricity in power grids and include, but are not limited to, electricity theft and faulty meters. In emerging countries, they may range up to 40% of the total electricity distributed. In order to detect NTLs, machine learning methods are used that learn irregular consumption patterns from customer data and inspection results. The Big Data paradigm followed in modern machine learning reflects the desire of deriving better conclusions from simply analyzing more data, without the necessity of looking at theory and models. However, the sample of inspected customers may be biased, i.e. it does not represent the population of all customers. As a consequence, machine learning models trained on these inspection results are biased as well and therefore lead to unreliable predictions of whether customers cause NTL or not. In machine learning, this issue is called covariate shift and has not been addressed in the literature on NTL detection yet. In this work, we present a novel framework for quantifying and visualizing covariate shift. We apply it to a commercial data set from Brazil that consists of 3.6M customers and 820K inspection results. We show that some features have a stronger covariate shift than others, making predictions less reliable. In particular, previous inspections were focused on certain neighborhoods or customer classes and that they were not sufficiently spread among the population of customers. This framework is about to be deployed in a commercial product for NTL detection.
Tasks
Published	2017-02-13
URL	http://arxiv.org/abs/1702.03767v2
PDF	http://arxiv.org/pdf/1702.03767v2.pdf
PWC	https://paperswithcode.com/paper/is-big-data-sufficient-for-a-reliable
Repo	https://github.com/pglauner/SpatialBiasNTL
Framework	none

Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation


Title	Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Authors	Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel
Abstract	Imitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills.
Tasks	Imitation Learning
Published	2017-10-12
URL	http://arxiv.org/abs/1710.04615v2
PDF	http://arxiv.org/pdf/1710.04615v2.pdf
PWC	https://paperswithcode.com/paper/deep-imitation-learning-for-complex
Repo	https://github.com/h2r/parameterized-imitation-learning
Framework	pytorch