October 21, 2019

2783 words 14 mins read

Paper Group AWR 81

Noise2Noise: Learning Image Restoration without Clean Data. Semi-Supervised Sequence Modeling with Cross-View Training. GPSP: Graph Partition and Space Projection based Approach for Heterogeneous Network Embedding. Learning Deep Features for One-Class Classification. The Limitations of Cross-language Word Embeddings Evaluation. Markov Game Modeling …

Noise2Noise: Learning Image Restoration without Clean Data


Title	Noise2Noise: Learning Image Restoration without Clean Data
Authors	Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, Timo Aila
Abstract	We apply basic statistical reasoning to signal reconstruction by machine learning – learning to map corrupted observations to clean signals – with a simple and powerful conclusion: it is possible to learn to restore images by only looking at corrupted examples, at performance at and sometimes exceeding training using clean data, without explicit image priors or likelihood models of the corruption. In practice, we show that a single model learns photographic noise removal, denoising synthetic Monte Carlo images, and reconstruction of undersampled MRI scans – all corrupted by different processes – based on noisy data only.
Tasks	Denoising, Image Restoration
Published	2018-03-12
URL	http://arxiv.org/abs/1803.04189v3
PDF	http://arxiv.org/pdf/1803.04189v3.pdf
PWC	https://paperswithcode.com/paper/noise2noise-learning-image-restoration
Repo	https://github.com/itsuki8914/simply-noise2noise-TF
Framework	tf

Semi-Supervised Sequence Modeling with Cross-View Training


Title	Semi-Supervised Sequence Modeling with Cross-View Training
Authors	Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Quoc V. Le
Abstract	Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. However, the supervised models only learn from task-specific labeled data during the main training phase. We therefore propose Cross-View Training (CVT), a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. On labeled examples, standard supervised learning is used. On unlabeled examples, CVT teaches auxiliary prediction modules that see restricted views of the input (e.g., only part of a sentence) to match the predictions of the full model seeing the whole input. Since the auxiliary modules and the full model share intermediate representations, this in turn improves the full model. Moreover, we show that CVT is particularly effective when combined with multi-task learning. We evaluate CVT on five sequence tagging tasks, machine translation, and dependency parsing, achieving state-of-the-art results.
Tasks	CCG Supertagging, Dependency Parsing, Machine Translation, Multi-Task Learning, Named Entity Recognition, Representation Learning, Unsupervised Representation Learning
Published	2018-09-22
URL	http://arxiv.org/abs/1809.08370v1
PDF	http://arxiv.org/pdf/1809.08370v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-sequence-modeling-with-cross
Repo	https://github.com/tensorflow/models
Framework	tf

GPSP: Graph Partition and Space Projection based Approach for Heterogeneous Network Embedding


Title	GPSP: Graph Partition and Space Projection based Approach for Heterogeneous Network Embedding
Authors	Wenyu Du, Shuai Yu, Min Yang, Qiang Qu, Jia Zhu
Abstract	In this paper, we propose GPSP, a novel Graph Partition and Space Projection based approach, to learn the representation of a heterogeneous network that consists of multiple types of nodes and links. Concretely, we first partition the heterogeneous network into homogeneous and bipartite subnetworks. Then, the projective relations hidden in bipartite subnetworks are extracted by learning the projective embedding vectors. Finally, we concatenate the projective vectors from bipartite subnetworks with the ones learned from homogeneous subnetworks to form the final representation of the heterogeneous network. Extensive experiments are conducted on a real-life dataset. The results demonstrate that GPSP outperforms the state-of-the-art baselines in two key network mining tasks: node classification and clustering.
Tasks	Network Embedding, Node Classification
Published	2018-03-07
URL	http://arxiv.org/abs/1803.02590v1
PDF	http://arxiv.org/pdf/1803.02590v1.pdf
PWC	https://paperswithcode.com/paper/gpsp-graph-partition-and-space-projection
Repo	https://github.com/Ange1o/GPSP
Framework	tf

Learning Deep Features for One-Class Classification


Title	Learning Deep Features for One-Class Classification
Authors	Pramuditha Perera, Vishal M. Patel
Abstract	We propose a deep learning-based solution for the problem of feature learning in one-class classification. The proposed method operates on top of a Convolutional Neural Network (CNN) of choice and produces descriptive features while maintaining a low intra-class variance in the feature space for the given class. For this purpose two loss functions, compactness loss and descriptiveness loss are proposed along with a parallel CNN architecture. A template matching-based framework is introduced to facilitate the testing process. Extensive experiments on publicly available anomaly detection, novelty detection and mobile active authentication datasets show that the proposed Deep One-Class (DOC) classification method achieves significant improvements over the state-of-the-art.
Tasks	Anomaly Detection
Published	2018-01-16
URL	https://arxiv.org/abs/1801.05365v2
PDF	https://arxiv.org/pdf/1801.05365v2.pdf
PWC	https://paperswithcode.com/paper/learning-deep-features-for-one-class
Repo	https://github.com/PramuPerera/DeepOneClass
Framework	caffe2

The Limitations of Cross-language Word Embeddings Evaluation


Title	The Limitations of Cross-language Word Embeddings Evaluation
Authors	Amir Bakarov, Roman Suvorov, Ilya Sochenkov
Abstract	The aim of this work is to explore the possible limitations of existing methods of cross-language word embeddings evaluation, addressing the lack of correlation between intrinsic and extrinsic cross-language evaluation methods. To prove this hypothesis, we construct English-Russian datasets for extrinsic and intrinsic evaluation tasks and compare performances of 5 different cross-language models on them. The results say that the scores even on different intrinsic benchmarks do not correlate to each other. We can conclude that the use of human references as ground truth for cross-language word embeddings is not proper unless one does not understand how do native speakers process semantics in their cognition.
Tasks	Word Embeddings
Published	2018-06-06
URL	http://arxiv.org/abs/1806.02253v1
PDF	http://arxiv.org/pdf/1806.02253v1.pdf
PWC	https://paperswithcode.com/paper/the-limitations-of-cross-language-word
Repo	https://github.com/bakarov/cross-lang-embeddings
Framework	none

Markov Game Modeling of Moving Target Defense for Strategic Detection of Threats in Cloud Networks


Title	Markov Game Modeling of Moving Target Defense for Strategic Detection of Threats in Cloud Networks
Authors	Ankur Chowdhary, Sailik Sengupta, Dijiang Huang, Subbarao Kambhampati
Abstract	The processing and storage of critical data in large-scale cloud networks necessitate the need for scalable security solutions. It has been shown that deploying all possible security measures incurs a cost on performance by using up valuable computing and networking resources which are the primary selling points for cloud service providers. Thus, there has been a recent interest in developing Moving Target Defense (MTD) mechanisms that helps one optimize the joint objective of maximizing security while ensuring that the impact on performance is minimized. Often, these techniques model the problem of multi-stage attacks by stealthy adversaries as a single-step attack detection game using graph connectivity measures as a heuristic to measure performance, thereby (1) losing out on valuable information that is inherently present in graph-theoretic models designed for large cloud networks, and (2) coming up with certain strategies that have asymmetric impacts on performance. In this work, we leverage knowledge in attack graphs of a cloud network in formulating a zero-sum Markov Game and use the Common Vulnerability Scoring System (CVSS) to come up with meaningful utility values for this game. Then, we show that the optimal strategy of placing detecting mechanisms against an adversary is equivalent to computing the mixed Min-max Equilibrium of the Markov Game. We compare the gains obtained by using our method to other techniques presently used in cloud network security, thereby showing its effectiveness. Finally, we highlight how the method was used for a small real-world cloud system.
Tasks
Published	2018-12-23
URL	http://arxiv.org/abs/1812.09660v2
PDF	http://arxiv.org/pdf/1812.09660v2.pdf
PWC	https://paperswithcode.com/paper/markov-game-modeling-of-moving-target-defense
Repo	https://github.com/facundo-p/Markov-Game-Model
Framework	none

Occluded Person Re-identification


Title	Occluded Person Re-identification
Authors	Jiaxuan Zhuo, Zeyu Chen, Jianhuang Lai, Guangcong Wang
Abstract	Person re-identification (re-id) suffers from a serious occlusion problem when applied to crowded public places. In this paper, we propose to retrieve a full-body person image by using a person image with occlusions. This differs significantly from the conventional person re-id problem where it is assumed that person images are detected without any occlusion. We thus call this new problem the occluded person re-identitification. To address this new problem, we propose a novel Attention Framework of Person Body (AFPB) based on deep learning, consisting of 1) an Occlusion Simulator (OS) which automatically generates artificial occlusions for full-body person images, and 2) multi-task losses that force the neural network not only to discriminate a person’s identity but also to determine whether a sample is from the occluded data distribution or the full-body data distribution. Experiments on a new occluded person re-id dataset and three existing benchmarks modified to include full-body person images and occluded person images show the superiority of the proposed method.
Tasks	Person Re-Identification
Published	2018-04-09
URL	http://arxiv.org/abs/1804.02792v3
PDF	http://arxiv.org/pdf/1804.02792v3.pdf
PWC	https://paperswithcode.com/paper/occluded-person-re-identification
Repo	https://github.com/tinajia2012/ICME2018_Occluded-Person-Reidentification_datasets
Framework	none

Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments


Title	Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments
Authors	Thierry Moreau, Anton Lokhmotov, Grigori Fursin
Abstract	Co-designing efficient machine learning based systems across the whole hardware/software stack to trade off speed, accuracy, energy and costs is becoming extremely complex and time consuming. Researchers often struggle to evaluate and compare different published works across rapidly evolving software frameworks, heterogeneous hardware platforms, compilers, libraries, algorithms, data sets, models, and environments. We present our community effort to develop an open co-design tournament platform with an online public scoreboard. It will gradually incorporate best research practices while providing a common way for multidisciplinary researchers to optimize and compare the quality vs. efficiency Pareto optimality of various workloads on diverse and complete hardware/software systems. We want to leverage the open-source Collective Knowledge framework and the ACM artifact evaluation methodology to validate and share the complete machine learning system implementations in a standardized, portable, and reproducible fashion. We plan to hold regular multi-objective optimization and co-design tournaments for emerging workloads such as deep learning, starting with ASPLOS’18 (ACM conference on Architectural Support for Programming Languages and Operating Systems - the premier forum for multidisciplinary systems research spanning computer architecture and hardware, programming languages and compilers, operating systems and networking) to build a public repository of the most efficient machine learning algorithms and systems which can be easily customized, reused and built upon.
Tasks
Published	2018-01-19
URL	http://arxiv.org/abs/1801.06378v1
PDF	http://arxiv.org/pdf/1801.06378v1.pdf
PWC	https://paperswithcode.com/paper/introducing-request-an-open-platform-for
Repo	https://github.com/ctuning/ck-request
Framework	tf

Zero-Shot Skill Composition and Simulation-to-Real Transfer by Learning Task Representations


Title	Zero-Shot Skill Composition and Simulation-to-Real Transfer by Learning Task Representations
Authors	Zhanpeng He, Ryan Julian, Eric Heiden, Hejia Zhang, Stefan Schaal, Joseph J. Lim, Gaurav Sukhatme, Karol Hausman
Abstract	Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks, despite training for thousands of hours equivalent real robot time. To address this shortcoming, we present a novel approach to efficiently learning new robotic skills directly on a real robot, based on model-predictive control (MPC) and an algorithm for learning task representations. In short, we show how to reuse the simulation from the pre-training step of sim-to-real methods as a tool for foresight, allowing the sim-to-real policy adapt to unseen tasks. Rather than end-to-end learning policies for single tasks and attempting to transfer them, we first use simulation to simultaneously learn (1) a continuous parameterization (i.e. a skill embedding or latent) of task-appropriate primitive skills, and (2) a single policy for these skills which is conditioned on this representation. We then directly transfer our multi-skill policy to a real robot, and actuate the robot by choosing sequences of skill latents which actuate the policy, with each latent corresponding to a pre-learned primitive skill controller. We complete unseen tasks by choosing new sequences of skill latents to control the robot using MPC, where our MPC model is composed of the pre-trained skill policy executed in the simulation environment, run in parallel with the real robot. We discuss the background and principles of our method, detail its practical implementation, and evaluate its performance by using our method to train a real Sawyer Robot to achieve motion tasks such as drawing and block pushing.
Tasks
Published	2018-10-04
URL	http://arxiv.org/abs/1810.02422v2
PDF	http://arxiv.org/pdf/1810.02422v2.pdf
PWC	https://paperswithcode.com/paper/zero-shot-skill-composition-and-simulation-to
Repo	https://github.com/ryanjulian/embed2learn
Framework	tf

Deep Neural Nets with Interpolating Function as Output Activation


Title	Deep Neural Nets with Interpolating Function as Output Activation
Authors	Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, Stanley J. Osher
Abstract	We replace the output layer of deep neural nets, typically the softmax function, by a novel interpolating function. And we propose end-to-end training and testing algorithms for this new architecture. Compared to classical neural nets with softmax function as output activation, the surrogate with interpolating function as output activation combines advantages of both deep and manifold learning. The new framework demonstrates the following major advantages: First, it is better applicable to the case with insufficient training data. Second, it significantly improves the generalization accuracy on a wide variety of networks. The algorithm is implemented in PyTorch, and code will be made publicly available.
Tasks
Published	2018-02-01
URL	http://arxiv.org/abs/1802.00168v3
PDF	http://arxiv.org/pdf/1802.00168v3.pdf
PWC	https://paperswithcode.com/paper/deep-neural-nets-with-interpolating-function
Repo	https://github.com/BaoWangMath/DNN-DataDependentActivation
Framework	pytorch

Using reinforcement learning to learn how to play text-based games


Title	Using reinforcement learning to learn how to play text-based games
Authors	Mikuláš Zelinka
Abstract	The ability to learn optimal control policies in systems where action space is defined by sentences in natural language would allow many interesting real-world applications such as automatic optimisation of dialogue systems. Text-based games with multiple endings and rewards are a promising platform for this task, since their feedback allows us to employ reinforcement learning techniques to jointly learn text representations and control policies. We present a general text game playing agent, testing its generalisation and transfer learning performance and showing its ability to play multiple games at once. We also present pyfiction, an open-source library for universal access to different text games that could, together with our agent that implements its interface, serve as a baseline for future research.
Tasks	Transfer Learning
Published	2018-01-06
URL	http://arxiv.org/abs/1801.01999v1
PDF	http://arxiv.org/pdf/1801.01999v1.pdf
PWC	https://paperswithcode.com/paper/using-reinforcement-learning-to-learn-how-to
Repo	https://github.com/MikulasZelinka/pyfiction
Framework	tf

A Weakly Supervised Approach for Estimating Spatial Density Functions from High-Resolution Satellite Imagery


Title	A Weakly Supervised Approach for Estimating Spatial Density Functions from High-Resolution Satellite Imagery
Authors	Nathan Jacobs, Adam Kraft, Muhammad Usman Rafique, Ranti Dev Sharma
Abstract	We propose a neural network component, the regional aggregation layer, that makes it possible to train a pixel-level density estimator using only coarse-grained density aggregates, which reflect the number of objects in an image region. Our approach is simple to use and does not require domain-specific assumptions about the nature of the density function. We evaluate our approach on several synthetic datasets. In addition, we use this approach to learn to estimate high-resolution population and housing density from satellite imagery. In all cases, we find that our approach results in better density estimates than a commonly used baseline. We also show how our housing density estimator can be used to classify buildings as residential or non-residential.
Tasks
Published	2018-10-22
URL	http://arxiv.org/abs/1810.09528v1
PDF	http://arxiv.org/pdf/1810.09528v1.pdf
PWC	https://paperswithcode.com/paper/a-weakly-supervised-approach-for-estimating
Repo	https://github.com/orbitalinsight/region-aggregation-public
Framework	tf

Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)


Title	Flex-Convolution (Million-Scale Point-Cloud Learning Beyond Grid-Worlds)
Authors	Fabian Groh, Patrick Wieschollek, Hendrik P. A. Lensch
Abstract	Traditional convolution layers are specifically designed to exploit the natural data representation of images – a fixed and regular grid. However, unstructured data like 3D point clouds containing irregular neighborhoods constantly breaks the grid-based data assumption. Therefore applying best-practices and design choices from 2D-image learning methods towards processing point clouds are not readily possible. In this work, we introduce a natural generalization flex-convolution of the conventional convolution layer along with an efficient GPU implementation. We demonstrate competitive performance on rather small benchmark sets using fewer parameters and lower memory consumption and obtain significant improvements on a million-scale real-world dataset. Ours is the first which allows to efficiently process 7 million points concurrently.
Tasks	Classify 3D Point Clouds, Semantic Segmentation
Published	2018-03-20
URL	http://arxiv.org/abs/1803.07289v3
PDF	http://arxiv.org/pdf/1803.07289v3.pdf
PWC	https://paperswithcode.com/paper/flex-convolution-million-scale-point-cloud
Repo	https://github.com/cgtuebingen/Flex-Convolution
Framework	tf

Semi-Supervised Learning with GANs: Revisiting Manifold Regularization


Title	Semi-Supervised Learning with GANs: Revisiting Manifold Regularization
Authors	Bruno Lecouat, Chuan-Sheng Foo, Houssam Zenati, Vijay R. Chandrasekhar
Abstract	GANS are powerful generative models that are able to model the manifold of natural images. We leverage this property to perform manifold regularization by approximating the Laplacian norm using a Monte Carlo approximation that is easily computed with the GAN. When incorporated into the feature-matching GAN of Improved GAN, we achieve state-of-the-art results for GAN-based semi-supervised learning on the CIFAR-10 dataset, with a method that is significantly easier to implement than competing methods.
Tasks
Published	2018-05-23
URL	http://arxiv.org/abs/1805.08957v1
PDF	http://arxiv.org/pdf/1805.08957v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-learning-with-gans-revisiting
Repo	https://github.com/UCI-ML-course-team/GAN-manifold-regularization-PyTorch
Framework	pytorch

Banach Wasserstein GAN


Title	Banach Wasserstein GAN
Authors	Jonas Adler, Sebastian Lunz
Abstract	Wasserstein Generative Adversarial Networks (WGANs) can be used to generate realistic samples from complicated image distributions. The Wasserstein metric used in WGANs is based on a notion of distance between individual images, which induces a notion of distance between probability distributions of images. So far the community has considered $\ell^2$ as the underlying distance. We generalize the theory of WGAN with gradient penalty to Banach spaces, allowing practitioners to select the features to emphasize in the generator. We further discuss the effect of some particular choices of underlying norms, focusing on Sobolev norms. Finally, we demonstrate a boost in performance for an appropriate choice of norm on CIFAR-10 and CelebA.
Tasks
Published	2018-06-18
URL	http://arxiv.org/abs/1806.06621v2
PDF	http://arxiv.org/pdf/1806.06621v2.pdf
PWC	https://paperswithcode.com/paper/banach-wasserstein-gan
Repo	https://github.com/adler-j/bwgan
Framework	tf