January 29, 2020

3326 words 16 mins read

Paper Group ANR 533

Prediction and optimization of mechanical properties of composites using convolutional neural networks. Learning to Clean: A GAN Perspective. An encoding framework with brain inner state for natural image identification. Does Knowledge Transfer Always Help to Learn a Better Policy?. Likelihood Ratios and Generative Classifiers for Unsupervised Out- …

Prediction and optimization of mechanical properties of composites using convolutional neural networks


Title	Prediction and optimization of mechanical properties of composites using convolutional neural networks
Authors	Diab W. Abueidda, Mohammad Almasri, Rami Ammourah, Umberto Ravaioli, Iwona M. Jasiuk, Nahil A. Sobh
Abstract	In this paper, we develop a convolutional neural network model to predict the mechanical properties of a two-dimensional checkerboard composite quantitatively. The checkerboard composite possesses two phases, one phase is soft and ductile while the other is stiff and brittle. The ground-truth data used in the training process are obtained from finite element analyses under the assumption of plane stress. Monte Carlo simulations and central limit theorem are used to find the size of the dataset needed. Once the training process is completed, the developed model is validated using data unseen during training. The developed neural network model captures the stiffness, strength, and toughness of checkerboard composites with high accuracy. Also, we integrate the developed model with a genetic algorithm (GA) optimizer to identify the optimal microstructural designs. The genetic algorithm optimizer adopted here has several operators, selection, crossover, mutation, and elitism. The optimizer converges to configurations with highly enhanced properties. For the case of the modulus and starting from randomly-initialized generation, the GA optimizer converges to the global maximum which involves no soft elements. Also, the GA optimizers, when used to maximize strength and toughness, tend towards having soft elements in the region next to the crack tip.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1906.00094v1
PDF	https://arxiv.org/pdf/1906.00094v1.pdf
PWC	https://paperswithcode.com/paper/190600094
Repo
Framework

Learning to Clean: A GAN Perspective


Title	Learning to Clean: A GAN Perspective
Authors	Monika Sharma, Abhishek Verma, Lovekesh Vig
Abstract	In the big data era, the impetus to digitize the vast reservoirs of data trapped in unstructured scanned documents such as invoices, bank documents and courier receipts has gained fresh momentum. The scanning process often results in the introduction of artifacts such as background noise, blur due to camera motion, watermarkings, coffee stains, or faded text. These artifacts pose many readability challenges to current text recognition algorithms and significantly degrade their performance. Existing learning based denoising techniques require a dataset comprising of noisy documents paired with cleaned versions. In such scenarios, a model can be trained to generate clean documents from noisy versions. However, very often in the real world such a paired dataset is not available, and all we have for training our denoising model are unpaired sets of noisy and clean images. This paper explores the use of GANs to generate denoised versions of the noisy documents. In particular, where paired information is available, we formulate the problem as an image-to-image translation task i.e, translating a document from noisy domain ( i.e., background noise, blurred, faded, watermarked ) to a target clean document using Generative Adversarial Networks (GAN). However, in the absence of paired images for training, we employed CycleGAN which is known to learn a mapping between the distributions of the noisy images to the denoised images using unpaired data to achieve image-to-image translation for cleaning the noisy documents. We compare the performance of CycleGAN for document cleaning tasks using unpaired images with a Conditional GAN trained on paired data from the same dataset. Experiments were performed on a public document dataset on which different types of noise were artificially induced, results demonstrate that CycleGAN learns a more robust mapping from the space of noisy to clean documents.
Tasks	Denoising, Image-to-Image Translation
Published	2019-01-28
URL	http://arxiv.org/abs/1901.11382v1
PDF	http://arxiv.org/pdf/1901.11382v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-clean-a-gan-perspective
Repo
Framework

An encoding framework with brain inner state for natural image identification


Title	An encoding framework with brain inner state for natural image identification
Authors	Hao Wu, Ziyu Zhu, Jiayi Wang, Nanning Zheng, Badong Chen
Abstract	Neural encoding and decoding, which aim to characterize the relationship between stimuli and brain activities, have emerged as an important area in cognitive neuroscience. Traditional encoding models, which focus on feature extraction and mapping, consider the brain as an input-output mapper without inner states. In this work, inspired by the fact that human brain acts like a state machine, we proposed a novel encoding framework that combines information from both the external world and the inner state to predict brain activity. The framework comprises two parts: forward encoding model that deals with visual stimuli and inner state model that captures influence from intrinsic connections in the brain. The forward model can be any traditional encoding model, making the framework flexible. The inner state model is a linear model to utilize information in the prediction residuals of the forward model. The proposed encoding framework can achieve much better performance on natural image identification from fMRI response than forwardonly models. The identification accuracy will decrease slightly with the dataset size increasing, but remain relatively stable with different identification methods. The results confirm that the new encoding framework is effective and robust when used for brain decoding.
Tasks	Brain Decoding
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08807v1
PDF	https://arxiv.org/pdf/1908.08807v1.pdf
PWC	https://paperswithcode.com/paper/an-encoding-framework-with-brain-inner-state
Repo
Framework

Does Knowledge Transfer Always Help to Learn a Better Policy?


Title	Does Knowledge Transfer Always Help to Learn a Better Policy?
Authors	Fei Feng, Wotao Yin, Lin F. Yang
Abstract	One of the key approaches to save samples when learning a policy for a reinforcement learning problem is to use knowledge from an approximate model such as its simulator. However, does knowledge transfer from approximate models always help to learn a better policy? Despite numerous empirical studies of transfer reinforcement learning, an answer to this question is still elusive. In this paper, we provide a strong negative result, showing that even the full knowledge of an approximate model may not help reduce the number of samples for learning an accurate policy of the true model. We construct an example of reinforcement learning models and show that the complexity with or without knowledge transfer has the same order. On the bright side, effective knowledge transferring is still possible under additional assumptions. In particular, we demonstrate that knowing the (linear) bases of the true model significantly reduces the number of samples for learning an accurate policy.
Tasks	Transfer Learning, Transfer Reinforcement Learning
Published	2019-12-06
URL	https://arxiv.org/abs/1912.02986v1
PDF	https://arxiv.org/pdf/1912.02986v1.pdf
PWC	https://paperswithcode.com/paper/does-knowledge-transfer-always-help-to-learn
Repo
Framework

Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog


Title	Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog
Authors	Varun Gangal, Abhinav Arora, Arash Einolghozati, Sonal Gupta
Abstract	The task of identifying out-of-domain (OOD) input examples directly at test-time has seen renewed interest recently due to increased real world deployment of models. In this work, we focus on OOD detection for natural language sentence inputs to task-based dialog systems. Our findings are three-fold: First, we curate and release ROSTD (Real Out-of-Domain Sentences From Task-oriented Dialog) - a dataset of 4K OOD examples for the publicly available dataset from (Schuster et al. 2019). In contrast to existing settings which synthesize OOD examples by holding out a subset of classes, our examples were authored by annotators with apriori instructions to be out-of-domain with respect to the sentences in an existing dataset. Second, we explore likelihood ratio based approaches as an alternative to currently prevalent paradigms. Specifically, we reformulate and apply these approaches to natural language inputs. We find that they match or outperform the latter on all datasets, with larger improvements on non-artificial OOD benchmarks such as our dataset. Our ablations validate that specifically using likelihood ratios rather than plain likelihood is necessary to discriminate well between OOD and in-domain data. Third, we propose learning a generative classifier and computing a marginal likelihood (ratio) for OOD detection. This allows us to use a principled likelihood while at the same time exploiting training-time labels. We find that this approach outperforms both simple likelihood (ratio) based and other prior approaches. We are hitherto the first to investigate the use of generative classifiers for OOD detection at test-time.
Tasks
Published	2019-12-30
URL	https://arxiv.org/abs/1912.12800v1
PDF	https://arxiv.org/pdf/1912.12800v1.pdf
PWC	https://paperswithcode.com/paper/likelihood-ratios-and-generative-classifiers
Repo
Framework

Identity-preserving Face Recovery from Stylized Portraits


Title	Identity-preserving Face Recovery from Stylized Portraits
Authors	Fatemeh Shiri, Xin Yu, Fatih Porikli, Richard Hartley, Piotr Koniusz
Abstract	Given an artistic portrait, recovering the latent photorealistic face that preserves the subject’s identity is challenging because the facial details are often distorted or fully lost in artistic portraits. We develop an Identity-preserving Face Recovery from Portraits (IFRP) method that utilizes a Style Removal network (SRN) and a Discriminative Network (DN). Our SRN, composed of an autoencoder with residual block-embedded skip connections, is designed to transfer feature maps of stylized images to the feature maps of the corresponding photorealistic faces. Owing to the Spatial Transformer Network (STN), SRN automatically compensates for misalignments of stylized portraits to output aligned realistic face images. To ensure the identity preservation, we promote the recovered and ground truth faces to share similar visual features via a distance measure which compares features of recovered and ground truth faces extracted from a pre-trained FaceNet network. DN has multiple convolutional and fully-connected layers, and its role is to enforce recovered faces to be similar to authentic faces. Thus, we can recover high-quality photorealistic faces from unaligned portraits while preserving the identity of the face in an image. By conducting extensive evaluations on a large-scale synthesized dataset and a hand-drawn sketch dataset, we demonstrate that our method achieves superior face recovery and attains state-of-the-art results. In addition, our method can recover photorealistic faces from unseen stylized portraits, artistic paintings, and hand-drawn sketches.
Tasks
Published	2019-04-07
URL	http://arxiv.org/abs/1904.04241v1
PDF	http://arxiv.org/pdf/1904.04241v1.pdf
PWC	https://paperswithcode.com/paper/identity-preserving-face-recovery-from-1
Repo
Framework

Zero-Shot Semantic Segmentation


Title	Zero-Shot Semantic Segmentation
Authors	Maxime Bucher, Tuan-Hung Vu, Matthieu Cord, Patrick Pérez
Abstract	Semantic segmentation models are limited in their ability to scale to large numbers of object classes. In this paper, we introduce the new task of zero-shot semantic segmentation: learning pixel-wise classifiers for never-seen object categories with zero training examples. To this end, we present a novel architecture, ZS3Net, combining a deep visual segmentation model with an approach to generate visual representations from semantic word embeddings. By this way, ZS3Net addresses pixel classification tasks where both seen and unseen categories are faced at test time (so called “generalized” zero-shot classification). Performance is further improved by a self-training step that relies on automatic pseudo-labeling of pixels from unseen classes. On the two standard segmentation datasets, Pascal-VOC and Pascal-Context, we propose zero-shot benchmarks and set competitive baselines. For complex scenes as ones in the Pascal-Context dataset, we extend our approach by using a graph-context encoding to fully leverage spatial context priors coming from class-wise segmentation maps.
Tasks	Semantic Segmentation, Word Embeddings, Zero-Shot Learning
Published	2019-06-03
URL	https://arxiv.org/abs/1906.00817v2
PDF	https://arxiv.org/pdf/1906.00817v2.pdf
PWC	https://paperswithcode.com/paper/190600817
Repo
Framework

Unsupervised Context Retrieval for Long-tail Entities


Title	Unsupervised Context Retrieval for Long-tail Entities
Authors	Darío Garigliotti, Dyaa Albakour, Miguel Martinez, Krisztian Balog
Abstract	Monitoring entities in media streams often relies on rich entity representations, like structured information available in a knowledge base (KB). For long-tail entities, such monitoring is highly challenging, due to their limited, if not entirely missing, representation in the reference KB. In this paper, we address the problem of retrieving textual contexts for monitoring long-tail entities. We propose an unsupervised method to overcome the limited representation of long-tail entities by leveraging established entities and their contexts as support information. Evaluation on a purpose-built test collection shows the suitability of our approach and its robustness for out-of-KB entities.
Tasks
Published	2019-08-05
URL	https://arxiv.org/abs/1908.01798v1
PDF	https://arxiv.org/pdf/1908.01798v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-context-retrieval-for-long-tail
Repo
Framework

DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling


Title	DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling
Authors	Kevin Karsch, Ce Liu, Sing Bing Kang
Abstract	We describe a technique that automatically generates plausible depth maps from videos using non-parametric depth sampling. We demonstrate our technique in cases where past methods fail (non-translating cameras and dynamic scenes). Our technique is applicable to single images as well as videos. For videos, we use local motion cues to improve the inferred depth maps, while optical flow is used to ensure temporal depth consistency. For training and evaluation, we use a Kinect-based system to collect a large dataset containing stereoscopic videos with known depths. We show that our depth estimation technique outperforms the state-of-the-art on benchmark databases. Our technique can be used to automatically convert a monoscopic video into stereo for 3D visualization, and we demonstrate this through a variety of visually pleasing results for indoor and outdoor scenes, including results from the feature film Charade.
Tasks	Depth Estimation, Optical Flow Estimation
Published	2019-12-24
URL	https://arxiv.org/abs/2001.00987v1
PDF	https://arxiv.org/pdf/2001.00987v1.pdf
PWC	https://paperswithcode.com/paper/depthtransfer-depth-extraction-from-video
Repo
Framework

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design


Title	MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design
Authors	Ziyao Zhang, Liang Ma, Konstantinos Poularakis, Kin K. Leung, Jeremy Tucker, Ananthram Swami
Abstract	In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distributed SDN controller architectures, most existing works only aim at eliminating anomalies arising from the inconsistencies in different controllers’ network views. However, the performance aspect of controller synchronization designs with respect to given SDN applications are generally missing. To fill this gap, we formulate the controller synchronization problem as a Markov decision process (MDP) and apply reinforcement learning techniques combined with deep neural networks (DNNs) to train a smart, scalable, and fine-grained controller synchronization policy, called the Multi-Armed Cooperative Synchronization (MACS), whose goal is to maximise the performance enhancements brought by controller synchronizations. Evaluation results confirm the DNN’s exceptional ability in abstracting latent patterns in the distributed SDN environment, rendering significant superiority to MACS-based synchronization policy, which are 56% and 30% performance improvements over ONOS and greedy SDN controller synchronization heuristics.
Tasks
Published	2019-09-19
URL	https://arxiv.org/abs/1909.09063v1
PDF	https://arxiv.org/pdf/1909.09063v1.pdf
PWC	https://paperswithcode.com/paper/macs-deep-reinforcement-learning-based-sdn
Repo
Framework

A Fundamental Performance Limitation for Adversarial Classification


Title	A Fundamental Performance Limitation for Adversarial Classification
Authors	Abed AlRahman Al Makdah, Vaibhav Katewa, Fabio Pasqualetti
Abstract	Despite the widespread use of machine learning algorithms to solve problems of technological, economic, and social relevance, provable guarantees on the performance of these data-driven algorithms are critically lacking, especially when the data originates from unreliable sources and is transmitted over unprotected and easily accessible channels. In this paper we take an important step to bridge this gap and formally show that, in a quest to optimize their accuracy, binary classification algorithms – including those based on machine-learning techniques – inevitably become more sensitive to adversarial manipulation of the data. Further, for a given class of algorithms with the same complexity (i.e., number of classification boundaries), the fundamental tradeoff curve between accuracy and sensitivity depends solely on the statistics of the data, and cannot be improved by tuning the algorithm.
Tasks
Published	2019-03-04
URL	http://arxiv.org/abs/1903.01032v2
PDF	http://arxiv.org/pdf/1903.01032v2.pdf
PWC	https://paperswithcode.com/paper/a-fundamental-performance-limitation-for
Repo
Framework

Progressive-Growing of Generative Adversarial Networks for Metasurface Optimization


Title	Progressive-Growing of Generative Adversarial Networks for Metasurface Optimization
Authors	Fufang Wen, Jiaqi Jiang, Jonathan A. Fan
Abstract	Generative adversarial networks, which can generate metasurfaces based on a training set of high performance device layouts, have the potential to significantly reduce the computational cost of the metasurface design process. However, basic GAN architectures are unable to fully capture the detailed features of topologically complex metasurfaces, and generated devices therefore require additional computationally-expensive design refinement. In this Letter, we show that GANs can better learn spatially fine features from high-resolution training data by progressively growing its network architecture and training set. Our results indicate that with this training methodology, the best generated devices have performances that compare well with the best devices produced by gradient-based topology optimization, thereby eliminating the need for additional design refinement. We envision that this network training method can generalize to other physical systems where device performance is strongly correlated with fine geometric structuring.
Tasks
Published	2019-11-29
URL	https://arxiv.org/abs/1911.13029v2
PDF	https://arxiv.org/pdf/1911.13029v2.pdf
PWC	https://paperswithcode.com/paper/progressive-growing-of-generative-adversarial
Repo
Framework

Joint Extraction of Entities and Relations with a Hierarchical Multi-task Tagging Model


Title	Joint Extraction of Entities and Relations with a Hierarchical Multi-task Tagging Model
Authors	Zhepei Wei, Yantao Jia, Yuan Tian, Mohammad Javad Hosseini, Mark Steedman, Yi Chang
Abstract	Entity extraction and relation extraction are two indispensable building blocks for knowledge graph construction. Recent works on entity and relation extraction have shown the superiority of solving the two problems in a joint manner, where entities and relations are extracted simultaneously to form relational triples in a knowledge graph. However, existing methods ignore the hierarchical semantic interdependency between entity extraction (EE) and joint extraction (JE), which leaves much to be desired in real applications. In this work, we propose a hierarchical multi-task tagging model, called HMT, which captures such interdependency and achieves better performance for joint extraction of entities and relations. Specifically, the EE task is organized at the bottom layer and JE task at the top layer in a hierarchical structure. Furthermore, the learned semantic representation at the lower level can be shared by the upper level via multi-task learning. Experimental results demonstrate the effectiveness of the proposed model for joint extraction in comparison with the state-of-the-art methods.
Tasks	Entity Extraction, graph construction, Multi-Task Learning, Relation Extraction
Published	2019-08-23
URL	https://arxiv.org/abs/1908.08672v1
PDF	https://arxiv.org/pdf/1908.08672v1.pdf
PWC	https://paperswithcode.com/paper/joint-extraction-of-entities-and-relations-2
Repo
Framework

Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approach


Title	Distributed Reinforcement Learning for Decentralized Linear Quadratic Control: A Derivative-Free Policy Optimization Approach
Authors	Yingying Li, Yujie Tang, Runyu Zhang, Na Li
Abstract	This paper considers a distributed reinforcement learning problem for decentralized linear quadratic control with partial state observations and local costs. We propose the Zero-Order Distributed Policy Optimization algorithm (ZODPO) that learns linear local controllers in a distributed fashion, leveraging the ideas of policy gradient, zero-order optimization and consensus algorithms. In ZODPO, each agent estimates the global cost by consensus, and then conducts local policy gradient in parallel based on zero-order gradient estimation. ZODPO only requires limited communication and storage even in large-scale systems. Further, we investigate the nonasymptotic performance of ZODPO and show that the sample complexity to approach a stationary point is polynomial with the error tolerance’s inverse and the problem dimensions, demonstrating the scalability of ZODPO. We also show that the controllers generated by ZODPO are stabilizing with high probability. Lastly, we numerically test ZODPO on a multi-zone HVAC system.
Tasks
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09135v2
PDF	https://arxiv.org/pdf/1912.09135v2.pdf
PWC	https://paperswithcode.com/paper/distributed-reinforcement-learning-for
Repo
Framework

Weakly Supervised Clustering by Exploiting Unique Class Count


Title	Weakly Supervised Clustering by Exploiting Unique Class Count
Authors	Mustafa Umit Oner, Hwee Kuan Lee, Wing-Kin Sung
Abstract	A weakly supervised learning based clustering framework is proposed in this paper. As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count ($ucc$), which is the number of unique classes among all instances inside the bag. In this task, no annotations on individual instances inside the bag are needed during training of the models. We mathematically prove that with a perfect $ucc$ classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are given during training. We have constructed a neural network based $ucc$ classifier and experimentally shown that the clustering performance of our framework with our weakly supervised $ucc$ classifier is comparable to that of fully supervised learning models where labels for all instances are known. Furthermore, we have tested the applicability of our framework to a real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.
Tasks	Multiple Instance Learning, Semantic Segmentation, Zero-Shot Learning
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07647v2
PDF	https://arxiv.org/pdf/1906.07647v2.pdf
PWC	https://paperswithcode.com/paper/a-weakly-supervised-learning-based-clustering
Repo
Framework