October 18, 2019

3132 words 15 mins read

Paper Group ANR 630

Interpreting Adversarial Robustness: A View from Decision Surface in Input Space. Learning Graph Representation via Formal Concept Analysis. Quantum Entanglement in Deep Learning Architectures. Flipped-Adversarial AutoEncoders. Denoising Prior Driven Deep Neural Network for Image Restoration. Mimetic vs Anchored Value Alignment in Artificial Intell …

Interpreting Adversarial Robustness: A View from Decision Surface in Input Space


Title	Interpreting Adversarial Robustness: A View from Decision Surface in Input Space
Authors	Fuxun Yu, Chenchen Liu, Yanzhi Wang, Liang Zhao, Xiang Chen
Abstract	One popular hypothesis of neural network generalization is that the flat local minima of loss surface in parameter space leads to good generalization. However, we demonstrate that loss surface in parameter space has no obvious relationship with generalization, especially under adversarial settings. Through visualizing decision surfaces in both parameter space and input space, we instead show that the geometry property of decision surface in input space correlates well with the adversarial robustness. We then propose an adversarial robustness indicator, which can evaluate a neural network’s intrinsic robustness property without testing its accuracy under adversarial attacks. Guided by it, we further propose our robust training method. Without involving adversarial training, our method could enhance network’s intrinsic adversarial robustness against various adversarial attacks.
Tasks
Published	2018-09-29
URL	http://arxiv.org/abs/1810.00144v2
PDF	http://arxiv.org/pdf/1810.00144v2.pdf
PWC	https://paperswithcode.com/paper/interpreting-adversarial-robustness-a-view
Repo
Framework

Learning Graph Representation via Formal Concept Analysis


Title	Learning Graph Representation via Formal Concept Analysis
Authors	Yuka Yoneda, Mahito Sugiyama, Takashi Washio
Abstract	We present a novel method that can learn a graph representation from multivariate data. In our representation, each node represents a cluster of data points and each edge represents the subset-superset relationship between clusters, which can be mutually overlapped. The key to our method is to use formal concept analysis (FCA), which can extract hierarchical relationships between clusters based on the algebraic closedness property. We empirically show that our method can effectively extract hierarchical structures of clusters compared to the baseline method.
Tasks
Published	2018-12-08
URL	http://arxiv.org/abs/1812.03395v1
PDF	http://arxiv.org/pdf/1812.03395v1.pdf
PWC	https://paperswithcode.com/paper/learning-graph-representation-via-formal
Repo
Framework

Quantum Entanglement in Deep Learning Architectures


Title	Quantum Entanglement in Deep Learning Architectures
Authors	Yoav Levine, Or Sharir, Nadav Cohen, Amnon Shashua
Abstract	Modern deep learning has enabled unprecedented achievements in various domains. Nonetheless, employment of machine learning for wave function representations is focused on more traditional architectures such as restricted Boltzmann machines (RBMs) and fully-connected neural networks. In this letter, we establish that contemporary deep learning architectures, in the form of deep convolutional and recurrent networks, can efficiently represent highly entangled quantum systems. By constructing Tensor Network equivalents of these architectures, we identify an inherent reuse of information in the network operation as a key trait which distinguishes them from standard Tensor Network based representations, and which enhances their entanglement capacity. Our results show that such architectures can support volume-law entanglement scaling, polynomially more efficiently than presently employed RBMs. Thus, beyond a quantification of the entanglement capacity of leading deep learning architectures, our analysis formally motivates a shift of trending neural-network based wave function representations closer to the state-of-the-art in machine learning.
Tasks	Image Classification
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09780v3
PDF	http://arxiv.org/pdf/1803.09780v3.pdf
PWC	https://paperswithcode.com/paper/quantum-entanglement-in-deep-learning
Repo
Framework

Flipped-Adversarial AutoEncoders


Title	Flipped-Adversarial AutoEncoders
Authors	Jiyi Zhang, Hung Dang, Hwee Kuan Lee, Ee-Chien Chang
Abstract	We propose a flipped-Adversarial AutoEncoder (FAAE) that simultaneously trains a generative model G that maps an arbitrary latent code distribution to a data distribution and an encoder E that embodies an “inverse mapping” that encodes a data sample into a latent code vector. Unlike previous hybrid approaches that leverage adversarial training criterion in constructing autoencoders, FAAE minimizes re-encoding errors in the latent space and exploits adversarial criterion in the data space. Experimental evaluations demonstrate that the proposed framework produces sharper reconstructed images while at the same time enabling inference that captures rich semantic representation of data.
Tasks
Published	2018-02-13
URL	http://arxiv.org/abs/1802.04504v5
PDF	http://arxiv.org/pdf/1802.04504v5.pdf
PWC	https://paperswithcode.com/paper/flipped-adversarial-autoencoders
Repo
Framework

Denoising Prior Driven Deep Neural Network for Image Restoration


Title	Denoising Prior Driven Deep Neural Network for Image Restoration
Authors	Weisheng Dong, Peiyao Wang, Wotao Yin, Guangming Shi, Fangfang Wu, Xiaotong Lu
Abstract	Deep neural networks (DNNs) have shown very promising results for various image restoration (IR) tasks. However, the design of network architectures remains a major challenging for achieving further improvements. While most existing DNN-based methods solve the IR problems by directly mapping low quality images to desirable high-quality images, the observation models characterizing the image degradation processes have been largely ignored. In this paper, we first propose a denoising-based IR algorithm, whose iterative steps can be computed efficiently. Then, the iterative process is unfolded into a deep neural network, which is composed of multiple denoisers modules interleaved with back-projection (BP) modules that ensure the observation consistencies. A convolutional neural network (CNN) based denoiser that can exploit the multi-scale redundancies of natural images is proposed. As such, the proposed network not only exploits the powerful denoising ability of DNNs, but also leverages the prior of the observation model. Through end-to-end training, both the denoisers and the BP modules can be jointly optimized. Experimental results on several IR tasks, e.g., image denoising, super-resolution and deblurring show that the proposed method can lead to very competitive and often state-of-the-art results on several IR tasks, including image denoising, deblurring and super-resolution.
Tasks	Deblurring, Denoising, Image Denoising, Image Restoration, Super-Resolution
Published	2018-01-21
URL	http://arxiv.org/abs/1801.06756v1
PDF	http://arxiv.org/pdf/1801.06756v1.pdf
PWC	https://paperswithcode.com/paper/denoising-prior-driven-deep-neural-network
Repo
Framework

Mimetic vs Anchored Value Alignment in Artificial Intelligence


Title	Mimetic vs Anchored Value Alignment in Artificial Intelligence
Authors	Tae Wan Kim, Thomas Donaldson, John Hooker
Abstract	“Value alignment” (VA) is considered as one of the top priorities in AI research. Much of the existing research focuses on the “A” part and not the “V” part of “value alignment.” This paper corrects that neglect by emphasizing the “value” side of VA and analyzes VA from the vantage point of requirements in value theory, in particular, of avoiding the “naturalistic fallacy”–a major epistemic caveat. The paper begins by isolating two distinct forms of VA: “mimetic” and “anchored.” Then it discusses which VA approach better avoids the naturalistic fallacy. The discussion reveals stumbling blocks for VA approaches that neglect implications of the naturalistic fallacy. Such problems are more serious in mimetic VA since the mimetic process imitates human behavior that may or may not rise to the level of correct ethical behavior. Anchored VA, including hybrid VA, in contrast, holds more promise for future VA since it anchors alignment by normative concepts of intrinsic value.
Tasks
Published	2018-10-25
URL	http://arxiv.org/abs/1810.11116v1
PDF	http://arxiv.org/pdf/1810.11116v1.pdf
PWC	https://paperswithcode.com/paper/mimetic-vs-anchored-value-alignment-in
Repo
Framework

Algorithms and Conditional Lower Bounds for Planning Problems


Title	Algorithms and Conditional Lower Bounds for Planning Problems
Authors	Krishnendu Chatterjee, Wolfgang Dvořák, Monika Henzinger, Alexander Svozil
Abstract	We consider planning problems for graphs, Markov decision processes (MDPs), and games on graphs. While graphs represent the most basic planning model, MDPs represent interaction with nature and games on graphs represent interaction with an adversarial environment. We consider two planning problems where there are k different target sets, and the problems are as follows: (a) the coverage problem asks whether there is a plan for each individual target set, and (b) the sequential target reachability problem asks whether the targets can be reached in sequence. For the coverage problem, we present a linear-time algorithm for graphs and quadratic conditional lower bound for MDPs and games on graphs. For the sequential target problem, we present a linear-time algorithm for graphs, a sub-quadratic algorithm for MDPs, and a quadratic conditional lower bound for games on graphs. Our results with conditional lower bounds establish (i) model-separation results showing that for the coverage problem MDPs and games on graphs are harder than graphs and for the sequential reachability problem games on graphs are harder than MDPs and graphs; (ii) objective-separation results showing that for MDPs the coverage problem is harder than the sequential target problem.
Tasks
Published	2018-04-19
URL	http://arxiv.org/abs/1804.07031v1
PDF	http://arxiv.org/pdf/1804.07031v1.pdf
PWC	https://paperswithcode.com/paper/algorithms-and-conditional-lower-bounds-for
Repo
Framework


Title	Textbook Question Answering with Multi-modal Context Graph Understanding and Self-supervised Open-set Comprehension
Authors	Daesik Kim, Seonhoon Kim, Nojun Kwak
Abstract	In this work, we introduce a novel algorithm for solving the textbook question answering (TQA) task which describes more realistic QA problems compared to other recent tasks. We mainly focus on two related issues with analysis of the TQA dataset. First, solving the TQA problems requires to comprehend multi-modal contexts in complicated input data. To tackle this issue of extracting knowledge features from long text lessons and merging them with visual features, we establish a context graph from texts and images, and propose a new module f-GCN based on graph convolutional networks (GCN). Second, scientific terms are not spread over the chapters and subjects are split in the TQA dataset. To overcome this so called “out-of-domain” issue, before learning QA problems, we introduce a novel self-supervised open-set learning process without any annotations. The experimental results show that our model significantly outperforms prior state-of-the-art methods. Moreover, ablation studies validate that both methods of incorporating f-GCN for extracting knowledge from multi-modal contexts and our newly proposed self-supervised learning process are effective for TQA problems.
Tasks	Open Set Learning, Question Answering, Reading Comprehension
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00232v2
PDF	https://arxiv.org/pdf/1811.00232v2.pdf
PWC	https://paperswithcode.com/paper/textbook-question-answering-with-knowledge
Repo
Framework

Complex Relations in a Deep Structured Prediction Model for Fine Image Segmentation


Title	Complex Relations in a Deep Structured Prediction Model for Fine Image Segmentation
Authors	Cristina Mata, Guy Ben-Yosef, Boris Katz
Abstract	Many deep learning architectures for semantic segmentation involve a Fully Convolutional Neural Network (FCN) followed by a Conditional Random Field (CRF) to carry out inference over an image. These models typically involve unary potentials based on local appearance features computed by FCNs, and binary potentials based on the displacement between pixels. We show that while current methods succeed in segmenting whole objects, they perform poorly in situations involving a large number of object parts. We therefore suggest incorporating into the inference algorithm additional higher-order potentials inspired by the way humans identify and localize parts. We incorporate two relations that were shown to be useful to human object identification - containment and attachment - into the energy term of the CRF and evaluate their performance on the Pascal VOC Parts dataset. Our experimental results show that the segmentation of fine parts is positively affected by the addition of these two relations, and that the segmentation of fine parts can be further influenced by complex structural features.
Tasks	Semantic Segmentation, Structured Prediction
Published	2018-05-24
URL	http://arxiv.org/abs/1805.09462v1
PDF	http://arxiv.org/pdf/1805.09462v1.pdf
PWC	https://paperswithcode.com/paper/complex-relations-in-a-deep-structured
Repo
Framework

Encoding prior knowledge in the structure of the likelihood


Title	Encoding prior knowledge in the structure of the likelihood
Authors	Jakob Knollmüller, Torsten A. Enßlin
Abstract	The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy and leads to a standard Gaussian prior on all resulting parameters. The transformation also transfers all the prior information into the structure of the likelihood, hereby decoupling the transformed parameters a priori from each other. A variational Gaussian approximation in this standardized space will be excellent in situations of relatively uninformative data. Additionally, the curvature of the log-posterior is well-conditioned in directions that are weakly constrained by the data, allowing for fast inference in such a scenario. In an example we perform the transformation explicitly for Gaussian process regression with a priori unknown correlation structure. Deep models are inferred rapidly in highly and slowly in poorly informed situations. The flat model show exactly the opposite performance pattern. A synthesis of both, the deep and the flat perspective, provides their combined advantages and overcomes the individual limitations, leading to a faster inference.
Tasks
Published	2018-12-11
URL	http://arxiv.org/abs/1812.04403v1
PDF	http://arxiv.org/pdf/1812.04403v1.pdf
PWC	https://paperswithcode.com/paper/encoding-prior-knowledge-in-the-structure-of
Repo
Framework

Convolutional Networks in Visual Environments


Title	Convolutional Networks in Visual Environments
Authors	Alessandro Betti, Marco Gori
Abstract	The puzzle of computer vision might find new challenging solutions when we realize that most successful methods are working at image level, which is remarkably more difficult than processing directly visual streams. In this paper, we claim that their processing naturally leads to formulate the motion invariance principle, which enables the construction of a new theory of learning with convolutional networks. The theory addresses a number of intriguing questions that arise in natural vision, and offers a well-posed computational scheme for the discovery of convolutional filters over the retina. They are driven by differential equations derived from the principle of least cognitive action. Unlike traditional convolutional networks, which need massive supervision, the proposed theory offers a truly new scenario in which feature learning takes place by unsupervised processing of video signals. It is pointed out that an opportune blurring of the video, along the interleaving of segments of null signal, make it possible to conceive a novel learning mechanism that yields the minimum of the cognitive action. Basically, while the theory enables the implementation of novel computer vision systems, it is also provides an intriguing explanation of the solution that evolution has discovered for humans, where it looks like that the video blurring in newborns and the day-night rhythm seem to emerge in a general computational framework, regardless of biology.
Tasks
Published	2018-01-16
URL	http://arxiv.org/abs/1801.07110v1
PDF	http://arxiv.org/pdf/1801.07110v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-networks-in-visual-environments
Repo
Framework

A Deterministic Algorithm for Bridging Anaphora Resolution


Title	A Deterministic Algorithm for Bridging Anaphora Resolution
Authors	Yufang Hou
Abstract	Previous work on bridging anaphora resolution (Poesio et al., 2004; Hou et al., 2013b) use syntactic preposition patterns to calculate word relatedness. However, such patterns only consider NPs’ head nouns and hence do not fully capture the semantics of NPs. Recently, Hou (2018) created word embeddings (embeddings_PP) to capture associative similarity (ie, relatedness) between nouns by exploring the syntactic structure of noun phrases. But embeddings_PP only contains word representations for nouns. In this paper, we create new word vectors by combining embeddings_PP with GloVe. This new word embeddings (embeddings_bridging) are a more general lexical knowledge resource for bridging and allow us to represent the meaning of an NP beyond its head easily. We therefore develop a deterministic approach for bridging anaphora resolution, which represents the semantics of an NP based on its head noun and modifications. We show that this simple approach achieves the competitive results compared to the best system in Hou et al.(2013b) which explores Markov Logic Networks to model the problem. Additionally, we further improve the results for bridging anaphora resolution reported in Hou (2018) by combining our simple deterministic approach with Hou et al.(2013b)‘s best system MLN II.
Tasks	Bridging Anaphora Resolution, Word Embeddings
Published	2018-11-14
URL	http://arxiv.org/abs/1811.05721v1
PDF	http://arxiv.org/pdf/1811.05721v1.pdf
PWC	https://paperswithcode.com/paper/a-deterministic-algorithm-for-bridging
Repo
Framework

Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning


Title	Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning
Authors	Rise Ooi, Chao-Han Huck Yang, Pin-Yu Chen, Vìctor Eguìluz, Narsis Kiani, Hector Zenil, David Gomez-Cabrero, Jesper Tegnèr
Abstract	Networks are fundamental building blocks for representing data, and computations. Remarkable progress in learning in structurally defined (shallow or deep) networks has recently been achieved. Here we introduce evolutionary exploratory search and learning method of topologically flexible networks under the constraint of producing elementary computational steady-state input-output operations. Our results include; (1) the identification of networks, over four orders of magnitude, implementing computation of steady-state input-output functions, such as a band-pass filter, a threshold function, and an inverse band-pass function. Next, (2) the learned networks are technically controllable as only a small number of driver nodes are required to move the system to a new state. Furthermore, we find that the fraction of required driver nodes is constant during evolutionary learning, suggesting a stable system design. (3), our framework allows multiplexing of different computations using the same network. For example, using a binary representation of the inputs, the network can readily compute three different input-output functions. Finally, (4) the proposed evolutionary learning demonstrates transfer learning. If the system learns one function A, then learning B requires on average less number of steps as compared to learning B from tabula rasa. We conclude that the constrained evolutionary learning produces large robust controllable circuits, capable of multiplexing and transfer learning. Our study suggests that network-based computations of steady-state functions, representing either cellular modules of cell-to-cell communication networks or internal molecular circuits communicating within a cell, could be a powerful model for biologically inspired computing. This complements conceptualizations such as attractor based models, or reservoir computing.
Tasks	Transfer Learning
Published	2018-11-14
URL	https://arxiv.org/abs/1811.05592v2
PDF	https://arxiv.org/pdf/1811.05592v2.pdf
PWC	https://paperswithcode.com/paper/controllability-multiplexing-and-transfer
Repo
Framework

Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations


Title	Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations
Authors	Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, Oren Sar Shalom
Abstract	Latent factor models for recommender systems represent users and items as low dimensional vectors. Privacy risks of such systems have previously been studied mostly in the context of recovery of personal information in the form of usage records from the training data. However, the user representations themselves may be used together with external data to recover private user information such as gender and age. In this paper we show that user vectors calculated by a common recommender system can be exploited in this way. We propose the privacy-adversarial framework to eliminate such leakage of private information, and study the trade-off between recommender performance and leakage both theoretically and empirically using a benchmark dataset. An advantage of the proposed method is that it also helps guarantee fairness of results, since all implicit knowledge of a set of attributes is scrubbed from the representations used by the model, and thus can’t enter into the decision making. We discuss further applications of this method towards the generation of deeper and more insightful recommendations.
Tasks	Decision Making, Recommendation Systems
Published	2018-07-10
URL	http://arxiv.org/abs/1807.03521v3
PDF	http://arxiv.org/pdf/1807.03521v3.pdf
PWC	https://paperswithcode.com/paper/privacy-and-fairness-in-recommender-systems
Repo
Framework

Weighted Low-Rank Approximation of Matrices and Background Modeling


Title	Weighted Low-Rank Approximation of Matrices and Background Modeling
Authors	Aritra Dutta, Xin Li, Peter Richtarik
Abstract	We primarily study a special a weighted low-rank approximation of matrices and then apply it to solve the background modeling problem. We propose two algorithms for this purpose: one operates in the batch mode on the entire data and the other one operates in the batch-incremental mode on the data and naturally captures more background variations and computationally more effective. Moreover, we propose a robust technique that learns the background frame indices from the data and does not require any training frames. We demonstrate through extensive experiments that by inserting a simple weight in the Frobenius norm, it can be made robust to the outliers similar to the $\ell_1$ norm. Our methods match or outperform several state-of-the-art online and batch background modeling methods in virtually all quantitative and qualitative measures.
Tasks
Published	2018-04-15
URL	http://arxiv.org/abs/1804.06252v1
PDF	http://arxiv.org/pdf/1804.06252v1.pdf
PWC	https://paperswithcode.com/paper/weighted-low-rank-approximation-of-matrices
Repo
Framework