April 1, 2020

3269 words 16 mins read

Paper Group ANR 432

UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision. Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^*$-Modules and Applications. Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations. Weighting NT …

UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision


Title	UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision
Authors	Tsun-Yi Yang, Duy-Kien Nguyen, Huub Heijnen, Vassileios Balntas
Abstract	In this paper, we explore how three related tasks, namely keypoint detection, description, and image retrieval can be jointly tackled using a single unified framework, which is trained without the need of training data with point to point correspondences. By leveraging diverse information from sequential layers of a standard ResNet-based architecture, we are able to extract keypoints and descriptors that encode local information using generic techniques such as local activation norms, channel grouping and dropping, and self-distillation. Subsequently, global information for image retrieval is encoded in an end-to-end pipeline, based on pooling of the aforementioned local responses. In contrast to previous methods in local matching, our method does not depend on pointwise/pixelwise correspondences, and requires no such supervision at all i.e. no depth-maps from an SfM model nor manually created synthetic affine transformations. We illustrate that this simple and direct paradigm, is able to achieve very competitive results against the state-of-the-art methods in various challenging benchmark conditions such as viewpoint changes, scale changes, and day-night shifting localization.
Tasks	Image Retrieval, Keypoint Detection
Published	2020-01-20
URL	https://arxiv.org/abs/2001.07252v1
PDF	https://arxiv.org/pdf/2001.07252v1.pdf
PWC	https://paperswithcode.com/paper/ur2kid-unifying-retrieval-keypoint-detection
Repo
Framework

Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^*$-Modules and Applications


Title	Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^*$-Modules and Applications
Authors	Yuka Hashimoto, Isao Ishikawa, Masahiro Ikeda, Fuyuta Komura, Takeshi Katsura, Yoshinobu Kawahara
Abstract	Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert $C^$-module (RKHM), which is another generalization of RKHS than vector-valued RKHS (vv-RKHS). Analysis with RKHMs enables us to deal with structures among variables more explicitly than vv-RKHS. We show the theoretical validity for the construction of orthonormal systems in Hilbert $C^$-modules, and derive concrete procedures for orthonormalization in RKHMs with those theoretical properties in numerical computations. Moreover, we apply those to generalize with RKHM kernel principal component analysis and the analysis of dynamical systems with Perron-Frobenius operators. The empirical performance of our methods is also investigated by using synthetic and real-world data.
Tasks
Published	2020-03-02
URL	https://arxiv.org/abs/2003.00738v1
PDF	https://arxiv.org/pdf/2003.00738v1.pdf
PWC	https://paperswithcode.com/paper/analysis-via-orthonormal-systems-in
Repo
Framework

Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations


Title	Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations
Authors	Saima Sharmin, Nitin Rathi, Priyadarshini Panda, Kaushik Roy
Abstract	In the recent quest for trustworthy neural networks, we present Spiking Neural Network (SNN) as a potential candidate for inherent robustness against adversarial attacks. In this work, we demonstrate that accuracy degradation is less severe in SNNs than in their non-spiking counterparts for CIFAR10 and CIFAR100 datasets on deep VGG architectures. We attribute this robustness to two fundamental characteristics of SNNs and analyze their effects. First, we exhibit that input discretization introduced by the Poisson encoder improves adversarial robustness with reduced number of timesteps. Second, we quantify the amount of adversarial accuracy with increased leak rate in Leaky-Integrate-Fire (LIF) neurons. Our results suggest that SNNs trained with LIF neurons and smaller number of timesteps are more robust than the ones with IF (Integrate-Fire) neurons and larger number of timesteps. We overcome the bottleneck of creating gradient-based adversarial inputs in temporal domain by proposing a technique for crafting attacks from SNN.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10399v1
PDF	https://arxiv.org/pdf/2003.10399v1.pdf
PWC	https://paperswithcode.com/paper/inherent-adversarial-robustness-of-deep
Repo
Framework

Weighting NTBEA for Game AI Optimisation


Title	Weighting NTBEA for Game AI Optimisation
Authors	James Goodman, Simon Lucas
Abstract	The N-Tuple Bandit Evolutionary Algorithm (NTBEA) has proven very effective in optimising algorithm parameters in Game AI. A potential weakness is the use of a simple average of all component Tuples in the model. This study investigates a refinement to the N-Tuple model used in NTBEA by weighting these component Tuples by their level of information and specificity of match. We introduce weighting functions to the model to obtain Weighted- NTBEA and test this on four benchmark functions and two game environments. These tests show that vanilla NTBEA is the most reliable and performant of the algorithms tested. Furthermore we show that given an iteration budget it is better to execute several independent NTBEA runs, and use part of the budget to find the best recommendation from these runs.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10378v2
PDF	https://arxiv.org/pdf/2003.10378v2.pdf
PWC	https://paperswithcode.com/paper/weighting-ntbea-for-game-ai-optimisation
Repo
Framework

Data-driven control of micro-climate in buildings; an event-triggered reinforcement learning approach


Title	Data-driven control of micro-climate in buildings; an event-triggered reinforcement learning approach
Authors	Ashkan Haji Hosseinloo, Alexander Ryzhov, Aldo Bischi, Henni Ouerdane, Konstantin Turitsyn, Munther A. Dahleh
Abstract	Smart buildings have great potential for shaping an energy-efficient, sustainable, and more economic future for our planet as buildings account for approximately 40% of the global energy consumption. A key challenge for large-scale plug and play deployment of the smart building technology is the ability to learn a good control policy in a short period of time, i.e. having a low sample complexity for the learning control agent. Motivated by this problem and to remedy the issue of high sample complexity in the general context of cyber-physical systems, we propose an event-triggered paradigm for learning and control with variable-time intervals, as opposed to the traditional constant-time sampling. The events occur when the system state crosses the a priori-parameterized switching manifolds; this crossing triggers the learning as well as the control processes. Policy gradient and temporal difference methods are employed to learn the optimal switching manifolds which define the optimal control policy. We propose two event-triggered learning algorithms for stochastic and deterministic control policies. We show the efficacy of our proposed approach via designing a smart learning thermostat for autonomous micro-climate control in buildings. The event-triggered algorithms are implemented on a single-zone building to decrease buildings’ energy consumption as well as to increase occupants’ comfort. Simulation results confirm the efficacy and improved sample efficiency of the proposed event-triggered approach for online learning and control.
Tasks
Published	2020-01-28
URL	https://arxiv.org/abs/2001.10505v1
PDF	https://arxiv.org/pdf/2001.10505v1.pdf
PWC	https://paperswithcode.com/paper/data-driven-control-of-micro-climate-in
Repo
Framework

On Interactive Machine Learning and the Potential of Cognitive Feedback


Title	On Interactive Machine Learning and the Potential of Cognitive Feedback
Authors	Chris J. Michael, Dina Acklin, Jaelle Scheuerman
Abstract	In order to increase productivity, capability, and data exploitation, numerous defense applications are experiencing an integration of state-of-the-art machine learning and AI into their architectures. Especially for defense applications, having a human analyst in the loop is of high interest due to quality control, accountability, and complex subject matter expertise not readily automated or replicated by AI. However, many applications are suffering from a very slow transition. This may be in large part due to lack of trust, usability, and productivity, especially when adapting to unforeseen classes and changes in mission context. Interactive machine learning is a newly emerging field in which machine learning implementations are trained, optimized, evaluated, and exploited through an intuitive human-computer interface. In this paper, we introduce interactive machine learning and explain its advantages and limitations within the context of defense applications. Furthermore, we address several of the shortcomings of interactive machine learning by discussing how cognitive feedback may inform features, data, and results in the state of the art. We define the three techniques by which cognitive feedback may be employed: self reporting, implicit cognitive feedback, and modeled cognitive feedback. The advantages and disadvantages of each technique are discussed.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10365v1
PDF	https://arxiv.org/pdf/2003.10365v1.pdf
PWC	https://paperswithcode.com/paper/on-interactive-machine-learning-and-the
Repo
Framework

Support recovery and sup-norm convergence rates for sparse pivotal estimation


Title	Support recovery and sup-norm convergence rates for sparse pivotal estimation
Authors	Mathurin Massias, Quentin Bertrand, Alexandre Gramfort, Joseph Salmon
Abstract	In high dimensional sparse regression, pivotal estimators are estimators for which the optimal regularization parameter is independent of the noise level. The canonical pivotal estimator is the square-root Lasso, formulated along with its derivatives as a ``non-smooth + non-smooth’’ optimization problem. Modern techniques to solve these include smoothing the datafitting term, to benefit from fast efficient proximal algorithms. In this work we show minimax sup-norm convergence rates for non smoothed and smoothed, single task and multitask square-root Lasso-type estimators. Thanks to our theoretical analysis, we provide some guidelines on how to set the smoothing hyperparameter, and illustrate on synthetic data the interest of such guidelines. \|
Tasks
Published	2020-01-15
URL	https://arxiv.org/abs/2001.05401v1
PDF	https://arxiv.org/pdf/2001.05401v1.pdf
PWC	https://paperswithcode.com/paper/support-recovery-and-sup-norm-convergence
Repo
Framework

Blank Language Models


Title	Blank Language Models
Authors	Tianxiao Shen, Victor Quach, Regina Barzilay, Tommi Jaakkola
Abstract	We propose Blank Language Model (BLM), a model that generates sequences by dynamically creating and filling in blanks. Unlike previous masked language models or the Insertion Transformer, BLM uses blanks to control which part of the sequence to expand. This fine-grained control of generation is ideal for a variety of text editing and rewriting tasks. The model can start from a single blank or partially completed text with blanks at specified locations. It iteratively determines which word to place in a blank and whether to insert new blanks, and stops generating when no blanks are left to fill. BLM can be efficiently trained using a lower bound of the marginal data likelihood, and achieves perplexity comparable to traditional left-to-right language models on the Penn Treebank and WikiText datasets. On the task of filling missing text snippets, BLM significantly outperforms all other baselines in terms of both accuracy and fluency. Experiments on style transfer and damaged ancient text restoration demonstrate the potential of this framework for a wide range of applications.
Tasks	Ancient Text Restoration, Language Modelling, Style Transfer
Published	2020-02-08
URL	https://arxiv.org/abs/2002.03079v1
PDF	https://arxiv.org/pdf/2002.03079v1.pdf
PWC	https://paperswithcode.com/paper/blank-language-models
Repo
Framework

How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction


Title	How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction
Authors	Se-eun Yoon, Hyungseok Song, Kijung Shin, Yung Yi
Abstract	Hypergraphs provide a natural way of representing group relations, whose complexity motivates an extensive array of prior work to adopt some form of abstraction and simplification of higher-order interactions. However, the following question has yet to be addressed: How much abstraction of group interactions is sufficient in solving a hypergraph task, and how different such results become across datasets? This question, if properly answered, provides a useful engineering guideline on how to trade off between complexity and accuracy of solving a downstream task. To this end, we propose a method of incrementally representing group interactions using a notion of n-projected graph whose accumulation contains information on up to n-way interactions, and quantify the accuracy of solving a task as n grows for various datasets. As a downstream task, we consider hyperedge prediction, an extension of link prediction, which is a canonical task for evaluating graph models. Through experiments on 15 real-world datasets, we draw the following messages: (a) Diminishing returns: small n is enough to achieve accuracy comparable with near-perfect approximations, (b) Troubleshooter: as the task becomes more challenging, larger n brings more benefit, and (c) Irreducibility: datasets whose pairwise interactions do not tell much about higher-order interactions lose much accuracy when reduced to pairwise abstractions.
Tasks	Link Prediction
Published	2020-01-30
URL	https://arxiv.org/abs/2001.11181v2
PDF	https://arxiv.org/pdf/2001.11181v2.pdf
PWC	https://paperswithcode.com/paper/how-much-and-when-do-we-need-higher-order
Repo
Framework

Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild


Title	Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild
Authors	Canjie Luo, Qingxiang Lin, Yuliang Liu, Lianwen Jin, Chunhua Shen
Abstract	In this work we propose to improve text recognition from a new perspective by separating text content from complex backgrounds. We exploit the generative adversarial networks (GANs) for removing backgrounds while retaining the text content. As vanilla GANs are not sufficiently robust to generate sequence-like characters in natural images, we propose an adversarial learning framework for the generation and recognition of multiple characters in an image. The proposed framework consists of an attention-based recognizer and a generative adversarial architecture. Furthermore, to tackle the lack of paired training samples, we design an interactive joint training scheme, which shares attention masks from the recognizer to the discriminator, and enables the discriminator to extract the features of every character for further adversarial training. Benefiting from the character-level adversarial training, our framework requires only unpaired simple data for style supervision. Every target style sample containing only one randomly chosen character can be simply synthesized online during the training. This is significant as the training does not require costly paired samples or character-level annotations. Thus, only the input images and corresponding text labels are needed. In addition to the style transfer of the backgrounds, we refine character patterns to ease the recognition task. A feedback mechanism is proposed to bridge the gap between the discriminator and the recognizer. Therefore, the discriminator can guide the generator according to the confusion of the recognizer. The generated patterns are thus clearer for recognition. Experiments on various benchmarks, including both regular and irregular text, demonstrate that our method significantly reduces the difficulty of recognition. Our framework can be integrated with recent recognition methods to achieve new state-of-the-art performance.
Tasks	Style Transfer
Published	2020-01-13
URL	https://arxiv.org/abs/2001.04189v1
PDF	https://arxiv.org/pdf/2001.04189v1.pdf
PWC	https://paperswithcode.com/paper/separating-content-from-style-using
Repo
Framework

A termination criterion for stochastic gradient descent for binary classification


Title	A termination criterion for stochastic gradient descent for binary classification
Authors	Sina Baghal, Courtney Paquette, Stephen A. Vavasis
Abstract	We propose a new, simple, and computationally inexpensive termination test for constant step-size stochastic gradient descent (SGD) applied to binary classification on the logistic and hinge loss with homogeneous linear predictors. Our theoretical results support the effectiveness of our stopping criterion when the data is Gaussian distributed. This presence of noise allows for the possibility of non-separable data. We show that our test terminates in a finite number of iterations and when the noise in the data is not too large, the expected classifier at termination nearly minimizes the probability of misclassification. Finally, numerical experiments indicate for both real and synthetic data sets that our termination test exhibits a good degree of predictability on accuracy and running time.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10312v1
PDF	https://arxiv.org/pdf/2003.10312v1.pdf
PWC	https://paperswithcode.com/paper/a-termination-criterion-for-stochastic
Repo
Framework

Searching for polarization in signed graphs: a local spectral approach


Title	Searching for polarization in signed graphs: a local spectral approach
Authors	Han Xiao, Bruno Ordozgoiti, Aristides Gionis
Abstract	Signed graphs have been used to model interactions in social net-works, which can be either positive (friendly) or negative (antagonistic). The model has been used to study polarization and other related phenomena in social networks, which can be harmful to the process of democratic deliberation in our society. An interesting and challenging task in this application domain is to detect polarized communities in signed graphs. A number of different methods have been proposed for this task. However, existing approaches aim at finding globally optimal solutions. Instead, in this paper we are interested in finding polarized communities that are related to a small set of seed nodes provided as input. Seed nodes may consist of two sets, which constitute the two sides of a polarized structure. In this paper we formulate the problem of finding local polarized communities in signed graphs as a locally-biased eigen-problem. By viewing the eigenvector associated with the smallest eigenvalue of the Laplacian matrix as the solution of a constrained optimization problem, we are able to incorporate the local information as an additional constraint. In addition, we show that the locally-biased vector can be used to find communities with approximation guarantee with respect to a local analogue of the Cheeger constant on signed graphs. By exploiting the sparsity in the input graph, an indicator vector for the polarized communities can be found in time linear to the graph size. Our experiments on real-world networks validate the proposed algorithm and demonstrate its usefulness in finding local structures in this semi-supervised manner.
Tasks
Published	2020-01-26
URL	https://arxiv.org/abs/2001.09410v1
PDF	https://arxiv.org/pdf/2001.09410v1.pdf
PWC	https://paperswithcode.com/paper/searching-for-polarization-in-signed-graphs-a
Repo
Framework

Generating new concepts with hybrid neuro-symbolic models


Title	Generating new concepts with hybrid neuro-symbolic models
Authors	Reuben Feinman, Brenden M. Lake
Abstract	Human conceptual knowledge supports the ability to generate novel yet highly structured concepts, and the form of this conceptual knowledge is of great interest to cognitive scientists. One tradition has emphasized structured knowledge, viewing concepts as embedded in intuitive theories or organized in complex symbolic knowledge structures. A second tradition has emphasized statistical knowledge, viewing conceptual knowledge as an emerging from the rich correlational structure captured by training neural networks and other statistical models. In this paper, we explore a synthesis of these two traditions through a novel neuro-symbolic model for generating new concepts. Using simple visual concepts as a testbed, we bring together neural networks and symbolic probabilistic programs to learn a generative model of novel handwritten characters. Two alternative models are explored with more generic neural network architectures. We compare each of these three models for their likelihoods on held-out character classes and for the quality of their productions, finding that our hybrid model learns the most convincing representation and generalizes further from the training observations.
Tasks
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08978v2
PDF	https://arxiv.org/pdf/2003.08978v2.pdf
PWC	https://paperswithcode.com/paper/generating-new-concepts-with-hybrid-neuro
Repo
Framework

Online Preselection with Context Information under the Plackett-Luce Model


Title	Online Preselection with Context Information under the Plackett-Luce Model
Authors	Adil El Mesaoudi-Paul, Viktor Bengs, Eyke Hüllermeier
Abstract	We consider an extension of the contextual multi-armed bandit problem, in which, instead of selecting a single alternative (arm), a learner is supposed to make a preselection in the form of a subset of alternatives. More specifically, in each iteration, the learner is presented a set of arms and a context, both described in terms of feature vectors. The task of the learner is to preselect $k$ of these arms, among which a final choice is made in a second step. In our setup, we assume that each arm has a latent (context-dependent) utility, and that feedback on a preselection is produced according to a Plackett-Luce model. We propose the CPPL algorithm, which is inspired by the well-known UCB algorithm, and evaluate this algorithm on synthetic and real data. In particular, we consider an online algorithm selection scenario, which served as a main motivation of our problem setting. Here, an instance (which defines the context) from a certain problem class (such as SAT) can be solved by different algorithms (the arms), but only $k$ of these algorithms can actually be run.
Tasks
Published	2020-02-11
URL	https://arxiv.org/abs/2002.04275v1
PDF	https://arxiv.org/pdf/2002.04275v1.pdf
PWC	https://paperswithcode.com/paper/online-preselection-with-context-information
Repo
Framework

Analysis of Genetic Algorithm on Bearings-Only Target Motion Analysis


Title	Analysis of Genetic Algorithm on Bearings-Only Target Motion Analysis
Authors	Erdem Kose
Abstract	Target motion analysis using only bearing angles is an important study for tracking targets in water. Several methods including Kalman-like filters and evolutionary strategies are used to get a good predictor. Kalman-like filters couldn’t get the expected results thus evolutionary strategies have been using in this area for a long time. Target Motion Analysis with Genetic Algorithm is the most successful method for Bearings-Only Target Motion Analysis and we investigated it. We found that Covariance Matrix Adaptation Evolutionary Strategies does the similar work with Target Motion Analysis with Genetic Algorithm and tried it; but it has statistical feedback mechanism and converges faster than other methods. In this study, we compared and criticize the methods.
Tasks
Published	2020-01-15
URL	https://arxiv.org/abs/2001.05381v1
PDF	https://arxiv.org/pdf/2001.05381v1.pdf
PWC	https://paperswithcode.com/paper/analysis-of-genetic-algorithm-on-bearings
Repo
Framework