January 30, 2020

2876 words 14 mins read

Paper Group ANR 402

Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm. Learning Rhyming Constraints using Structured Adversaries. Terminology-based Text Embedding for Computing Document Similarities on Technical Content. Can NetGAN be improved on short random walks?. JANOS: An Integrated Predictive and Prescriptive Mode …

Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm


Title	Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm
Authors	Zhiyuan Liu, Guohui Ding, Lijun Chen, Enoch Yeung
Abstract	We propose an alternating optimization algorithm to the nonconvex Koopman operator learning problem for nonlinear dynamic systems. We show that the proposed algorithm will converge to a critical point with rate $O(1/T)$ and $O(\frac{1}{\log T})$ for the constant and diminishing learning rates, respectively, under some mild conditions. To cope with the high dimensional nonlinear dynamical systems, we present the first-ever distributed Koopman operator learning algorithm. We show that the distributed Koopman operator learning has the same convergence properties as the centralized Koopman operator learning, in the absence of optimal tracker, so long as the basis functions satisfy a set of state-based decomposition conditions. Numerical experiments are provided to complement our theoretical results.
Tasks
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13455v2
PDF	https://arxiv.org/pdf/1909.13455v2.pdf
PWC	https://paperswithcode.com/paper/towards-scalable-koopman-operator-learning
Repo
Framework

Learning Rhyming Constraints using Structured Adversaries


Title	Learning Rhyming Constraints using Structured Adversaries
Authors	Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell, Taylor Berg-Kirkpatrick
Abstract	Existing recurrent neural language models often fail to capture higher-level structure present in text: for example, rhyming patterns present in poetry. Much prior work on poetry generation uses manually defined constraints which are satisfied during decoding using either specialized decoding procedures or rejection sampling. The rhyming constraints themselves are typically not learned by the generator. We propose an alternate approach that uses a structured discriminator to learn a poetry generator that directly captures rhyming constraints in a generative adversarial setup. By causing the discriminator to compare poems based only on a learned similarity matrix of pairs of line ending words, the proposed approach is able to successfully learn rhyming patterns in two different English poetry datasets (Sonnet and Limerick) without explicitly being provided with any phonetic information.
Tasks
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06743v1
PDF	https://arxiv.org/pdf/1909.06743v1.pdf
PWC	https://paperswithcode.com/paper/learning-rhyming-constraints-using-structured
Repo
Framework

Terminology-based Text Embedding for Computing Document Similarities on Technical Content


Title	Terminology-based Text Embedding for Computing Document Similarities on Technical Content
Authors	Hamid Mirisaee, Eric Gaussier, Cedric Lagnier, Agnes Guerraz
Abstract	We propose in this paper a new, hybrid document embedding approach in order to address the problem of document similarities with respect to the technical content. To do so, we employ a state-of-the-art graph techniques to first extract the keyphrases (composite keywords) of documents and, then, use them to score the sentences. Using the ranked sentences, we propose two approaches to embed documents and show their performances with respect to two baselines. With domain expert annotations, we illustrate that the proposed methods can find more relevant documents and outperform the baselines up to 27% in terms of NDCG.
Tasks	Document Embedding
Published	2019-06-05
URL	https://arxiv.org/abs/1906.01874v2
PDF	https://arxiv.org/pdf/1906.01874v2.pdf
PWC	https://paperswithcode.com/paper/terminology-based-text-embedding-for
Repo
Framework

Can NetGAN be improved on short random walks?


Title	Can NetGAN be improved on short random walks?
Authors	Amir Jalilifard, Vinicius Caridá, Alex Mansano, Rogers Cristo
Abstract	Graphs are useful structures that can model several important real-world problems. Recently, learning graphs have drawn considerable attention, leading to the proposal of new methods for learning these data structures. One of these studies produced NetGAN, a new approach for generating graphs via random walks. Although NetGAN has shown promising results in terms of accuracy in the tasks of generating graphs and link prediction, the choice of vertices from which it starts random walks can lead to inconsistent and highly variable results, especially when the length of walks is short. As an alternative to random starting, this study aims to establish a new method for initializing random walks from a set of dense vertices. We purpose estimating the importance of a node based on the inverse of its influence over the whole vertices of its neighborhood through random walks of different sizes. The proposed method manages to achieve significantly better accuracy, less variance and lesser outliers.
Tasks	Link Prediction
Published	2019-05-13
URL	https://arxiv.org/abs/1905.05298v2
PDF	https://arxiv.org/pdf/1905.05298v2.pdf
PWC	https://paperswithcode.com/paper/can-netgan-be-improved-by-short-random-walks
Repo
Framework

JANOS: An Integrated Predictive and Prescriptive Modeling Framework


Title	JANOS: An Integrated Predictive and Prescriptive Modeling Framework
Authors	David Bergman, Teng Huang, Philip Brooks, Andrea Lodi, Arvind U. Raghunathan
Abstract	Business research practice is witnessing a surge in the integration of predictive modeling and prescriptive analysis. We describe a modeling framework JANOS that seamlessly integrates the two streams of analytics, for the first time allowing researchers and practitioners to embed machine learning models in an optimization framework. JANOS allows for specifying a prescriptive model using standard optimization modeling elements such as constraints and variables. The key novelty lies in providing modeling constructs that allow for the specification of commonly used predictive models and their features as constraints and variables in the optimization model. The framework considers two sets of decision variables; regular and predicted. The relationship between the regular and the predicted variables are specified by the user as pre-trained predictive models. JANOS currently supports linear regression, logistic regression, and neural network with rectified linear activation functions, but we plan to expand on this set in the future. In this paper, we demonstrate the flexibility of the framework through an example on scholarship allocation in a student enrollment problem and provide a numeric performance evaluation.
Tasks
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09461v1
PDF	https://arxiv.org/pdf/1911.09461v1.pdf
PWC	https://paperswithcode.com/paper/janos-an-integrated-predictive-and
Repo
Framework

Revisiting Visual Grounding


Title	Revisiting Visual Grounding
Authors	Erik Conser, Kennedy Hahn, Chandler M. Watson, Melanie Mitchell
Abstract	We revisit a particular visual grounding method: the “Image Retrieval Using Scene Graphs” (IRSG) system of Johnson et al. (2015). Our experiments indicate that the system does not effectively use its learned object-relationship models. We also look closely at the IRSG dataset, as well as the widely used Visual Relationship Dataset (VRD) that is adapted from it. We find that these datasets exhibit biases that allow methods that ignore relationships to perform relatively well. We also describe several other problems with the IRSG dataset, and report on experiments using a subset of the dataset in which the biases and other problems are removed. Our studies contribute to a more general effort: that of better understanding what machine learning methods that combine language and vision actually learn and what popular datasets actually test.
Tasks	Image Retrieval
Published	2019-04-03
URL	http://arxiv.org/abs/1904.02225v1
PDF	http://arxiv.org/pdf/1904.02225v1.pdf
PWC	https://paperswithcode.com/paper/revisiting-visual-grounding
Repo
Framework

A Novel Orthogonal Direction Mesh Adaptive Direct Search Approach for SVM Hyperparameter Tuning


Title	A Novel Orthogonal Direction Mesh Adaptive Direct Search Approach for SVM Hyperparameter Tuning
Authors	Alexandre Reeberg Mello, Jonathan de Matos, Marcelo R. Stemmer, Alceu de Souza Britto Jr., Alessandro Lameiras Koerich
Abstract	In this paper, we propose the use of a black-box optimization method called deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal directions (Ortho-MADS) for the selection of hyperparameters of Support Vector Machines with a Gaussian kernel. Different from most of the methods in the literature that exploit the properties of the data or attempt to minimize the accuracy of a validation dataset over the first quadrant of (C, gamma), the Ortho-MADS provides convergence proof. We present the MADS, followed by the Ortho-MADS, the dynamic stopping criterion defined by the MADS mesh size and two different search strategies (Nelder-Mead and Variable Neighborhood Search) that contribute to a competitive convergence rate as well as a mechanism to escape from undesired local minima. We have investigated the practical selection of hyperparameters for the Support Vector Machine with a Gaussian kernel, i.e., properly choose the hyperparameters gamma (bandwidth) and C (trade-off) on several benchmark datasets. The experimental results have shown that the proposed approach for hyperparameter tuning consistently finds comparable or better solutions, when using a common configuration, than other methods. We have also evaluated the accuracy and the number of function evaluations of the Ortho-MADS with the Nelder-Mead search strategy and the Variable Neighborhood Search strategy using the mesh size as a stopping criterion, and we have achieved accuracy that no other method for hyperparameters optimization could reach.
Tasks
Published	2019-04-26
URL	http://arxiv.org/abs/1904.11649v1
PDF	http://arxiv.org/pdf/1904.11649v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-orthogonal-direction-mesh-adaptive
Repo
Framework

Super-resolved Chromatic Mapping of Snapshot Mosaic Image Sensors via a Texture Sensitive Residual Network


Title	Super-resolved Chromatic Mapping of Snapshot Mosaic Image Sensors via a Texture Sensitive Residual Network
Authors	Mehrdad Shoeiby, Lars Petersson, Mohammad Ali Armin, Sadegh Aliakbarian, Antonio Robles-Kelly
Abstract	This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fact that the spatial resolution of the imagery acquired by these sensors is low. Moreover, chromatic mapping in snapshot mosaic sensors is not straightforward since the bands delivered by the sensor tend to be narrow and unevenly distributed across the range in which they operate. We tackle this drawback as applied to chromatic mapping by using a residual channel attention network equipped with a texture sensitive block. Our method significantly outperforms the traditional approach of interpolating the image and, afterwards, applying a colour matching function. This work establishes state-of-the-art in this domain while also making available to the research community a dataset containing 296 registered stereo multi-spectral/RGB images pairs.
Tasks
Published	2019-09-05
URL	https://arxiv.org/abs/1909.02221v1
PDF	https://arxiv.org/pdf/1909.02221v1.pdf
PWC	https://paperswithcode.com/paper/super-resolved-chromatic-mapping-of-snapshot
Repo
Framework

FAQ-based Question Answering via Knowledge Anchors


Title	FAQ-based Question Answering via Knowledge Anchors
Authors	Ruobing Xie, Yanan Lu, Fen Lin, Leyu Lin
Abstract	Question answering (QA) aims to understand user questions and find appropriate answers. In real-world QA systems, Frequently Asked Question (FAQ) based QA is usually a practical and effective solution, especially for some complicated questions (e.g., How and Why). Recent years have witnessed the great successes of knowledge graphs (KGs) utilized in KBQA systems, while there are still few works focusing on making full use of KGs in FAQ-based QA. In this paper, we propose a novel Knowledge Anchor based Question Answering (KAQA) framework for FAQ-based QA to better understand questions and retrieve more appropriate answers. More specifically, KAQA mainly consists of three parts: knowledge graph construction, query anchoring and query-document matching. We consider entities and triples of KGs in texts as knowledge anchors to precisely capture the core semantics, which brings in higher precision and better interpretability. The multi-channel matching strategy also enable most sentence matching models to be flexibly plugged in out KAQA framework to fit different real-world computation costs. In experiments, we evaluate our models on a query-document matching task over a real-world FAQ-based QA dataset, with detailed analysis over different settings and cases. The results confirm the effectiveness and robustness of the KAQA framework in real-world FAQ-based QA.
Tasks	graph construction, Knowledge Graphs, Question Answering
Published	2019-11-14
URL	https://arxiv.org/abs/1911.05930v1
PDF	https://arxiv.org/pdf/1911.05930v1.pdf
PWC	https://paperswithcode.com/paper/faq-based-question-answering-via-knowledge
Repo
Framework

Generalized Clustering by Learning to Optimize Expected Normalized Cuts


Title	Generalized Clustering by Learning to Optimize Expected Normalized Cuts
Authors	Azade Nazi, Will Hang, Anna Goldie, Sujith Ravi, Azalia Mirhoseini
Abstract	We introduce a novel end-to-end approach for learning to cluster in the absence of labeled examples. Our clustering objective is based on optimizing normalized cuts, a criterion which measures both intra-cluster similarity as well as inter-cluster dissimilarity. We define a differentiable loss function equivalent to the expected normalized cuts. Unlike much of the work in unsupervised deep learning, our trained model directly outputs final cluster assignments, rather than embeddings that need further processing to be usable. Our approach generalizes to unseen datasets across a wide variety of domains, including text, and image. Specifically, we achieve state-of-the-art results on popular unsupervised clustering benchmarks (e.g., MNIST, Reuters, CIFAR-10, and CIFAR-100), outperforming the strongest baselines by up to 10.9%. Our generalization results are superior (by up to 21.9%) to the recent top-performing clustering approach with the ability to generalize.
Tasks
Published	2019-10-16
URL	https://arxiv.org/abs/1910.07623v1
PDF	https://arxiv.org/pdf/1910.07623v1.pdf
PWC	https://paperswithcode.com/paper/generalized-clustering-by-learning-to
Repo
Framework

Implicit Label Augmentation on Partially Annotated Clips via Temporally-Adaptive Features Learning


Title	Implicit Label Augmentation on Partially Annotated Clips via Temporally-Adaptive Features Learning
Authors	Yongxi Lu, Ziyao Tang, Tara Javidi
Abstract	Partially annotated clips contain rich temporal contexts that can complement the sparse key frame annotations in providing supervision for model training. We present a novel paradigm called Temporally-Adaptive Features (TAF) learning that can utilize such data to learn better single frame models. By imposing distinct temporal change rate constraints on different factors in the model, TAF enables learning from unlabeled frames using context to enhance model accuracy. TAF generalizes “slow feature” learning and we present much stronger empirical evidence than prior works, showing convincing gains for the challenging semantic segmentation task over a variety of architecture designs and on two popular datasets. TAF can be interpreted as an implicit label augmentation method but is a more principled formulation compared to existing explicit augmentation techniques. Our work thus connects two promising methods that utilize partially annotated clips for single frame model training and can inspire future explorations in this direction.
Tasks	Semantic Segmentation
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10000v1
PDF	https://arxiv.org/pdf/1905.10000v1.pdf
PWC	https://paperswithcode.com/paper/implicit-label-augmentation-on-partially
Repo
Framework

A unified variance-reduced accelerated gradient method for convex optimization


Title	A unified variance-reduced accelerated gradient method for convex optimization
Authors	Guanghui Lan, Zhize Li, Yi Zhou
Abstract	We propose a novel randomized incremental gradient algorithm, namely, VAriance-Reduced Accelerated Gradient (Varag), for finite-sum optimization. Equipped with a unified step-size policy that adjusts itself to the value of the condition number, Varag exhibits the unified optimal rates of convergence for solving smooth convex finite-sum problems directly regardless of their strong convexity. Moreover, Varag is the first accelerated randomized incremental gradient method that benefits from the strong convexity of the data-fidelity term to achieve the optimal linear convergence. It also establishes an optimal linear rate of convergence for solving a wide class of problems only satisfying a certain error bound condition rather than strong convexity. Varag can also be extended to solve stochastic finite-sum problems.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12412v3
PDF	https://arxiv.org/pdf/1905.12412v3.pdf
PWC	https://paperswithcode.com/paper/a-unified-variance-reduced-accelerated
Repo
Framework

Autonomous exploration for navigating in non-stationary CMPs


Title	Autonomous exploration for navigating in non-stationary CMPs
Authors	Pratik Gajane, Ronald Ortner, Peter Auer, Csaba Szepesvari
Abstract	We consider a setting in which the objective is to learn to navigate in a controlled Markov process (CMP) where transition probabilities may abruptly change. For this setting, we propose a performance measure called exploration steps which counts the time steps at which the learner lacks sufficient knowledge to navigate its environment efficiently. We devise a learning meta-algorithm, MNM and prove an upper bound on the exploration steps in terms of the number of changes.
Tasks
Published	2019-10-18
URL	https://arxiv.org/abs/1910.08446v1
PDF	https://arxiv.org/pdf/1910.08446v1.pdf
PWC	https://paperswithcode.com/paper/autonomous-exploration-for-navigating-in-non
Repo
Framework

Semantic-guided Encoder Feature Learning for Blurry Boundary Delineation


Title	Semantic-guided Encoder Feature Learning for Blurry Boundary Delineation
Authors	Dong Nie, Dinggang Shen
Abstract	Encoder-decoder architectures are widely adopted for medical image segmentation tasks. With the lateral skip connection, the models can obtain and fuse both semantic and resolution information in deep layers to achieve more accurate segmentation performance. However, in many applications (e.g., blurry boundary images), these models often cannot precisely locate complex boundaries and segment tiny isolated parts. To solve this challenging problem, we firstly analyze why simple skip connections are not enough to help accurately locate indistinct boundaries and argue that it is due to the fuzzy information in the skip connection provided in the encoder layers. Then we propose a semantic-guided encoder feature learning strategy to learn both high resolution and rich semantic encoder features so that we can more accurately locate the blurry boundaries, which can also enhance the network by selectively learning discriminative features. Besides, we further propose a soft contour constraint mechanism to model the blurry boundary detection. Experimental results on real clinical datasets show that our proposed method can achieve state-of-the-art segmentation accuracy, especially for the blurry regions. Further analysis also indicates that our proposed network components indeed contribute to the improvement of performance. Experiments on additional datasets validate the generalization ability of our proposed method.
Tasks	Boundary Detection, Medical Image Segmentation, Semantic Segmentation
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04306v1
PDF	https://arxiv.org/pdf/1906.04306v1.pdf
PWC	https://paperswithcode.com/paper/semantic-guided-encoder-feature-learning-for
Repo
Framework

Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy


Title	Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy
Authors	Kisuk Lee, Nicholas Turner, Thomas Macrina, Jingpeng Wu, Ran Lu, H. Sebastian Seung
Abstract	Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.
Tasks	Boundary Detection
Published	2019-04-29
URL	http://arxiv.org/abs/1904.12966v1
PDF	http://arxiv.org/pdf/1904.12966v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-nets-for-reconstructing-neural
Repo
Framework