January 28, 2020

3220 words 16 mins read

Paper Group ANR 771

Automatic Layout Generation with Applications in Machine Learning Engine Evaluation. OntoPlot: A Novel Visualisation for Non-hierarchical Associations in Large Ontologies. A Neural Network Architecture for Learning Word-Referent Associations in Multiple Contexts. Learning from Multi-User Activity Trails for B2B Ad Targeting. Property Invariant Embe …

Automatic Layout Generation with Applications in Machine Learning Engine Evaluation


Title	Automatic Layout Generation with Applications in Machine Learning Engine Evaluation
Authors	Haoyu Yang, Wen Chen, Piyush Pathak, Frank Gennari, Ya-Chieh Lai, Bei Yu
Abstract	Machine learning-based lithography hotspot detection has been deeply studied recently, from varies feature extraction techniques to efficient learning models. It has been observed that such machine learning-based frameworks are providing satisfactory metal layer hotspot prediction results on known public metal layer benchmarks. In this work, we seek to evaluate how these machine learning-based hotspot detectors generalize to complicated patterns. We first introduce a automatic layout generation tool that can synthesize varies layout patterns given a set of design rules. The tool currently supports both metal layer and via layer generation. As a case study, we conduct hotspot detection on the generated via layer layouts with representative machine learning-based hotspot detectors, which shows that continuous study on model robustness and generality is necessary to prototype and integrate the learning engines in DFM flows. The source code of the layout generation tool will be available at https://github. com/phdyang007/layout-generation.
Tasks
Published	2019-12-12
URL	https://arxiv.org/abs/1912.05796v1
PDF	https://arxiv.org/pdf/1912.05796v1.pdf
PWC	https://paperswithcode.com/paper/automatic-layout-generation-with-applications
Repo
Framework

OntoPlot: A Novel Visualisation for Non-hierarchical Associations in Large Ontologies


Title	OntoPlot: A Novel Visualisation for Non-hierarchical Associations in Large Ontologies
Authors	Ying Yang, Michael Wybrow, Yuan-Fang Li, Tobias Czauderna, Yongqun He
Abstract	Ontologies are formal representations of concepts and complex relationships among them. They have been widely used to capture comprehensive domain knowledge in areas such as biology and medicine, where large and complex ontologies can contain hundreds of thousands of concepts. Especially due to the large size of ontologies, visualisation is useful for authoring, exploring and understanding their underlying data. Existing ontology visualisation tools generally focus on the hierarchical structure, giving much less emphasis to non-hierarchical associations. In this paper we present OntoPlot, a novel visualisation specifically designed to facilitate the exploration of all concept associations whilst still showing an ontology’s large hierarchical structure. This hybrid visualisation combines icicle plots, visual compression techniques and interactivity, improving space-efficiency and reducing visual structural complexity. We conducted a user study with domain experts to evaluate the usability of OntoPlot, comparing it with the de facto ontology editor Prot{'e}g{'e}. The results confirm that OntoPlot attains our design goals for association-related tasks and is strongly favoured by domain experts.
Tasks
Published	2019-08-02
URL	https://arxiv.org/abs/1908.00688v2
PDF	https://arxiv.org/pdf/1908.00688v2.pdf
PWC	https://paperswithcode.com/paper/ontoplot-a-novel-visualisation-for-non
Repo
Framework

A Neural Network Architecture for Learning Word-Referent Associations in Multiple Contexts


Title	A Neural Network Architecture for Learning Word-Referent Associations in Multiple Contexts
Authors	Hansenclever F. Bassani, Aluizio F. R. Araujo
Abstract	This article proposes a biologically inspired neurocomputational architecture which learns associations between words and referents in different contexts, considering evidence collected from the literature of Psycholinguistics and Neurolinguistics. The multi-layered architecture takes as input raw images of objects (referents) and streams of word’s phonemes (labels), builds an adequate representation, recognizes the current context, and associates label with referents incrementally, by employing a Self-Organizing Map which creates new association nodes (prototypes) as required, adjusts the existing prototypes to better represent the input stimuli and removes prototypes that become obsolete/unused. The model takes into account the current context to retrieve the correct meaning of words with multiple meanings. Simulations show that the model can reach up to 78% of word-referent association accuracy in ambiguous situations and approximates well the learning rates of humans as reported by three different authors in five Cross-Situational Word Learning experiments, also displaying similar learning patterns in the different learning conditions.
Tasks
Published	2019-05-20
URL	https://arxiv.org/abs/1905.08300v1
PDF	https://arxiv.org/pdf/1905.08300v1.pdf
PWC	https://paperswithcode.com/paper/a-neural-network-architecture-for-learning
Repo
Framework

Learning from Multi-User Activity Trails for B2B Ad Targeting


Title	Learning from Multi-User Activity Trails for B2B Ad Targeting
Authors	Shaunak Mishra, Jelena Gligorijevic, Narayan Bhamidipati
Abstract	Online purchase decisions in organizations can go through a complex journey with multiple agents involved in the decision making process. Depending on the product being purchased, and the organizational structure, the process may involve employees who first conduct market research, and then influence decision makers who place the online purchase order. In such cases, the online activity trail of a single individual in the organization may only provide partial information for predicting purchases (conversions). To refine conversion prediction for business-to-business (B2B) products using online activity trails, we introduce the notion of relevant users in an organization with respect to a given B2B advertiser, and leverage the collective activity trails of such relevant users to predict conversions. In particular, our notion of relevant users is tied to a seed list of relevant activities for a B2B advertiser, and we propose a method using distributed activity representations to build such a seed list. Experiments using data from Yahoo Gemini demonstrate that the proposed methods can improve conversion prediction AUC by 8.8%, and provide an interpretable advertiser specific list of activities useful for B2B ad targeting.
Tasks	Decision Making
Published	2019-08-29
URL	https://arxiv.org/abs/1909.00057v1
PDF	https://arxiv.org/pdf/1909.00057v1.pdf
PWC	https://paperswithcode.com/paper/learning-from-multi-user-activity-trails-for
Repo
Framework

Property Invariant Embedding for Automated Reasoning


Title	Property Invariant Embedding for Automated Reasoning
Authors	Miroslav Olšák, Cezary Kaliszyk, Josef Urban
Abstract	Automated reasoning and theorem proving have recently become major challenges for machine learning. In other domains, representations that are able to abstract over unimportant transformations, such as abstraction over translations and rotations in vision, are becoming more common. Standard methods of embedding mathematical formulas for learning theorem proving are however yet unable to handle many important transformations. In particular, embedding previously unseen labels, that often arise in definitional encodings and in Skolemization, has been very weak so far. Similar problems appear when transferring knowledge between known symbols. We propose a novel encoding of formulas that extends existing graph neural network models. This encoding represents symbols only by nodes in the graph, without giving the network any knowledge of the original labels. We provide additional links between such nodes that allow the network to recover the meaning and therefore correctly embed such nodes irrespective of the given labels. We test the proposed encoding in an automated theorem prover based on the tableaux connection calculus, and show that it improves on the best characterizations used so far. The encoding is further evaluated on the premise selection task and a newly introduced symbol guessing task, and shown to correctly predict 65% of the symbol names.
Tasks	Automated Theorem Proving
Published	2019-11-27
URL	https://arxiv.org/abs/1911.12073v1
PDF	https://arxiv.org/pdf/1911.12073v1.pdf
PWC	https://paperswithcode.com/paper/property-invariant-embedding-for-automated
Repo
Framework

Clustered Gaussian Graphical Model via Symmetric Convex Clustering


Title	Clustered Gaussian Graphical Model via Symmetric Convex Clustering
Authors	Tianyi Yao, Genevera I. Allen
Abstract	Knowledge of functional groupings of neurons can shed light on structures of neural circuits and is valuable in many types of neuroimaging studies. However, accurately determining which neurons carry out similar neurological tasks via controlled experiments is both labor-intensive and prohibitively expensive on a large scale. Thus, it is of great interest to cluster neurons that have similar connectivity profiles into functionally coherent groups in a data-driven manner. In this work, we propose the clustered Gaussian graphical model (GGM) and a novel symmetric convex clustering penalty in an unified convex optimization framework for inferring functional clusters among neurons from neural activity data. A parallelizable multi-block Alternating Direction Method of Multipliers (ADMM) algorithm is used to solve the corresponding convex optimization problem. In addition, we establish convergence guarantees for the proposed ADMM algorithm. Experimental results on both synthetic data and real-world neuroscientific data demonstrate the effectiveness of our approach.
Tasks
Published	2019-05-30
URL	https://arxiv.org/abs/1905.13251v1
PDF	https://arxiv.org/pdf/1905.13251v1.pdf
PWC	https://paperswithcode.com/paper/clustered-gaussian-graphical-model-via
Repo
Framework

Classification of EEG Signals using Genetic Programming for Feature Construction


Title	Classification of EEG Signals using Genetic Programming for Feature Construction
Authors	Icaro Marcelino Miranda, Claus Aranha, Marcelo Ladeira
Abstract	The analysis of electroencephalogram (EEG) waves is of critical importance for the diagnosis of sleep disorders, such as sleep apnea and insomnia, besides that, seizures, epilepsy, head injuries, dizziness, headaches and brain tumors. In this context, one important task is the identification of visible structures in the EEG signal, such as sleep spindles and K-complexes. The identification of these structures is usually performed by visual inspection from human experts, a process that can be error prone and susceptible to biases. Therefore there is interest in developing technologies for the automated analysis of EEG. In this paper, we propose a new Genetic Programming (GP) framework for feature construction and dimensionality reduction from EEG signals. We use these features to automatically identify spindles and K-complexes on data from the DREAMS project. Using 5 different classifiers, the set of attributes produced by GP obtained better AUC scores than those obtained from PCA or the full set of attributes. Also, the results obtained from the proposed framework obtained a better balance of Specificity and Recall than other models recently proposed in the literature. Analysis of the features most used by GP also suggested improvements for data acquisition protocols in future EEG examinations.
Tasks	Dimensionality Reduction, EEG
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04403v1
PDF	https://arxiv.org/pdf/1906.04403v1.pdf
PWC	https://paperswithcode.com/paper/classification-of-eeg-signals-using-genetic
Repo
Framework

Sky pixel detection in outdoor imagery using an adaptive algorithm and machine learning


Title	Sky pixel detection in outdoor imagery using an adaptive algorithm and machine learning
Authors	Kerry A. Nice, Jasper S. Wijnands, Ariane Middel, Jingcheng Wang, Yiming Qiu, Nan Zhao, Jason Thompson, Gideon D. P. A. Aschwanden, Haifeng Zhao, Mark Stevenson
Abstract	Computer vision techniques enable automated detection of sky pixels in outdoor imagery. In urban climate, sky detection is an important first step in gathering information about urban morphology and sky view factors. However, obtaining accurate results remains challenging and becomes even more complex using imagery captured under a variety of lighting and weather conditions. To address this problem, we present a new sky pixel detection system demonstrated to produce accurate results using a wide range of outdoor imagery types. Images are processed using a selection of mean-shift segmentation, K-means clustering, and Sobel filters to mark sky pixels in the scene. The algorithm for a specific image is chosen by a convolutional neural network, trained with 25,000 images from the Skyfinder data set, reaching 82% accuracy for the top three classes. This selection step allows the sky marking to follow an adaptive process and to use different techniques and parameters to best suit a particular image. An evaluation of fourteen different techniques and parameter sets shows that no single technique can perform with high accuracy across varied Skyfinder and Google Street View data sets. However, by using our adaptive process, large increases in accuracy are observed. The resulting system is shown to perform better than other published techniques.
Tasks
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03182v2
PDF	https://arxiv.org/pdf/1910.03182v2.pdf
PWC	https://paperswithcode.com/paper/sky-pixel-detection-in-outdoor-imagery-using
Repo
Framework

Sibling Neural Estimators: Improving Iterative Image Decoding with Gradient Communication


Title	Sibling Neural Estimators: Improving Iterative Image Decoding with Gradient Communication
Authors	Ankur Mali, Alexander G. Ororbia, Clyde Lee Giles
Abstract	For lossy image compression, we develop a neural-based system which learns a nonlinear estimator for decoding from quantized representations. The system links two recurrent networks that \help” each other reconstruct same target image patches using complementary portions of spatial context that communicate via gradient signals. This dual agent system builds upon prior work that proposed the iterative refinement algorithm for recurrent neural network (RNN)based decoding which improved image reconstruction compared to standard decoding techniques. Our approach, which works with any encoder, neural or non-neural, This system progressively reduces image patch reconstruction error over a fixed number of steps. Experiment with variants of RNN memory cells, with and without future information, find that our model consistently creates lower distortion images of higher perceptual quality compared to other approaches. Specifically, on the Kodak Lossless True Color Image Suite, we observe as much as a 1:64 decibel (dB) gain over JPEG, a 1:46 dB gain over JPEG 2000, a 1:34 dB gain over the GOOG neural baseline, 0:36 over E2E (a modern competitive neural compression model), and 0:37 over a single iterative neural decoder.
Tasks	Image Compression, Image Reconstruction
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08478v1
PDF	https://arxiv.org/pdf/1911.08478v1.pdf
PWC	https://paperswithcode.com/paper/sibling-neural-estimators-improving-iterative
Repo
Framework

BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction


Title	BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction
Authors	Weipeng Huang, Xingyi Cheng, Taifeng Wang, Wei Chu
Abstract	In this paper, we report our method for the Information Extraction task in 2019 Language and Intelligence Challenge. We incorporate BERT into the multi-head selection framework for joint entity-relation extraction. This model extends existing approaches from three perspectives. First, BERT is adopted as a feature extraction layer at the bottom of the multi-head selection framework. We further optimize BERT by introducing a semantic-enhanced task during BERT pre-training. Second, we introduce a large-scale Baidu Baike corpus for entity recognition pre-training, which is of weekly supervised learning since there is no actual named entity label. Third, soft label embedding is proposed to effectively transmit information between entity recognition and relation extraction. Combining these three contributions, we enhance the information extracting ability of the multi-head selection model and achieve F1-score 0.876 on testset-1 with a single model. By ensembling four variants of our model, we finally achieve F1 score 0.892 (1st place) on testset-1 and F1 score 0.8924 (2nd place) on testset-2.
Tasks	Relation Extraction
Published	2019-08-16
URL	https://arxiv.org/abs/1908.05908v2
PDF	https://arxiv.org/pdf/1908.05908v2.pdf
PWC	https://paperswithcode.com/paper/bert-based-multi-head-selection-for-joint
Repo
Framework

A system of different layers of abstraction for artificial intelligence


Title	A system of different layers of abstraction for artificial intelligence
Authors	Alexander Serb, Themistoklis Prodromakis
Abstract	The field of artificial intelligence (AI) represents an enormous endeavour of humankind that is currently transforming our societies down to their very foundations. Its task, building truly intelligent systems, is underpinned by a vast array of subfields ranging from the development of new electronic components to mathematical formulations of highly abstract and complex reasoning. This breadth of subfields renders it often difficult to understand how they all fit together into a bigger picture and hides the multi-faceted, multi-layered conceptual structure that in a sense can be said to be what AI truly is. In this perspective we propose a system of five levels/layers of abstraction that underpin many AI implementations. We further posit that each layer is subject to a complexity-performance trade-off whilst different layers are interlocked with one another in a control-complexity trade-off. This overview provides a conceptual map that can help to identify how and where innovation should be targeted in order to achieve different levels of functionality, assure them for safety, optimise performance under various operating constraints and map the opportunity space for social and economic exploitation.
Tasks
Published	2019-07-22
URL	https://arxiv.org/abs/1907.10508v1
PDF	https://arxiv.org/pdf/1907.10508v1.pdf
PWC	https://paperswithcode.com/paper/a-system-of-different-layers-of-abstraction
Repo
Framework

Wasserstein Robust Reinforcement Learning


Title	Wasserstein Robust Reinforcement Learning
Authors	Mohammed Amin Abdullah, Hang Ren, Haitham Bou Ammar, Vladimir Milenkovic, Rui Luo, Mingtian Zhang, Jun Wang
Abstract	Reinforcement learning algorithms, though successful, tend to over-fit to training environments hampering their application to the real-world. This paper proposes $\text{W}\text{R}^{2}\text{L}$ – a robust reinforcement learning algorithm with significant robust performance on low and high-dimensional control tasks. Our method formalises robust reinforcement learning as a novel min-max game with a Wasserstein constraint for a correct and convergent solver. Apart from the formulation, we also propose an efficient and scalable solver following a novel zero-order optimisation method that we believe can be useful to numerical optimisation in general. We empirically demonstrate significant gains compared to standard and robust state-of-the-art algorithms on high-dimensional MuJuCo environments.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1907.13196v4
PDF	https://arxiv.org/pdf/1907.13196v4.pdf
PWC	https://paperswithcode.com/paper/wasserstein-robust-reinforcement-learning
Repo
Framework

The Perils of Exploration under Competition: A Computational Modeling Approach


Title	The Perils of Exploration under Competition: A Computational Modeling Approach
Authors	Guy Aridor, Kevin Liu, Aleksandrs Slivkins, Zhiwei Steven Wu
Abstract	We empirically study the interplay between exploration and competition. Systems that learn from interactions with users often engage in exploration: making potentially suboptimal decisions in order to acquire new information for future decisions. However, when multiple systems are competing for the same market of users, exploration may hurt a system’s reputation in the near term, with adverse competitive effects. In particular, a system may enter a “death spiral”, when the short-term reputation cost decreases the number of users for the system to learn from, which degrades its performance relative to competition and further decreases its market share. We ask whether better exploration algorithms are incentivized under competition. We run extensive numerical experiments in a stylized duopoly model in which two firms deploy multi-armed bandit algorithms and compete for myopic users. We find that duopoly and monopoly tend to favor a primitive “greedy algorithm” that does not explore and leads to low consumer welfare, whereas a temporary monopoly (a duopoly with an early entrant) may incentivize better bandit algorithms and lead to higher consumer welfare. Our findings shed light on the first-mover advantage in the digital economy by exploring the role that data can play as a barrier to entry in online markets.
Tasks
Published	2019-02-14
URL	http://arxiv.org/abs/1902.05590v2
PDF	http://arxiv.org/pdf/1902.05590v2.pdf
PWC	https://paperswithcode.com/paper/competing-bandits-the-perils-of-exploration
Repo
Framework

Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections


Title	Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections
Authors	Weinan E, Chao Ma, Qingcan Wang, Lei Wu
Abstract	The behavior of the gradient descent (GD) algorithm is analyzed for a deep neural network model with skip-connections. It is proved that in the over-parametrized regime, for a suitable initialization, with high probability GD can find a global minimum exponentially fast. Generalization error estimates along the GD path are also established. As a consequence, it is shown that when the target function is in the reproducing kernel Hilbert space (RKHS) with a kernel defined by the initialization, there exist generalizable early-stopping solutions along the GD path. In addition, it is also shown that the GD path is uniformly close to the functions given by the related random feature model. Consequently, in this “implicit regularization” setting, the deep neural network model deteriorates to a random feature model. Our results hold for neural networks of any width larger than the input dimension.
Tasks
Published	2019-04-10
URL	http://arxiv.org/abs/1904.05263v3
PDF	http://arxiv.org/pdf/1904.05263v3.pdf
PWC	https://paperswithcode.com/paper/analysis-of-the-gradient-descent-algorithm
Repo
Framework

VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction


Title	VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction
Authors	Hongsong Wang, Jiashi Feng
Abstract	Human motion prediction, which aims to predict future human poses given past poses, has recently seen increased interest. Many recent approaches are based on Recurrent Neural Networks (RNN) which model human poses with exponential maps. These approaches neglect the pose velocity as well as temporal relation of different poses, and tend to converge to the mean pose or fail to generate natural-looking poses. We therefore propose a novel Position-Velocity Recurrent Encoder-Decoder (PVRED) for human motion prediction, which makes full use of pose velocities and temporal positional information. A temporal position embedding method is presented and a Position-Velocity RNN (PVRNN) is proposed. We also emphasize the benefits of quaternion parameterization of poses and design a novel trainable Quaternion Transformation (QT) layer, which is combined with a robust loss function during training. Experiments on two human motion prediction benchmarks show that our approach considerably outperforms the state-of-the-art methods for both short-term prediction and long-term prediction. In particular, our proposed approach can predict future human-like and meaningful poses in 4000 milliseconds.
Tasks	motion prediction
Published	2019-06-15
URL	https://arxiv.org/abs/1906.06514v1
PDF	https://arxiv.org/pdf/1906.06514v1.pdf
PWC	https://paperswithcode.com/paper/vred-a-position-velocity-recurrent-encoder
Repo
Framework