Paper Group ANR 1502
Unsupervised Learning and Exploration of Reachable Outcome Space. Data Science through the looking glass and what we found there. Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings. Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks. Native Banach spaces for splines and variational in …
Unsupervised Learning and Exploration of Reachable Outcome Space
| Title | Unsupervised Learning and Exploration of Reachable Outcome Space |
| Authors | Giuseppe Paolo, Alban Laflaquière, Alexandre Coninx, Stephane Doncieux |
| Abstract | Performing Reinforcement Learning in sparse rewards settings, with very little prior knowledge, is a challenging problem since there is no signal to properly guide the learning process. In such situations, a good search strategy is fundamental. At the same time, not having to adapt the algorithm to every single problem is very desirable. Here we introduce TAXONS, a Task Agnostic eXploration of Outcome spaces through Novelty and Surprise algorithm. Based on a population-based divergent-search approach, it learns a set of diverse policies directly from high-dimensional observations, without any task-specific information. TAXONS builds a repertoire of policies while training an autoencoder on the high-dimensional observation of the final state of the system to build a low-dimensional outcome space. The learned outcome space, combined with the reconstruction error, is used to drive the search for new policies. Results show that TAXONS can find a diverse set of controllers, covering a good part of the ground-truth outcome space, while having no information about such space. |
| Tasks | |
| Published | 2019-09-12 |
| URL | https://arxiv.org/abs/1909.05508v3 |
| https://arxiv.org/pdf/1909.05508v3.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-learning-and-exploration-of |
| Repo | |
| Framework | |
Data Science through the looking glass and what we found there
| Title | Data Science through the looking glass and what we found there |
| Authors | Fotis Psallidas, Yiwen Zhu, Bojan Karlas, Matteo Interlandi, Avrilia Floratou, Konstantinos Karanasos, Wentao Wu, Ce Zhang, Subru Krishnan, Carlo Curino, Markus Weimer |
| Abstract | The recent success of machine learning (ML) has led to an explosive growth both in terms of new systems and algorithms built in industry and academia, and new applications built by an ever-growing community of data science (DS) practitioners. This quickly shifting panorama of technologies and applications is challenging for builders and practitioners alike to follow. In this paper, we set out to capture this panorama through a wide-angle lens, by performing the largest analysis of DS projects to date, focusing on questions that can help determine investments on either side. Specifically, we download and analyze: (a) over 6M Python notebooks publicly available on GITHUB, (b) over 2M enterprise DS pipelines developed within COMPANYX, and (c) the source code and metadata of over 900 releases from 12 important DS libraries. The analysis we perform ranges from coarse-grained statistical characterizations to analysis of library imports, pipelines, and comparative studies across datasets and time. We report a large number of measurements for our readers to interpret, and dare to draw a few (actionable, yet subjective) conclusions on (a) what systems builders should focus on to better serve practitioners, and (b) what technologies should practitioners bet on given current trends. We plan to automate this analysis and release associated tools and results periodically. |
| Tasks | |
| Published | 2019-12-19 |
| URL | https://arxiv.org/abs/1912.09536v1 |
| https://arxiv.org/pdf/1912.09536v1.pdf | |
| PWC | https://paperswithcode.com/paper/data-science-through-the-looking-glass-and |
| Repo | |
| Framework | |
Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings
| Title | Weakly-Supervised Concept-based Adversarial Learning for Cross-lingual Word Embeddings |
| Authors | Haozhou Wang, James Henderson, Paola Merlo |
| Abstract | Distributed representations of words which map each word to a continuous vector have proven useful in capturing important linguistic information not only in a single language but also across different languages. Current unsupervised adversarial approaches show that it is possible to build a mapping matrix that align two sets of monolingual word embeddings together without high quality parallel data such as a dictionary or a sentence-aligned corpus. However, without post refinement, the performance of these methods’ preliminary mapping is not good, leading to poor performance for typologically distant languages. In this paper, we propose a weakly-supervised adversarial training method to overcome this limitation, based on the intuition that mapping across languages is better done at the concept level than at the word level. We propose a concept-based adversarial training method which for most languages improves the performance of previous unsupervised adversarial methods, especially for typologically distant language pairs. |
| Tasks | Word Embeddings |
| Published | 2019-04-20 |
| URL | http://arxiv.org/abs/1904.09446v1 |
| http://arxiv.org/pdf/1904.09446v1.pdf | |
| PWC | https://paperswithcode.com/paper/weakly-supervised-concept-based-adversarial |
| Repo | |
| Framework | |
Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks
| Title | Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networks |
| Authors | Gauthier Gidel, Francis Bach, Simon Lacoste-Julien |
| Abstract | When optimizing over-parameterized models, such as deep neural networks, a large set of parameters can achieve zero training error. In such cases, the choice of the optimization algorithm and its respective hyper-parameters introduces biases that will lead to convergence to specific minimizers of the objective. Consequently, this choice can be considered as an implicit regularization for the training of over-parametrized models. In this work, we push this idea further by studying the discrete gradient dynamics of the training of a two-layer linear network with the least-squares loss. Using a time rescaling, we show that, with a vanishing initialization and a small enough step size, this dynamics sequentially learns the solutions of a reduced-rank regression with a gradually increasing rank. |
| Tasks | |
| Published | 2019-04-30 |
| URL | https://arxiv.org/abs/1904.13262v2 |
| https://arxiv.org/pdf/1904.13262v2.pdf | |
| PWC | https://paperswithcode.com/paper/implicit-regularization-of-discrete-gradient |
| Repo | |
| Framework | |
Native Banach spaces for splines and variational inverse problems
| Title | Native Banach spaces for splines and variational inverse problems |
| Authors | Michael Unser, Julien Fageot |
| Abstract | We propose a systematic construction of native Banach spaces for general spline-admissible operators ${\rm L}$. In short, the native space for ${\rm L}$ and the (dual) norm $\cdot_{\mathcal{X}'}$ is the largest space of functions $f: \mathbb{R}^d \to \mathbb{R}$ such that ${\rm L} f_{\mathcal{X}'}<\infty$, subject to the constraint that the growth-restricted null space of ${\rm L}$be finite-dimensional. This space, denoted by $\mathcal{X}'{\rm L}$, is specified as the dual of the pre-native space $\mathcal{X}{\rm L}$, which is itself obtained through a suitable completion process. The main difference with prior constructions (e.g., reproducing kernel Hilbert spaces) is that our approach involves test functions rather than sums of atoms (e.g, kernels), which makes it applicable to a much broader class of norms, including total variation. Under specific admissibility and compatibility hypotheses, we lay out the direct-sum topology of $\mathcal{X}{\rm L}$ and $\mathcal{X}'{\rm L}$, and identify the whole family of equivalent norms. Our construction ensures that the native space and its pre-dual are endowed with a fundamental Schwartz-Banach property. In practical terms, this means that $\mathcal{X}'_{\rm L}$ is rich enough to reproduce any function with an arbitrary degree of precision. |
| Tasks | |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10818v1 |
| http://arxiv.org/pdf/1904.10818v1.pdf | |
| PWC | https://paperswithcode.com/paper/native-banach-spaces-for-splines-and |
| Repo | |
| Framework | |
Three-dimensional Segmentation of Trees Through a Flexible Multi-Class Graph Cut Algorithm (MCGC)
| Title | Three-dimensional Segmentation of Trees Through a Flexible Multi-Class Graph Cut Algorithm (MCGC) |
| Authors | Jonathan Williams, Carola-Bibiane Schönlieb, Tom Swinfield, Juheon Lee, Xiaohao Cai, Lan Qie, David A. Coomes |
| Abstract | Developing a robust algorithm for automatic individual tree crown (ITC) detection from laser scanning datasets is important for tracking the responses of trees to anthropogenic change. Such approaches allow the size, growth and mortality of individual trees to be measured, enabling forest carbon stocks and dynamics to be tracked and understood. Many algorithms exist for structurally simple forests including coniferous forests and plantations. Finding a robust solution for structurally complex, species-rich tropical forests remains a challenge; existing segmentation algorithms often perform less well than simple area-based approaches when estimating plot-level biomass. Here we describe a Multi-Class Graph Cut (MCGC) approach to tree crown delineation. This uses local three-dimensional geometry and density information, alongside knowledge of crown allometries, to segment individual tree crowns from LiDAR point clouds. Our approach robustly identifies trees in the top and intermediate layers of the canopy, but cannot recognise small trees. From these three-dimensional crowns, we are able to measure individual tree biomass. Comparing these estimates to those from permanent inventory plots, our algorithm is able to produce robust estimates of hectare-scale carbon density, demonstrating the power of ITC approaches in monitoring forests. The flexibility of our method to add additional dimensions of information, such as spectral reflectance, make this approach an obvious avenue for future development and extension to other sources of three-dimensional data, such as structure from motion datasets. |
| Tasks | |
| Published | 2019-03-20 |
| URL | http://arxiv.org/abs/1903.08481v1 |
| http://arxiv.org/pdf/1903.08481v1.pdf | |
| PWC | https://paperswithcode.com/paper/three-dimensional-segmentation-of-trees |
| Repo | |
| Framework | |
End-to-End Learning from Complex Multigraphs with Latent Graph Convolutional Networks
| Title | End-to-End Learning from Complex Multigraphs with Latent Graph Convolutional Networks |
| Authors | Floris Hermsen, Peter Bloem, Fabian Jansen, Wolf Vos |
| Abstract | We study the problem of end-to-end learning from complex multigraphs with potentially very large numbers of edges between two vertices, each edge labeled with rich information. Examples of such graphs include financial transactions, communication networks, or flights between airports. We propose Latent-Graph Convolutional Networks (L-GCNs), which can successfully propagate information from these edge labels to a latent adjacency tensor, after which further propagation and downstream tasks can be performed, such as node classification. We evaluate the performance of several variations of the model on two synthetic datasets simulating fraud in financial transaction networks, to ensure that the model must make use of edge labels in order to achieve good classification performance. We find that allowing for nonlinear interactions on a per-neighbor basis enhances performance significantly, while also showing promising results in an inductive setting. |
| Tasks | Node Classification |
| Published | 2019-08-14 |
| URL | https://arxiv.org/abs/1908.05365v1 |
| https://arxiv.org/pdf/1908.05365v1.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-learning-from-complex-multigraphs |
| Repo | |
| Framework | |
Artificial Neural Network Algorithm based Skyrmion Material Design of Chiral Crystals
| Title | Artificial Neural Network Algorithm based Skyrmion Material Design of Chiral Crystals |
| Authors | B. U. V Prashanth, Mohammed Riyaz Ahmed |
| Abstract | The model presented in this research predicts ideal chiral crystal and propose a new direction of designing chiral crystals. Skyrmions are topologically protected and structurally assymetric materials with an exotic spin composition. This work presents deep learning method for skyrmion material design of chiral crystals. This paper presents an approach to construct a probabilistic classifier and an Artificial Neural Network(ANN) from a true or false chirality dataset consisting of chiral and achiral compounds with ‘A’ and ‘B’ type elements. A quantitative predictor for accuracy of forming the chiral crystals is illustrated. The feasibility of ANN method is tested in a comprehensive manner by comparing with probalistic classifier method. Throughout this manuscript we present deep learnig algorithm design with modelling and simulations of materials. This research work elucidated paves a way to develop sophisticated software tool to make an indicator of crystal design. |
| Tasks | |
| Published | 2019-07-19 |
| URL | https://arxiv.org/abs/1907.09314v1 |
| https://arxiv.org/pdf/1907.09314v1.pdf | |
| PWC | https://paperswithcode.com/paper/artificial-neural-network-algorithm-based |
| Repo | |
| Framework | |
Back to the Future: Predicting Traffic Shockwave Formation and Propagation Using a Convolutional Encoder-Decoder Network
| Title | Back to the Future: Predicting Traffic Shockwave Formation and Propagation Using a Convolutional Encoder-Decoder Network |
| Authors | Mohammadreza Khajeh-Hosseini, Alireza Talebpour |
| Abstract | This study proposes a deep learning methodology to predict the propagation of traffic shockwaves. The input to the deep neural network is time-space diagram of the study segment, and the output of the network is the predicted (future) propagation of the shockwave on the study segment in the form of time-space diagram. The main feature of the proposed methodology is the ability to extract the features embedded in the time-space diagram to predict the propagation of traffic shockwaves. |
| Tasks | |
| Published | 2019-05-04 |
| URL | https://arxiv.org/abs/1905.02197v1 |
| https://arxiv.org/pdf/1905.02197v1.pdf | |
| PWC | https://paperswithcode.com/paper/back-to-the-future-predicting-traffic |
| Repo | |
| Framework | |
On the Limitations of the Univariate Marginal Distribution Algorithm to Deception and Where Bivariate EDAs might help
| Title | On the Limitations of the Univariate Marginal Distribution Algorithm to Deception and Where Bivariate EDAs might help |
| Authors | Per Kristian Lehre, Phan Trung Hai Nguyen |
| Abstract | We introduce a new benchmark problem called Deceptive Leading Blocks (DLB) to rigorously study the runtime of the Univariate Marginal Distribution Algorithm (UMDA) in the presence of epistasis and deception. We show that simple Evolutionary Algorithms (EAs) outperform the UMDA unless the selective pressure $\mu/\lambda$ is extremely high, where $\mu$ and $\lambda$ are the parent and offspring population sizes, respectively. More precisely, we show that the UMDA with a parent population size of $\mu=\Omega(\log n)$ has an expected runtime of $e^{\Omega(\mu)}$ on the DLB problem assuming any selective pressure $\frac{\mu}{\lambda} \geq \frac{14}{1000}$, as opposed to the expected runtime of $\mathcal{O}(n\lambda\log \lambda+n^3)$ for the non-elitist $(\mu,\lambda)~\text{EA}$ with $\mu/\lambda\leq 1/e$. These results illustrate inherent limitations of univariate EDAs against deception and epistasis, which are common characteristics of real-world problems. In contrast, empirical evidence reveals the efficiency of the bi-variate MIMIC algorithm on the DLB problem. Our results suggest that one should consider EDAs with more complex probabilistic models when optimising problems with some degree of epistasis and deception. |
| Tasks | |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12438v1 |
| https://arxiv.org/pdf/1907.12438v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-limitations-of-the-univariate-marginal |
| Repo | |
| Framework | |
A Decision-Based Dynamic Ensemble Selection Method for Concept Drift
| Title | A Decision-Based Dynamic Ensemble Selection Method for Concept Drift |
| Authors | Regis Antonio Saraiva Albuquerque, Albert Franca Josua Costa, Eulanda Miranda dos Santos, Robert Sabourin, Rafael Giusti |
| Abstract | We propose an online method for concept driftdetection based on dynamic classifier ensemble selection. Theproposed method generates a pool of ensembles by promotingdiversity among classifier members and chooses expert ensemblesaccording to global prequential accuracy values. Unlike currentdynamic ensemble selection approaches that use only local knowl-edge to select the most competent ensemble for each instance,our method focuses on selection taking into account the decisionspace. Consequently, it is well adapted to the context of driftdetection in data stream problems. The results of the experimentsshow that the proposed method attained the highest detection pre-cision and the lowest number of false alarms, besides competitiveclassification accuracy rates, in artificial datasets representingdifferent types of drifts. Moreover, it outperformed baselines indifferent real-problem datasets in terms of classification accuracy. |
| Tasks | |
| Published | 2019-09-26 |
| URL | https://arxiv.org/abs/1909.12185v1 |
| https://arxiv.org/pdf/1909.12185v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-decision-based-dynamic-ensemble-selection |
| Repo | |
| Framework | |
A tale of two toolkits, report the second: bake off redux. Chapter 1. dictionary based classifiers
| Title | A tale of two toolkits, report the second: bake off redux. Chapter 1. dictionary based classifiers |
| Authors | Anthony Bagnall, James Large, Matthew Middlehurst |
| Abstract | Time series classification (TSC) is the problem of learning labels from time dependent data. One class of algorithms is derived from a bag of words approach. A window is run along a series, the subseries is shortened and discretised to form a word, then features are formed from the histogram of frequency of occurrence of words. We call this type of approach to TSC dictionary based classification. We compare four dictionary based algorithms in the context of a wider project to update the great time series classification bakeoff, a comparative study published in 2017. We experimentally characterise the algorithms in terms of predictive performance, time complexity and space complexity. We find that we can improve on the previous best in terms of accuracy, but this comes at the cost of time and space. Alternatively, the same performance can be achieved with far less cost. We review the relative merits of the four algorithms before suggesting a path to possible improvement. |
| Tasks | Time Series, Time Series Classification |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.12008v1 |
| https://arxiv.org/pdf/1911.12008v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-tale-of-two-toolkits-report-the-second-bake |
| Repo | |
| Framework | |
Transfer Learning for Scientific Data Chain Extraction in Small Chemical Corpus with BERT-CRF Model
| Title | Transfer Learning for Scientific Data Chain Extraction in Small Chemical Corpus with BERT-CRF Model |
| Authors | Na Pang, Li Qian, Weimin Lyu, Jin-Dong Yang |
| Abstract | Computational chemistry develops fast in recent years due to the rapid growth and breakthroughs in AI. Thanks for the progress in natural language processing, researchers can extract more fine-grained knowledge in publications to stimulate the development in computational chemistry. While the works and corpora in chemical entity extraction have been restricted in the biomedicine or life science field instead of the chemistry field, we build a new corpus in chemical bond field annotated for 7 types of entities: compound, solvent, method, bond, reaction, pKa and pKa value. This paper presents a novel BERT-CRF model to build scientific chemical data chains by extracting 7 chemical entities and relations from publications. And we propose a joint model to extract the entities and relations simultaneously. Experimental results on our Chemical Special Corpus demonstrate that we achieve state-of-art and competitive NER performance. |
| Tasks | Entity Extraction, Transfer Learning |
| Published | 2019-05-13 |
| URL | https://arxiv.org/abs/1905.05615v1 |
| https://arxiv.org/pdf/1905.05615v1.pdf | |
| PWC | https://paperswithcode.com/paper/transfer-learning-for-scientific-data-chain |
| Repo | |
| Framework | |
Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification
| Title | Invariant Attribute Profiles: A Spatial-Frequency Joint Feature Extractor for Hyperspectral Image Classification |
| Authors | Danfeng Hong, Xin Wu, Pedram Ghamisi, Jocelyn Chanussot, Naoto Yokoya, Xiao Xiang Zhu |
| Abstract | Up to the present, an enormous number of advanced techniques have been developed to enhance and extract the spatially semantic information in hyperspectral image processing and analysis. However, locally semantic change, such as scene composition, relative position between objects, spectral variability caused by illumination, atmospheric effects, and material mixture, has been less frequently investigated in modeling spatial information. As a consequence, identifying the same materials from spatially different scenes or positions can be difficult. In this paper, we propose a solution to address this issue by locally extracting invariant features from hyperspectral imagery (HSI) in both spatial and frequency domains, using a method called invariant attribute profiles (IAPs). IAPs extract the spatial invariant features by exploiting isotropic filter banks or convolutional kernels on HSI and spatial aggregation techniques (e.g., superpixel segmentation) in the Cartesian coordinate system. Furthermore, they model invariant behaviors (e.g., shift, rotation) by the means of a continuous histogram of oriented gradients constructed in a Fourier polar coordinate. This yields a combinatorial representation of spatial-frequency invariant features with application to HSI classification. Extensive experiments conducted on three promising hyperspectral datasets (Houston2013 and Houston2018) demonstrate the superiority and effectiveness of the proposed IAP method in comparison with several state-of-the-art profile-related techniques. The codes will be available from the website: https://sites.google.com/view/danfeng-hong/data-code. |
| Tasks | Hyperspectral Image Classification, Image Classification |
| Published | 2019-12-18 |
| URL | https://arxiv.org/abs/1912.08847v1 |
| https://arxiv.org/pdf/1912.08847v1.pdf | |
| PWC | https://paperswithcode.com/paper/invariant-attribute-profiles-a-spatial |
| Repo | |
| Framework | |
Specular- and Diffuse-reflection-based Face Spoofing Detection for Mobile Devices
| Title | Specular- and Diffuse-reflection-based Face Spoofing Detection for Mobile Devices |
| Authors | Akinori F. Ebihara, Kazuyuki Sakurai, Hitoshi Imaoka |
| Abstract | In light of the rising demand for biometric-authentication systems, preventing face spoofing attacks is a critical issue for the safe deployment of face recognition systems. Here, we propose an efficient face presentation attack detection (PAD) algorithm that requires minimal hardware and only a small database, making it suitable for resource-constrained devices such as mobile phones. Utilizing one monocular visible light camera, the proposed algorithm takes two facial photos, one taken with a flash, the other without a flash. The proposed $SpecDiff$ descriptor is constructed by leveraging two types of reflection: (i) specular reflections from the iris region that have a specific intensity distribution depending on liveness, and (ii) diffuse reflections from the entire face region that represents the 3D structure of a subject’s face. Classifiers trained with $SpecDiff$ descriptor outperforms other flash-based PAD algorithms on both an in-house database and on publicly available NUAA, Replay-Attack, and SiW databases. Moreover, the proposed algorithm achieves statistically significantly better accuracy to that of an end-to-end, deep neural network classifier, while being approximately six-times faster execution speed. |
| Tasks | Face Presentation Attack Detection, Face Recognition |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12400v2 |
| https://arxiv.org/pdf/1907.12400v2.pdf | |
| PWC | https://paperswithcode.com/paper/specular-and-diffuse-reflection-based-face |
| Repo | |
| Framework | |
