Paper Group ANR 1387
Towards a computer-interpretable actionable formal model to encode data governance rules. TopoAct: Exploring the Shape of Activations in Deep Learning. Bifurcation Spiking Neural Network. BERTQA – Attention on Steroids. Interpreting Deep Learning Features for Myoelectric Control: A Comparison with Handcrafted Features. Machine Learning meets Numbe …
Towards a computer-interpretable actionable formal model to encode data governance rules
| Title | Towards a computer-interpretable actionable formal model to encode data governance rules |
| Authors | Rui Zhao, Malcolm Atkinson |
| Abstract | With the needs of science and business, data sharing and re-use has become an intensive activity for various areas. In many cases, governance imposes rules concerning data use, but there is no existing computational technique to help data-users comply with such rules. We argue that intelligent systems can be used to improve the situation, by recording provenance records during processing, encoding the rules and performing reasoning. We present our initial work, designing formal models for data rules and flow rules and the reasoning system, as the first step towards helping data providers and data users sustain productive relationships. |
| Tasks | |
| Published | 2019-11-19 |
| URL | https://arxiv.org/abs/1911.08439v1 |
| https://arxiv.org/pdf/1911.08439v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-a-computer-interpretable-actionable |
| Repo | |
| Framework | |
TopoAct: Exploring the Shape of Activations in Deep Learning
| Title | TopoAct: Exploring the Shape of Activations in Deep Learning |
| Authors | Archit Rathore, Nithin Chalapathi, Sourabh Palande, Bei Wang |
| Abstract | Deep neural networks such as GoogLeNet and ResNet have achieved superhuman performance in tasks like image classification. To understand how such superior performance is achieved, we can probe a trained deep neural network by studying neuron activations, that is, combinations of neuron firings, at any layer of the network in response to a particular input. With a large set of input images, we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the space of activations? That is, what is the organizational principle behind neuron activations, and how are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system used to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present visual exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier. |
| Tasks | Image Classification, Topological Data Analysis |
| Published | 2019-12-13 |
| URL | https://arxiv.org/abs/1912.06332v1 |
| https://arxiv.org/pdf/1912.06332v1.pdf | |
| PWC | https://paperswithcode.com/paper/topoact-exploring-the-shape-of-activations-in |
| Repo | |
| Framework | |
Bifurcation Spiking Neural Network
| Title | Bifurcation Spiking Neural Network |
| Authors | Shao-Qun Zhang, Zhao-Yu Zhang, Zhi-Hua Zhou |
| Abstract | Spiking neural networks (SNNs) has attracted much attention due to its great potential of modeling time-dependent signals. The firing rate of spiking neurons is decided by control rate which is fixed manually in advance, and thus, whether the firing rate is adequate for modeling actual time series relies on fortune. Though it is demanded to have an adaptive control rate, it is a non-trivial task because the control rate and the connection weights learned during the training process are usually entangled. In this paper, we show that the firing rate is related to the eigenvalue of the spike generation function. Inspired by this insight, by enabling the spike generation function to have adaptable eigenvalues rather than parametric control rates, we develop the Bifurcation Spiking Neural Network (BSNN), which has an adaptive firing rate and is insensitive to the setting of control rates. Experiments validate the effectiveness of BSNN on a broad range of tasks, showing that BSNN achieves superior performance to existing SNNs and is robust to the setting of control rates. |
| Tasks | Time Series |
| Published | 2019-09-18 |
| URL | https://arxiv.org/abs/1909.08341v2 |
| https://arxiv.org/pdf/1909.08341v2.pdf | |
| PWC | https://paperswithcode.com/paper/bifurcation-spiking-neural-network |
| Repo | |
| Framework | |
BERTQA – Attention on Steroids
| Title | BERTQA – Attention on Steroids |
| Authors | Ankit Chadha, Rewa Sood |
| Abstract | In this work, we extend the Bidirectional Encoder Representations from Transformers (BERT) with an emphasis on directed coattention to obtain an improved F1 performance on the SQUAD2.0 dataset. The Transformer architecture on which BERT is based places hierarchical global attention on the concatenation of the context and query. Our additions to the BERT architecture augment this attention with a more focused context to query (C2Q) and query to context (Q2C) attention via a set of modified Transformer encoder units. In addition, we explore adding convolution-based feature extraction within the coattention architecture to add localized information to self-attention. We found that coattention significantly improves the no answer F1 by 4 points in the base and 1 point in the large architecture. After adding skip connections the no answer F1 improved further without causing an additional loss in has answer F1. The addition of localized feature extraction added to attention produced an overall dev F1 of 77.03 in the base architecture. We applied our findings to the large BERT model which contains twice as many layers and further used our own augmented version of the SQUAD 2.0 dataset created by back translation, which we have named SQUAD 2.Q. Finally, we performed hyperparameter tuning and ensembled our best models for a final F1/EM of 82.317/79.442 (Attention on Steroids, PCE Test Leaderboard). |
| Tasks | |
| Published | 2019-12-14 |
| URL | https://arxiv.org/abs/1912.10435v1 |
| https://arxiv.org/pdf/1912.10435v1.pdf | |
| PWC | https://paperswithcode.com/paper/bertqa-attention-on-steroids |
| Repo | |
| Framework | |
Interpreting Deep Learning Features for Myoelectric Control: A Comparison with Handcrafted Features
| Title | Interpreting Deep Learning Features for Myoelectric Control: A Comparison with Handcrafted Features |
| Authors | Ulysse Côté-Allard, Evan Campbell, Angkoon Phinyomark, François Laviolette, Benoit Gosselin, Erik Scheme |
| Abstract | The research in myoelectric control systems primarily focuses on extracting discriminative representations from the electromyographic (EMG) signal by designing handcrafted features. Recently, deep learning techniques have been applied to the challenging task of EMG-based gesture recognition. The adoption of these techniques slowly shifts the focus from feature engineering to feature learning. However, the black-box nature of deep learning makes it hard to understand the type of information learned by the network and how it relates to handcrafted features. Additionally, due to the high variability in EMG recordings between participants, deep features tend to generalize poorly across subjects using standard training methods. Consequently, this work introduces a new multi-domain learning algorithm, named ADANN, which significantly enhances (p=0.00004) inter-subject classification accuracy by an average of 19.40% compared to standard training. Using ADANN-generated features, the main contribution of this work is to provide the first topological data analysis of EMG-based gesture recognition for the characterisation of the information encoded within a deep network, using handcrafted features as landmarks. This analysis reveals that handcrafted features and the learned features (in the earlier layers) both try to discriminate between all gestures, but do not encode the same information to do so. Furthermore, using convolutional network visualization techniques reveal that learned features tend to ignore the most activated channel during gesture contraction, which is in stark contrast with the prevalence of handcrafted features designed to capture amplitude information. Overall, this work paves the way for hybrid feature sets by providing a clear guideline of complementary information encoded within learned and handcrafted features. |
| Tasks | Feature Engineering, Gesture Recognition, Topological Data Analysis |
| Published | 2019-11-30 |
| URL | https://arxiv.org/abs/1912.00283v2 |
| https://arxiv.org/pdf/1912.00283v2.pdf | |
| PWC | https://paperswithcode.com/paper/interpreting-deep-learning-features-for |
| Repo | |
| Framework | |
Machine Learning meets Number Theory: The Data Science of Birch-Swinnerton-Dyer
| Title | Machine Learning meets Number Theory: The Data Science of Birch-Swinnerton-Dyer |
| Authors | Laura Alessandretti, Andrea Baronchelli, Yang-Hui He |
| Abstract | Empirical analysis is often the first step towards the birth of a conjecture. This is the case of the Birch-Swinnerton-Dyer (BSD) Conjecture describing the rational points on an elliptic curve, one of the most celebrated unsolved problems in mathematics. Here we extend the original empirical approach, to the analysis of the Cremona database of quantities relevant to BSD, inspecting more than 2.5 million elliptic curves by means of the latest techniques in data science, machine-learning and topological data analysis. Key quantities such as rank, Weierstrass coefficients, period, conductor, Tamagawa number, regulator and order of the Tate-Shafarevich group give rise to a high-dimensional point-cloud whose statistical properties we investigate. We reveal patterns and distributions in the rank versus Weierstrass coefficients, as well as the Beta distribution of the BSD ratio of the quantities. Via gradient boosted trees, machine learning is applied in finding inter-correlation amongst the various quantities. We anticipate that our approach will spark further research on the statistical properties of large datasets in Number Theory and more in general in pure Mathematics. |
| Tasks | Topological Data Analysis |
| Published | 2019-11-04 |
| URL | https://arxiv.org/abs/1911.02008v1 |
| https://arxiv.org/pdf/1911.02008v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-meets-number-theory-the-data |
| Repo | |
| Framework | |
Harnessing the power of Topological Data Analysis to detect change points in time series
| Title | Harnessing the power of Topological Data Analysis to detect change points in time series |
| Authors | Umar Islambekov, Monisha Yuvaraj, Yulia R. Gel |
| Abstract | We introduce a novel geometry-oriented methodology, based on the emerging tools of topological data analysis, into the change point detection framework. The key rationale is that change points are likely to be associated with changes in geometry behind the data generating process. While the applications of topological data analysis to change point detection are potentially very broad, in this paper we primarily focus on integrating topological concepts with the existing nonparametric methods for change point detection. In particular, the proposed new geometry-oriented approach aims to enhance detection accuracy of distributional regime shift locations. Our simulation studies suggest that integration of topological data analysis with some existing algorithms for change point detection leads to consistently more accurate detection results. We illustrate our new methodology in application to the two closely related environmental time series datasets -ice phenology of the Lake Baikal and the North Atlantic Oscillation indices, in a research query for a possible association between their estimated regime shift locations. |
| Tasks | Change Point Detection, Time Series, Topological Data Analysis |
| Published | 2019-10-28 |
| URL | https://arxiv.org/abs/1910.12939v1 |
| https://arxiv.org/pdf/1910.12939v1.pdf | |
| PWC | https://paperswithcode.com/paper/harnessing-the-power-of-topological-data |
| Repo | |
| Framework | |
Characterization of Posidonia Oceanica Seagrass Aerenchyma through Whole Slide Imaging: A Pilot Study
| Title | Characterization of Posidonia Oceanica Seagrass Aerenchyma through Whole Slide Imaging: A Pilot Study |
| Authors | Olivier Debeir, Justine Allard, Christine Decaestecker, Jean-Pierre Hermand |
| Abstract | Characterizing the tissue morphology and anatomy of seagrasses is essential to predicting their acoustic behavior. In this pilot study, we use histology techniques and whole slide imaging (WSI) to describe the composition and topology of the aerenchyma of an entire leaf blade in an automatic way combining the advantages of X-ray microtomography and optical microscopy. Paraffin blocks are prepared in such a way that microtome slices contain an arbitrarily large number of cross sections distributed along the full length of a blade. The sample organization in the paraffin block coupled with whole slide image analysis allows high throughput data extraction and an exhaustive characterization along the whole blade length. The core of the work are image processing algorithms that can identify cells and air lacunae (or void) from fiber strand, epidermis, mesophyll and vascular system. A set of specific features is developed to adequately describe the convexity of cells and voids where standard descriptors fail. The features scrutinize the local curvature of the object borders to allow an accurate discrimination between void and cell through machine learning. The algorithm allows to reconstruct the cells and cell membrane features that are relevant to tissue density, compressibility and rigidity. Size distribution of the different cell types and gas spaces, total biomass and total void volume fraction are then extracted from the high resolution slices to provide a complete characterization of the tissue along the leave from its base to the apex. |
| Tasks | |
| Published | 2019-03-07 |
| URL | http://arxiv.org/abs/1903.03044v2 |
| http://arxiv.org/pdf/1903.03044v2.pdf | |
| PWC | https://paperswithcode.com/paper/characterization-of-posidonia-oceanica |
| Repo | |
| Framework | |
Detection of gravitational waves using topological data analysis and convolutional neural network: An improved approach
| Title | Detection of gravitational waves using topological data analysis and convolutional neural network: An improved approach |
| Authors | Christopher Bresten, Jae-Hun Jung |
| Abstract | The gravitational wave detection problem is challenging because the noise is typically overwhelming. Convolutional neural networks (CNNs) have been successfully applied, but require a large training set and the accuracy suffers significantly in the case of low SNR. We propose an improved method that employs a feature extraction step using persistent homology. The resulting method is more resilient to noise, more capable of detecting signals with varied signatures and requires less training. This is a powerful improvement as the detection problem can be computationally intense and is concerned with a relatively large class of wave signatures. |
| Tasks | Gravitational Wave Detection, Topological Data Analysis |
| Published | 2019-10-18 |
| URL | https://arxiv.org/abs/1910.08245v1 |
| https://arxiv.org/pdf/1910.08245v1.pdf | |
| PWC | https://paperswithcode.com/paper/detection-of-gravitational-waves-using |
| Repo | |
| Framework | |
Random Restrictions of High-Dimensional Distributions and Uniformity Testing with Subcube Conditioning
| Title | Random Restrictions of High-Dimensional Distributions and Uniformity Testing with Subcube Conditioning |
| Authors | Clément L. Canonne, Xi Chen, Gautam Kamath, Amit Levi, Erik Waingarten |
| Abstract | We give a nearly-optimal algorithm for testing uniformity of distributions supported on ${-1,1}^n$, which makes $\tilde O (\sqrt{n}/\varepsilon^2)$ queries to a subcube conditional sampling oracle (Bhattacharyya and Chakraborty (2018)). The key technical component is a natural notion of random restriction for distributions on ${-1,1}^n$, and a quantitative analysis of how such a restriction affects the mean vector of the distribution. Along the way, we consider the problem of mean testing with independent samples and provide a nearly-optimal algorithm. |
| Tasks | |
| Published | 2019-11-17 |
| URL | https://arxiv.org/abs/1911.07357v1 |
| https://arxiv.org/pdf/1911.07357v1.pdf | |
| PWC | https://paperswithcode.com/paper/random-restrictions-of-high-dimensional |
| Repo | |
| Framework | |
Scene Text Recognition with Temporal Convolutional Encoder
| Title | Scene Text Recognition with Temporal Convolutional Encoder |
| Authors | Xiangcheng Du, Tianlong Ma, Yingbin Zheng, Hao Ye, Xingjiao Wu, Liang He |
| Abstract | Texts from scene images typically consist of several characters and exhibit a characteristic sequence structure. Existing methods capture the structure with the sequence-to-sequence models by an encoder to have the visual representations and then a decoder to translate the features into the label sequence. In this paper, we study text recognition framework by considering the long-term temporal dependencies in the encoder stage. We demonstrate that the proposed Temporal Convolutional Encoder with increased sequential extents improves the accuracy of text recognition. We also study the impact of different attention modules in convolutional blocks for learning accurate text representations. We conduct comparisons on seven datasets and the experiments demonstrate the effectiveness of our proposed approach. |
| Tasks | Scene Text Recognition |
| Published | 2019-11-04 |
| URL | https://arxiv.org/abs/1911.01051v2 |
| https://arxiv.org/pdf/1911.01051v2.pdf | |
| PWC | https://paperswithcode.com/paper/scene-text-recognition-with-temporal |
| Repo | |
| Framework | |
Document Hashing with Mixture-Prior Generative Models
| Title | Document Hashing with Mixture-Prior Generative Models |
| Authors | Wei Dong, Qinliang Su, Dinghan Shen, Changyou Chen |
| Abstract | Hashing is promising for large-scale information retrieval tasks thanks to the efficiency of distance evaluation between binary codes. Generative hashing is often used to generate hashing codes in an unsupervised way. However, existing generative hashing methods only considered the use of simple priors, like Gaussian and Bernoulli priors, which limits these methods to further improve their performance. In this paper, two mixture-prior generative models are proposed, under the objective to produce high-quality hashing codes for documents. Specifically, a Gaussian mixture prior is first imposed onto the variational auto-encoder (VAE), followed by a separate step to cast the continuous latent representation of VAE into binary code. To avoid the performance loss caused by the separate casting, a model using a Bernoulli mixture prior is further developed, in which an end-to-end training is admitted by resorting to the straight-through (ST) discrete gradient estimator. Experimental results on several benchmark datasets demonstrate that the proposed methods, especially the one using Bernoulli mixture priors, consistently outperform existing ones by a substantial margin. |
| Tasks | Information Retrieval |
| Published | 2019-08-29 |
| URL | https://arxiv.org/abs/1908.11078v1 |
| https://arxiv.org/pdf/1908.11078v1.pdf | |
| PWC | https://paperswithcode.com/paper/document-hashing-with-mixture-prior |
| Repo | |
| Framework | |
Open Information Extraction from Question-Answer Pairs
| Title | Open Information Extraction from Question-Answer Pairs |
| Authors | Nikita Bhutani, Yoshihiko Suhara, Wang-Chiew Tan, Alon Halevy, H. V. Jagadish |
| Abstract | Open Information Extraction (OpenIE) extracts meaningful structured tuples from free-form text. Most previous work on OpenIE considers extracting data from one sentence at a time. We describe NeurON, a system for extracting tuples from question-answer pairs. Since real questions and answers often contain precisely the information that users care about, such information is particularly desirable to extend a knowledge base with. NeurON addresses several challenges. First, an answer text is often hard to understand without knowing the question, and second, relevant information can span multiple sentences. To address these, NeurON formulates extraction as a multi-source sequence-to-sequence learning task, wherein it combines distributed representations of a question and an answer to generate knowledge facts. We describe experiments on two real-world datasets that demonstrate that NeurON can find a significant number of new and interesting facts to extend a knowledge base compared to state-of-the-art OpenIE methods. |
| Tasks | Open Information Extraction |
| Published | 2019-03-01 |
| URL | http://arxiv.org/abs/1903.00172v2 |
| http://arxiv.org/pdf/1903.00172v2.pdf | |
| PWC | https://paperswithcode.com/paper/open-information-extraction-from-question |
| Repo | |
| Framework | |
CAMUS: A Framework to Build Formal Specifications for Deep Perception Systems Using Simulators
| Title | CAMUS: A Framework to Build Formal Specifications for Deep Perception Systems Using Simulators |
| Authors | Julien Girard-Satabin, Guillaume Charpiat, Zakaria Chihani, Marc Schoenauer |
| Abstract | The topic of provable deep neural network robustness has raised considerable interest in recent years. Most research has focused on adversarial robustness, which studies the robustness of perceptive models in the neighbourhood of particular samples. However, other works have proved global properties of smaller neural networks. Yet, formally verifying perception remains uncharted. This is due notably to the lack of relevant properties to verify, as the distribution of possible inputs cannot be formally specified. We propose to take advantage of the simulators often used either to train machine learning models or to check them with statistical tests, a growing trend in industry. Our formulation allows us to formally express and verify safety properties on perception units, covering all cases that could ever be generated by the simulator, to the difference of statistical tests which cover only seen examples. Along with this theoretical formulation , we provide a tool to translate deep learning models into standard logical formulae. As a proof of concept, we train a toy example mimicking an autonomous car perceptive unit, and we formally verify that it will never fail to capture the relevant information in the provided inputs. |
| Tasks | |
| Published | 2019-11-25 |
| URL | https://arxiv.org/abs/1911.10735v1 |
| https://arxiv.org/pdf/1911.10735v1.pdf | |
| PWC | https://paperswithcode.com/paper/camus-a-framework-to-build-formal |
| Repo | |
| Framework | |
Efficient Optimization of Loops and Limits with Randomized Telescoping Sums
| Title | Efficient Optimization of Loops and Limits with Randomized Telescoping Sums |
| Authors | Alex Beatson, Ryan P. Adams |
| Abstract | We consider optimization problems in which the objective requires an inner loop with many steps or is the limit of a sequence of increasingly costly approximations. Meta-learning, training recurrent neural networks, and optimization of the solutions to differential equations are all examples of optimization problems with this character. In such problems, it can be expensive to compute the objective function value and its gradient, but truncating the loop or using less accurate approximations can induce biases that damage the overall solution. We propose randomized telescope (RT) gradient estimators, which represent the objective as the sum of a telescoping series and sample linear combinations of terms to provide cheap unbiased gradient estimates. We identify conditions under which RT estimators achieve optimization convergence rates independent of the length of the loop or the required accuracy of the approximation. We also derive a method for tuning RT estimators online to maximize a lower bound on the expected decrease in loss per unit of computation. We evaluate our adaptive RT estimators on a range of applications including meta-optimization of learning rates, variational inference of ODE parameters, and training an LSTM to model long sequences. |
| Tasks | Meta-Learning |
| Published | 2019-05-16 |
| URL | https://arxiv.org/abs/1905.07006v1 |
| https://arxiv.org/pdf/1905.07006v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-optimization-of-loops-and-limits |
| Repo | |
| Framework | |
