Paper Group ANR 586
Learning big Gaussian Bayesian networks: partition, estimation, and fusion. Adapting Convolutional Neural Networks for Geographical Domain Shift. Improving Model Training by Periodic Sampling over Weight Distributions. Convergence rates for ordinal embedding. Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation. Distribu …
Learning big Gaussian Bayesian networks: partition, estimation, and fusion
| Title | Learning big Gaussian Bayesian networks: partition, estimation, and fusion |
| Authors | Jiaying Gu, Qing Zhou |
| Abstract | Structure learning of Bayesian networks has always been a challenging problem. Nowadays, massive-size networks with thousands or more of nodes but fewer samples frequently appear in many areas. We develop a divide-and-conquer framework, called partition-estimation-fusion (PEF), for structure learning of such big networks. The proposed method first partitions nodes into clusters, then learns a subgraph on each cluster of nodes, and finally fuses all learned subgraphs into one Bayesian network. The PEF method is designed in a flexible way so that any structure learning method may be used in the second step to learn a subgraph structure as either a DAG or a CPDAG. In the clustering step, we adapt the hierarchical clustering method to automatically choose a proper number of clusters. In the fusion step, we propose a novel hybrid method that sequentially add edges between subgraphs. Extensive numerical experiments demonstrate the competitive performance of our PEF method, in terms of both speed and accuracy compared to existing methods. Our method can improve the accuracy of structure learning by 20% or more, while reducing running time up to two orders-of-magnitude. |
| Tasks | |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10900v1 |
| http://arxiv.org/pdf/1904.10900v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-big-gaussian-bayesian-networks |
| Repo | |
| Framework | |
Adapting Convolutional Neural Networks for Geographical Domain Shift
| Title | Adapting Convolutional Neural Networks for Geographical Domain Shift |
| Authors | Pavel Ostyakov, Sergey I. Nikolenko |
| Abstract | We present the winning solution for the Inclusive Images Competition organized as part of the Conference on Neural Information Processing Systems (NeurIPS 2018) Competition Track. The competition was organized to study ways to cope with domain shift in image processing, specifically geographical shift: the training and two test sets in the competition had different geographical distributions. Our solution has proven to be relatively straightforward and simple: it is an ensemble of several CNNs where only the last layer is fine-tuned with the help of a small labeled set of tuning labels made available by the organizers. We believe that while domain shift remains a formidable problem, our approach opens up new possibilities for alleviating this problem in practice, where small labeled datasets from the target domain are usually either available or can be obtained and labeled cheaply. |
| Tasks | |
| Published | 2019-01-18 |
| URL | http://arxiv.org/abs/1901.06345v1 |
| http://arxiv.org/pdf/1901.06345v1.pdf | |
| PWC | https://paperswithcode.com/paper/adapting-convolutional-neural-networks-for |
| Repo | |
| Framework | |
Improving Model Training by Periodic Sampling over Weight Distributions
| Title | Improving Model Training by Periodic Sampling over Weight Distributions |
| Authors | Samarth Tripathi, Jiayi Liu, Unmesh Kurup, Mohak Shah, Sauptik Dhar |
| Abstract | In this paper, we explore techniques centered around periodic sampling of model weights that provide convergence improvements on gradient update methods (vanilla \acs{SGD}, Momentum, Adam) for a variety of vision problems (classification, detection, segmentation). Importantly, our algorithms provide better, faster and more robust convergence and training performance with only a slight increase in computation time. Our techniques are independent of the neural network model, gradient optimization methods or existing optimal training policies and converge in a less volatile fashion with performance improvements that are approximately monotonic. We conduct a variety of experiments to quantify these improvements and identify scenarios where these techniques could be more useful. |
| Tasks | |
| Published | 2019-05-14 |
| URL | https://arxiv.org/abs/1905.05774v2 |
| https://arxiv.org/pdf/1905.05774v2.pdf | |
| PWC | https://paperswithcode.com/paper/robust-neural-network-training-using-periodic |
| Repo | |
| Framework | |
Convergence rates for ordinal embedding
| Title | Convergence rates for ordinal embedding |
| Authors | Jordan S. Ellenberg, Lalit Jain |
| Abstract | We prove optimal bounds for the convergence rate of ordinal embedding (also known as non-metric multidimensional scaling) in the 1-dimensional case. The examples witnessing optimality of our bounds arise from a result in additive number theory on sets of integers with no three-term arithmetic progressions. We also carry out some computational experiments aimed at developing a sense of what the convergence rate for ordinal embedding might look like in higher dimensions. |
| Tasks | |
| Published | 2019-04-30 |
| URL | http://arxiv.org/abs/1904.12994v1 |
| http://arxiv.org/pdf/1904.12994v1.pdf | |
| PWC | https://paperswithcode.com/paper/convergence-rates-for-ordinal-embedding |
| Repo | |
| Framework | |
Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation
| Title | Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation |
| Authors | Elizabeth Salesky, Matthias Sperber, Alan W Black |
| Abstract | Previous work on end-to-end translation from speech has primarily used frame-level features as speech representations, which creates longer, sparser sequences than text. We show that a naive method to create compressed phoneme-like speech representations is far more effective and efficient for translation than traditional frame-level speech features. Specifically, we generate phoneme labels for speech frames and average consecutive frames with the same label to create shorter, higher-level source sequences for translation. We see improvements of up to 5 BLEU on both our high and low resource language pairs, with a reduction in training time of 60%. Our improvements hold across multiple data sizes and two language pairs. |
| Tasks | |
| Published | 2019-06-04 |
| URL | https://arxiv.org/abs/1906.01199v1 |
| https://arxiv.org/pdf/1906.01199v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-phoneme-level-speech |
| Repo | |
| Framework | |
Distributional Clustering: A distribution-preserving clustering method
| Title | Distributional Clustering: A distribution-preserving clustering method |
| Authors | Arvind Krishna, Simon Mak, Roshan Joseph |
| Abstract | One key use of k-means clustering is to identify cluster prototypes which can serve as representative points for a dataset. However, a drawback of using k-means cluster centers as representative points is that such points distort the distribution of the underlying data. This can be highly disadvantageous in problems where the representative points are subsequently used to gain insights on the data distribution, as these points do not mimic the distribution of the data. To this end, we propose a new clustering method called “distributional clustering”, which ensures cluster centers capture the distribution of the underlying data. We first prove the asymptotic convergence of the proposed cluster centers to the data generating distribution, then present an efficient algorithm for computing these cluster centers in practice. Finally, we demonstrate the effectiveness of distributional clustering on synthetic and real datasets. |
| Tasks | |
| Published | 2019-11-14 |
| URL | https://arxiv.org/abs/1911.05940v1 |
| https://arxiv.org/pdf/1911.05940v1.pdf | |
| PWC | https://paperswithcode.com/paper/distributional-clustering-a-distribution |
| Repo | |
| Framework | |
Speaker Anonymization Using X-vector and Neural Waveform Models
| Title | Speaker Anonymization Using X-vector and Neural Waveform Models |
| Authors | Fuming Fang, Xin Wang, Junichi Yamagishi, Isao Echizen, Massimiliano Todisco, Nicholas Evans, Jean-Francois Bonastre |
| Abstract | The social media revolution has produced a plethora of web services to which users can easily upload and share multimedia documents. Despite the popularity and convenience of such services, the sharing of such inherently personal data, including speech data, raises obvious security and privacy concerns. In particular, a user’s speech data may be acquired and used with speech synthesis systems to produce high-quality speech utterances which reflect the same user’s speaker identity. These utterances may then be used to attack speaker verification systems. One solution to mitigate these concerns involves the concealing of speaker identities before the sharing of speech data. For this purpose, we present a new approach to speaker anonymization. The idea is to extract linguistic and speaker identity features from an utterance and then to use these with neural acoustic and waveform models to synthesize anonymized speech. The original speaker identity, in the form of timbre, is suppressed and replaced with that of an anonymous pseudo identity. The approach exploits state-of-the-art x-vector speaker representations. These are used to derive anonymized pseudo speaker identities through the combination of multiple, random speaker x-vectors. Experimental results show that the proposed approach is effective in concealing speaker identities. It increases the equal error rate of a speaker verification system while maintaining high quality, anonymized speech. |
| Tasks | Speaker Verification, Speech Synthesis |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.13561v1 |
| https://arxiv.org/pdf/1905.13561v1.pdf | |
| PWC | https://paperswithcode.com/paper/speaker-anonymization-using-x-vector-and |
| Repo | |
| Framework | |
Explainable artificial intelligence model to predict acute critical illness from electronic health records
| Title | Explainable artificial intelligence model to predict acute critical illness from electronic health records |
| Authors | Simon Meyer Lauritsen, Mads Kristensen, Mathias Vassard Olsen, Morten Skaarup Larsen, Katrine Meyer Lauritsen, Marianne Johansson Jørgensen, Jeppe Lange, Bo Thiesson |
| Abstract | We developed an explainable artificial intelligence (AI) early warning score (xAI-EWS) system for early detection of acute critical illness. While maintaining a high predictive performance, our system explains to the clinician on which relevant electronic health records (EHRs) data the prediction is grounded. Acute critical illness is often preceded by deterioration of routinely measured clinical parameters, e.g., blood pressure and heart rate. Early clinical prediction is typically based on manually calculated screening metrics that simply weigh these parameters, such as Early Warning Scores (EWS). The predictive performance of EWSs yields a tradeoff between sensitivity and specificity that can lead to negative outcomes for the patient. Previous work on EHR-trained AI systems offers promising results with high levels of predictive performance in relation to the early, real-time prediction of acute critical illness. However, without insight into the complex decisions by such system, clinical translation is hindered. In this letter, we present our xAI-EWS system, which potentiates clinical translation by accompanying a prediction with information on the EHR data explaining it. |
| Tasks | |
| Published | 2019-12-03 |
| URL | https://arxiv.org/abs/1912.01266v1 |
| https://arxiv.org/pdf/1912.01266v1.pdf | |
| PWC | https://paperswithcode.com/paper/explainable-artificial-intelligence-model-to |
| Repo | |
| Framework | |
A Split-and-Recombine Approach for Follow-up Query Analysis
| Title | A Split-and-Recombine Approach for Follow-up Query Analysis |
| Authors | Qian Liu, Bei Chen, Haoyan Liu, Lei Fang, Jian-Guang Lou, Bin Zhou, Dongmei Zhang |
| Abstract | Context-dependent semantic parsing has proven to be an important yet challenging task. To leverage the advances in context-independent semantic parsing, we propose to perform follow-up query analysis, aiming to restate context-dependent natural language queries with contextual information. To accomplish the task, we propose STAR, a novel approach with a well-designed two-phase process. It is parser-independent and able to handle multifarious follow-up scenarios in different domains. Experiments on the FollowUp dataset show that STAR outperforms the state-of-the-art baseline by a large margin of nearly 8%. The superiority on parsing results verifies the feasibility of follow-up query analysis. We also explore the extensibility of STAR on the SQA dataset, which is very promising. |
| Tasks | Semantic Parsing |
| Published | 2019-09-19 |
| URL | https://arxiv.org/abs/1909.08905v1 |
| https://arxiv.org/pdf/1909.08905v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-split-and-recombine-approach-for-follow-up |
| Repo | |
| Framework | |
Graph convolutional networks for learning with few clean and many noisy labels
| Title | Graph convolutional networks for learning with few clean and many noisy labels |
| Authors | Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Ondrej Chum, Cordelia Schmid |
| Abstract | In this work we consider the problem of learning a classifier from noisy labels when a few clean labeled examples are given. The structure of clean and noisy data is modeled by a graph per class and Graph Convolutional Networks (GCN) are used to predict class relevance of noisy examples. For each class, the GCN is treated as a binary classifier learning to discriminate clean from noisy examples using a weighted binary cross-entropy loss function, and then the GCN-inferred “clean” probability is exploited as a relevance measure. Each noisy example is weighted by its relevance when learning a classifier for the end task. We evaluate our method on an extended version of a few-shot learning problem, where the few clean examples of novel classes are supplemented with additional noisy data. Experimental results show that our GCN-based cleaning process significantly improves the classification accuracy over not cleaning the noisy data and standard few-shot classification where only few clean examples are used. The proposed GCN-based method outperforms the transductive approach (Douze et al., 2018) that is using the same additional data without labels. |
| Tasks | Few-Shot Learning |
| Published | 2019-10-01 |
| URL | https://arxiv.org/abs/1910.00324v1 |
| https://arxiv.org/pdf/1910.00324v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-convolutional-networks-for-learning |
| Repo | |
| Framework | |
Symbolic Regression Methods for Reinforcement Learning
| Title | Symbolic Regression Methods for Reinforcement Learning |
| Authors | Jiří Kubalík, Jan Žegklitz, Erik Derner, Robert Babuška |
| Abstract | Reinforcement learning algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: they are black-box models offering no insight in the mappings learned, and they require significant trial and error tuning of their meta-parameters. In this paper, we propose a new approach to constructing smooth value functions by means of symbolic regression. We introduce three off-line methods for finding value functions based on a state transition model: symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions not only yield well-performing policies, but also are compact, human-readable and mathematically tractable. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with alternative approaches using neural networks shows that our method constructs well-performing value functions with substantially fewer parameters. |
| Tasks | Decision Making |
| Published | 2019-03-22 |
| URL | http://arxiv.org/abs/1903.09688v1 |
| http://arxiv.org/pdf/1903.09688v1.pdf | |
| PWC | https://paperswithcode.com/paper/symbolic-regression-methods-for-reinforcement |
| Repo | |
| Framework | |
Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in Distributional Semantic Models
| Title | Bad Form: Comparing Context-Based and Form-Based Few-Shot Learning in Distributional Semantic Models |
| Authors | Jeroen Van Hautte, Guy Emerson, Marek Rei |
| Abstract | Word embeddings are an essential component in a wide range of natural language processing applications. However, distributional semantic models are known to struggle when only a small number of context sentences are available. Several methods have been proposed to obtain higher-quality vectors for these words, leveraging both this context information and sometimes the word forms themselves through a hybrid approach. We show that the current tasks do not suffice to evaluate models that use word-form information, as such models can easily leverage word forms in the training data that are related to word forms in the test data. We introduce 3 new tasks, allowing for a more balanced comparison between models. Furthermore, we show that hyperparameters that have largely been ignored in previous work can consistently improve the performance of both baseline and advanced models, achieving a new state of the art on 4 out of 6 tasks. |
| Tasks | Few-Shot Learning, Word Embeddings |
| Published | 2019-10-01 |
| URL | https://arxiv.org/abs/1910.00275v1 |
| https://arxiv.org/pdf/1910.00275v1.pdf | |
| PWC | https://paperswithcode.com/paper/bad-form-comparing-context-based-and-form |
| Repo | |
| Framework | |
Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
| Title | Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate |
| Authors | Sebastian Billaudelle, Yannik Stradmann, Korbinian Schreiber, Benjamin Cramer, Andreas Baumbach, Dominik Dold, Julian Göltz, Akos F. Kungl, Timo C. Wunderlich, Andreas Hartel, Eric Müller, Oliver Breitwieser, Christian Mauch, Mitja Kleider, Andreas Grübl, David Stöckel, Christian Pehle, Arthur Heimbrecht, Philipp Spilger, Gerd Kiene, Vitali Karasenko, Walter Senn, Mihai A. Petrovici, Johannes Schemmel, Karlheinz Meier |
| Abstract | We present first experimental results on the novel BrainScaleS-2 neuromorphic architecture based on an analog neuro-synaptic core and augmented by embedded microprocessors for complex plasticity and experiment control. The high acceleration factor of 1000 compared to biological dynamics enables the execution of computationally expensive tasks, by allowing the fast emulation of long-duration experiments or rapid iteration over many consecutive trials. The flexibility of our architecture is demonstrated in a suite of five distinct experiments, which emphasize different aspects of the BrainScaleS-2 system. |
| Tasks | |
| Published | 2019-12-30 |
| URL | https://arxiv.org/abs/1912.12980v1 |
| https://arxiv.org/pdf/1912.12980v1.pdf | |
| PWC | https://paperswithcode.com/paper/versatile-emulation-of-spiking-neural |
| Repo | |
| Framework | |
Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search
| Title | Query-Adaptive Hash Code Ranking for Large-Scale Multi-View Visual Search |
| Authors | Xianglong Liu, Lei Huang, Cheng Deng, Bo Lang, Dacheng Tao |
| Abstract | Hash based nearest neighbor search has become attractive in many applications. However, the quantization in hashing usually degenerates the discriminative power when using Hamming distance ranking. Besides, for large-scale visual search, existing hashing methods cannot directly support the efficient search over the data with multiple sources, and while the literature has shown that adaptively incorporating complementary information from diverse sources or views can significantly boost the search performance. To address the problems, this paper proposes a novel and generic approach to building multiple hash tables with multiple views and generating fine-grained ranking results at bitwise and tablewise levels. For each hash table, a query-adaptive bitwise weighting is introduced to alleviate the quantization loss by simultaneously exploiting the quality of hash functions and their complement for nearest neighbor search. From the tablewise aspect, multiple hash tables are built for different data views as a joint index, over which a query-specific rank fusion is proposed to rerank all results from the bitwise ranking by diffusing in a graph. Comprehensive experiments on image search over three well-known benchmarks show that the proposed method achieves up to 17.11% and 20.28% performance gains on single and multiple table search over state-of-the-art methods. |
| Tasks | Image Retrieval, Quantization |
| Published | 2019-04-18 |
| URL | http://arxiv.org/abs/1904.08623v1 |
| http://arxiv.org/pdf/1904.08623v1.pdf | |
| PWC | https://paperswithcode.com/paper/query-adaptive-hash-code-ranking-for-large |
| Repo | |
| Framework | |
Deep learning within a priori temporal feature spaces for large-scale dynamic MR image reconstruction: Application to 5-D cardiac MR Multitasking
| Title | Deep learning within a priori temporal feature spaces for large-scale dynamic MR image reconstruction: Application to 5-D cardiac MR Multitasking |
| Authors | Yuhua Chen, Jaime L. Shaw, Yibin Xie, Debiao Li, Anthony G. Christodoulou |
| Abstract | High spatiotemporal resolution dynamic magnetic resonance imaging (MRI) is a powerful clinical tool for imaging moving structures as well as to reveal and quantify other physical and physiological dynamics. The low speed of MRI necessitates acceleration methods such as deep learning reconstruction from under-sampled data. However, the massive size of many dynamic MRI problems prevents deep learning networks from directly exploiting global temporal relationships. In this work, we show that by applying deep neural networks inside a priori calculated temporal feature spaces, we enable deep learning reconstruction with global temporal modeling even for image sequences with >40,000 frames. One proposed variation of our approach using dilated multi-level Densely Connected Network (mDCN) speeds up feature space coordinate calculation by 3000x compared to conventional iterative methods, from 20 minutes to 0.39 seconds. Thus, the combination of low-rank tensor and deep learning models not only makes large-scale dynamic MRI feasible but also practical for routine clinical application. |
| Tasks | Image Reconstruction |
| Published | 2019-10-02 |
| URL | https://arxiv.org/abs/1910.00956v1 |
| https://arxiv.org/pdf/1910.00956v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-within-a-priori-temporal |
| Repo | |
| Framework | |
