Paper Group ANR 1542
Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets. Self-Attention with Structural Position Representations. Inspecting and Interacting with Meaningful Music Representations using VAE. Gextext: Disease Network Extraction from Biomedical Literature. On the Convergence of Memory-Based Distributed SGD. An Extensible In …
Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets
| Title | Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets |
| Authors | Nicolas Roulet, Diego Fernandez Slezak, Enzo Ferrante |
| Abstract | Brain lesion and anatomy segmentation in magnetic resonance images are fundamental tasks in neuroimaging research and clinical practice. Given enough training data, convolutional neuronal networks (CNN) proved to outperform all existent techniques in both tasks independently. However, to date, little work has been done regarding simultaneous learning of brain lesion and anatomy segmentation from disjoint datasets. In this work we focus on training a single CNN model to predict brain tissue and lesion segmentations using heterogeneous datasets labeled independently, according to only one of these tasks (a common scenario when using publicly available datasets). We show that label contradiction issues can arise in this case, and propose a novel adaptive cross entropy (ACE) loss function that makes such training possible. We provide quantitative evaluation in two different scenarios, benchmarking the proposed method in comparison with a multi-network approach. Our experiments suggest that ACE loss enables training of single models when standard cross entropy and Dice loss functions tend to fail. Moreover, we show that it is possible to achieve competitive results when comparing with multiple networks trained for independent tasks. |
| Tasks | |
| Published | 2019-03-08 |
| URL | http://arxiv.org/abs/1903.03445v2 |
| http://arxiv.org/pdf/1903.03445v2.pdf | |
| PWC | https://paperswithcode.com/paper/joint-learning-of-brain-lesion-and-anatomy |
| Repo | |
| Framework | |
Self-Attention with Structural Position Representations
| Title | Self-Attention with Structural Position Representations |
| Authors | Xing Wang, Zhaopeng Tu, Longyue Wang, Shuming Shi |
| Abstract | Although self-attention networks (SANs) have advanced the state-of-the-art on various NLP tasks, one criticism of SANs is their ability of encoding positions of input words (Shaw et al., 2018). In this work, we propose to augment SANs with structural position representations to model the latent structure of the input sentence, which is complementary to the standard sequential positional representations. Specifically, we use dependency tree to represent the grammatical structure of a sentence, and propose two strategies to encode the positional relationships among words in the dependency tree. Experimental results on NIST Chinese-to-English and WMT14 English-to-German translation tasks show that the proposed approach consistently boosts performance over both the absolute and relative sequential position representations. |
| Tasks | |
| Published | 2019-09-01 |
| URL | https://arxiv.org/abs/1909.00383v1 |
| https://arxiv.org/pdf/1909.00383v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-attention-with-structural-position |
| Repo | |
| Framework | |
Inspecting and Interacting with Meaningful Music Representations using VAE
| Title | Inspecting and Interacting with Meaningful Music Representations using VAE |
| Authors | Ruihan Yang, Tianyao Chen, Yiyi Zhang, Gus Xia |
| Abstract | Variational Autoencoders(VAEs) have already achieved great results on image generation and recently made promising progress on music generation. However, the generation process is still quite difficult to control in the sense that the learned latent representations lack meaningful music semantics. It would be much more useful if people can modify certain music features, such as rhythm and pitch contour, via latent representations to test different composition ideas. In this paper, we propose a new method to inspect the pitch and rhythm interpretations of the latent representations and we name it disentanglement by augmentation. Based on the interpretable representations, an intuitive graphical user interface is designed for users to better direct the music creation process by manipulating the pitch contours and rhythmic complexity. |
| Tasks | Image Generation, Music Generation |
| Published | 2019-04-18 |
| URL | http://arxiv.org/abs/1904.08842v1 |
| http://arxiv.org/pdf/1904.08842v1.pdf | |
| PWC | https://paperswithcode.com/paper/inspecting-and-interacting-with-meaningful |
| Repo | |
| Framework | |
Gextext: Disease Network Extraction from Biomedical Literature
| Title | Gextext: Disease Network Extraction from Biomedical Literature |
| Authors | Robert O’Shea |
| Abstract | PURPOSE: We propose a fully unsupervised method to learn latent disease networks directly from unstructured biomedical text corpora. This method addresses current challenges in unsupervised knowledge extraction, such as the detection of long-range dependencies and requirements for large training corpora. METHODS: Let C be a corpus of n text chunks. Let V be a set of p disease terms occurring in the corpus. Let X indicate the occurrence of V in C. Gextext identifies disease similarities by positively correlated occurrence patterns. This information is combined to generate a graph on which geodesic distance describes dissimilarity. Diseasomes were learned by Gextext and GloVE on corpora of 100-1000 PubMed abstracts. Similarity matrix estimates were validated against biomedical semantic similarity metrics and gene profile similarity. RESULTS: Geodesic distance on Gextext-inferred diseasomes correlated inversely with external measures of semantic similarity. Gene profile similarity also correlated significant with proximity on the inferred graph. Gextext outperformed GloVE in our experiments. The information contained on the Gextext graph exceeded the explicit information content within the text. CONCLUSIONS: Gextext extracts latent relationships from unstructured text, enabling fully unsupervised modelling of diseasome graphs from PubMed abstracts. |
| Tasks | Semantic Similarity, Semantic Textual Similarity |
| Published | 2019-11-06 |
| URL | https://arxiv.org/abs/1911.02562v2 |
| https://arxiv.org/pdf/1911.02562v2.pdf | |
| PWC | https://paperswithcode.com/paper/gextext-unsupervised-knowledge-modelling-in |
| Repo | |
| Framework | |
On the Convergence of Memory-Based Distributed SGD
| Title | On the Convergence of Memory-Based Distributed SGD |
| Authors | Shen-Yi Zhao, Hao Gao, Wu-Jun Li |
| Abstract | Distributed stochastic gradient descent~(DSGD) has been widely used for optimizing large-scale machine learning models, including both convex and non-convex models. With the rapid growth of model size, huge communication cost has been the bottleneck of traditional DSGD. Recently, many communication compression methods have been proposed. Memory-based distributed stochastic gradient descent~(M-DSGD) is one of the efficient methods since each worker communicates a sparse vector in each iteration so that the communication cost is small. Recent works propose the convergence rate of M-DSGD when it adopts vanilla SGD. However, there is still a lack of convergence theory for M-DSGD when it adopts momentum SGD. In this paper, we propose a universal convergence analysis for M-DSGD by introducing \emph{transformation equation}. The transformation equation describes the relation between traditional DSGD and M-DSGD so that we can transform M-DSGD to its corresponding DSGD. Hence we get the convergence rate of M-DSGD with momentum for both convex and non-convex problems. Furthermore, we combine M-DSGD and stagewise learning that the learning rate of M-DSGD in each stage is a constant and is decreased by stage, instead of iteration. Using the transformation equation, we propose the convergence rate of stagewise M-DSGD which bridges the gap between theory and practice. |
| Tasks | |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.12960v1 |
| https://arxiv.org/pdf/1905.12960v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-convergence-of-memory-based |
| Repo | |
| Framework | |
An Extensible Interactive Interface for Agent Design
| Title | An Extensible Interactive Interface for Agent Design |
| Authors | Matthew Rahtz, James Fang, Anca D. Dragan, Dylan Hadfield-Menell |
| Abstract | In artificial intelligence, we often specify tasks through a reward function. While this works well in some settings, many tasks are hard to specify this way. In deep reinforcement learning, for example, directly specifying a reward as a function of a high-dimensional observation is challenging. Instead, we present an interface for specifying tasks interactively using demonstrations. Our approach defines a set of increasingly complex policies. The interface allows the user to switch between these policies at fixed intervals to generate demonstrations of novel, more complex, tasks. We train new policies based on these demonstrations and repeat the process. We present a case study of our approach in the Lunar Lander domain, and show that this simple approach can quickly learn a successful landing policy and outperforms an existing comparison-based deep RL method. |
| Tasks | |
| Published | 2019-06-06 |
| URL | https://arxiv.org/abs/1906.02641v3 |
| https://arxiv.org/pdf/1906.02641v3.pdf | |
| PWC | https://paperswithcode.com/paper/an-extensible-interactive-interface-for-agent |
| Repo | |
| Framework | |
Angular Learning: Toward Discriminative Embedded Features
| Title | Angular Learning: Toward Discriminative Embedded Features |
| Authors | JT Wu, L. Wang |
| Abstract | The margin-based softmax loss functions greatly enhance intra-class compactness and perform well on the tasks of face recognition and object classification. Outperformance, however, depends on the careful hyperparameter selection. Moreover, the hard angle restriction also increases the risk of overfitting. In this paper, angular loss suggested by maximizing the angular gradient to promote intra-class compactness avoids overfitting. Besides, our method has only one adjustable constant for intra-class compactness control. We define three metrics to measure inter-class separability and intra-class compactness. In experiments, we test our method, as well as other methods, on many well-known datasets. Experimental results reveal that our method has the superiority of accuracy improvement, discriminative information, and time-consumption. |
| Tasks | Face Recognition, Object Classification |
| Published | 2019-12-17 |
| URL | https://arxiv.org/abs/1912.07819v1 |
| https://arxiv.org/pdf/1912.07819v1.pdf | |
| PWC | https://paperswithcode.com/paper/angular-learning-toward-discriminative |
| Repo | |
| Framework | |
DNN-based cross-lingual voice conversion using Bottleneck Features
| Title | DNN-based cross-lingual voice conversion using Bottleneck Features |
| Authors | M Kiran Reddy, K Sreenivasa Rao |
| Abstract | Cross-lingual voice conversion (CLVC) is a quite challenging task since the source and target speakers speak different languages. This paper proposes a CLVC framework based on bottleneck features and deep neural network (DNN). In the proposed method, the bottleneck features extracted from a deep auto-encoder (DAE) are used to represent speaker-independent features of speech signals from different languages. A DNN model is trained to learn the mapping between bottleneck features and the corresponding spectral features of the target speaker. The proposed method can capture speaker-specific characteristics of a target speaker, and hence requires no speech data from source speaker during training. The performance of the proposed method is evaluated using data from three Indian languages: Telugu, Tamil and Malayalam. The experimental results show that the proposed method outperforms the baseline Gaussian mixture model (GMM)-based CLVC approach. |
| Tasks | Voice Conversion |
| Published | 2019-09-09 |
| URL | https://arxiv.org/abs/1909.03974v2 |
| https://arxiv.org/pdf/1909.03974v2.pdf | |
| PWC | https://paperswithcode.com/paper/dnn-based-cross-lingual-voice-conversion |
| Repo | |
| Framework | |
Comparing linear structure-based and data-driven latent spatial representations for sequence prediction
| Title | Comparing linear structure-based and data-driven latent spatial representations for sequence prediction |
| Authors | Myriam Bontonou, Carlos Lassance, Vincent Gripon, Nicolas Farrugia |
| Abstract | Predicting the future of Graph-supported Time Series (GTS) is a key challenge in many domains, such as climate monitoring, finance or neuroimaging. Yet it is a highly difficult problem as it requires to account jointly for time and graph (spatial) dependencies. To simplify this process, it is common to use a two-step procedure in which spatial and time dependencies are dealt with separately. In this paper, we are interested in comparing various linear spatial representations, namely structure-based ones and data-driven ones, in terms of how they help predict the future of GTS. To that end, we perform experiments with various datasets including spontaneous brain activity and raw videos. |
| Tasks | Time Series |
| Published | 2019-08-19 |
| URL | https://arxiv.org/abs/1908.06868v1 |
| https://arxiv.org/pdf/1908.06868v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparing-linear-structure-based-and-data |
| Repo | |
| Framework | |
Seasonally-Adjusted Auto-Regression of Vector Time Series
| Title | Seasonally-Adjusted Auto-Regression of Vector Time Series |
| Authors | Enzo Busseti |
| Abstract | We present a simple algorithm to forecast vector time series, that is robust against missing data, in both training and inference. It models seasonal annual, weekly, and daily baselines, and a Gaussian process for the seasonally-adjusted residuals. We develop a custom truncated eigendecomposition to fit a low-rank plus block-diagonal Gaussian kernel. Inference is performed with the Schur complement, using Tikhonov regularization to prevent overfit, and the Woodbury formula to invert sub-matrices of the kernel efficiently. Inference requires an amount of memory and computation linear in the dimension of the time series, and so the model can scale to very large datasets. We also propose a simple “greedy” grid search for automatic hyper-parameter tuning. The paper is accompanied by tsar (i.e., time series auto-regressor), a Python library that implements the algorithm. |
| Tasks | Time Series |
| Published | 2019-11-04 |
| URL | https://arxiv.org/abs/1911.01010v1 |
| https://arxiv.org/pdf/1911.01010v1.pdf | |
| PWC | https://paperswithcode.com/paper/seasonally-adjusted-auto-regression-of-vector |
| Repo | |
| Framework | |
Reinforcement Learning is not a Causal problem
| Title | Reinforcement Learning is not a Causal problem |
| Authors | Mauricio Gonzalez-Soto, Felipe Orihuela Espina |
| Abstract | We use an analogy between non-isomorphic mathematical structures defined over the same set and the algebras induced by associative and causal levels of information in order to argue that Reinforcement Learning, in its current formulation, is not a causal problem, independently if the motivation behind it has to do with an agent taking actions. |
| Tasks | |
| Published | 2019-08-20 |
| URL | https://arxiv.org/abs/1908.07617v2 |
| https://arxiv.org/pdf/1908.07617v2.pdf | |
| PWC | https://paperswithcode.com/paper/190807617 |
| Repo | |
| Framework | |
Code-Mixed to Monolingual Translation Framework
| Title | Code-Mixed to Monolingual Translation Framework |
| Authors | Sainik Kumar Mahata, Soumil Mandal, Dipankar Das, Sivaji Bandyopadhyay |
| Abstract | The use of multilingualism in the new generation is widespread in the form of code-mixed data on social media, and therefore a robust translation system is required for catering to the monolingual users, as well as for easier comprehension by language processing models. In this work, we present a translation framework that uses a translation-transliteration strategy for translating code-mixed data into their equivalent monolingual instances. For converting the output to a more fluent form, it is reordered using a target language model. The most important advantage of the proposed framework is that it does not require a code-mixed to monolingual parallel corpus at any point. On testing the framework, it achieved BLEU and TER scores of 16.47 and 55.45, respectively. Since the proposed framework deals with various sub-modules, we dive deeper into the importance of each of them, analyze the errors and finally, discuss some improvement strategies. |
| Tasks | Language Modelling, Transliteration |
| Published | 2019-11-09 |
| URL | https://arxiv.org/abs/1911.03772v2 |
| https://arxiv.org/pdf/1911.03772v2.pdf | |
| PWC | https://paperswithcode.com/paper/code-mixed-to-monolingual-translation |
| Repo | |
| Framework | |
Network Parameter Learning Using Nonlinear Transforms, Local Representation Goals and Local Propagation Constraints
| Title | Network Parameter Learning Using Nonlinear Transforms, Local Representation Goals and Local Propagation Constraints |
| Authors | Dimche Kostadinov, Behrooz Razdehi, Slava Voloshynovskiy |
| Abstract | In this paper, we introduce a novel concept for learning of the parameters in a neural network. Our idea is grounded on modeling a learning problem that addresses a trade-off between (i) satisfying local objectives at each node and (ii) achieving desired data propagation through the network under (iii) local propagation constraints. We consider two types of nonlinear transforms which describe the network representations. One of the nonlinear transforms serves as activation function. The other one enables a locally adjusted, deviation corrective components to be included in the update of the network weights in order to enable attaining target specific representations at the last network node. Our learning principle not only provides insight into the understanding and the interpretation of the learning dynamics, but it offers theoretical guarantees over decoupled and parallel parameter estimation strategy that enables learning in synchronous and asynchronous mode. Numerical experiments validate the potential of our approach on image recognition task. The preliminary results show advantages in comparison to the state-of-the-art methods, w.r.t. the learning time and the network size while having competitive recognition accuracy. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1902.00016v1 |
| http://arxiv.org/pdf/1902.00016v1.pdf | |
| PWC | https://paperswithcode.com/paper/network-parameter-learning-using-nonlinear |
| Repo | |
| Framework | |
Visualisation of Pareto Front Approximation: A Short Survey and Empirical Comparisons
| Title | Visualisation of Pareto Front Approximation: A Short Survey and Empirical Comparisons |
| Authors | Huiru Gao, Haifeng Nie, Ke Li |
| Abstract | Visualisation is an effective way to facilitate the analysis and understanding of multivariate data. In the context of multi-objective optimisation, comparing to quantitative performance metrics, visualisation is, in principle, able to provide a decision maker better insights about Pareto front approximation sets (e.g. the distribution of solutions, the geometric characteristics of Pareto front approximation) thus to facilitate the decision-making (e.g. the exploration of trade-off relationship, the knee region or region of interest). In this paper, we overview some currently prevalent visualisation techniques according to the way how data is represented. To have a better understanding of the pros and cons of different visualisation techniques, we empirically compare six representative visualisation techniques for the exploratory analysis of different Pareto front approximation sets obtained by four state-of-the-art evolutionary multi-objective optimisation algorithms on the classic DTLZ benchmark test problems. From the empirical results, we find that visual comparisons also follow the \textit{No-Free-Lunch} theorem where no single visualisation technique is able to provide a comprehensive understanding of the characteristics of a Pareto front approximation set. In other words, a specific type of visualisation technique is only good at exploring a particular aspect of the data. |
| Tasks | Decision Making |
| Published | 2019-03-05 |
| URL | http://arxiv.org/abs/1903.01768v1 |
| http://arxiv.org/pdf/1903.01768v1.pdf | |
| PWC | https://paperswithcode.com/paper/visualisation-of-pareto-front-approximation-a |
| Repo | |
| Framework | |
Deconstructing Word Embeddings
| Title | Deconstructing Word Embeddings |
| Authors | Koushik Varma Kalidindi |
| Abstract | A review of Word Embedding Models through a deconstructive approach reveals their several shortcomings and inconsistencies. These include instability of the vector representations, a distorted analogical reasoning, geometric incompatibility with linguistic features, and the inconsistencies in the corpus data. A new theoretical embedding model, Derridian Embedding, is proposed in this paper. Contemporary embedding models are evaluated qualitatively in terms of how adequate they are in relation to the capabilities of a Derridian Embedding. |
| Tasks | Word Embeddings |
| Published | 2019-01-08 |
| URL | http://arxiv.org/abs/1902.00551v1 |
| http://arxiv.org/pdf/1902.00551v1.pdf | |
| PWC | https://paperswithcode.com/paper/deconstructing-word-embeddings |
| Repo | |
| Framework | |
