January 27, 2020

2983 words 15 mins read

Paper Group ANR 1229

Automated Multi-sequence Cardiac MRI Segmentation Using Supervised Domain Adaptation. Spectral Graph Wavelet Transform as Feature Extractor for Machine Learning in Neuroimaging. BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA. Novelty Detection via Network Saliency in Visual-based Deep Learning. Vis …

Automated Multi-sequence Cardiac MRI Segmentation Using Supervised Domain Adaptation


Title	Automated Multi-sequence Cardiac MRI Segmentation Using Supervised Domain Adaptation
Authors	Sulaiman Vesal, Nishant Ravikumar, Andreas Maier
Abstract	Left ventricle segmentation and morphological assessment are essential for improving diagnosis and our understanding of cardiomyopathy, which in turn is imperative for reducing risk of myocardial infarctions in patients. Convolutional neural network (CNN) based methods for cardiac magnetic resonance (CMR) image segmentation rely on supervision with pixel-level annotations, and may not generalize well to images from a different domain. These methods are typically sensitive to variations in imaging protocols and data acquisition. Since annotating multi-sequence CMR images is tedious and subject to inter- and intra-observer variations, developing methods that can automatically adapt from one domain to the target domain is of great interest. In this paper, we propose an approach for domain adaptation in multi-sequence CMR segmentation task using transfer learning that combines multi-source image information. We first train an encoder-decoder CNN on T2-weighted and balanced-Steady State Free Precession (bSSFP) MR images with pixel-level annotation and fine-tune the same network with a limited number of Late Gadolinium Enhanced-MR (LGE-MR) subjects, to adapt the domain features. The domain-adapted network was trained with just four LGE-MR training samples and obtained an average Dice score of $\sim$85.0% on the test set comprises of 40 LGE-MR subjects. The proposed method significantly outperformed a network without adaptation trained from scratch on the same set of LGE-MR training data.
Tasks	Domain Adaptation, Semantic Segmentation, Transfer Learning
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07726v1
PDF	https://arxiv.org/pdf/1908.07726v1.pdf
PWC	https://paperswithcode.com/paper/automated-multi-sequence-cardiac-mri
Repo
Framework

Spectral Graph Wavelet Transform as Feature Extractor for Machine Learning in Neuroimaging


Title	Spectral Graph Wavelet Transform as Feature Extractor for Machine Learning in Neuroimaging
Authors	Yusuf Pilavci, Nicolas Farrugia
Abstract	Graph Signal Processing has become a very useful framework for signal operations and representations defined on irregular domains. Exploiting transformations that are defined on graph models can be highly beneficial when the graph encodes relationships between signals. In this work, we present the benefits of using Spectral Graph Wavelet Transform (SGWT) as a feature extractor for machine learning on brain graphs. First, we consider a synthetic regression problem in which the smooth graph signals are generated as input with additive noise, and the target is derived from the input without noise. This enables us to optimize the spectrum coverage using different wavelet shapes. Finally, we present the benefits obtained by SGWT on a functional Magnetic Resonance Imaging (fMRI) open dataset on human subjects, with several graphs and wavelet shapes, by demonstrating significant performance improvements compared to the state of the art.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05149v1
PDF	https://arxiv.org/pdf/1910.05149v1.pdf
PWC	https://paperswithcode.com/paper/spectral-graph-wavelet-transform-as-feature
Repo
Framework

BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA


Title	BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
Authors	Nina Poerner, Ulli Waltinger, Hinrich Schütze
Abstract	The BERT language model (LM) (Devlin et al., 2019) is surprisingly good at answering cloze-style questions about relational facts. Petroni et al. (2019) take this as evidence that BERT memorizes factual knowledge during pre-training. We take issue with this interpretation and argue that the performance of BERT is partly due to reasoning about (the surface form of) entity names, e.g., guessing that a person with an Italian-sounding name speaks Italian. More specifically, we show that BERT’s precision drops dramatically when we filter certain easy-to-guess facts. As a remedy, we propose E-BERT, an extension of BERT that replaces entity mentions with symbolic entity embeddings. E-BERT outperforms both BERT and ERNIE (Zhang et al., 2019) on hard-to-guess queries. We take this as evidence that E-BERT is richer in factual knowledge, and we show two ways of ensembling BERT and E-BERT.
Tasks	Entity Embeddings, Language Modelling
Published	2019-11-09
URL	https://arxiv.org/abs/1911.03681v1
PDF	https://arxiv.org/pdf/1911.03681v1.pdf
PWC	https://paperswithcode.com/paper/bert-is-not-a-knowledge-base-yet-factual
Repo
Framework

Novelty Detection via Network Saliency in Visual-based Deep Learning


Title	Novelty Detection via Network Saliency in Visual-based Deep Learning
Authors	Valerie Chen, Man-Ki Yoon, Zhong Shao
Abstract	Machine-learning driven safety-critical autonomous systems, such as self-driving cars, must be able to detect situations where its trained model is not able to make a trustworthy prediction. Often viewed as a black-box, it is non-obvious to determine when a model will make a safe decision and when it will make an erroneous, perhaps life-threatening one. Prior work on novelty detection deal with highly structured data and do not translate well to dynamic, real-world situations. This paper proposes a multi-step framework for the detection of novel scenarios in vision-based autonomous systems by leveraging information learned by the trained prediction model and a new image similarity metric. We demonstrate the efficacy of this method through experiments on a real-world driving dataset as well as on our in-house indoor racing environment.
Tasks	Self-Driving Cars
Published	2019-06-09
URL	https://arxiv.org/abs/1906.03685v1
PDF	https://arxiv.org/pdf/1906.03685v1.pdf
PWC	https://paperswithcode.com/paper/novelty-detection-via-network-saliency-in
Repo
Framework

Visualization of Multi-Objective Switched Reluctance Machine Optimization at Multiple Operating Conditions with t-SNE


Title	Visualization of Multi-Objective Switched Reluctance Machine Optimization at Multiple Operating Conditions with t-SNE
Authors	Shen Zhang, Shibo Zhang, Sufei Li, Liang Du, Thomas G. Habetler
Abstract	The optimization of electric machines at multiple operating points is crucial for applications that require frequent changes on speeds and loads, such as the electric vehicles, to strive for the machine optimal performance across the entire driving cycle. However, the number of objectives that would need to be optimized would significantly increase with the number of operating points considered in the optimization, thus posting a potential problem in regards to the visualization techniques currently in use, such as in the scatter plots of Pareto fronts, the parallel coordinates, and in the principal component analysis (PCA), inhibiting their ability to provide machine designers with intuitive and informative visualizations of all of the design candidates and their ability to pick a few for further fine-tuning with performance verification. Therefore, this paper proposes the utilization of t-distributed stochastic neighbor embedding (t-SNE) to visualize all of the optimization objectives of various electric machines design candidates with various operating conditions, which constitute a high-dimensional set of data that would lie on several different, but related, low-dimensional manifolds. Finally, two case studies of switched reluctance machines (SRM) are presented to illustrate the superiority of then t-SNE when compared to traditional visualization techniques used in electric machine optimizations.
Tasks
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01024v1
PDF	https://arxiv.org/pdf/1911.01024v1.pdf
PWC	https://paperswithcode.com/paper/visualization-of-multi-objective-switched
Repo
Framework

Using U-Nets to Create High-Fidelity Virtual Observations of the Solar Corona


Title	Using U-Nets to Create High-Fidelity Virtual Observations of the Solar Corona
Authors	Valentina Salvatelli, Souvik Bose, Brad Neuberg, Luiz F. G. dos Santos, Mark Cheung, Miho Janvier, Atilim Gunes Baydin, Yarin Gal, Meng Jin
Abstract	Understanding and monitoring the complex and dynamic processes of the Sun is important for a number of human activities on Earth and in space. For this reason, NASA’s Solar Dynamics Observatory (SDO) has been continuously monitoring the multi-layered Sun’s atmosphere in high-resolution since its launch in 2010, generating terabytes of observational data every day. The synergy between machine learning and this enormous amount of data has the potential, still largely unexploited, to advance our understanding of the Sun and extend the capabilities of heliophysics missions. In the present work, we show that deep learning applied to SDO data can be successfully used to create a high-fidelity virtual telescope that generates synthetic observations of the solar corona by image translation. Towards this end we developed a deep neural network, structured as an encoder-decoder with skip connections (U-Net), that reconstructs the Sun’s image of one instrument channel given temporally aligned images in three other channels. The approach we present has the potential to reduce the telemetry needs of SDO, enhance the capabilities of missions that have less observing channels, and transform the concept development of future missions.
Tasks
Published	2019-11-10
URL	https://arxiv.org/abs/1911.04006v1
PDF	https://arxiv.org/pdf/1911.04006v1.pdf
PWC	https://paperswithcode.com/paper/using-u-nets-to-create-high-fidelity-virtual
Repo
Framework

Morphological Word Embeddings


Title	Morphological Word Embeddings
Authors	Ryan Cotterell, Hinrich Schütze
Abstract	Linguistic similarity is multi-faceted. For instance, two words may be similar with respect to semantics, syntax, or morphology inter alia. Continuous word-embeddings have been shown to capture most of these shades of similarity to some degree. This work considers guiding word-embeddings with morphologically annotated data, a form of semi-supervised learning, encouraging the vectors to encode a word’s morphology, i.e., words close in the embedded space share morphological features. We extend the log-bilinear model to this end and show that indeed our learned embeddings achieve this, using German as a case study.
Tasks	Word Embeddings
Published	2019-07-04
URL	https://arxiv.org/abs/1907.02423v1
PDF	https://arxiv.org/pdf/1907.02423v1.pdf
PWC	https://paperswithcode.com/paper/morphological-word-embeddings-1
Repo
Framework

On the optimality of kernels for high-dimensional clustering


Title	On the optimality of kernels for high-dimensional clustering
Authors	Leena Chennuru Vankadara, Debarghya Ghoshdastidar
Abstract	This paper studies the optimality of kernel methods in high-dimensional data clustering. Recent works have studied the large sample performance of kernel clustering in the high-dimensional regime, where Euclidean distance becomes less informative. However, it is unknown whether popular methods, such as kernel k-means, are optimal in this regime. We consider the problem of high-dimensional Gaussian clustering and show that, with the exponential kernel function, the sufficient conditions for partial recovery of clusters using the NP-hard kernel k-means objective matches the known information-theoretic limit up to a factor of $\sqrt{2}$ for large $k$. It also exactly matches the known upper bounds for the non-kernel setting. We also show that a semi-definite relaxation of the kernel k-means procedure matches up to constant factors, the spectral threshold, below which no polynomial-time algorithm is known to succeed. This is the first work that provides such optimality guarantees for the kernel k-means as well as its convex relaxation. Our proofs demonstrate the utility of the less known polynomial concentration results for random variables with exponentially decaying tails in a higher-order analysis of kernel methods.
Tasks
Published	2019-12-01
URL	https://arxiv.org/abs/1912.00458v1
PDF	https://arxiv.org/pdf/1912.00458v1.pdf
PWC	https://paperswithcode.com/paper/on-the-optimality-of-kernels-for-high
Repo
Framework

Hunting for Troll Comments in News Community Forums


Title	Hunting for Troll Comments in News Community Forums
Authors	Todor Mihaylov, Preslav Nakov
Abstract	There are different definitions of what a troll is. Certainly, a troll can be somebody who teases people to make them angry, or somebody who offends people, or somebody who wants to dominate any single discussion, or somebody who tries to manipulate people’s opinion (sometimes for money), etc. The last definition is the one that dominates the public discourse in Bulgaria and Eastern Europe, and this is our focus in this paper. In our work, we examine two types of opinion manipulation trolls: paid trolls that have been revealed from leaked reputation management contracts and mentioned trolls that have been called such by several different people. We show that these definitions are sensible: we build two classifiers that can distinguish a post by such a paid troll from one by a non-troll with 81-82% accuracy; the same classifier achieves 81-82% accuracy on so called mentioned troll vs. non-troll posts.
Tasks
Published	2019-11-19
URL	https://arxiv.org/abs/1911.08113v1
PDF	https://arxiv.org/pdf/1911.08113v1.pdf
PWC	https://paperswithcode.com/paper/hunting-for-troll-comments-in-news-community-1
Repo
Framework

An Empirical Evaluation of Two General Game Systems: Ludii and RBG


Title	An Empirical Evaluation of Two General Game Systems: Ludii and RBG
Authors	Éric Piette, Matthew Stephenson, Dennis J. N. J. Soemers, Cameron Browne
Abstract	Although General Game Playing (GGP) systems can facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often computationally inefficient and somewhat specialised to a specific class of games. However, since the start of this year, two General Game Systems have emerged that provide efficient alternatives to the academic state of the art – the Game Description Language (GDL). In order of publication, these are the Regular Boardgames language (RBG), and the Ludii system. This paper offers an experimental evaluation of Ludii. Here, we focus mainly on a comparison between the two new systems in terms of two key properties for any GGP system: simplicity/clarity (e.g. human-readability), and efficiency.
Tasks
Published	2019-06-29
URL	https://arxiv.org/abs/1907.00244v1
PDF	https://arxiv.org/pdf/1907.00244v1.pdf
PWC	https://paperswithcode.com/paper/an-empirical-evaluation-of-two-general-game
Repo
Framework

Cascading Non-Stationary Bandits: Online Learning to Rank in the Non-Stationary Cascade Model


Title	Cascading Non-Stationary Bandits: Online Learning to Rank in the Non-Stationary Cascade Model
Authors	Chang Li, Maarten de Rijke
Abstract	Non-stationarity appears in many online applications such as web search and advertising. In this paper, we study the online learning to rank problem in a non-stationary environment where user preferences change abruptly at an unknown moment in time. We consider the problem of identifying the K most attractive items and propose cascading non-stationary bandits, an online learning variant of the cascading model, where a user browses a ranked list from top to bottom and clicks on the first attractive item. We propose two algorithms for solving this non-stationary problem: CascadeDUCB and CascadeSWUCB. We analyze their performance and derive gap-dependent upper bounds on the n-step regret of these algorithms. We also establish a lower bound on the regret for cascading non-stationary bandits and show that both algorithms match the lower bound up to a logarithmic factor. Finally, we evaluate their performance on a real-world web search click dataset.
Tasks	Learning-To-Rank
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12370v3
PDF	https://arxiv.org/pdf/1905.12370v3.pdf
PWC	https://paperswithcode.com/paper/cascading-non-stationary-bandits-online
Repo
Framework

Speech Emotion Recognition via Contrastive Loss under Siamese Networks


Title	Speech Emotion Recognition via Contrastive Loss under Siamese Networks
Authors	Zheng Lian, Ya Li, Jianhua Tao, Jian Huang
Abstract	Speech emotion recognition is an important aspect of human-computer interaction. Prior work proposes various end-to-end models to improve the classification performance. However, most of them rely on the cross-entropy loss together with softmax as the supervision component, which does not explicitly encourage discriminative learning of features. In this paper, we introduce the contrastive loss function to encourage intra-class compactness and inter-class separability between learnable features. Furthermore, multiple feature selection methods and pairwise sample selection methods are evaluated. To verify the performance of the proposed system, we conduct experiments on The Interactive Emotional Dyadic Motion Capture (IEMOCAP) database, a common evaluation corpus. Experimental results reveal the advantages of the proposed method, which reaches 62.19% in the weighted accuracy and 63.21% in the unweighted accuracy. It outperforms the baseline system that is optimized without the contrastive loss function with 1.14% and 2.55% in the weighted accuracy and the unweighted accuracy, respectively.
Tasks	Emotion Recognition, Feature Selection, Motion Capture, Speech Emotion Recognition
Published	2019-10-23
URL	https://arxiv.org/abs/1910.11174v1
PDF	https://arxiv.org/pdf/1910.11174v1.pdf
PWC	https://paperswithcode.com/paper/speech-emotion-recognition-via-contrastive
Repo
Framework

From Video Game to Real Robot: The Transfer between Action Spaces


Title	From Video Game to Real Robot: The Transfer between Action Spaces
Authors	Janne Karttunen, Anssi Kanervisto, Ville Kyrki, Ville Hautamäki
Abstract	Deep reinforcement learning has proven to be successful for learning tasks in simulated environments, but applying same techniques for robots in real-world domain is more challenging, as they require hours of training. To address this, transfer learning can be used to train the policy first in a simulated environment and then transfer it to physical agent. As the simulation never matches reality perfectly, the physics, visuals and action spaces by necessity differ between these environments to some degree. In this work, we study how general video games can be directly used instead of fine-tuned simulations for the sim-to-real transfer. Especially, we study how the agent can learn the new action space autonomously, when the game actions do not match the robot actions. Our results show that the different action space can be learned by re-training only part of neural network and we obtain above 90% mean success rate in simulation and robot experiments.
Tasks	Transfer Learning
Published	2019-05-02
URL	https://arxiv.org/abs/1905.00741v2
PDF	https://arxiv.org/pdf/1905.00741v2.pdf
PWC	https://paperswithcode.com/paper/from-video-game-to-real-robot-the-transfer
Repo
Framework

Predicting Customer Call Intent by Analyzing Phone Call Transcripts based on CNN for Multi-Class Classification


Title	Predicting Customer Call Intent by Analyzing Phone Call Transcripts based on CNN for Multi-Class Classification
Authors	Junmei Zhong, William Li
Abstract	Auto dealerships receive thousands of calls daily from customers who are interested in sales, service, vendors and jobseekers. With so many calls, it is very important for auto dealers to understand the intent of these calls to provide positive customer experiences that ensure customer satisfaction, deep customer engagement to boost sales and revenue, and optimum allocation of agents or customer service representatives across the business. In this paper, we define the problem of customer phone call intent as a multi-class classification problem stemming from the large database of recorded phone call transcripts. To solve this problem, we develop a convolutional neural network (CNN)-based supervised learning model to classify the customer calls into four intent categories: sales, service, vendor and jobseeker. Experimental results show that with the thrust of our scalable data labeling method to provide sufficient training data, the CNN-based predictive model performs very well on long text classification according to the quantitative metrics of F1-Score, precision, recall, and accuracy.
Tasks	Text Classification
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03715v1
PDF	https://arxiv.org/pdf/1907.03715v1.pdf
PWC	https://paperswithcode.com/paper/predicting-customer-call-intent-by-analyzing
Repo
Framework

Graph Informer Networks for Molecules


Title	Graph Informer Networks for Molecules
Authors	Jaak Simm, Adam Arany, Edward De Brouwer, Yves Moreau
Abstract	In machine learning, chemical molecules are often represented by sparse high-dimensional vectorial fingerprints. However, a more natural mathematical object for molecule representation is a graph, which is much more challenging to handle from a machine learning perspective. In recent years, several deep learning architectures have been proposed to directly learn from the graph structure of chemical molecules, including graph convolution (Duvenaud et al., 2015) and graph gating networks (Li et al., 2015). Here, we introduce Graph Informer, a route-based multi-head attention mechanism inspired by transformer networks (Vaswani et al., 2017), which incorporates features for node pairs. We show empirically that the proposed method gives significant improvements over existing approaches in prediction tasks for 13C nuclear magnetic resonance spectra and for drug bioactivity. These results indicate that our method is well suited for both node-level and graph-level prediction tasks.
Tasks
Published	2019-07-25
URL	https://arxiv.org/abs/1907.11318v1
PDF	https://arxiv.org/pdf/1907.11318v1.pdf
PWC	https://paperswithcode.com/paper/graph-informer-networks-for-molecules
Repo
Framework