January 28, 2020

3276 words 16 mins read

Paper Group ANR 1066

Learning Representations by Maximizing Mutual Information in Variational Autoencoders. CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning. Cognitive Systems Approach to Smart Cities. Efforts estimation of doctors annotating medical image. Brain-Network Clustering via Kernel-ARMA Modeling and the Grassmannian. T …

Learning Representations by Maximizing Mutual Information in Variational Autoencoders


Title	Learning Representations by Maximizing Mutual Information in Variational Autoencoders
Authors	Ali Lotfi Rezaabad, Sriram Vishwanath
Abstract	Variational autoencoders (VAEs) have ushered in a new era of unsupervised learning methods for complex distributions. Although these techniques are elegant in their approach, they are typically not useful for representation learning. In this work, we propose a simple yet powerful class of VAEs that simultaneously result in meaningful learned representations. Our solution is to combine traditional VAEs with mutual information maximization, with the goal to enhance amortized inference in VAEs using Information Theoretic techniques. We call this approach InfoMax-VAE, and such an approach can significantly boost the quality of learned high-level representations. We realize this through the explicit maximization of information measures associated with the representation. Using extensive experiments on varied datasets and setups, we show that InfoMax-VAE outperforms contemporary popular approaches, including Info-VAE and $\beta$-VAE.
Tasks	Representation Learning
Published	2019-12-21
URL	https://arxiv.org/abs/1912.13361v2
PDF	https://arxiv.org/pdf/1912.13361v2.pdf
PWC	https://paperswithcode.com/paper/learning-representations-by-maximizing-mutual-1
Repo
Framework

CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning


Title	CodedPrivateML: A Fast and Privacy-Preserving Framework for Distributed Machine Learning
Authors	Jinhyun So, Basak Guler, A. Salman Avestimehr, Payman Mohassel
Abstract	How to train a machine learning model while keeping the data private and secure? We present CodedPrivateML, a fast and scalable approach to this critical problem. CodedPrivateML keeps both the data and the model information-theoretically private, while allowing efficient parallelization of training across distributed workers. We characterize CodedPrivateML’s privacy threshold and prove its convergence for logistic (and linear) regression. Furthermore, via experiments over Amazon EC2, we demonstrate that CodedPrivateML can provide an order of magnitude speedup (up to $\sim 34\times$) over the state-of-the-art cryptographic approaches.
Tasks
Published	2019-02-02
URL	http://arxiv.org/abs/1902.00641v1
PDF	http://arxiv.org/pdf/1902.00641v1.pdf
PWC	https://paperswithcode.com/paper/codedprivateml-a-fast-and-privacy-preserving
Repo
Framework

Cognitive Systems Approach to Smart Cities


Title	Cognitive Systems Approach to Smart Cities
Authors	Aladdin Ayesh
Abstract	In our connected world, services are expected to be delivered at speed through multiple means with seamless communication. To put it in day to day conversational terms, ‘there is an app for it’ attitude prevails. Several technologies are needed to meet this growing demand and indeed these technologies are being developed. The first noteworthy is Internet of Things (IoT), which is in itself coupled technologies to deliver seamless communication with ‘anywhere, anytime’ as an underlying objective. The ‘anywhere, anytime’ service delivery paradigm requires a new type of smart systems in developing these services with better capabilities to interact with the human user, such as personalisation, affect state recognition, etc. Here enter cognitive systems, where AI meets cognitive sciences (e.g. cognitive psychology, linguistics, social cognition, etc.). In this paper we will examine the requirements imposed by smart cities development, e.g. intelligent logistics, sensor networks and domestic appliances connectivity, data streams and media delivery, to mention but few. Then we will explore how cognitive systems can meet the challenges these requirements present to the development of new systems. Throughout our discussion here, examples from our recent and current projects will be given supplemented by examples from the literature.
Tasks
Published	2019-06-26
URL	https://arxiv.org/abs/1906.11032v1
PDF	https://arxiv.org/pdf/1906.11032v1.pdf
PWC	https://paperswithcode.com/paper/cognitive-systems-approach-to-smart-cities
Repo
Framework

Efforts estimation of doctors annotating medical image


Title	Efforts estimation of doctors annotating medical image
Authors	Yang Deng, Yao Sun, Yongpei Zhu, Yue Xu, Qianxi Yang, Shuo Zhang, Mingwang Zhu, Jirang Sun, Weiling Zhao, Xiaobo Zhou, Kehong Yuan
Abstract	Accurate annotation of medical image is the crucial step for image AI clinical application. However, annotating medical image will incur a great deal of annotation effort and expense due to its high complexity and needing experienced doctors. To alleviate annotation cost, some active learning methods are proposed. But such methods just cut the number of annotation candidates and do not study how many efforts the doctor will exactly take, which is not enough since even annotating a small amount of medical data will take a lot of time for the doctor. In this paper, we propose a new criterion to evaluate efforts of doctors annotating medical image. First, by coming active learning and U-shape network, we employ a suggestive annotation strategy to choose the most effective annotation candidates. Then we exploit a fine annotation platform to alleviate annotating efforts on each candidate and first utilize a new criterion to quantitatively calculate the efforts taken by doctors. In our work, we take MR brain tissue segmentation as an example to evaluate the proposed method. Extensive experiments on the well-known IBSR18 dataset and MRBrainS18 Challenge dataset show that, using proposed strategy, state-of-the-art segmentation performance can be achieved by using only 60% annotation candidates and annotation efforts can be alleviated by at least 44%, 44%, 47% on CSF, GM, WM separately.
Tasks	Active Learning
Published	2019-01-06
URL	http://arxiv.org/abs/1901.02355v1
PDF	http://arxiv.org/pdf/1901.02355v1.pdf
PWC	https://paperswithcode.com/paper/efforts-estimation-of-doctors-annotating
Repo
Framework

Brain-Network Clustering via Kernel-ARMA Modeling and the Grassmannian


Title	Brain-Network Clustering via Kernel-ARMA Modeling and the Grassmannian
Authors	Cong Ye, Konstantinos Slavakis, Pratik V. Patil, Sarah F. Muldoon, John Medaglia
Abstract	Recent advances in neuroscience and in the technology of functional magnetic resonance imaging (fMRI) and electro-encephalography (EEG) have propelled a growing interest in brain-network clustering via time-series analysis. Notwithstanding, most of the brain-network clustering methods revolve around state clustering and/or node clustering (a.k.a. community detection or topology inference) within states. This work answers first the need of capturing non-linear nodal dependencies by bringing forth a novel feature-extraction mechanism via kernel autoregressive-moving-average modeling. The extracted features are mapped to the Grassmann manifold (Grassmannian), which consists of all linear subspaces of a fixed rank. By virtue of the Riemannian geometry of the Grassmannian, a unifying clustering framework is offered to tackle all possible clustering problems in a network: Cluster multiple states, detect communities within states, and even identify/track subnetwork state sequences. The effectiveness of the proposed approach is underlined by extensive numerical tests on synthetic and real fMRI/EEG data which demonstrate that the advocated learning method compares favorably versus several state-of-the-art clustering schemes.
Tasks	Community Detection, EEG, Time Series, Time Series Analysis
Published	2019-06-05
URL	https://arxiv.org/abs/1906.02292v1
PDF	https://arxiv.org/pdf/1906.02292v1.pdf
PWC	https://paperswithcode.com/paper/brain-network-clustering-via-kernel-arma
Repo
Framework

Towards Physically Safe Reinforcement Learning under Supervision


Title	Towards Physically Safe Reinforcement Learning under Supervision
Authors	Yinan Zhang, Devin Balkcom, Haoxiang Li
Abstract	This paper addresses the question of how a previously available control policy $\pi_s$ can be used as a supervisor to more quickly and safely train a new learned control policy $\pi_L$ for a robot. A weighted average of the supervisor and learned policies is used during trials, with a heavier weight initially on the supervisor, in order to allow safe and useful physical trials while the learned policy is still ineffective. During the process, the weight is adjusted to favor the learned policy. As weights are adjusted, the learned network must compensate so as to give safe and reasonable outputs under the different weights. A pioneer network is introduced that pre-learns a policy that performs similarly to the current learned policy under the planned next step for new weights; this pioneer network then replaces the currently learned network in the next set of trials. Experiments in OpenAI Gym demonstrate the effectiveness of the proposed method.
Tasks
Published	2019-01-19
URL	http://arxiv.org/abs/1901.06576v1
PDF	http://arxiv.org/pdf/1901.06576v1.pdf
PWC	https://paperswithcode.com/paper/towards-physically-safe-reinforcement
Repo
Framework

Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection


Title	Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection
Authors	Pan Li, Eli Chien, Olgica Milenkovic
Abstract	Landing probabilities (LP) of random walks (RW) over graphs encode rich information regarding graph topology. Generalized PageRanks (GPR), which represent weighted sums of LPs of RWs, utilize the discriminative power of LP features to enable many graph-based learning studies. Previous work in the area has mostly focused on evaluating suitable weights for GPRs, and only a few studies so far have attempted to derive the optimal weights of GRPs for a given application. We take a fundamental step forward in this direction by using random graph models to better our understanding of the behavior of GPRs. In this context, we provide a rigorous non-asymptotic analysis for the convergence of LPs and GPRs to their mean-field values on edge-independent random graphs. Although our theoretical results apply to many problem settings, we focus on the task of seed-expansion community detection over stochastic block models. There, we find that the predictive power of LPs decreases significantly slower than previously reported based on asymptotic findings. Given this result, we propose a new GPR, termed Inverse PR (IPR), with LP weights that increase for the initial few steps of the walks. Extensive experiments on both synthetic and real, large-scale networks illustrate the superiority of IPR compared to other GPRs for seeded community detection.
Tasks	Community Detection
Published	2019-05-26
URL	https://arxiv.org/abs/1905.10881v3
PDF	https://arxiv.org/pdf/1905.10881v3.pdf
PWC	https://paperswithcode.com/paper/optimizing-generalized-pagerank-methods-for
Repo
Framework

Accuracy-Memory Tradeoffs and Phase Transitions in Belief Propagation


Title	Accuracy-Memory Tradeoffs and Phase Transitions in Belief Propagation
Authors	Vishesh Jain, Frederic Koehler, Jingbo Liu, Elchanan Mossel
Abstract	The analysis of Belief Propagation and other algorithms for the {\em reconstruction problem} plays a key role in the analysis of community detection in inference on graphs, phylogenetic reconstruction in bioinformatics, and the cavity method in statistical physics. We prove a conjecture of Evans, Kenyon, Peres, and Schulman (2000) which states that any bounded memory message passing algorithm is statistically much weaker than Belief Propagation for the reconstruction problem. More formally, any recursive algorithm with bounded memory for the reconstruction problem on the trees with the binary symmetric channel has a phase transition strictly below the Belief Propagation threshold, also known as the Kesten-Stigum bound. The proof combines in novel fashion tools from recursive reconstruction, information theory, and optimal transport, and also establishes an asymptotic normality result for BP and other message-passing algorithms near the critical threshold.
Tasks	Community Detection
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10031v1
PDF	https://arxiv.org/pdf/1905.10031v1.pdf
PWC	https://paperswithcode.com/paper/accuracy-memory-tradeoffs-and-phase
Repo
Framework

Pair Matching: When bandits meet stochastic block model


Title	Pair Matching: When bandits meet stochastic block model
Authors	Christophe Giraud, Yann Issartel, Luc Lehéricy, Matthieu Lerasle
Abstract	The pair-matching problem appears in many applications where one wants to discover good matches between pairs of individuals. Formally, the set of individuals is represented by the nodes of a graph where the edges, unobserved at first, represent the good matches. The algorithm queries pairs of nodes and observes the presence/absence of edges. Its goal is to discover as many edges as possible with a fixed budget of queries. Pair-matching is a particular instance of multi-armed bandit problem in which the arms are pairs of individuals and the rewards are edges linking these pairs. This bandit problem is non-standard though, as each arm can only be played once. Given this last constraint, sublinear regret can be expected only if the graph presents some underlying structure. This paper shows that sublinear regret is achievable in the case where the graph is generated according to a Stochastic Block Model (SBM) with two communities. Optimal regret bounds are computed for this pair-matching problem. They exhibit a phase transition related to the Kesten-Stigund threshold for community detection in SBM. To avoid undesirable features of optimal solutions, the pair-matching problem is also considered in the case where each node is constrained to be sampled less than a given amount of times. We show how this constraint deteriorates optimal regret rates. The paper is concluded by a conjecture regarding the optimal regret when the number of communities is larger than $2$. Contrary to the two communities case, we believe that a statistical-computational gap would appear in this problem.
Tasks	Community Detection
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07342v1
PDF	https://arxiv.org/pdf/1905.07342v1.pdf
PWC	https://paperswithcode.com/paper/pair-matching-when-bandits-meet-stochastic
Repo
Framework

Detecting inter-sectional accuracy differences in driver drowsiness detection algorithms


Title	Detecting inter-sectional accuracy differences in driver drowsiness detection algorithms
Authors	Mkhuseli Ngxande, Jule-Raymond Tapamo, Michael Burke
Abstract	Convolutional Neural Networks (CNNs) have been used successfully across a broad range of areas including data mining, object detection, and in business. The dominance of CNNs follows a breakthrough by Alex Krizhevsky which showed improvements by dramatically reducing the error rate obtained in a general image classification task from 26.2% to 15.4%. In road safety, CNNs have been applied widely to the detection of traffic signs, obstacle detection, and lane departure checking. In addition, CNNs have been used in data mining systems that monitor driving patterns and recommend rest breaks when appropriate. This paper presents a driver drowsiness detection system and shows that there are potential social challenges regarding the application of these techniques, by highlighting problems in detecting dark-skinned driver’s faces. This is a particularly important challenge in African contexts, where there are more dark-skinned drivers. Unfortunately, publicly available datasets are often captured in different cultural contexts, and therefore do not cover all ethnicities, which can lead to false detections or racially biased models. This work evaluates the performance obtained when training convolutional neural network models on commonly used driver drowsiness detection datasets and testing on datasets specifically chosen for broader representation. Results show that models trained using publicly available datasets suffer extensively from over-fitting, and can exhibit racial bias, as shown by testing on a more representative dataset. We propose a novel visualisation technique that can assist in identifying groups of people where there might be the potential of discrimination, using Principal Component Analysis (PCA) to produce a grid of faces sorted by similarity, and combining these with a model accuracy overlay.
Tasks	Image Classification, Object Detection
Published	2019-04-23
URL	http://arxiv.org/abs/1904.12631v1
PDF	http://arxiv.org/pdf/1904.12631v1.pdf
PWC	https://paperswithcode.com/paper/190412631
Repo
Framework

Topological Analysis of Syntactic Structures


Title	Topological Analysis of Syntactic Structures
Authors	Alexander Port, Taelin Karidi, Matilde Marcolli
Abstract	We use the persistent homology method of topological data analysis and dimensional analysis techniques to study data of syntactic structures of world languages. We analyze relations between syntactic parameters in terms of dimensionality, of hierarchical clustering structures, and of non-trivial loops. We show there are relations that hold across language families and additional relations that are family-specific. We then analyze the trees describing the merging structure of persistent connected components for languages in different language families and we show that they partly correlate to historical phylogenetic trees but with significant differences. We also show the existence of interesting non-trivial persistent first homology groups in various language families. We give examples where explicit generators for the persistent first homology can be identified, some of which appear to correspond to homoplasy phenomena, while others may have an explanation in terms of historical linguistics, corresponding to known cases of syntactic borrowing across different language subfamilies.
Tasks	Topological Data Analysis
Published	2019-03-12
URL	http://arxiv.org/abs/1903.05181v1
PDF	http://arxiv.org/pdf/1903.05181v1.pdf
PWC	https://paperswithcode.com/paper/topological-analysis-of-syntactic-structures
Repo
Framework

Homograph Disambiguation Through Selective Diacritic Restoration


Title	Homograph Disambiguation Through Selective Diacritic Restoration
Authors	Sawsan Alqahtani, Hanan Aldarmaki, Mona Diab
Abstract	Lexical ambiguity, a challenging phenomenon in all natural languages, is particularly prevalent for languages with diacritics that tend to be omitted in writing, such as Arabic. Omitting diacritics leads to an increase in the number of homographs: different words with the same spelling. Diacritic restoration could theoretically help disambiguate these words, but in practice, the increase in overall sparsity leads to performance degradation in NLP applications. In this paper, we propose approaches for automatically marking a subset of words for diacritic restoration, which leads to selective homograph disambiguation. Compared to full or no diacritic restoration, these approaches yield selectively-diacritized datasets that balance sparsity and lexical disambiguation. We evaluate the various selection strategies extrinsically on several downstream applications: neural machine translation, part-of-speech tagging, and semantic textual similarity. Our experiments on Arabic show promising results, where our devised strategies on selective diacritization lead to a more balanced and consistent performance in downstream applications.
Tasks	Machine Translation, Part-Of-Speech Tagging, Semantic Textual Similarity
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04479v1
PDF	https://arxiv.org/pdf/1912.04479v1.pdf
PWC	https://paperswithcode.com/paper/homograph-disambiguation-through-selective-1
Repo
Framework

Generalized Learning with Rejection for Classification and Regression Problems


Title	Generalized Learning with Rejection for Classification and Regression Problems
Authors	Amina Asif, Fayyaz ul Amir Afsar Minhas
Abstract	Learning with rejection (LWR) allows development of machine learning systems with the ability to discard low confidence decisions generated by a prediction model. That is, just like human experts, LWR allows machine models to abstain from generating a prediction when reliability of the prediction is expected to be low. Several frameworks for this learning with rejection have been proposed in the literature. However, most of them work for classification problems only and regression with rejection has not been studied in much detail. In this work, we present a neural framework for LWR based on a generalized meta-loss function that involves simultaneous training of two neural network models: a predictor model for generating predictions and a rejecter model for deciding whether the prediction should be accepted or rejected. The proposed framework can be used for classification as well as regression and other related machine learning tasks. We have demonstrated the applicability and effectiveness of the method on synthetically generated data as well as benchmark datasets from UCI machine learning repository for both classification and regression problems. Despite being simpler in implementation, the proposed scheme for learning with rejection has shown to perform at par or better than previously proposed methods. Furthermore, we have applied the method to the problem of hurricane intensity prediction from satellite imagery. Significant improvement in performance as compared to conventional supervised methods shows the effectiveness of the proposed scheme in real-world regression problems.
Tasks
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00896v1
PDF	https://arxiv.org/pdf/1911.00896v1.pdf
PWC	https://paperswithcode.com/paper/generalized-learning-with-rejection-for
Repo
Framework

Learning higher-order sequential structure with cloned HMMs


Title	Learning higher-order sequential structure with cloned HMMs
Authors	Antoine Dedieu, Nishad Gothoskar, Scott Swingle, Wolfgang Lehrach, Miguel Lázaro-Gredilla, Dileep George
Abstract	Variable order sequence modeling is an important problem in artificial and natural intelligence. While overcomplete Hidden Markov Models (HMMs), in theory, have the capacity to represent long-term temporal structure, they often fail to learn and converge to local minima. We show that by constraining HMMs with a simple sparsity structure inspired by biology, we can make it learn variable order sequences efficiently. We call this model cloned HMM (CHMM) because the sparsity structure enforces that many hidden states map deterministically to the same emission state. CHMMs with over 1 billion parameters can be efficiently trained on GPUs without being severely affected by the credit diffusion problem of standard HMMs. Unlike n-grams and sequence memoizers, CHMMs can model temporal dependencies at arbitrarily long distances and recognize contexts with ‘holes’ in them. Compared to Recurrent Neural Networks and their Long Short-Term Memory extensions (LSTMs), CHMMs are generative models that can natively deal with uncertainty. Moreover, CHMMs return a higher-order graph that represents the temporal structure of the data which can be useful for community detection, and for building hierarchical models. Our experiments show that CHMMs can beat n-grams, sequence memoizers, and LSTMs on character-level language modeling tasks. CHMMs can be a viable alternative to these methods in some tasks that require variable order sequence modeling and the handling of uncertainty.
Tasks	Community Detection, Language Modelling
Published	2019-05-01
URL	https://arxiv.org/abs/1905.00507v4
PDF	https://arxiv.org/pdf/1905.00507v4.pdf
PWC	https://paperswithcode.com/paper/learning-higher-order-sequential-structure
Repo
Framework

Feature-Rich Part-of-speech Tagging for Morphologically Complex Languages: Application to Bulgarian


Title	Feature-Rich Part-of-speech Tagging for Morphologically Complex Languages: Application to Bulgarian
Authors	Georgi Georgiev, Valentin Zhikov, Petya Osenova, Kiril Simov, Preslav Nakov
Abstract	We present experiments with part-of-speech tagging for Bulgarian, a Slavic language with rich inflectional and derivational morphology. Unlike most previous work, which has used a small number of grammatical categories, we work with 680 morpho-syntactic tags. We combine a large morphological lexicon with prior linguistic knowledge and guided learning from a POS-annotated corpus, achieving accuracy of 97.98%, which is a significant improvement over the state-of-the-art for Bulgarian.
Tasks	Part-Of-Speech Tagging
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11503v1
PDF	https://arxiv.org/pdf/1911.11503v1.pdf
PWC	https://paperswithcode.com/paper/feature-rich-part-of-speech-tagging-for-1
Repo
Framework