January 27, 2020

3148 words 15 mins read

Paper Group ANR 1147

Realistic Channel Models Pre-training. Neural Topographic Factor Analysis for fMRI Data. Position-Aware Convolutional Networks for Traffic Prediction. Adversarial Examples with Difficult Common Words for Paraphrase Identification. HalalNet: A Deep Neural Network that Classifies the Halalness Slaughtered Chicken from their Images. Automatic detectio …

Realistic Channel Models Pre-training


Title	Realistic Channel Models Pre-training
Authors	Yourui Huangfu, Jian Wang, Chen Xu, Rong Li, Yiqun Ge, Xianbin Wang, Huazi Zhang, Jun Wang
Abstract	In this paper, we propose a neural-network-based realistic channel model with both the similar accuracy as deterministic channel models and uniformity as stochastic channel models. To facilitate this realistic channel modeling, a multi-domain channel embedding method combined with self-attention mechanism is proposed to extract channel features from multiple domains simultaneously. This ‘one model to fit them all’ solution employs available wireless channel data as the only data set for self-supervised pre-training. With the permission of users, network operators or other organizations can make use of some available user specific data to fine-tune this pre-trained realistic channel model for applications on channel-related downstream tasks. Moreover, even without fine-tuning, we show that the pre-trained realistic channel model itself is a great tool with its understanding of wireless channel.
Tasks
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09117v1
PDF	https://arxiv.org/pdf/1907.09117v1.pdf
PWC	https://paperswithcode.com/paper/realistic-channel-models-pre-training
Repo
Framework

Neural Topographic Factor Analysis for fMRI Data


Title	Neural Topographic Factor Analysis for fMRI Data
Authors	Eli Sennesh, Zulqarnain Khan, Yiyu Wang, Amirreza Farnoosh, Sarah Ostadabbas, Jennifer Dy, Ajay B. Satpute, J. Benjamin Hutchinson, Jan-Willem van de Meent
Abstract	Neuroimaging studies produce gigabytes of spatio-temporal data for a small number of participants and stimuli. Recent work increasingly suggests that the common practice of averaging across participants and stimuli leaves out systematic and meaningful information. We propose Neural Topographic Factor Analysis (NTFA), a deep generative model that parameterizes factors in terms of embeddings for participants and stimuli. We evaluate NTFA on data from an in-house pilot experiment, as well as two publicly available datasets. We demonstrate that inferring representations for participants and stimuli improves predictive generalization to unseen data when compared to existing methods. NTFA infers meaningful embeddings without supervision, circumventing the assumptions that experimenters defined stimulus conditions and that subject variance is error. We also demonstrate that the inferred latent factor representations are useful for downstream tasks such as multivoxel pattern analysis and functional connectivity.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.08901v3
PDF	https://arxiv.org/pdf/1906.08901v3.pdf
PWC	https://paperswithcode.com/paper/neural-topographic-factor-analysis-for-fmri
Repo
Framework

Position-Aware Convolutional Networks for Traffic Prediction


Title	Position-Aware Convolutional Networks for Traffic Prediction
Authors	Shiheng Ma, Jingcai Guo, Song Guo, Minyi Guo
Abstract	Forecasting the future traffic flow distribution in an area is an important issue for traffic management in an intelligent transportation system. The key challenge of traffic prediction is to capture spatial and temporal relations between future traffic flows and historical traffic due to highly dynamical patterns of human activities. Most existing methods explore such relations by fusing spatial and temporal features extracted from multi-source data. However, they neglect position information which helps distinguish patterns on different positions. In this paper, we propose a position-aware neural network that integrates data features and position information. Our approach employs the inception backbone network to capture rich features of traffic distribution on the whole area. The novelty lies in that under the backbone network, we apply position embedding technique used in neural language processing to represent position information as embedding vectors which are learned during the training. With these embedding vectors, we design position-aware convolution which allows different kernels to process features of different positions. Extensive experiments on two real-world datasets show that our approach outperforms previous methods even with fewer data sources.
Tasks	Traffic Prediction
Published	2019-04-12
URL	http://arxiv.org/abs/1904.06187v1
PDF	http://arxiv.org/pdf/1904.06187v1.pdf
PWC	https://paperswithcode.com/paper/position-aware-convolutional-networks-for
Repo
Framework

Adversarial Examples with Difficult Common Words for Paraphrase Identification


Title	Adversarial Examples with Difficult Common Words for Paraphrase Identification
Authors	Zhouxing Shi, Minlie Huang, Ting Yao, Jingfang Xu
Abstract	Deep models are commonly vulnerable to adversarial examples. In this paper, we propose the first algorithm for effectively generating both positive and negative adversarial examples for paraphrase identification. We first sample an original sentence pair from the dataset and then adversarially replace some word pairs with difficult common words. We take multiple steps and use beam search to find a modification that makes the target model fail, and thereby obtain an adversarial example. The word replacement is also constrained by heuristic rules and a language model, to preserve the label and language quality during modification. Experiments show that the performance of the target models has a severe drop on our adversarially modified examples.Meanwhile, human annotators are much less affected, and the generated sentences retain a good language quality. We also show that adversarial training with generated adversarial examples can improve model robustness, while previous work provides little improvement on our adversarial examples.
Tasks	Language Modelling, Paraphrase Identification
Published	2019-09-05
URL	https://arxiv.org/abs/1909.02560v3
PDF	https://arxiv.org/pdf/1909.02560v3.pdf
PWC	https://paperswithcode.com/paper/adversarial-examples-with-difficult-common
Repo
Framework

HalalNet: A Deep Neural Network that Classifies the Halalness Slaughtered Chicken from their Images


Title	HalalNet: A Deep Neural Network that Classifies the Halalness Slaughtered Chicken from their Images
Authors	A. Elfakharany, R. Yusof, N. Ismail, R. Arfa, M. Yunus
Abstract	Halal requirement in food is important for millions of Muslims worldwide especially for meat and chicken products, insuring that slaughter houses adhere to this requirement is a challenging task to do manually. In this paper a method is proposed that uses a camera that takes images of slaughtered chicken on the conveyor in a slaughter house, the images are then analyzed by a deep neural network to classify if the image is of a halal slaughtered chicken or not. However, traditional deep learning models require large amounts of data to train on, which in this case these amounts of data were challenging to collect especially the images of non-halal slaughtered chicken, hence this paper shows how the use of one shot learning [1] and transfer learning [2] can reach high accuracy on the few amounts of data that were available. The architecture used is based on the Siamese neural networks architecture which ranks the similarity between two inputs [3] while using the Xception network [4] as the twin networks. We call it HalalNet. This work was done as part of SYCUT (syriah compliant slaughtering system) which is a monitoring system that monitors the halalness of the slaughtered chicken in a slaughter house. The data used to train and validate HalalNet was collected from the Azain slaughtering site (Semenyih, Selangor, Malaysia) containing images of both halal and non-halal slaughtered chicken.
Tasks	One-Shot Learning, Transfer Learning
Published	2019-06-10
URL	https://arxiv.org/abs/1906.11893v1
PDF	https://arxiv.org/pdf/1906.11893v1.pdf
PWC	https://paperswithcode.com/paper/halalnet-a-deep-neural-network-that
Repo
Framework

Automatic detection of rare pathologies in fundus photographs using few-shot learning


Title	Automatic detection of rare pathologies in fundus photographs using few-shot learning
Authors	Gwenolé Quellec, Mathieu Lamard, Pierre-Henri Conze, Pascale Massin, Béatrice Cochener
Abstract	In the last decades, large datasets of fundus photographs have been collected in diabetic retinopathy (DR) screening networks. Through deep learning, these datasets were used to train automatic detectors for DR and a few other frequent pathologies, with the goal to automate screening. One challenge limits the adoption of such systems so far: automatic detectors ignore rare conditions that ophthalmologists currently detect, such as papilledema or anterior ischemic optic neuropathy. The reason is that standard deep learning requires too many examples of these conditions. However, this limitation can be addressed with few-shot learning, a machine learning paradigm where a classifier has to generalize to a new category not seen in training, given only a few examples of this category. This paper presents a new few-shot learning framework that extends convolutional neural networks (CNNs), trained for frequent conditions, with an unsupervised probabilistic model for rare condition detection. It is based on the observation that CNNs often perceive photographs containing the same anomalies as similar, even though these CNNs were trained to detect unrelated conditions. This observation was based on the t-SNE visualization tool, which we decided to incorporate in our probabilistic model. Experiments on a dataset of 164,660 screening examinations from the OPHDIAT screening network show that 37 conditions, out of 41, can be detected with an area under the ROC curve (AUC) greater than 0.8 (average AUC: 0.938). In particular, this framework significantly outperforms other frameworks for detecting rare conditions, including multitask learning, transfer learning and Siamese networks, another few-shot learning solution. We expect these richer predictions to trigger the adoption of automated eye pathology screening, which will revolutionize clinical practice in ophthalmology.
Tasks	Few-Shot Learning, One-Shot Learning, Transfer Learning
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09449v3
PDF	https://arxiv.org/pdf/1907.09449v3.pdf
PWC	https://paperswithcode.com/paper/automatic-detection-of-multiple-pathologies
Repo
Framework

Transfer Fine-Tuning: A BERT Case Study


Title	Transfer Fine-Tuning: A BERT Case Study
Authors	Yuki Arase, Junichi Tsujii
Abstract	A semantic equivalence assessment is defined as a task that assesses semantic equivalence in a sentence pair by binary judgment (i.e., paraphrase identification) or grading (i.e., semantic textual similarity measurement). It constitutes a set of tasks crucial for research on natural language understanding. Recently, BERT realized a breakthrough in sentence representation learning (Devlin et al., 2019), which is broadly transferable to various NLP tasks. While BERT’s performance improves by increasing its model size, the required computational power is an obstacle preventing practical applications from adopting the technology. Herein, we propose to inject phrasal paraphrase relations into BERT in order to generate suitable representations for semantic equivalence assessment instead of increasing the model size. Experiments on standard natural language understanding tasks confirm that our method effectively improves a smaller BERT model while maintaining the model size. The generated model exhibits superior performance compared to a larger BERT model on semantic equivalence assessment tasks. Furthermore, it achieves larger performance gains on tasks with limited training datasets for fine-tuning, which is a property desirable for transfer learning.
Tasks	Paraphrase Identification, Representation Learning, Semantic Textual Similarity, Transfer Learning
Published	2019-09-03
URL	https://arxiv.org/abs/1909.00931v1
PDF	https://arxiv.org/pdf/1909.00931v1.pdf
PWC	https://paperswithcode.com/paper/transfer-fine-tuning-a-bert-case-study
Repo
Framework

Backpropagation in the Simply Typed Lambda-calculus with Linear Negation


Title	Backpropagation in the Simply Typed Lambda-calculus with Linear Negation
Authors	Alois Brunel, Damiano Mazza, Michele Pagani
Abstract	Backpropagation is a classic automatic differentiation algorithm computing the gradient of functions specified by a certain class of simple, first-order programs, called computational graphs. It is a fundamental tool in several fields, most notably machine learning, where it is the key for efficiently training (deep) neural networks. Recent years have witnessed the quick growth of a research field called differentiable programming, the aim of which is to express computational graphs more synthetically and modularly by resorting to actual programming languages endowed with control flow operators and higher-order combinators, such as map and fold. In this paper, we extend the backpropagation algorithm to a paradigmatic example of such a programming language: we define a compositional program transformation from the simply-typed lambda-calculus to itself augmented with a notion of linear negation, and prove that this computes the gradient of the source program with the same efficiency as first-order backpropagation. The transformation is completely effect-free and thus provides a purely logical understanding of the dynamics of backpropagation.
Tasks
Published	2019-09-27
URL	https://arxiv.org/abs/1909.13768v2
PDF	https://arxiv.org/pdf/1909.13768v2.pdf
PWC	https://paperswithcode.com/paper/backpropagation-in-the-simply-typed-lambda
Repo
Framework

A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching


Title	A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching
Authors	Jihun Choi, Taeuk Kim, Sang-goo Lee
Abstract	We present a latent variable model for predicting the relationship between a pair of text sequences. Unlike previous auto-encoding–based approaches that consider each sequence separately, our proposed framework utilizes both sequences within a single model by generating a sequence that has a given relationship with a source sequence. We further extend the cross-sentence generating framework to facilitate semi-supervised training. We also define novel semantic constraints that lead the decoder network to generate semantically plausible and diverse sequences. We demonstrate the effectiveness of the proposed model from quantitative and qualitative experiments, while achieving state-of-the-art results on semi-supervised natural language inference and paraphrase identification.
Tasks	Natural Language Inference, Paraphrase Identification
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01343v1
PDF	https://arxiv.org/pdf/1906.01343v1.pdf
PWC	https://paperswithcode.com/paper/a-cross-sentence-latent-variable-model-for
Repo
Framework

A Realistic Face-to-Face Conversation System based on Deep Neural Networks


Title	A Realistic Face-to-Face Conversation System based on Deep Neural Networks
Authors	Zezhou Chen, Zhaoxiang Liu, Huan Hu, Jinqiang Bai, Shiguo Lian, Fuyuan Shi, Kai Wang
Abstract	To improve the experiences of face-to-face conversation with avatar, this paper presents a novel conversation system. It is composed of two sequence-to-sequence models respectively for listening and speaking and a Generative Adversarial Network (GAN) based realistic avatar synthesizer. The models exploit the facial action and head pose to learn natural human reactions. Based on the models’ output, the synthesizer uses the Pixel2Pixel model to generate realistic facial images. To show the improvement of our system, we use a 3D model based avatar driving scheme as a reference. We train and evaluate our neural networks with the data from ESPN shows. Experimental results show that our conversation system can generate natural facial reactions and realistic facial images.
Tasks
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07750v1
PDF	https://arxiv.org/pdf/1908.07750v1.pdf
PWC	https://paperswithcode.com/paper/190807750
Repo
Framework

Multiresolution Graph Attention Networks for Relevance Matching


Title	Multiresolution Graph Attention Networks for Relevance Matching
Authors	Ting Zhang, Bang Liu, Di Niu, Kunfeng Lai, Yu Xu
Abstract	A large number of deep learning models have been proposed for the text matching problem, which is at the core of various typical natural language processing (NLP) tasks. However, existing deep models are mainly designed for the semantic matching between a pair of short texts, such as paraphrase identification and question answering, and do not perform well on the task of relevance matching between short-long text pairs. This is partially due to the fact that the essential characteristics of short-long text matching have not been well considered in these deep models. More specifically, these methods fail to handle extreme length discrepancy between text pieces and neither can they fully characterize the underlying structural information in long text documents. In this paper, we are especially interested in relevance matching between a piece of short text and a long document, which is critical to problems like query-document matching in information retrieval and web searching. To extract the structural information of documents, an undirected graph is constructed, with each vertex representing a keyword and the weight of an edge indicating the degree of interaction between keywords. Based on the keyword graph, we further propose a Multiresolution Graph Attention Network to learn multi-layered representations of vertices through a Graph Convolutional Network (GCN), and then match the short text snippet with the graphical representation of the document with the attention mechanisms applied over each layer of the GCN. Experimental results on two datasets demonstrate that our graph approach outperforms other state-of-the-art deep matching models.
Tasks	Information Retrieval, Paraphrase Identification, Question Answering, Text Matching
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10580v1
PDF	http://arxiv.org/pdf/1902.10580v1.pdf
PWC	https://paperswithcode.com/paper/multiresolution-graph-attention-networks-for
Repo
Framework

Effective LHC measurements with matrix elements and machine learning


Title	Effective LHC measurements with matrix elements and machine learning
Authors	Johann Brehmer, Kyle Cranmer, Irina Espejo, Felix Kling, Gilles Louppe, Juan Pavez
Abstract	One major challenge for the legacy measurements at the LHC is that the likelihood function is not tractable when the collected data is high-dimensional and the detector response has to be modeled. We review how different analysis strategies solve this issue, including the traditional histogram approach used in most particle physics analyses, the Matrix Element Method, Optimal Observables, and modern techniques based on neural density estimation. We then discuss powerful new inference methods that use a combination of matrix element information and machine learning to accurately estimate the likelihood function. The MadMiner package automates all necessary data-processing steps. In first studies we find that these new techniques have the potential to substantially improve the sensitivity of the LHC legacy measurements.
Tasks	Density Estimation
Published	2019-06-04
URL	https://arxiv.org/abs/1906.01578v1
PDF	https://arxiv.org/pdf/1906.01578v1.pdf
PWC	https://paperswithcode.com/paper/effective-lhc-measurements-with-matrix
Repo
Framework

Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation


Title	Task Decomposition and Synchronization for Semantic Biomedical Image Segmentation
Authors	Xuhua Ren, Lichi Zhang, Sahar Ahmad, Dong Nie, Fan Yang, Lei Xiang, Qian Wang, Dinggang Shen
Abstract	Semantic segmentation is essentially important to biomedical image analysis. Many recent works mainly focus on integrating the Fully Convolutional Network (FCN) architecture with sophisticated convolution implementation and deep supervision. In this paper, we propose to decompose the single segmentation task into three subsequent sub-tasks, including (1) pixel-wise image segmentation, (2) prediction of the class labels of the objects within the image, and (3) classification of the scene the image belonging to. While these three sub-tasks are trained to optimize their individual loss functions of different perceptual levels, we propose to let them interact by the task-task context ensemble. Moreover, we propose a novel sync-regularization to penalize the deviation between the outputs of the pixel-wise segmentation and the class prediction tasks. These effective regularizations help FCN utilize context information comprehensively and attain accurate semantic segmentation, even though the number of the images for training may be limited in many biomedical applications. We have successfully applied our framework to three diverse 2D/3D medical image datasets, including Robotic Scene Segmentation Challenge 18 (ROBOT18), Brain Tumor Segmentation Challenge 18 (BRATS18), and Retinal Fundus Glaucoma Challenge (REFUGE18). We have achieved top-tier performance in all three challenges.
Tasks	Brain Tumor Segmentation, Scene Segmentation, Semantic Segmentation
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08720v2
PDF	https://arxiv.org/pdf/1905.08720v2.pdf
PWC	https://paperswithcode.com/paper/task-decomposition-and-synchronization-for
Repo
Framework

Adapting Stochastic Block Models to Power-Law Degree Distributions


Title	Adapting Stochastic Block Models to Power-Law Degree Distributions
Authors	Maoying Qiao, Jun Yu, Wei Bian, Qiang Li, Dacheng Tao
Abstract	Stochastic block models (SBMs) have been playing an important role in modeling clusters or community structures of network data. But, it is incapable of handling several complex features ubiquitously exhibited in real-world networks, one of which is the power-law degree characteristic. To this end, we propose a new variant of SBM, termed power-law degree SBM (PLD-SBM), by introducing degree decay variables to explicitly encode the varying degree distribution over all nodes. With an exponential prior, it is proved that PLD-SBM approximately preserves the scale-free feature in real networks. In addition, from the inference of variational E-Step, PLD-SBM is indeed to correct the bias inherited in SBM with the introduced degree decay factors. Furthermore, experiments conducted on both synthetic networks and two real-world datasets including Adolescent Health Data and the political blogs network verify the effectiveness of the proposed model in terms of cluster prediction accuracies.
Tasks
Published	2019-04-05
URL	http://arxiv.org/abs/1904.05335v1
PDF	http://arxiv.org/pdf/1904.05335v1.pdf
PWC	https://paperswithcode.com/paper/adapting-stochastic-block-models-to-power-law
Repo
Framework

Distributed Generative Adversarial Net


Title	Distributed Generative Adversarial Net
Authors	Xiaoyu Wang, Ye Deng, Jinjun Wang
Abstract	Recently the Generative Adversarial Network has become a hot topic. Considering the application of GAN in multi-user environment, we propose Distributed-GAN. It enables multiple users to train with their own data locally and generates more diverse samples. Users don’t need to share data with each other to avoid the leakage of privacy. In recent years, commercial companies have launched cloud platforms based on artificial intelligence to provide model for users who lack computing power. We hope our work can inspire these companies to provide more powerful AI services.
Tasks
Published	2019-11-19
URL	https://arxiv.org/abs/1911.08128v1
PDF	https://arxiv.org/pdf/1911.08128v1.pdf
PWC	https://paperswithcode.com/paper/distributed-generative-adversarial-net
Repo
Framework