February 1, 2020

2956 words 14 mins read

Paper Group AWR 172

Convolutional Reservoir Computing for World Models. NASS-AI: Towards Digitization of Parliamentary Bills using Document Level Embedding and Bidirectional Long Short-Term Memory. POPQORN: Quantifying Robustness of Recurrent Neural Networks. FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents. CogniVal: A Framework for Cognitive Word E …

Convolutional Reservoir Computing for World Models


Title	Convolutional Reservoir Computing for World Models
Authors	Hanten Chang, Katsuya Futagami
Abstract	Recently, reinforcement learning models have achieved great success, completing complex tasks such as mastering Go and other games with higher scores than human players. Many of these models collect considerable data on the tasks and improve accuracy by extracting visual and time-series features using convolutional neural networks (CNNs) and recurrent neural networks, respectively. However, these networks have very high computational costs because they need to be trained by repeatedly using a large volume of past playing data. In this study, we propose a novel practical approach called reinforcement learning with convolutional reservoir computing (RCRC) model. The RCRC model has several desirable features: 1. it can extract visual and time-series features very fast because it uses random fixed-weight CNN and the reservoir computing model; 2. it does not require the training data to be stored because it extracts features without training and decides action with evolution strategy. Furthermore, the model achieves state of the art score in the popular reinforcement learning task. Incredibly, we find the random weight-fixed simple networks like only one dense layer network can also reach high score in the RL task.
Tasks	Time Series
Published	2019-07-18
URL	https://arxiv.org/abs/1907.08040v1
PDF	https://arxiv.org/pdf/1907.08040v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-reservoir-computing-for-world
Repo	https://github.com/Narsil/rl-baselines
Framework	pytorch

NASS-AI: Towards Digitization of Parliamentary Bills using Document Level Embedding and Bidirectional Long Short-Term Memory


Title	NASS-AI: Towards Digitization of Parliamentary Bills using Document Level Embedding and Bidirectional Long Short-Term Memory
Authors	Adewale Akinfaderin, Olamilekan Wahab
Abstract	There has been several reports in the Nigerian and International media about the Senators and House of Representative Members of the Nigerian National Assembly (NASS) being the highest paid in the world. Despite this high-level of parliamentary compensation and a lack of oversight, most of the legislative duties like bills introduced and vote proceedings are shrouded in mystery without an open and annotated corpus. In this paper, we present results from ongoing research on the categorization of bills introduced in the Nigerian parliament since the fourth republic (1999 - 2018). For this task, we employed a multi-step approach which involves extracting text from scanned and embedded pdfs with low to medium quality using Optical Character Recognition (OCR) tools and labeling them into eight categories. We investigate the performance of document level embedding for feature representation of the extracted texts before using a Bidirectional Long Short-Term Memory (Bi-LSTM) for our classifier. The performance was further compared with other feature representation and machine learning techniques. We believe that these results are well-positioned to have a substantial impact on the quest to meet the basic open data charter principles.
Tasks	Optical Character Recognition
Published	2019-10-02
URL	https://arxiv.org/abs/1910.04865v1
PDF	https://arxiv.org/pdf/1910.04865v1.pdf
PWC	https://paperswithcode.com/paper/nass-ai-towards-digitization-of-parliamentary
Repo	https://github.com/Olamyy/nass-ai
Framework	none

POPQORN: Quantifying Robustness of Recurrent Neural Networks


Title	POPQORN: Quantifying Robustness of Recurrent Neural Networks
Authors	Ching-Yun Ko, Zhaoyang Lyu, Tsui-Wei Weng, Luca Daniel, Ngai Wong, Dahua Lin
Abstract	The vulnerability to adversarial attacks has been a critical issue for deep neural networks. Addressing this issue requires a reliable way to evaluate the robustness of a network. Recently, several methods have been developed to compute $\textit{robustness quantification}$ for neural networks, namely, certified lower bounds of the minimum adversarial perturbation. Such methods, however, were devised for feed-forward networks, e.g. multi-layer perceptron or convolutional networks. It remains an open problem to quantify robustness for recurrent networks, especially LSTM and GRU. For such networks, there exist additional challenges in computing the robustness quantification, such as handling the inputs at multiple steps and the interaction between gates and states. In this work, we propose $\textit{POPQORN}$ ($\textbf{P}$ropagated-$\textbf{o}$ut$\textbf{p}$ut $\textbf{Q}$uantified R$\textbf{o}$bustness for $\textbf{RN}$Ns), a general algorithm to quantify robustness of RNNs, including vanilla RNNs, LSTMs, and GRUs. We demonstrate its effectiveness on different network architectures and show that the robustness quantification on individual steps can lead to new insights.
Tasks
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07387v1
PDF	https://arxiv.org/pdf/1905.07387v1.pdf
PWC	https://paperswithcode.com/paper/popqorn-quantifying-robustness-of-recurrent
Repo	https://github.com/ZhaoyangLyu/POPQORN
Framework	pytorch

FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents


Title	FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents
Authors	Guillaume Jaume, Hazim Kemal Ekenel, Jean-Philippe Thiran
Abstract	We present a new dataset for form understanding in noisy scanned documents (FUNSD) that aims at extracting and structuring the textual content of forms. The dataset comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging task. The proposed dataset can be used for various tasks, including text detection, optical character recognition, spatial layout analysis, and entity labeling/linking. To the best of our knowledge, this is the first publicly available dataset with comprehensive annotations to address FoUn task. We also present a set of baselines and introduce metrics to evaluate performance on the FUNSD dataset, which can be downloaded at https://guillaumejaume.github.io/FUNSD/.
Tasks	Optical Character Recognition
Published	2019-05-27
URL	https://arxiv.org/abs/1905.13538v2
PDF	https://arxiv.org/pdf/1905.13538v2.pdf
PWC	https://paperswithcode.com/paper/190513538
Repo	https://github.com/SPAIC-OCR/OCR-at-the-edge
Framework	none

CogniVal: A Framework for Cognitive Word Embedding Evaluation


Title	CogniVal: A Framework for Cognitive Word Embedding Evaluation
Authors	Nora Hollenstein, Antonio de la Torre, Nicolas Langer, Ce Zhang
Abstract	An interesting method of evaluating word representations is by how much they reflect the semantic representations in the human brain. However, most, if not all, previous works only focus on small datasets and a single modality. In this paper, we present the first multi-modal framework for evaluating English word representations based on cognitive lexical semantics. Six types of word embeddings are evaluated by fitting them to 15 datasets of eye-tracking, EEG and fMRI signals recorded during language processing. To achieve a global score over all evaluation hypotheses, we apply statistical significance testing accounting for the multiple comparisons problem. This framework is easily extensible and available to include other intrinsic and extrinsic evaluation methods. We find strong correlations in the results between cognitive datasets, across recording modalities and to their performance on extrinsic NLP tasks.
Tasks	EEG, Eye Tracking, Word Embeddings
Published	2019-09-19
URL	https://arxiv.org/abs/1909.09001v2
PDF	https://arxiv.org/pdf/1909.09001v2.pdf
PWC	https://paperswithcode.com/paper/cognival-a-framework-for-cognitive-word
Repo	https://github.com/DS3Lab/cognival
Framework	none

A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition


Title	A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition
Authors	Min Peng, Chongyang Wang, Tao Bi, Tong Chen, XiangDong Zhou, Yu shi
Abstract	The automatic recognition of micro-expression has been boosted ever since the successful introduction of deep learning approaches. As researchers working on such topics are moving to learn from the nature of micro-expression, the practice of using deep learning techniques has evolved from processing the entire video clip of micro-expression to the recognition on apex frame. Using the apex frame is able to get rid of redundant video frames, but the relevant temporal evidence of micro-expression would be thereby left out. This paper proposes a novel Apex-Time Network (ATNet) to recognize micro-expression based on spatial information from the apex frame as well as on temporal information from the respective-adjacent frames. Through extensive experiments on three benchmarks, we demonstrate the improvement achieved by learning such temporal information. Specially, the model with such temporal information is more robust in cross-dataset validations.
Tasks
Published	2019-04-07
URL	https://arxiv.org/abs/1904.03699v7
PDF	https://arxiv.org/pdf/1904.03699v7.pdf
PWC	https://paperswithcode.com/paper/a-novel-apex-time-network-for-cross-dataset
Repo	https://github.com/CodeShareBot/ACII19-Apex-Time-Network
Framework	caffe2

Detecting semantic anomalies


Title	Detecting semantic anomalies
Authors	Faruk Ahmed, Aaron Courville
Abstract	We critically appraise the recent interest in out-of-distribution (OOD) detection and question the practical relevance of existing benchmarks. While the currently prevalent trend is to consider different datasets as OOD, we argue that out-distributions of practical interest are ones where the distinction is semantic in nature for a specified context, and that evaluative tasks should reflect this more closely. Assuming a context of object recognition, we recommend a set of benchmarks, motivated by practical applications. We make progress on these benchmarks by exploring a multi-task learning based approach, showing that auxiliary objectives for improved semantic awareness result in improved semantic anomaly detection, with accompanying generalization benefits.
Tasks	Anomaly Detection, Multi-Task Learning, Object Recognition
Published	2019-08-13
URL	https://arxiv.org/abs/1908.04388v3
PDF	https://arxiv.org/pdf/1908.04388v3.pdf
PWC	https://paperswithcode.com/paper/detecting-semantic-anomalies
Repo	https://github.com/Faruk-Ahmed/detecting_semantic_anomalies
Framework	tf

The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes


Title	The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes
Authors	Nicholas Heller, Niranjan Sathianathen, Arveen Kalapara, Edward Walczak, Keenan Moore, Heather Kaluzniak, Joel Rosenberg, Paul Blake, Zachary Rengel, Makinna Oestreich, Joshua Dean, Michael Tradewell, Aneri Shah, Resha Tejpaul, Zachary Edgerton, Matthew Peterson, Shaneabbas Raza, Subodh Regmi, Nikolaos Papanikolopoulos, Christopher Weight
Abstract	The morphometry of a kidney tumor revealed by contrast-enhanced Computed Tomography (CT) imaging is an important factor in clinical decision making surrounding the lesion’s diagnosis and treatment. Quantitative study of the relationship between kidney tumor morphology and clinical outcomes is difficult due to data scarcity and the laborious nature of manually quantifying imaging predictors. Automatic semantic segmentation of kidneys and kidney tumors is a promising tool towards automatically quantifying a wide array of morphometric features, but no sizeable annotated dataset is currently available to train models for this task. We present the KiTS19 challenge dataset: A collection of multi-phase CT imaging, segmentation masks, and comprehensive clinical outcomes for 300 patients who underwent nephrectomy for kidney tumors at our center between 2010 and 2018. 210 (70%) of these patients were selected at random as the training set for the 2019 MICCAI KiTS Kidney Tumor Segmentation Challenge and have been released publicly. With the presence of clinical context and surgical outcomes, this data can serve not only for benchmarking semantic segmentation models, but also for developing and studying biomarkers which make use of the imaging and semantic segmentation masks.
Tasks	Computed Tomography (CT), Decision Making, Semantic Segmentation
Published	2019-03-31
URL	https://arxiv.org/abs/1904.00445v2
PDF	https://arxiv.org/pdf/1904.00445v2.pdf
PWC	https://paperswithcode.com/paper/the-kits19-challenge-data-300-kidney-tumor
Repo	https://github.com/fevalero19/Proyecto-imagenes
Framework	none

Stagewise Knowledge Distillation


Title	Stagewise Knowledge Distillation
Authors	Akshay Kulkarni, Navid Panchi, Shital Chiddarwar
Abstract	The deployment of modern Deep Learning models requires high computational power. However, many applications are targeted for embedded devices like smartphones and wearables which lack such computational abilities. This necessitates compact networks that reduce computations while preserving the performance. Knowledge Distillation is one of the methods used to achieve this. Traditional Knowledge Distillation methods transfer knowledge from teacher to student in a single stage. We propose progressive stagewise training to improve the transfer of knowledge. We also show that this method works even with a fraction of the data used for training the teacher model, without compromising on the metric. This method can complement other model compression methods and also can be viewed as a generalized model compression technique.
Tasks	Model Compression
Published	2019-11-15
URL	https://arxiv.org/abs/1911.06786v2
PDF	https://arxiv.org/pdf/1911.06786v2.pdf
PWC	https://paperswithcode.com/paper/stagewise-knowledge-distillation
Repo	https://github.com/akshaykvnit/stagewise-knowledge-distillation
Framework	pytorch

Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment


Title	Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
Authors	Di Jin, Zhijing Jin, Joey Tianyi Zhou, Peter Szolovits
Abstract	Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate natural adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate the advantages of this framework in three ways: (1) effective—it outperforms state-of-the-art attacks in terms of success rate and perturbation rate, (2) utility-preserving—it preserves semantic content and grammaticality, and remains correctly classified by humans, and (3) efficient—it generates adversarial text with computational complexity linear to the text length. *The code, pre-trained target models, and test examples are available at https://github.com/jind11/TextFooler.
Tasks	Adversarial Text, Natural Language Inference, Text Classification
Published	2019-07-27
URL	https://arxiv.org/abs/1907.11932v4
PDF	https://arxiv.org/pdf/1907.11932v4.pdf
PWC	https://paperswithcode.com/paper/is-bert-really-robust-natural-language-attack
Repo	https://github.com/jind11/TextFooler
Framework	pytorch

Deep Learning Predictive Band Switching in Wireless Networks


Title	Deep Learning Predictive Band Switching in Wireless Networks
Authors	Faris B. Mismar, Ahmad AlAmmouri, Ahmed Alkhateeb, Jeffrey G. Andrews, Brian L. Evans
Abstract	In cellular systems, the user equipment (UE) can request a change in the frequency band when its rate drops below a threshold on the current band. The UE is then instructed by the base station (BS) to measure the quality of candidate bands, which requires a measurement gap in the data transmission, thus lowering the data rate. We propose a band switching approach based on machine learning that does not require any measurement gap. Our proposed classifier-based band switching policy instead exploits spatial and spectral correlation between radio frequency signals in different bands based on knowledge of the UE location. We focus on switching between a lower (e.g. 3.5 GHz) band and a millimeter wave band (e.g. 28 GHz), and design and evaluate two classification models that are trained on a ray-tracing dataset. A key insight is that measurement gaps are overkill, in that only the relative order of the bands is necessary for band selection, rather than a full channel estimate. Our proposed machine learning-based policies achieve roughly 30% improvement in mean effective rates over those of the industry standard policy, while achieving misclassification errors well below 0.5%.
Tasks
Published	2019-10-02
URL	https://arxiv.org/abs/1910.05305v1
PDF	https://arxiv.org/pdf/1910.05305v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-predictive-band-switching-in
Repo	https://github.com/farismismar/Bandswitch-DeepMIMO
Framework	tf

MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction


Title	MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction
Authors	Oscar Araque, Lorenzo Gatti, Kyriaki Kalimeri
Abstract	Moral rhetoric plays a fundamental role in how we perceive and interpret the information we receive, greatly influencing our decision-making process. Especially when it comes to controversial social and political issues, our opinions and attitudes are hardly ever based on evidence alone. The Moral Foundations Dictionary (MFD) was developed to operationalize moral values in the text. In this study, we present MoralStrength, a lexicon of approximately 1,000 lemmas, obtained as an extension of the Moral Foundations Dictionary, based on WordNet synsets. Moreover, for each lemma it provides with a crowdsourced numeric assessment of Moral Valence, indicating the strength with which a lemma is expressing the specific value. We evaluated the predictive potentials of this moral lexicon, defining three utilization approaches of increased complexity, ranging from lemmas’ statistical properties to a deep learning approach of word embeddings based on semantic similarity. Logistic regression models trained on the features extracted from MoralStrength, significantly outperformed the current state-of-the-art, reaching an F1-score of 87.6% over the previous 62.4% (p-value<0.01), and an average F1-Score of 86.25% over six different datasets. Such findings pave the way for further research, allowing for an in-depth understanding of moral narratives in text for a wide range of social issues.
Tasks	Decision Making, Semantic Similarity, Semantic Textual Similarity, Word Embeddings
Published	2019-04-17
URL	https://arxiv.org/abs/1904.08314v2
PDF	https://arxiv.org/pdf/1904.08314v2.pdf
PWC	https://paperswithcode.com/paper/moralstrength-exploiting-a-moral-lexicon-and
Repo	https://github.com/oaraque/moral-foundations
Framework	none

Sensitivity of Deep Convolutional Networks to Gabor Noise


Title	Sensitivity of Deep Convolutional Networks to Gabor Noise
Authors	Kenneth T. Co, Luis Muñoz-González, Emil C. Lupu
Abstract	Deep Convolutional Networks (DCNs) have been shown to be sensitive to Universal Adversarial Perturbations (UAPs): input-agnostic perturbations that fool a model on large portions of a dataset. These UAPs exhibit interesting visual patterns, but this phenomena is, as yet, poorly understood. Our work shows that visually similar procedural noise patterns also act as UAPs. In particular, we demonstrate that different DCN architectures are sensitive to Gabor noise patterns. This behaviour, its causes, and implications deserve further in-depth study.
Tasks
Published	2019-06-08
URL	https://arxiv.org/abs/1906.03455v2
PDF	https://arxiv.org/pdf/1906.03455v2.pdf
PWC	https://paperswithcode.com/paper/sensitivity-of-deep-convolutional-networks-to
Repo	https://github.com/kenny-co/procedural-advml
Framework	tf

Mask Based Unsupervised Content Transfer


Title	Mask Based Unsupervised Content Transfer
Authors	Ron Mokady, Sagie Benaim, Lior Wolf, Amit Bermano
Abstract	We consider the problem of translating, in an unsupervised manner, between two domains where one contains some additional information compared to the other. The proposed method disentangles the common and separate parts of these domains and, through the generation of a mask, focuses the attention of the underlying network to the desired augmentation alone, without wastefully reconstructing the entire target. This enables state-of-the-art quality and variety of content translation, as demonstrated through extensive quantitative and qualitative evaluation. Our method is also capable of adding the separate content of different guide images and domains as well as remove existing separate content. Furthermore, our method enables weakly-supervised semantic segmentation of the separate part of each domain, where only class labels are provided. Our code is publicly available at https://github.com/rmokady/mbu-content-tansfer.
Tasks	Semantic Segmentation, Weakly-Supervised Semantic Segmentation
Published	2019-06-15
URL	https://arxiv.org/abs/1906.06558v2
PDF	https://arxiv.org/pdf/1906.06558v2.pdf
PWC	https://paperswithcode.com/paper/mask-based-unsupervised-content-transfer
Repo	https://github.com/rmokady/mbu-content-tansfer
Framework	pytorch

Is Feature Diversity Necessary in Neural Network Initialization?


Title	Is Feature Diversity Necessary in Neural Network Initialization?
Authors	Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Abstract	Standard practice in training neural networks involves initializing the weights in an independent fashion. The results of recent work suggest that feature “diversity” at initialization plays an important role in training the network. However, other initialization schemes with reduced feature diversity have also been shown to be viable. In this work, we conduct a series of experiments aimed at elucidating the importance of feature diversity at initialization. We show that a complete lack of diversity is harmful to training, but its effects can be counteracted by a relatively small addition of noise - even the noise in standard non-deterministic GPU computations is sufficient. Furthermore, we construct a deep convolutional network with identical features at initialization and almost all of the weights initialized at 0 that can be trained to reach accuracy matching its standard-initialized counterpart.
Tasks
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05137v2
PDF	https://arxiv.org/pdf/1912.05137v2.pdf
PWC	https://paperswithcode.com/paper/is-feature-diversity-necessary-in-neural
Repo	https://github.com/yanivbl6/fixup
Framework	pytorch