October 21, 2019

3316 words 16 mins read

Paper Group AWR 41

SWRL2SPIN: A tool for transforming SWRL rule bases in OWL ontologies to object-oriented SPIN rules. Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension. A Fast Proximal Point Method for Computing Exact Wasserstein Distance. Graph Convolutional Networks for Text Classification. Efficient Neural Netw …

SWRL2SPIN: A tool for transforming SWRL rule bases in OWL ontologies to object-oriented SPIN rules


Title	SWRL2SPIN: A tool for transforming SWRL rule bases in OWL ontologies to object-oriented SPIN rules
Authors	Nick Bassiliades
Abstract	Semantic Web Rule Language (SWRL) combines OWL (Web Ontology Language) ontologies with Horn Logic rules of the Rule Markup Language (RuleML) family. Being supported by ontology editors, rule engines and ontology reasoners, it has become a very popular choice for developing rule-based applications on top of ontologies. However, SWRL is probably not go-ing to become a WWW Consortium standard, prohibiting industrial acceptance. On the other hand, SPIN (SPARQL Inferencing Notation) has become a de-facto industry standard to rep-resent SPARQL rules and constraints on Semantic Web models, building on the widespread acceptance of SPARQL (SPARQL Protocol and RDF Query Language). In this paper, we ar-gue that the life of existing SWRL rule-based ontology applications can be prolonged by con-verting them to SPIN. To this end, we have developed the SWRL2SPIN tool in Prolog that transforms SWRL rules into SPIN rules, considering the object-orientation of SPIN, i.e. linking rules to the appropriate ontology classes and optimizing them, as derived by analysing the rule conditions.
Tasks
Published	2018-01-27
URL	http://arxiv.org/abs/1801.09061v3
PDF	http://arxiv.org/pdf/1801.09061v3.pdf
PWC	https://paperswithcode.com/paper/swrl2spin-a-tool-for-transforming-swrl-rule
Repo	https://github.com/nbassili/SWRL2SPIN
Framework	none

Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension


Title	Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension
Authors	Chia-Hsuan Li, Szu-Lin Wu, Chi-Liang Liu, Hung-yi Lee
Abstract	Reading comprehension has been widely studied. One of the most representative reading comprehension tasks is Stanford Question Answering Dataset (SQuAD), on which machine is already comparable with human. On the other hand, accessing large collections of multimedia or spoken content is much more difficult and time-consuming than plain text content for humans. It’s therefore highly attractive to develop machines which can automatically understand spoken content. In this paper, we propose a new listening comprehension task - Spoken SQuAD. On the new task, we found that speech recognition errors have catastrophic impact on machine comprehension, and several approaches are proposed to mitigate the impact.
Tasks	Question Answering, Reading Comprehension, Speech Recognition
Published	2018-04-01
URL	http://arxiv.org/abs/1804.00320v1
PDF	http://arxiv.org/pdf/1804.00320v1.pdf
PWC	https://paperswithcode.com/paper/spoken-squad-a-study-of-mitigating-the-impact
Repo	https://github.com/chiahsuan156/Spoken-SQuAD
Framework	none

A Fast Proximal Point Method for Computing Exact Wasserstein Distance


Title	A Fast Proximal Point Method for Computing Exact Wasserstein Distance
Authors	Yujia Xie, Xiangfeng Wang, Ruijia Wang, Hongyuan Zha
Abstract	Wasserstein distance plays increasingly important roles in machine learning, stochastic programming and image processing. Major efforts have been under way to address its high computational complexity, some leading to approximate or regularized variations such as Sinkhorn distance. However, as we will demonstrate, regularized variations with large regularization parameter will degradate the performance in several important machine learning applications, and small regularization parameter will fail due to numerical stability issues with existing algorithms. We address this challenge by developing an Inexact Proximal point method for exact Optimal Transport problem (IPOT) with the proximal operator approximately evaluated at each iteration using projections to the probability simplex. The algorithm (a) converges to exact Wasserstein distance with theoretical guarantee and robust regularization parameter selection, (b) alleviates numerical stability issue, (c) has similar computational complexity to Sinkhorn, and (d) avoids the shrinking problem when apply to generative models. Furthermore, a new algorithm is proposed based on IPOT to obtain sharper Wasserstein barycenter.
Tasks
Published	2018-02-12
URL	https://arxiv.org/abs/1802.04307v3
PDF	https://arxiv.org/pdf/1802.04307v3.pdf
PWC	https://paperswithcode.com/paper/a-fast-proximal-point-method-for-computing
Repo	https://github.com/xieyujia/IPOT
Framework	tf

Graph Convolutional Networks for Text Classification


Title	Graph Convolutional Networks for Text Classification
Authors	Liang Yao, Chengsheng Mao, Yuan Luo
Abstract	Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.
Tasks	Sentiment Analysis, Text Classification
Published	2018-09-15
URL	http://arxiv.org/abs/1809.05679v3
PDF	http://arxiv.org/pdf/1809.05679v3.pdf
PWC	https://paperswithcode.com/paper/graph-convolutional-networks-for-text
Repo	https://github.com/plkmo/Bible_Text_GCN
Framework	pytorch

Efficient Neural Network Robustness Certification with General Activation Functions


Title	Efficient Neural Network Robustness Certification with General Activation Functions
Authors	Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel
Abstract	Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem. Nevertheless, recently it has been shown to be possible to give a non-trivial certified lower bound of minimum adversarial distortion, and some recent progress has been made towards this direction by exploiting the piece-wise linear nature of ReLU activations. However, a generic robustness certification for general activation functions still remains largely unexplored. To address this issue, in this paper we introduce CROWN, a general framework to certify robustness of neural networks with general activation functions for given input data points. The novelty in our algorithm consists of bounding a given activation function with linear and quadratic functions, hence allowing it to tackle general activation functions including but not limited to four popular choices: ReLU, tanh, sigmoid and arctan. In addition, we facilitate the search for a tighter certified lower bound by adaptively selecting appropriate surrogates for each neuron activation. Experimental results show that CROWN on ReLU networks can notably improve the certified lower bounds compared to the current state-of-the-art algorithm Fast-Lin, while having comparable computational efficiency. Furthermore, CROWN also demonstrates its effectiveness and flexibility on networks with general activation functions, including tanh, sigmoid and arctan.
Tasks
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00866v1
PDF	http://arxiv.org/pdf/1811.00866v1.pdf
PWC	https://paperswithcode.com/paper/efficient-neural-network-robustness
Repo	https://github.com/huanzhang12/RecurJac-and-CROWN
Framework	tf

Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization


Title	Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization
Authors	Siyuan Qiao, Zhe Lin, Jianming Zhang, Alan Yuille
Abstract	In this paper, we study the problem of improving computational resource utilization of neural networks. Deep neural networks are usually over-parameterized for their tasks in order to achieve good performances, thus are likely to have underutilized computational resources. This observation motivates a lot of research topics, e.g. network pruning, architecture search, etc. As models with higher computational costs (e.g. more parameters or more computations) usually have better performances, we study the problem of improving the resource utilization of neural networks so that their potentials can be further realized. To this end, we propose a novel optimization method named Neural Rejuvenation. As its name suggests, our method detects dead neurons and computes resource utilization in real time, rejuvenates dead neurons by resource reallocation and reinitialization, and trains them with new training schemes. By simply replacing standard optimizers with Neural Rejuvenation, we are able to improve the performances of neural networks by a very large margin while using similar training efforts and maintaining their original resource usages.
Tasks	Network Pruning, Neural Architecture Search
Published	2018-12-02
URL	http://arxiv.org/abs/1812.00481v1
PDF	http://arxiv.org/pdf/1812.00481v1.pdf
PWC	https://paperswithcode.com/paper/neural-rejuvenation-improving-deep-network
Repo	https://github.com/joe-siyuan-qiao/NeuralRejuvenation-CVPR19
Framework	pytorch

On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition


Title	On the End-to-End Solution to Mandarin-English Code-switching Speech Recognition
Authors	Zhiping Zeng, Yerbolat Khassanov, Van Tung Pham, Haihua Xu, Eng Siong Chng, Haizhou Li
Abstract	Code-switching (CS) refers to a linguistic phenomenon where a speaker uses different languages in an utterance or between alternating utterances. In this work, we study end-to-end (E2E) approaches to the Mandarin-English code-switching speech recognition (CSSR) task. We first examine the effectiveness of using data augmentation and byte-pair encoding (BPE) subword units. More importantly, we propose a multitask learning recipe, where a language identification task is explicitly learned in addition to the E2E speech recognition task. Furthermore, we introduce an efficient word vocabulary expansion method for language modeling to alleviate data sparsity issues under the code-switching scenario. Experimental results on the SEAME data, a Mandarin-English CS corpus, demonstrate the effectiveness of the proposed methods.
Tasks	Data Augmentation, Language Identification, Language Modelling, Speech Recognition
Published	2018-11-01
URL	https://arxiv.org/abs/1811.00241v2
PDF	https://arxiv.org/pdf/1811.00241v2.pdf
PWC	https://paperswithcode.com/paper/on-the-end-to-end-solution-to-mandarin
Repo	https://github.com/zengzp0912/SEAME-dev-set
Framework	none

Nose, eyes and ears: Head pose estimation by locating facial keypoints


Title	Nose, eyes and ears: Head pose estimation by locating facial keypoints
Authors	Aryaman Gupta, Kalpit Thakkar, Vineet Gandhi, P J Narayanan
Abstract	Monocular head pose estimation requires learning a model that computes the intrinsic Euler angles for pose (yaw, pitch, roll) from an input image of human face. Annotating ground truth head pose angles for images in the wild is difficult and requires ad-hoc fitting procedures (which provides only coarse and approximate annotations). This highlights the need for approaches which can train on data captured in controlled environment and generalize on the images in the wild (with varying appearance and illumination of the face). Most present day deep learning approaches which learn a regression function directly on the input images fail to do so. To this end, we propose to use a higher level representation to regress the head pose while using deep learning architectures. More specifically, we use the uncertainty maps in the form of 2D soft localization heatmap images over five facial keypoints, namely left ear, right ear, left eye, right eye and nose, and pass them through an convolutional neural network to regress the head-pose. We show head pose estimation results on two challenging benchmarks BIWI and AFLW and our approach surpasses the state of the art on both the datasets.
Tasks	Head Pose Estimation, Pose Estimation
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00739v1
PDF	http://arxiv.org/pdf/1812.00739v1.pdf
PWC	https://paperswithcode.com/paper/nose-eyes-and-ears-head-pose-estimation-by
Repo	https://github.com/manoj901/HeadposeEstimation
Framework	pytorch

Learning Rich Features for Image Manipulation Detection


Title	Learning Rich Features for Image Manipulation Detection
Authors	Peng Zhou, Xintong Han, Vlad I. Morariu, Larry S. Davis
Abstract	Image manipulation detection is different from traditional semantic object detection because it pays more attention to tampering artifacts than to image content, which suggests that richer features need to be learned. We propose a two-stream Faster R-CNN network and train it endto- end to detect the tampered regions given a manipulated image. One of the two streams is an RGB stream whose purpose is to extract features from the RGB image input to find tampering artifacts like strong contrast difference, unnatural tampered boundaries, and so on. The other is a noise stream that leverages the noise features extracted from a steganalysis rich model filter layer to discover the noise inconsistency between authentic and tampered regions. We then fuse features from the two streams through a bilinear pooling layer to further incorporate spatial co-occurrence of these two modalities. Experiments on four standard image manipulation datasets demonstrate that our two-stream framework outperforms each individual stream, and also achieves state-of-the-art performance compared to alternative methods with robustness to resizing and compression.
Tasks	Image Manipulation Detection, Object Detection
Published	2018-05-13
URL	http://arxiv.org/abs/1805.04953v1
PDF	http://arxiv.org/pdf/1805.04953v1.pdf
PWC	https://paperswithcode.com/paper/learning-rich-features-for-image-manipulation
Repo	https://github.com/pengzhou1108/RGB-N
Framework	tf

Neural Abstractive Text Summarization with Sequence-to-Sequence Models: A Survey


Title	Neural Abstractive Text Summarization with Sequence-to-Sequence Models: A Survey
Authors	Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, Chandan K. Reddy
Abstract	In the past few years, neural abstractive text summarization with sequence-to-sequence (seq2seq) models have gained a lot of popularity. Many interesting techniques have been proposed to improve the seq2seq models, making them capable of handling different challenges, such as saliency, fluency and human readability, and generate high-quality summaries. Generally speaking, most of these techniques differ in one of these three categories: network structure, parameter inference, and decoding/generation. There are also other concerns, such as efficiency and parallelism for training a model. In this paper, we provide a comprehensive literature and technical survey on different seq2seq models for abstractive text summarization from viewpoint of network structures, training strategies, and summary generation algorithms. Many models were first proposed for language modeling and generation tasks, such as machine translation, and later applied to abstractive text summarization. Therefore, we also provide a brief review of these models. As part of this survey, we also develop an open source library, namely Neural Abstractive Text Summarizer (NATS) toolkit, for the abstractive text summarization. An extensive set of experiments have been conducted on the widely used CNN/Daily Mail dataset to examine the effectiveness of several different neural network components. Finally, we benchmark two models implemented in NATS on two recently released datasets, i.e., Newsroom and Bytecup.
Tasks	Abstractive Text Summarization, Language Modelling, Machine Translation, Text Summarization
Published	2018-12-05
URL	https://arxiv.org/abs/1812.02303v3
PDF	https://arxiv.org/pdf/1812.02303v3.pdf
PWC	https://paperswithcode.com/paper/neural-abstractive-text-summarization-with
Repo	https://github.com/sf-18/politicalsynthesis
Framework	tf

Discourse Coherence in the Wild: A Dataset, Evaluation and Methods


Title	Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
Authors	Alice Lai, Joel Tetreault
Abstract	To date there has been very little work on assessing discourse coherence methods on real-world data. To address this, we present a new corpus of real-world texts (GCDC) as well as the first large-scale evaluation of leading discourse coherence algorithms. We show that neural models, including two that we introduce here (SentAvg and ParSeq), tend to perform best. We analyze these performance differences and discuss patterns we observed in low coherence texts in four domains.
Tasks
Published	2018-05-14
URL	http://arxiv.org/abs/1805.04993v1
PDF	http://arxiv.org/pdf/1805.04993v1.pdf
PWC	https://paperswithcode.com/paper/discourse-coherence-in-the-wild-a-dataset
Repo	https://github.com/aylai/GCDC-corpus
Framework	none

Deep Neural Networks for Bot Detection


Title	Deep Neural Networks for Bot Detection
Authors	Sneha Kudugunta, Emilio Ferrara
Abstract	The problem of detecting bots, automated social media accounts governed by software but disguising as human users, has strong implications. For example, bots have been used to sway political elections by distorting online discourse, to manipulate the stock market, or to push anti-vaccine conspiracy theories that caused health epidemics. Most techniques proposed to date detect bots at the account level, by processing large amount of social media posts, and leveraging information from network structure, temporal dynamics, sentiment analysis, etc. In this paper, we propose a deep neural network based on contextual long short-term memory (LSTM) architecture that exploits both content and metadata to detect bots at the tweet level: contextual features are extracted from user metadata and fed as auxiliary input to LSTM deep nets processing the tweet text. Another contribution that we make is proposing a technique based on synthetic minority oversampling to generate a large labeled dataset, suitable for deep nets training, from a minimal amount of labeled data (roughly 3,000 examples of sophisticated Twitter bots). We demonstrate that, from just one single tweet, our architecture can achieve high classification accuracy (AUC > 96%) in separating bots from humans. We apply the same architecture to account-level bot detection, achieving nearly perfect classification accuracy (AUC > 99%). Our system outperforms previous state of the art while leveraging a small and interpretable set of features yet requiring minimal training data.
Tasks	Sentiment Analysis
Published	2018-02-12
URL	http://arxiv.org/abs/1802.04289v2
PDF	http://arxiv.org/pdf/1802.04289v2.pdf
PWC	https://paperswithcode.com/paper/deep-neural-networks-for-bot-detection
Repo	https://github.com/ji10bhatt/Capstone-Social-Media-Bot-Detection
Framework	none

Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics


Title	Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics
Authors	Yunbo Wang, Jianjin Zhang, Hongyu Zhu, Mingsheng Long, Jianmin Wang, Philip S Yu
Abstract	Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the accumulation, deformation or dissipation of radar echoes in precipitation forecasting. From Cramer’s Decomposition, any non-stationary process can be decomposed into deterministic, time-variant polynomials, plus a zero-mean stochastic term. By applying differencing operations appropriately, we may turn time-variant polynomials into a constant, making the deterministic component predictable. However, most previous recurrent neural networks for spatiotemporal prediction do not use the differential signals effectively, and their relatively simple state transition functions prevent them from learning too complicated variations in spacetime. We propose the Memory In Memory (MIM) networks and corresponding recurrent blocks for this purpose. The MIM blocks exploit the differential signals between adjacent recurrent states to model the non-stationary and approximately stationary properties in spatiotemporal dynamics with two cascaded, self-renewed memory modules. By stacking multiple MIM blocks, we could potentially handle higher-order non-stationarity. The MIM networks achieve the state-of-the-art results on four spatiotemporal prediction tasks across both synthetic and real-world datasets. We believe that the general idea of this work can be potentially applied to other time-series forecasting tasks.
Tasks	Time Series, Time Series Forecasting, Video Prediction
Published	2018-11-19
URL	http://arxiv.org/abs/1811.07490v3
PDF	http://arxiv.org/pdf/1811.07490v3.pdf
PWC	https://paperswithcode.com/paper/memory-in-memory-a-predictive-neural-network
Repo	https://github.com/Yunbo426/MIM
Framework	tf

Multi-Level Network Embedding with Boosted Low-Rank Matrix Approximation


Title	Multi-Level Network Embedding with Boosted Low-Rank Matrix Approximation
Authors	Jundong Li, Liang Wu, Huan Liu
Abstract	As opposed to manual feature engineering which is tedious and difficult to scale, network representation learning has attracted a surge of research interests as it automates the process of feature learning on graphs. The learned low-dimensional node vector representation is generalizable and eases the knowledge discovery process on graphs by enabling various off-the-shelf machine learning tools to be directly applied. Recent research has shown that the past decade of network embedding approaches either explicitly factorize a carefully designed matrix to obtain the low-dimensional node vector representation or are closely related to implicit matrix factorization, with the fundamental assumption that the factorized node connectivity matrix is low-rank. Nonetheless, the global low-rank assumption does not necessarily hold especially when the factorized matrix encodes complex node interactions, and the resultant single low-rank embedding matrix is insufficient to capture all the observed connectivity patterns. In this regard, we propose a novel multi-level network embedding framework BoostNE, which can learn multiple network embedding representations of different granularity from coarse to fine without imposing the prevalent global low-rank assumption. The proposed BoostNE method is also in line with the successful gradient boosting method in ensemble learning as multiple weak embeddings lead to a stronger and more effective one. We assess the effectiveness of the proposed BoostNE framework by comparing it with existing state-of-the-art network embedding methods on various datasets, and the experimental results corroborate the superiority of the proposed BoostNE network embedding framework.
Tasks	Feature Engineering, Network Embedding, Representation Learning
Published	2018-08-26
URL	http://arxiv.org/abs/1808.08627v1
PDF	http://arxiv.org/pdf/1808.08627v1.pdf
PWC	https://paperswithcode.com/paper/multi-level-network-embedding-with-boosted
Repo	https://github.com/benedekrozemberczki/karateclub
Framework	none

A multi-instance deep neural network classifier: application to Higgs boson CP measurement


Title	A multi-instance deep neural network classifier: application to Higgs boson CP measurement
Authors	P. Bialas, D. Nemeth, E. Richter-Wąs
Abstract	We investigate properties of a classifier applied to the measurements of the CP state of the Higgs boson in $H\rightarrow\tau\tau$ decays. The problem is framed as binary classifier applied to individual instances. Then the prior knowledge that the instances belong to the same class is used to define the multi-instance classifier. Its final score is calculated as multiplication of single instance scores for a given series of instances. In the paper we discuss properties of such classifier, notably its dependence on the number of instances in the series. This classifier exhibits very strong random dependence on the number of epochs used for training and requires careful tuning of the classification threshold. We derive formula for this optimal threshold.
Tasks
Published	2018-03-02
URL	http://arxiv.org/abs/1803.00838v2
PDF	http://arxiv.org/pdf/1803.00838v2.pdf
PWC	https://paperswithcode.com/paper/a-multi-instance-deep-neural-network
Repo	https://github.com/klasocha/HiggsCP
Framework	none