January 28, 2020

3346 words 16 mins read

Paper Group ANR 949

Retinal Vessels Segmentation Based on Dilated Multi-Scale Convolutional Neural Network. Prediction Uncertainty Estimation for Hate Speech Classification. Behavior Gated Language Models. It GAN DO Better: GAN-based Detection of Objects on Images with Varying Quality. Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and T …

Retinal Vessels Segmentation Based on Dilated Multi-Scale Convolutional Neural Network


Title	Retinal Vessels Segmentation Based on Dilated Multi-Scale Convolutional Neural Network
Authors	Yun Jiang, Ning Tan, Tingting Peng, Hai Zhang
Abstract	Accurate segmentation of retinal vessels is a basic step in Diabetic retinopathy(DR) detection. Most methods based on deep convolutional neural network (DCNN) have small receptive fields, and hence they are unable to capture global context information of larger regions, with difficult to identify lesions. The final segmented retina vessels contain more noise with low classification accuracy. Therefore, in this paper, we propose a DCNN structure named as D-Net. In the proposed D-Net, the dilation convolution is used in the backbone network to obtain a larger receptive field without losing spatial resolution, so as to reduce the loss of feature information and to reduce the difficulty of tiny thin vessels segmentation. The large receptive field can better distinguished between the lesion area and the blood vessel area. In the proposed Multi-Scale Information Fusion module (MSIF), parallel convolution layers with different dilation rates are used, so that the model can obtain more dense feature information and better capture retinal vessel information of different sizes. In the decoding module, the skip layer connection is used to propagate context information to higher resolution layers, so as to prevent low-level information from passing the entire network structure. Finally, our method was verified on DRIVE, STARE and CHASE dataset. The experimental results show that our network structure outperforms some state-of-art method, such as N4-fields, U-Net, and DRIU in terms of accuracy, sensitivity, specificity, and AUCROC. Particularly, D-Net outperforms U-Net by 1.04%, 1.23% and 2.79% in DRIVE, STARE, and CHASE three dataset, respectively.
Tasks
Published	2019-04-11
URL	http://arxiv.org/abs/1904.05644v1
PDF	http://arxiv.org/pdf/1904.05644v1.pdf
PWC	https://paperswithcode.com/paper/retinal-vessels-segmentation-based-on-dilated
Repo
Framework

Prediction Uncertainty Estimation for Hate Speech Classification


Title	Prediction Uncertainty Estimation for Hate Speech Classification
Authors	Kristian Miok, Dong Nguyen-Doan, Blaž Škrlj, Daniela Zaharie, Marko Robnik-Šikonja
Abstract	As a result of social network popularity, in recent years, hate speech phenomenon has significantly increased. Due to its harmful effect on minority groups as well as on large communities, there is a pressing need for hate speech detection and filtering. However, automatic approaches shall not jeopardize free speech, so they shall accompany their decisions with explanations and assessment of uncertainty. Thus, there is a need for predictive machine learning models that not only detect hate speech but also help users understand when texts cross the line and become unacceptable. The reliability of predictions is usually not addressed in text classification. We fill this gap by proposing the adaptation of deep neural networks that can efficiently estimate prediction uncertainty. To reliably detect hate speech, we use Monte Carlo dropout regularization, which mimics Bayesian inference within neural networks. We evaluate our approach using different text embedding methods. We visualize the reliability of results with a novel technique that aids in understanding the classification reliability and errors.
Tasks	Bayesian Inference, Hate Speech Detection, Text Classification
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07158v3
PDF	https://arxiv.org/pdf/1909.07158v3.pdf
PWC	https://paperswithcode.com/paper/prediction-uncertainty-estimation-for-hate
Repo
Framework

Behavior Gated Language Models


Title	Behavior Gated Language Models
Authors	Prashanth Gurunath Shivakumar, Shao-Yen Tseng, Panayiotis Georgiou, Shrikanth Narayanan
Abstract	Most current language modeling techniques only exploit co-occurrence, semantic and syntactic information from the sequence of words. However, a range of information such as the state of the speaker and dynamics of the interaction might be useful. In this work we derive motivation from psycholinguistics and propose the addition of behavioral information into the context of language modeling. We propose the augmentation of language models with an additional module which analyzes the behavioral state of the current context. This behavioral information is used to gate the outputs of the language model before the final word prediction output. We show that the addition of behavioral context in language models achieves lower perplexities on behavior-rich datasets. We also confirm the validity of the proposed models on a variety of model architectures and improve on previous state-of-the-art models with generic domain Penn Treebank Corpus.
Tasks	Language Modelling
Published	2019-08-31
URL	https://arxiv.org/abs/1909.00107v1
PDF	https://arxiv.org/pdf/1909.00107v1.pdf
PWC	https://paperswithcode.com/paper/behavior-gated-language-models
Repo
Framework

It GAN DO Better: GAN-based Detection of Objects on Images with Varying Quality


Title	It GAN DO Better: GAN-based Detection of Objects on Images with Varying Quality
Authors	Charan D. Prakash, Lina J. Karam
Abstract	In this paper, we propose in our novel generative framework the use of Generative Adversarial Networks (GANs) to generate features that provide robustness for object detection on reduced quality images. The proposed GAN-based Detection of Objects (GAN-DO) framework is not restricted to any particular architecture and can be generalized to several deep neural network (DNN) based architectures. The resulting deep neural network maintains the exact architecture as the selected baseline model without adding to the model parameter complexity or inference speed. We first evaluate the effect of image quality not only on the object classification but also on the object bounding box regression. We then test the models resulting from our proposed GAN-DO framework, using two state-of-the-art object detection architectures as the baseline models. We also evaluate the effect of the number of re-trained parameters in the generator of GAN-DO on the accuracy of the final trained model. Performance results provided using GAN-DO on object detection datasets establish an improved robustness to varying image quality and a higher mAP compared to the existing approaches.
Tasks	Object Classification, Object Detection
Published	2019-12-03
URL	https://arxiv.org/abs/1912.01707v1
PDF	https://arxiv.org/pdf/1912.01707v1.pdf
PWC	https://paperswithcode.com/paper/it-gan-do-better-gan-based-detection-of
Repo
Framework

Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools


Title	Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools
Authors	Ruben Mayer, Hans-Arno Jacobsen
Abstract	Deep Learning (DL) has had an immense success in the recent past, leading to state-of-the-art results in various domains such as image recognition and natural language processing. One of the reasons for this success is the increasing size of DL models and the proliferation of vast amounts of training data being available. To keep on improving the performance of DL, increasing the scalability of DL systems is necessary. In this survey, we perform a broad and thorough investigation on challenges, techniques and tools for scalable DL on distributed infrastructures. This incorporates infrastructures for DL, methods for parallel DL training, multi-tenant resource scheduling and the management of training and model data. Further, we analyze and compare 11 current open-source DL frameworks and tools and investigate which of the techniques are commonly implemented in practice. Finally, we highlight future research trends in DL systems that deserve further research.
Tasks
Published	2019-03-27
URL	https://arxiv.org/abs/1903.11314v2
PDF	https://arxiv.org/pdf/1903.11314v2.pdf
PWC	https://paperswithcode.com/paper/scalable-deep-learning-on-distributed
Repo
Framework

Massively Multilingual Neural Machine Translation


Title	Massively Multilingual Neural Machine Translation
Authors	Roee Aharoni, Melvin Johnson, Orhan Firat
Abstract	Multilingual neural machine translation (NMT) enables training a single model that supports translation from multiple source languages into multiple target languages. In this paper, we push the limits of multilingual NMT in terms of number of languages being used. We perform extensive experiments in training massively multilingual NMT models, translating up to 102 languages to and from English within a single model. We explore different setups for training such models and analyze the trade-offs between translation quality and various modeling decisions. We report results on the publicly available TED talks multilingual corpus where we show that massively multilingual many-to-many models are effective in low resource settings, outperforming the previous state-of-the-art while supporting up to 59 languages. Our experiments on a large-scale dataset with 102 languages to and from English and up to one million examples per direction also show promising results, surpassing strong bilingual baselines and encouraging future work on massively multilingual NMT.
Tasks	Machine Translation
Published	2019-02-28
URL	https://arxiv.org/abs/1903.00089v3
PDF	https://arxiv.org/pdf/1903.00089v3.pdf
PWC	https://paperswithcode.com/paper/massively-multilingual-neural-machine
Repo
Framework

Knowledge-incorporating ESIM models for Response Selection in Retrieval-based Dialog Systems


Title	Knowledge-incorporating ESIM models for Response Selection in Retrieval-based Dialog Systems
Authors	Jatin Ganhotra, Siva Sankalp Patel, Kshitij Fadnis
Abstract	Goal-oriented dialog systems, which can be trained end-to-end without manually encoding domain-specific features, show tremendous promise in the customer support use-case e.g. flight booking, hotel reservation, technical support, student advising etc. These dialog systems must learn to interact with external domain knowledge to achieve the desired goal e.g. recommending courses to a student, booking a table at a restaurant etc. This paper presents extended Enhanced Sequential Inference Model (ESIM) models: a) K-ESIM (Knowledge-ESIM), which incorporates the external domain knowledge and b) T-ESIM (Targeted-ESIM), which leverages information from similar conversations to improve the prediction accuracy. Our proposed models and the baseline ESIM model are evaluated on the Ubuntu and Advising datasets in the Sentence Selection track of the latest Dialog System Technology Challenge (DSTC7), where the goal is to find the correct next utterance, given a partial conversation, from a set of candidates. Our preliminary results suggest that incorporating external knowledge sources and leveraging information from similar dialogs leads to performance improvements for predicting the next utterance.
Tasks	Goal-Oriented Dialog
Published	2019-07-11
URL	https://arxiv.org/abs/1907.05792v1
PDF	https://arxiv.org/pdf/1907.05792v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-incorporating-esim-models-for
Repo
Framework

Adaptively stacking ensembles for influenza forecasting with incomplete data


Title	Adaptively stacking ensembles for influenza forecasting with incomplete data
Authors	Thomas McAndrew, Nicholas G. Reich
Abstract	Seasonal influenza infects between 10 and 50 million people in the United States every year, overburdening hospitals during weeks of peak incidence. Named by the CDC as an important tool to fight the damaging effects of these epidemics, accurate forecasts of influenza and influenza-like illness (ILI) forewarn public health officials about when, and where, seasonal influenza outbreaks will hit hardest. Multi-model ensemble forecasts—weighted combinations of component models—have shown positive results in forecasting. Ensemble forecasts of influenza outbreaks have been static, training on all past ILI data at the beginning of a season, generating a set of optimal weights for each model in the ensemble, and keeping the weights constant. We propose an adaptive ensemble forecast that (i) changes model weights week-by-week throughout the influenza season, (ii) only needs the current influenza season’s data to make predictions, and (iii) by introducing a prior distribution, shrinks weights toward the reference equal weighting approach and adjusts for observed ILI percentages that are subject to future revisions. We investigate the prior’s ability to impact adaptive ensemble performance and, after finding an optimal prior via a cross-validation approach, compare our adaptive ensemble’s performance to equal-weighted and static ensembles. Applied to forecasts of short-term ILI incidence at the regional and national level in the US, our adaptive model outperforms a naive equal-weighted ensemble, and has similar or better performance to the static ensemble, which requires multiple years of training data. Adaptive ensembles are able to quickly train and forecast during epidemics, and provide a practical tool to public health officials looking for forecasts that can conform to unique features of a specific season.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1908.01675v1
PDF	https://arxiv.org/pdf/1908.01675v1.pdf
PWC	https://paperswithcode.com/paper/adaptively-stacking-ensembles-for-influenza
Repo
Framework

Efficient Inference of CNNs via Channel Pruning


Title	Efficient Inference of CNNs via Channel Pruning
Authors	Boyu Zhang, Azadeh Davoodi, Yu Hen Hu
Abstract	The deployment of Convolutional Neural Networks (CNNs) on resource constrained platforms such as mobile devices and embedded systems has been greatly hindered by their high implementation cost, and thus motivated a lot research interest in compressing and accelerating trained CNN models. Among various techniques proposed in literature, structured pruning, especially channel pruning, has gain a lot focus due to 1) its superior performance in memory, computation, and energy reduction; and 2) it is friendly to existing hardware and software libraries. In this paper, we investigate the intermediate results of convolutional layers and present a novel pivoted QR factorization based channel pruning technique that can prune any specified number of input channels of any layer. We also explore more pruning opportunities in ResNet-like architectures by applying two tweaks to our technique. Experiment results on VGG-16 and ResNet-50 models with ImageNet ILSVRC 2012 dataset are very impressive with 4.29X and 2.84X computation reduction while only sacrificing about 1.40% top-5 accuracy. Compared to many prior works, the pruned models produced by our technique require up to 47.7% less computation while still achieve higher accuracies.
Tasks
Published	2019-08-08
URL	https://arxiv.org/abs/1908.03266v1
PDF	https://arxiv.org/pdf/1908.03266v1.pdf
PWC	https://paperswithcode.com/paper/efficient-inference-of-cnns-via-channel
Repo
Framework

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era


Title	Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era
Authors	Brian Nord, Andrew J. Connolly, Jamie Kinney, Jeremy Kubica, Gautaum Narayan, Joshua E. G. Peek, Chad Schafer, Erik J. Tollerud, Camille Avestruz, G. Jogesh Babu, Simon Birrer, Douglas Burke, João Caldeira, Douglas A. Caldwell, Joleen K. Carlberg, Yen-Chi Chen, Chuanfei Dong, Eric D. Feigelson, V. Zach Golkhou, Vinay Kashyap, T. S. Li, Thomas Loredo, Luisa Lucie-Smith, Kaisey S. Mandel, J. R. Martínez-Galarza, Adam A. Miller, Priyamvada Natarajan, Michelle Ntampaka, Andy Ptak, David Rapetti, Lior Shamir, Aneta Siemiginowska, Brigitta M. Sipőcz, Arfon M. Smith, Nhan Tran, Ricardo Vilalta, Lucianne M. Walkowicz, John ZuHone
Abstract	The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt — e.g., via the onset of artificial intelligence — which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists’ contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/).
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.02479v1
PDF	https://arxiv.org/pdf/1911.02479v1.pdf
PWC	https://paperswithcode.com/paper/algorithms-and-statistical-models-for
Repo
Framework

What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks


Title	What goes around comes around: Cycle-Consistency-based Short-Term Motion Prediction for Anomaly Detection using Generative Adversarial Networks
Authors	Thomas Golda, Nils Murzyn, Chengchao Qu, Kristian Kroschel
Abstract	Anomaly detection plays in many fields of research, along with the strongly related task of outlier detection, a very important role. Especially within the context of the automated analysis of video material recorded by surveillance cameras, abnormal situations can be of very different nature. For this purpose this work investigates Generative-Adversarial-Network-based methods (GAN) for anomaly detection related to surveillance applications. The focus is on the usage of static camera setups, since this kind of camera is one of the most often used and belongs to the lower price segment. In order to address this task, multiple subtasks are evaluated, including the influence of existing optical flow methods for the incorporation of short-term temporal information, different forms of network setups and losses for GANs, and the use of morphological operations for further performance improvement. With these extension we achieved up to 2.4% better results. Furthermore, the final method reduced the anomaly detection error for GAN-based methods by about 42.8%.
Tasks	Anomaly Detection, motion prediction, Optical Flow Estimation, Outlier Detection
Published	2019-08-08
URL	https://arxiv.org/abs/1908.03055v1
PDF	https://arxiv.org/pdf/1908.03055v1.pdf
PWC	https://paperswithcode.com/paper/what-goes-around-comes-around-cycle
Repo
Framework

A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications


Title	A Self-Attention Joint Model for Spoken Language Understanding in Situational Dialog Applications
Authors	Mengyang Chen, Jin Zeng, Jie Lou
Abstract	Spoken language understanding (SLU) acts as a critical component in goal-oriented dialog systems. It typically involves identifying the speakers intent and extracting semantic slots from user utterances, which are known as intent detection (ID) and slot filling (SF). SLU problem has been intensively investigated in recent years. However, these methods just constrain SF results grammatically, solve ID and SF independently, or do not fully utilize the mutual impact of the two tasks. This paper proposes a multi-head self-attention joint model with a conditional random field (CRF) layer and a prior mask. The experiments show the effectiveness of our model, as compared with state-of-the-art models. Meanwhile, online education in China has made great progress in the last few years. But there are few intelligent educational dialog applications for students to learn foreign languages. Hence, we design an intelligent dialog robot equipped with different scenario settings to help students learn communication skills.
Tasks	Goal-Oriented Dialog, Intent Detection, Slot Filling, Spoken Language Understanding
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11393v1
PDF	https://arxiv.org/pdf/1905.11393v1.pdf
PWC	https://paperswithcode.com/paper/a-self-attention-joint-model-for-spoken
Repo
Framework

LGN-CNN: a biologically inspired CNN architecture


Title	LGN-CNN: a biologically inspired CNN architecture
Authors	Federico Bertoni, Giovanna Citti, Alessandro Sarti
Abstract	In this paper we introduce a biologically inspired CNN architecture that has a first convolutional layer that mimics the role of the LGN. The first layer of the net shows a rotationally symmetric pattern justified by the structure of the net itself that turns up to be an approximation of a Laplacian of Gaussian. The latter function is in turn a good approximation of the receptive profiles of the cells in the LGN. The analogy with respect to the visual system structure is established, emerging directly from the architecture of the net.
Tasks
Published	2019-11-14
URL	https://arxiv.org/abs/1911.06276v1
PDF	https://arxiv.org/pdf/1911.06276v1.pdf
PWC	https://paperswithcode.com/paper/lgn-cnn-a-biologically-inspired-cnn
Repo
Framework

Hyperbox based machine learning algorithms: A comprehensive survey


Title	Hyperbox based machine learning algorithms: A comprehensive survey
Authors	Thanh Tung Khuat, Dymitr Ruta, Bogdan Gabrys
Abstract	With the rapid development of digital information, the data volume generated by humans and machines is growing exponentially. Along with this trend, machine learning algorithms have been formed and evolved continuously to discover new information and knowledge from different data sources. Learning algorithms using hyperboxes as fundamental representational and building blocks are a branch of machine learning methods. These algorithms have enormous potential for high scalability and online adaptation of predictors built using hyperbox data representations to the dynamically changing environments and streaming data. This paper aims to give a comprehensive survey of literature on hyperbox-based machine learning models. In general, according to the architecture and characteristic features of the resulting models, the existing hyperbox-based learning algorithms may be grouped into three major categories: fuzzy min-max neural networks, hyperbox-based hybrid models, and other algorithms based on hyperbox representations. Within each of these groups, this paper shows a brief description of the structure of models, associated learning algorithms, and an analysis of their advantages and drawbacks. Main applications of these hyperbox-based models to the real-world problems are also described in this paper. Finally, we discuss some open problems and identify potential future research directions in this field.
Tasks
Published	2019-01-31
URL	http://arxiv.org/abs/1901.11303v3
PDF	http://arxiv.org/pdf/1901.11303v3.pdf
PWC	https://paperswithcode.com/paper/hyperbox-based-machine-learning-algorithms-a
Repo
Framework

Perturbation Sensitivity Analysis to Detect Unintended Model Biases


Title	Perturbation Sensitivity Analysis to Detect Unintended Model Biases
Authors	Vinodkumar Prabhakaran, Ben Hutchinson, Margaret Mitchell
Abstract	Data-driven statistical Natural Language Processing (NLP) techniques leverage large amounts of language data to build models that can understand language. However, most language data reflect the public discourse at the time the data was produced, and hence NLP models are susceptible to learning incidental associations around named referents at a particular point in time, in addition to general linguistic meaning. An NLP system designed to model notions such as sentiment and toxicity should ideally produce scores that are independent of the identity of such entities mentioned in text and their social associations. For example, in a general purpose sentiment analysis system, a phrase such as I hate Katy Perry should be interpreted as having the same sentiment as I hate Taylor Swift. Based on this idea, we propose a generic evaluation framework, Perturbation Sensitivity Analysis, which detects unintended model biases related to named entities, and requires no new annotations or corpora. We demonstrate the utility of this analysis by employing it on two different NLP models — a sentiment model and a toxicity model — applied on online comments in English language from four different genres.
Tasks	Sentiment Analysis
Published	2019-10-09
URL	https://arxiv.org/abs/1910.04210v1
PDF	https://arxiv.org/pdf/1910.04210v1.pdf
PWC	https://paperswithcode.com/paper/perturbation-sensitivity-analysis-to-detect
Repo
Framework