April 3, 2020

2994 words 15 mins read

Paper Group AWR 2

StructSum: Incorporating Latent and Explicit Sentence Dependencies for Single Document Summarization. The gap between theory and practice in function approximation with deep neural networks. Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool. Flashlight CNN Image Denoising. RIDE: Rewarding Impact-Dr …

StructSum: Incorporating Latent and Explicit Sentence Dependencies for Single Document Summarization


Title	StructSum: Incorporating Latent and Explicit Sentence Dependencies for Single Document Summarization
Authors	Vidhisha Balachandran, Artidoro Pagnoni, Jay Yoon Lee, Dheeraj Rajagopal, Jaime Carbonell, Yulia Tsvetkov
Abstract	Traditional preneural approaches to single document summarization relied on modeling the intermediate structure of a document before generating the summary. In contrast, the current state of the art neural summarization models do not preserve any intermediate structure, resorting to encoding the document as a sequence of tokens. The goal of this work is two-fold: to improve the quality of generated summaries and to learn interpretable document representations for summarization. To this end, we propose incorporating latent and explicit sentence dependencies into single-document summarization models. We use structure-aware encoders to induce latent sentence relations, and inject explicit coreferring mention graph across sentences to incorporate explicit structure. On the CNN/DM dataset, our model outperforms standard baselines and provides intermediate latent structures for analysis. We present an extensive analysis of our summaries and show that modeling document structure reduces copying long sequences and incorporates richer content from the source document while maintaining comparable summary lengths and an increased degree of abstraction.
Tasks	Document Summarization
Published	2020-03-01
URL	https://arxiv.org/abs/2003.00576v1
PDF	https://arxiv.org/pdf/2003.00576v1.pdf
PWC	https://paperswithcode.com/paper/structsum-incorporating-latent-and-explicit
Repo	https://github.com/atulkum/pointer_summarizer
Framework	tf

The gap between theory and practice in function approximation with deep neural networks


Title	The gap between theory and practice in function approximation with deep neural networks
Authors	Ben Adcock, Nick Dexter
Abstract	Deep learning (DL) is transforming whole industries as complicated decision-making processes are being automated by Deep Neural Networks (DNNs) trained on real-world data. Driven in part by a rapidly-expanding literature on DNN approximation theory showing that DNNs can approximate a rich variety of functions, these tools are increasingly being considered for problems in scientific computing. Yet, unlike more traditional algorithms in this field, relatively little is known about DNNs from the principles of numerical analysis, namely, stability, accuracy, computational efficiency and sample complexity. In this paper we introduce a computational framework for examining DNNs in practice, and use it to study their empirical performance with regard to these issues. We examine the performance of DNNs of different widths and depths on a variety of test functions in various dimensions, including smooth and piecewise smooth functions. We also compare DL against best-in-class methods for smooth function approximation based on compressed sensing. Our main conclusion is that there is a crucial gap between the approximation theory of DNNs and their practical performance, with trained DNNs performing relatively poorly on functions for which there are strong approximation results (e.g. smooth functions), yet performing well in comparison to best-in-class methods for other functions. Finally, we present a novel practical existence theorem, which asserts the existence of a DNN architecture and training procedure which offers the same performance as current best-in-class schemes. This result indicates the potential for practical DNN approximation, and the need for future research into practical architecture design and training strategies.
Tasks	Decision Making
Published	2020-01-16
URL	https://arxiv.org/abs/2001.07523v1
PDF	https://arxiv.org/pdf/2001.07523v1.pdf
PWC	https://paperswithcode.com/paper/the-gap-between-theory-and-practice-in
Repo	https://github.com/ndexter/MLFA
Framework	tf

Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool


Title	Resources for Turkish Dependency Parsing: Introducing the BOUN Treebank and the BoAT Annotation Tool
Authors	Utku Türk, Furkan Atmaca, Şaziye Betül Özateş, Gözde Berk, Seyyit Talha Bedir, Abdullatif Köksal, Balkız Öztürk Başaran, Tunga Güngör, Arzucan Özgür
Abstract	In this paper, we describe our contributions and efforts to develop Turkish resources, which include a new treebank (BOUN Treebank) with novel sentences, along with the guidelines we adopted and a new annotation tool we developed (BoAT). The manual annotation process we employed was shaped and implemented by a team of four linguists and five NLP specialists. Decisions regarding the annotation of the BOUN Treebank were made in line with the Universal Dependencies framework, which originated from the works of De Marneffe et al. (2014) and Nivre et al. (2016). We took into account the recent unifying efforts based on the re-annotation of other Turkish treebanks in the UD framework (T"urk et al., 2019). Through the BOUN Treebank, we introduced a total of 9,757 sentences from various topics including biographical texts, national newspapers, instructional texts, popular culture articles, and essays. In addition, we report the parsing results of a graph-based dependency parser obtained over each text type, the total of the BOUN Treebank, and all Turkish treebanks that we either re-annotated or introduced. We show that a state-of-the-art dependency parser has improved scores for identifying the proper head and the syntactic relationships between the heads and the dependents. In light of these results, we have observed that the unification of the Turkish annotation scheme and introducing a more comprehensive treebank improves performance with regards to dependency parsing
Tasks	Dependency Parsing
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10416v1
PDF	https://arxiv.org/pdf/2002.10416v1.pdf
PWC	https://paperswithcode.com/paper/resources-for-turkish-dependency-parsing
Repo	https://github.com/boun-tabi/UD_Turkish-BOUN
Framework	none

Flashlight CNN Image Denoising


Title	Flashlight CNN Image Denoising
Authors	Pham Huu Thanh Binh, Cristóvão Cruz, Karen Egiazarian
Abstract	This paper proposes a learning-based denoising method called FlashLight CNN (FLCNN) that implements a deep neural network for image denoising. The proposed approach is based on deep residual networks and inception networks and it is able to leverage many more parameters than residual networks alone for denoising grayscale images corrupted by additive white Gaussian noise (AWGN). FlashLight CNN demonstrates state of the art performance when compared quantitatively and visually with the current state of the art image denoising methods.
Tasks	Denoising, Image Denoising
Published	2020-03-02
URL	https://arxiv.org/abs/2003.00762v1
PDF	https://arxiv.org/pdf/2003.00762v1.pdf
PWC	https://paperswithcode.com/paper/flashlight-cnn-image-denoising
Repo	https://github.com/binhpht/flashlightCNN
Framework	none

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments


Title	RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
Authors	Roberta Raileanu, Tim Rocktäschel
Abstract	Exploration in sparse reward environments remains one of the key challenges of model-free reinforcement learning. Instead of solely relying on extrinsic rewards provided by the environment, many state-of-the-art methods use intrinsic rewards to encourage exploration. However, we show that existing methods fall short in procedurally-generated environments where an agent is unlikely to visit a state more than once. We propose a novel type of intrinsic reward which encourages the agent to take actions that lead to significant changes in its learned state representation. We evaluate our method on multiple challenging procedurally-generated tasks in MiniGrid, as well as on tasks with high-dimensional observations used in prior work. Our experiments demonstrate that this approach is more sample efficient than existing exploration methods, particularly for procedurally-generated MiniGrid environments. Furthermore, we analyze the learned behavior as well as the intrinsic reward received by our agent. In contrast to previous approaches, our intrinsic reward does not diminish during the course of training and it rewards the agent substantially more for interacting with objects that it can control.
Tasks
Published	2020-02-27
URL	https://arxiv.org/abs/2002.12292v2
PDF	https://arxiv.org/pdf/2002.12292v2.pdf
PWC	https://paperswithcode.com/paper/ride-rewarding-impact-driven-exploration-for-1
Repo	https://github.com/maximecb/gym-minigrid
Framework	pytorch

TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning


Title	TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
Authors	Fernando Pérez-García, Rachel Sparks, Sebastien Ourselin
Abstract	We present TorchIO, an open-source Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. It follows the design of PyTorch and relies on standard medical image processing libraries such as SimpleITK or NiBabel to efficiently process large 3D images during the training of convolutional neural networks. We provide multiple generic as well as magnetic-resonance-imaging-specific operations for preprocessing and augmentation of medical images. TorchIO is an open-source project with code, comprehensive examples and extensive documentation shared at https://github.com/fepegar/torchio.
Tasks
Published	2020-03-09
URL	https://arxiv.org/abs/2003.04696v1
PDF	https://arxiv.org/pdf/2003.04696v1.pdf
PWC	https://paperswithcode.com/paper/torchio-a-python-library-for-efficient
Repo	https://github.com/fepegar/torchio
Framework	pytorch

The Instantaneous Accuracy: a Novel Metric for the Problem of Online Human Behaviour Recognition in Untrimmed Videos


Title	The Instantaneous Accuracy: a Novel Metric for the Problem of Online Human Behaviour Recognition in Untrimmed Videos
Authors	Marcos Baptista Rios, Roberto J. López-Sastre, Fabian Caba Heilbron, Jan van Gemert, Francisco Javier Acevedo-Rodríguez, Saturnino Maldonado-Bascón
Abstract	The problem of Online Human Behaviour Recognition in untrimmed videos, aka Online Action Detection (OAD), needs to be revisited. Unlike traditional offline action detection approaches, where the evaluation metrics are clear and well established, in the OAD setting we find few works and no consensus on the evaluation protocols to be used. In this paper we introduce a novel online metric, the Instantaneous Accuracy ($IA$), that exhibits an \emph{online} nature, solving most of the limitations of the previous (offline) metrics. We conduct a thorough experimental evaluation on TVSeries dataset, comparing the performance of various baseline methods to the state of the art. Our results confirm the problems of previous evaluation protocols, and suggest that an IA-based protocol is more adequate to the online scenario for human behaviour understanding. Code of the metric available https://github.com/gramuah/ia
Tasks	Action Detection
Published	2020-03-22
URL	https://arxiv.org/abs/2003.09970v2
PDF	https://arxiv.org/pdf/2003.09970v2.pdf
PWC	https://paperswithcode.com/paper/the-instantaneous-accuracy-a-novel-metric-for
Repo	https://github.com/gramuah/ia
Framework	none

Harvesting Ambient RF for Presence Detection Through Deep Learning


Title	Harvesting Ambient RF for Presence Detection Through Deep Learning
Authors	Yang Liu, Tiexing Wang, Yuexin Jiang, Biao Chen
Abstract	This paper explores the use of ambient radio frequency (RF) signals for human presence detection through deep learning. Using WiFi signal as an example, we demonstrate that the channel state information (CSI) obtained at the receiver contains rich information about the propagation environment. Through judicious pre-processing of the estimated CSI followed by deep learning, reliable presence detection can be achieved. Several challenges in passive RF sensing are addressed. With presence detection, how to collect training data with human presence can have a significant impact on the performance. This is in contrast to activity detection when a specific motion pattern is of interest. A second challenge is that RF signals are complex-valued. Handling complex-valued input in deep learning requires careful data representation and network architecture design. Finally, human presence affects CSI variation along multiple dimensions; such variation, however, is often masked by system impediments such as timing or frequency offset. Addressing these challenges, the proposed learning system uses pre-processing to preserve human motion induced channel variation while insulating against other impairments. A convolutional neural network (CNN) properly trained with both magnitude and phase information is then designed to achieve reliable presence detection. Extensive experiments are conducted. Using off-the-shelf WiFi devices, the proposed deep learning based RF sensing achieves near perfect presence detection during multiple extended periods of test and exhibits superior performance compared with leading edge passive infrared sensors. The learning based passive RF sensing thus provides a viable and promising alternative for presence or occupancy detection.
Tasks	Action Detection, Activity Detection
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05770v1
PDF	https://arxiv.org/pdf/2002.05770v1.pdf
PWC	https://paperswithcode.com/paper/harvesting-ambient-rf-for-presence-detection
Repo	https://github.com/bigtreeyanger/presence_detection_cnn
Framework	tf

VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions


Title	VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions
Authors	Oytun Ulutan, A S M Iftekhar, B. S. Manjunath
Abstract	Comprehensive visual understanding requires detection frameworks that can effectively learn and utilize object interactions while analyzing objects individually. This is the main objective in Human-Object Interaction (HOI) detection task. In particular, relative spatial reasoning and structural connections between objects are essential cues for analyzing interactions, which is addressed by the proposed Visual-Spatial-Graph Network (VSGNet) architecture. VSGNet extracts visual features from the human-object pairs, refines the features with spatial configurations of the pair, and utilizes the structural connections between the pair via graph convolutions. The performance of VSGNet is thoroughly evaluated using the Verbs in COCO (V-COCO) and HICO-DET datasets. Experimental results indicate that VSGNet outperforms state-of-the-art solutions by 8% or 4 mAP in V-COCO and 16% or 3 mAP in HICO-DET.
Tasks	Human-Object Interaction Detection
Published	2020-03-11
URL	https://arxiv.org/abs/2003.05541v1
PDF	https://arxiv.org/pdf/2003.05541v1.pdf
PWC	https://paperswithcode.com/paper/vsgnet-spatial-attention-network-for
Repo	https://github.com/ASMIftekhar/VSGNet
Framework	none

An Unsupervised Learning Model for Medical Image Segmentation


Title	An Unsupervised Learning Model for Medical Image Segmentation
Authors	Junyu Chen, Eric C. Frey
Abstract	For the majority of the learning-based segmentation methods, a large quantity of high-quality training data is required. In this paper, we present a novel learning-based segmentation model that could be trained semi- or un- supervised. Specifically, in the unsupervised setting, we parameterize the Active contour without edges (ACWE) framework via a convolutional neural network (ConvNet), and optimize the parameters of the ConvNet using a self-supervised method. In another setting (semi-supervised), the auxiliary segmentation ground truth is used during training. We show that the method provides fast and high-quality bone segmentation in the context of single-photon emission computed tomography (SPECT) image.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2020-01-28
URL	https://arxiv.org/abs/2001.10155v1
PDF	https://arxiv.org/pdf/2001.10155v1.pdf
PWC	https://paperswithcode.com/paper/an-unsupervised-learning-model-for-medical
Repo	https://github.com/junyuchen245/Unsuprevised_Seg_via_CNN
Framework	tf

Deep Unfolding Network for Image Super-Resolution


Title	Deep Unfolding Network for Image Super-Resolution
Authors	Kai Zhang, Luc Van Gool, Radu Timofte
Abstract	Learning-based single image super-resolution (SISR) methods are continuously showing superior effectiveness and efficiency over traditional model-based methods, largely due to the end-to-end training. However, different from model-based methods that can handle the SISR problem with different scale factors, blur kernels and noise levels under a unified MAP (maximum a posteriori) framework, learning-based methods generally lack such flexibility. To address this issue, this paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods. Specifically, by unfolding the MAP inference via a half-quadratic splitting algorithm, a fixed number of iterations consisting of alternately solving a data subproblem and a prior subproblem can be obtained. The two subproblems then can be solved with neural modules, resulting in an end-to-end trainable, iterative network. As a result, the proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model, while maintaining the advantages of learning-based methods. Extensive experiments demonstrate the superiority of the proposed deep unfolding network in terms of flexibility, effectiveness and also generalizability.
Tasks	Image Super-Resolution, Super-Resolution
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10428v1
PDF	https://arxiv.org/pdf/2003.10428v1.pdf
PWC	https://paperswithcode.com/paper/deep-unfolding-network-for-image-super
Repo	https://github.com/cszn/USRNet
Framework	pytorch

Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation


Title	Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
Authors	Yige Xu, Xipeng Qiu, Ligao Zhou, Xuanjing Huang
Abstract	Fine-tuning pre-trained language models like BERT has become an effective way in NLP and yields state-of-the-art results on many downstream tasks. Recent studies on adapting BERT to new tasks mainly focus on modifying the model structure, re-designing the pre-train tasks, and leveraging external data and knowledge. The fine-tuning strategy itself has yet to be fully explored. In this paper, we improve the fine-tuning of BERT with two effective mechanisms: self-ensemble and self-distillation. The experiments on text classification and natural language inference tasks show our proposed methods can significantly improve the adaption of BERT without any external data or knowledge.
Tasks	Natural Language Inference, Text Classification
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10345v1
PDF	https://arxiv.org/pdf/2002.10345v1.pdf
PWC	https://paperswithcode.com/paper/improving-bert-fine-tuning-via-self-ensemble
Repo	https://github.com/lonePatient/BERT-SDA
Framework	pytorch

Tensor Graph Convolutional Networks for Text Classification


Title	Tensor Graph Convolutional Networks for Text Classification
Authors	Xien Liu, Xinxin You, Xiao Zhang, Ji Wu, Ping Lv
Abstract	Compared to sequential learning models, graph-based neural networks exhibit some excellent properties, such as ability capturing global information. In this paper, we investigate graph-based neural networks for text classification problem. A new framework TensorGCN (tensor graph convolutional networks), is presented for this task. A text graph tensor is firstly constructed to describe semantic, syntactic, and sequential contextual information. Then, two kinds of propagation learning perform on the text graph tensor. The first is intra-graph propagation used for aggregating information from neighborhood nodes in a single graph. The second is inter-graph propagation used for harmonizing heterogeneous information between graphs. Extensive experiments are conducted on benchmark datasets, and the results illustrate the effectiveness of our proposed framework. Our proposed TensorGCN presents an effective way to harmonize and integrate heterogeneous information from different kinds of graphs.
Tasks	Text Classification
Published	2020-01-12
URL	https://arxiv.org/abs/2001.05313v1
PDF	https://arxiv.org/pdf/2001.05313v1.pdf
PWC	https://paperswithcode.com/paper/tensor-graph-convolutional-networks-for-text
Repo	https://github.com/xienliu/tensor-gcn-text-classification-tensorflow
Framework	tf

Aggregated Learning: A Vector-Quantization Approach to Learning Neural Network Classifiers


Title	Aggregated Learning: A Vector-Quantization Approach to Learning Neural Network Classifiers
Authors	Masoumeh Soflaei, Hongyu Guo, Ali Al-Bashabsheh, Yongyi Mao, Richong Zhang
Abstract	We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call “IB learning”. We show that IB learning is, in fact, equivalent to a special class of the quantization problem. The classical results in rate-distortion theory then suggest that IB learning can benefit from a “vector quantization” approach, namely, simultaneously learning the representations of multiple input objects. Such an approach assisted with some variational techniques, result in a novel learning framework, “Aggregated Learning”, for classification with neural network models. In this framework, several objects are jointly classified by a single neural network. The effectiveness of this framework is verified through extensive experiments on standard image recognition and text classification tasks.
Tasks	Quantization, Representation Learning, Text Classification
Published	2020-01-12
URL	https://arxiv.org/abs/2001.03955v1
PDF	https://arxiv.org/pdf/2001.03955v1.pdf
PWC	https://paperswithcode.com/paper/aggregated-learning-a-vector-quantization
Repo	https://github.com/SITE5039/AgrLearn
Framework	none

Regularizing Class-wise Predictions via Self-knowledge Distillation


Title	Regularizing Class-wise Predictions via Self-knowledge Distillation
Authors	Sukmin Yun, Jongjin Park, Kimin Lee, Jinwoo Shin
Abstract	Deep neural networks with millions of parameters may suffer from poor generalization due to overfitting. To mitigate the issue, we propose a new regularization method that penalizes the predictive distribution between similar samples. In particular, we distill the predictive distribution between different samples of the same label during training. This results in regularizing the dark knowledge (i.e., the knowledge on wrong predictions) of a single network (i.e., a self-knowledge distillation) by forcing it to produce more meaningful and consistent predictions in a class-wise manner. Consequently, it mitigates overconfident predictions and reduces intra-class variations. Our experimental results on various image classification tasks demonstrate that the simple yet powerful method can significantly improve not only the generalization ability but also the calibration performance of modern convolutional neural networks.
Tasks	Calibration, Image Classification
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13964v1
PDF	https://arxiv.org/pdf/2003.13964v1.pdf
PWC	https://paperswithcode.com/paper/regularizing-class-wise-predictions-via-self
Repo	https://github.com/alinlab/cs-kd
Framework	pytorch