October 19, 2019

3125 words 15 mins read

Paper Group ANR 309

Approaches for Enriching and Improving Textual Knowledge Bases. On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent. Functionality-Oriented Convolutional Filter Pruning. Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering. A Provably Correct Algorithm for Deep Learning that A …

Approaches for Enriching and Improving Textual Knowledge Bases


Title	Approaches for Enriching and Improving Textual Knowledge Bases
Authors	Besnik Fetahu
Abstract	Verifiability is one of the core editing principles in Wikipedia, where editors are encouraged to provide citations for the added statements. Statements can be any arbitrary piece of text, ranging from a sentence up to a paragraph. However, in many cases, citations are either outdated, missing, or link to non-existing references (e.g. dead URL, moved content etc.). In total, 20% of the cases such citations refer to news articles and represent the second most cited source. Even in cases where citations are provided, there are no explicit indicators for the span of a citation for a given piece of text. In addition to issues related with the verifiability principle, many Wikipedia entity pages are incomplete, with relevant information that is already available in online news sources missing. Even for the already existing citations, there is often a delay between the news publication time and the reference time. In this thesis, we address the aforementioned issues and propose automated approaches that enforce the verifiability principle in Wikipedia, and suggest relevant and missing news references for further enriching Wikipedia entity pages.
Tasks
Published	2018-04-20
URL	http://arxiv.org/abs/1804.07583v2
PDF	http://arxiv.org/pdf/1804.07583v2.pdf
PWC	https://paperswithcode.com/paper/approaches-for-enriching-and-improving
Repo
Framework

On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent


Title	On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Authors	Noah Golmant, Nikita Vemuri, Zhewei Yao, Vladimir Feinberg, Amir Gholami, Kai Rothauge, Michael W. Mahoney, Joseph Gonzalez
Abstract	Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and systems challenges that impede the widespread success of this technique. We investigate these issues, with an emphasis on time to convergence and total computational cost, through an extensive empirical analysis of network training across several architectures and problem domains, including image classification, image segmentation, and language modeling. Although it is common practice to increase the batch size in order to fully exploit available computational resources, we find a substantially more nuanced picture. Our main finding is that across a wide range of network architectures and problem domains, increasing the batch size beyond a certain point yields no decrease in wall-clock time to convergence for \emph{either} train or test loss. This batch size is usually substantially below the capacity of current systems. We show that popular training strategies for large batch size optimization begin to fail before we can populate all available compute resources, and we show that the point at which these methods break down depends more on attributes like model architecture and data complexity than it does directly on the size of the dataset.
Tasks	Image Classification, Language Modelling, Semantic Segmentation
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12941v1
PDF	http://arxiv.org/pdf/1811.12941v1.pdf
PWC	https://paperswithcode.com/paper/on-the-computational-inefficiency-of-large
Repo
Framework

Functionality-Oriented Convolutional Filter Pruning


Title	Functionality-Oriented Convolutional Filter Pruning
Authors	Zhuwei Qin, Fuxun Yu, Chenchen Liu, Xiang Chen
Abstract	The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although extremely effective, most works are based only on quantitative characteristics of the convolutional filters, and highly overlook the qualitative interpretation of individual filter’s specific functionality. In this work, we interpreted the functionality and redundancy of the convolutional filters from different perspectives, and proposed a functionality-oriented filter pruning method. With extensive experiment results, we proved the convolutional filters’ qualitative significance regardless of magnitude, demonstrated significant neural network redundancy due to repetitive filter functions, and analyzed the filter functionality defection under inappropriate retraining process. Such an interpretable pruning approach not only offers outstanding computation cost optimization over previous filter pruning methods, but also interprets filter pruning process.
Tasks
Published	2018-10-12
URL	https://arxiv.org/abs/1810.07322v2
PDF	https://arxiv.org/pdf/1810.07322v2.pdf
PWC	https://paperswithcode.com/paper/interpretable-convolutional-filter-pruning
Repo
Framework

Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering


Title	Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering
Authors	Vikas Yadav, Rebecca Sharp, Mihai Surdeanu
Abstract	While increasingly complex approaches to question answering (QA) have been proposed, the true gain of these systems, particularly with respect to their expensive training requirements, can be inflated when they are not compared to adequate baselines. Here we propose an unsupervised, simple, and fast alignment and information retrieval baseline that incorporates two novel contributions: a \textit{one-to-many alignment} between query and document terms and \textit{negative alignment} as a proxy for discriminative information. Our approach not only outperforms all conventional baselines as well as many supervised recurrent neural networks, but also approaches the state of the art for supervised systems on three QA datasets. With only three hyperparameters, we achieve 47% P@1 on an 8th grade Science QA dataset, 32.9% P@1 on a Yahoo! answers QA dataset and 64% MAP on WikiQA. We also achieve 26.56% and 58.36% on ARC challenge and easy dataset respectively. In addition to including the additional ARC results in this version of the paper, for the ARC easy set only we also experimented with one additional parameter – number of justifications retrieved.
Tasks	Information Retrieval, Question Answering
Published	2018-07-05
URL	http://arxiv.org/abs/1807.01836v1
PDF	http://arxiv.org/pdf/1807.01836v1.pdf
PWC	https://paperswithcode.com/paper/sanity-check-a-strong-alignment-and
Repo
Framework

A Provably Correct Algorithm for Deep Learning that Actually Works


Title	A Provably Correct Algorithm for Deep Learning that Actually Works
Authors	Eran Malach, Shai Shalev-Shwartz
Abstract	We describe a layer-by-layer algorithm for training deep convolutional networks, where each step involves gradient updates for a two layer network followed by a simple clustering algorithm. Our algorithm stems from a deep generative model that generates mages level by level, where lower resolution images correspond to latent semantic classes. We analyze the convergence rate of our algorithm assuming that the data is indeed generated according to this model (as well as additional assumptions). While we do not pretend to claim that the assumptions are realistic for natural images, we do believe that they capture some true properties of real data. Furthermore, we show that our algorithm actually works in practice (on the CIFAR dataset), achieving results in the same ballpark as that of vanilla convolutional neural networks that are being trained by stochastic gradient descent. Finally, our proof techniques may be of independent interest.
Tasks
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09522v2
PDF	http://arxiv.org/pdf/1803.09522v2.pdf
PWC	https://paperswithcode.com/paper/a-provably-correct-algorithm-for-deep
Repo
Framework

A Solvable High-Dimensional Model of GAN


Title	A Solvable High-Dimensional Model of GAN
Authors	Chuang Wang, Hong Hu, Yue M. Lu
Abstract	We present a theoretical analysis of the training process for a single-layer GAN fed by high-dimensional input data. The training dynamics of the proposed model at both microscopic and macroscopic scales can be exactly analyzed in the high-dimensional limit. In particular, we prove that the macroscopic quantities measuring the quality of the training process converge to a deterministic process characterized by an ordinary differential equation (ODE), whereas the microscopic states containing all the detailed weights remain stochastic, whose dynamics can be described by a stochastic differential equation (SDE). This analysis provides a new perspective different from recent analyses in the limit of small learning rate, where the microscopic state is always considered deterministic, and the contribution of noise is ignored. From our analysis, we show that the level of the background noise is essential to the convergence of the training process: setting the noise level too strong leads to failure of feature recovery, whereas setting the noise too weak causes oscillation. Although this work focuses on a simple copy model of GAN, we believe the analysis methods and insights developed here would prove useful in the theoretical understanding of other variants of GANs with more advanced training algorithms.
Tasks
Published	2018-05-22
URL	https://arxiv.org/abs/1805.08349v2
PDF	https://arxiv.org/pdf/1805.08349v2.pdf
PWC	https://paperswithcode.com/paper/a-solvable-high-dimensional-model-of-gan
Repo
Framework

Connectivity-Driven Parcellation Methods for the Human Cerebral Cortex


Title	Connectivity-Driven Parcellation Methods for the Human Cerebral Cortex
Authors	Salim Arslan
Abstract	In this thesis, we present robust and fully-automated methods for the subdivision of the entire human cerebral cortex based on connectivity information. Our contributions are four-fold: First, we propose a clustering approach to delineate a cortical parcellation that provides a reliable abstraction of the brain’s functional organisation. Second, we cast the parcellation problem as a feature reduction problem and make use of manifold learning and image segmentation techniques to identify cortical regions with distinct structural connectivity patterns. Third, we present a multi-layer graphical model that combines within- and between-subject connectivity, which is then decomposed into a cortical parcellation that can represent the whole population, while accounting for the variability across subjects. Finally, we conduct a large-scale, systematic comparison of existing parcellation methods, with a focus on providing some insight into the reliability of brain parcellations in terms of reflecting the underlying connectivity, as well as, revealing their impact on network analysis. We evaluate the proposed parcellation methods on publicly available data from the Human Connectome Project and a plethora of quantitative and qualitative evaluation techniques investigated in the literature. Experiments across multiple resolutions demonstrate the accuracy of the presented methods at both subject and group levels with regards to reproducibility and fidelity to the data. The neuro-biological interpretation of the proposed parcellations is also investigated by comparing parcel boundaries with well-structured properties of the cerebral cortex. Results show the advantage of connectivity-driven parcellations over traditional approaches in terms of better fitting the underlying connectivity.
Tasks	Semantic Segmentation
Published	2018-02-17
URL	http://arxiv.org/abs/1802.06772v1
PDF	http://arxiv.org/pdf/1802.06772v1.pdf
PWC	https://paperswithcode.com/paper/connectivity-driven-parcellation-methods-for
Repo
Framework

Hierarchical Block Sparse Neural Networks


Title	Hierarchical Block Sparse Neural Networks
Authors	Dharma Teja Vooturi, Dheevatsa Mudigere, Sasikanth Avancha
Abstract	Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies are much lower than that of dense DNNs on regular parallel hardware such as TPU. This inefficiency leads to poor/no performance benefits for sparse DNNs. Performance issue for sparse DNNs can be alleviated by bringing structure to the sparsity and leveraging it for improving runtime efficiency. But such structural constraints often lead to suboptimal accuracies. In this work, we jointly address both accuracy and performance of sparse DNNs using our proposed class of sparse neural networks called HBsNN (Hierarchical Block sparse Neural Networks). For a given sparsity, HBsNN models achieve better runtime performance than unstructured sparse models and better accuracy than highly structured sparse models.
Tasks
Published	2018-08-10
URL	http://arxiv.org/abs/1808.03420v2
PDF	http://arxiv.org/pdf/1808.03420v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-block-sparse-neural-networks
Repo
Framework

TextMountain: Accurate Scene Text Detection via Instance Segmentation


Title	TextMountain: Accurate Scene Text Detection via Instance Segmentation
Authors	Yixing Zhu, Jun Du
Abstract	In this paper, we propose a novel scene text detection method named TextMountain. The key idea of TextMountain is making full use of border-center information. Different from previous works that treat center-border as a binary classification problem, we predict text center-border probability (TCBP) and text center-direction (TCD). The TCBP is just like a mountain whose top is text center and foot is text border. The mountaintop can separate text instances which cannot be easily achieved using semantic segmentation map and its rising direction can plan a road to top for each pixel on mountain foot at the group stage. The TCD helps TCBP learning better. Our label rules will not lead to the ambiguous problem with the transformation of angle, so the proposed method is robust to multi-oriented text and can also handle well with curved text. In inference stage, each pixel at the mountain foot needs to search the path to the mountaintop and this process can be efficiently completed in parallel, yielding the efficiency of our method compared with others. The experiments on MLT, ICDAR2015, RCTW-17 and SCUT-CTW1500 databases demonstrate that the proposed method achieves better or comparable performance in terms of both accuracy and efficiency. It is worth mentioning our method achieves an F-measure of 76.85% on MLT which outperforms the previous methods by a large margin. Code will be made available.
Tasks	Instance Segmentation, Scene Text Detection, Semantic Segmentation
Published	2018-11-30
URL	http://arxiv.org/abs/1811.12786v1
PDF	http://arxiv.org/pdf/1811.12786v1.pdf
PWC	https://paperswithcode.com/paper/textmountain-accurate-scene-text-detection
Repo
Framework

Lexical Bias In Essay Level Prediction


Title	Lexical Bias In Essay Level Prediction
Authors	Georgios Balikas
Abstract	Automatically predicting the level of non-native English speakers given their written essays is an interesting machine learning problem. In this work I present the system “balikasg” that achieved the state-of-the-art performance in the CAp 2018 data science challenge among 14 systems. I detail the feature extraction, feature engineering and model selection steps and I evaluate how these decisions impact the system’s performance. The paper concludes with remarks for future work.
Tasks	Feature Engineering, Model Selection
Published	2018-09-21
URL	http://arxiv.org/abs/1809.08935v1
PDF	http://arxiv.org/pdf/1809.08935v1.pdf
PWC	https://paperswithcode.com/paper/lexical-bias-in-essay-level-prediction
Repo
Framework

Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques


Title	Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques
Authors	Rateb Jabbar, Khalifa Al-Khalifa, Mohamed Kharbeche, Wael Alhajyaseen, Mohsen Jafari, Shan Jiang
Abstract	Road crashes and related forms of accidents are a common cause of injury and death among the human population. According to 2015 data from the World Health Organization, road traffic injuries resulted in approximately 1.25 million deaths worldwide, i.e. approximately every 25 seconds an individual will experience a fatal crash. While the cost of traffic accidents in Europe is estimated at around 160 billion Euros, driver drowsiness accounts for approximately 100,000 accidents per year in the United States alone as reported by The American National Highway Traffic Safety Administration (NHTSA). In this paper, a novel approach towards real-time drowsiness detection is proposed. This approach is based on a deep learning method that can be implemented on Android applications with high accuracy. The main contribution of this work is the compression of heavy baseline model to a lightweight model. Moreover, minimal network structure is designed based on facial landmark key point detection to recognize whether the driver is drowsy. The proposed model is able to achieve an accuracy of more than 80%. Keywords: Driver Monitoring System; Drowsiness Detection; Deep Learning; Real-time Deep Neural Network; Android.
Tasks
Published	2018-11-05
URL	http://arxiv.org/abs/1811.01627v1
PDF	http://arxiv.org/pdf/1811.01627v1.pdf
PWC	https://paperswithcode.com/paper/real-time-driver-drowsiness-detection-for
Repo
Framework

High Dimensional Classification through $\ell_0$-Penalized Empirical Risk Minimization


Title	High Dimensional Classification through $\ell_0$-Penalized Empirical Risk Minimization
Authors	Le-Yu Chen, Sokbae Lee
Abstract	We consider a high dimensional binary classification problem and construct a classification procedure by minimizing the empirical misclassification risk with a penalty on the number of selected features. We derive non-asymptotic probability bounds on the estimated sparsity as well as on the excess misclassification risk. In particular, we show that our method yields a sparse solution whose l0-norm can be arbitrarily close to true sparsity with high probability and obtain the rates of convergence for the excess misclassification risk. The proposed procedure is implemented via the method of mixed integer linear programming. Its numerical performance is illustrated in Monte Carlo experiments.
Tasks
Published	2018-11-23
URL	http://arxiv.org/abs/1811.09540v1
PDF	http://arxiv.org/pdf/1811.09540v1.pdf
PWC	https://paperswithcode.com/paper/high-dimensional-classification-through-ell_0
Repo
Framework

Bounding Bloat in Genetic Programming


Title	Bounding Bloat in Genetic Programming
Authors	Benjamin Doerr, Timo Kötzing, J. A. Gregor Lagodzinski, Johannes Lengler
Abstract	While many optimization problems work with a fixed number of decision variables and thus a fixed-length representation of possible solutions, genetic programming (GP) works on variable-length representations. A naturally occurring problem is that of bloat (unnecessary growth of solutions) slowing down optimization. Theoretical analyses could so far not bound bloat and required explicit assumptions on the magnitude of bloat. In this paper we analyze bloat in mutation-based genetic programming for the two test functions ORDER and MAJORITY. We overcome previous assumptions on the magnitude of bloat and give matching or close-to-matching upper and lower bounds for the expected optimization time. In particular, we show that the (1+1) GP takes (i) $\Theta(T_{init} + n \log n)$ iterations with bloat control on ORDER as well as MAJORITY; and (ii) $O(T_{init} \log T_{init} + n (\log n)^3)$ and $\Omega(T_{init} + n \log n)$ (and $\Omega(T_{init} \log T_{init})$ for $n=1$) iterations without bloat control on MAJORITY.
Tasks
Published	2018-06-06
URL	http://arxiv.org/abs/1806.02112v1
PDF	http://arxiv.org/pdf/1806.02112v1.pdf
PWC	https://paperswithcode.com/paper/bounding-bloat-in-genetic-programming
Repo
Framework

Linear density-based clustering with a discrete density model


Title	Linear density-based clustering with a discrete density model
Authors	Roberto Pirrone, Vincenzo Cannella, Sergio Monteleone, Gabriella Giordano
Abstract	Density-based clustering techniques are used in a wide range of data mining applications. One of their most attractive features con- sists in not making use of prior knowledge of the number of clusters that a dataset contains along with their shape. In this paper we propose a new algorithm named Linear DBSCAN (Lin-DBSCAN), a simple approach to clustering inspired by the density model introduced with the well known algorithm DBSCAN. Designed to minimize the computational cost of density based clustering on geospatial data, Lin-DBSCAN features a linear time complexity that makes it suitable for real-time applications on low-resource devices. Lin-DBSCAN uses a discrete version of the density model of DBSCAN that takes ad- vantage of a grid-based scan and merge approach. The name of the algorithm stems exactly from its main features outlined above. The algorithm was tested with well known data sets. Experimental results prove the efficiency and the validity of this approach over DBSCAN in the context of spatial data clustering, enabling the use of a density-based clustering technique on large datasets with low computational cost.
Tasks
Published	2018-07-21
URL	http://arxiv.org/abs/1807.08158v1
PDF	http://arxiv.org/pdf/1807.08158v1.pdf
PWC	https://paperswithcode.com/paper/linear-density-based-clustering-with-a
Repo
Framework

A Memory Network Approach for Story-based Temporal Summarization of 360° Videos


Title	A Memory Network Approach for Story-based Temporal Summarization of 360° Videos
Authors	Sangho Lee, Jinyoung Sung, Youngjae Yu, Gunhee Kim
Abstract	We address the problem of story-based temporal summarization of long 360{\deg} videos. We propose a novel memory network model named Past-Future Memory Network (PFMN), in which we first compute the scores of 81 normal field of view (NFOV) region proposals cropped from the input 360{\deg} video, and then recover a latent, collective summary using the network with two external memories that store the embeddings of previously selected subshots and future candidate subshots. Our major contributions are two-fold. First, our work is the first to address story-based temporal summarization of 360{\deg} videos. Second, our model is the first attempt to leverage memory networks for video summarization tasks. For evaluation, we perform three sets of experiments. First, we investigate the view selection capability of our model on the Pano2Vid dataset. Second, we evaluate the temporal summarization with a newly collected 360{\deg} video dataset. Finally, we experiment our model’s performance in another domain, with image-based storytelling VIST dataset. We verify that our model achieves state-of-the-art performance on all the tasks.
Tasks	Video Summarization
Published	2018-05-08
URL	http://arxiv.org/abs/1805.02838v3
PDF	http://arxiv.org/pdf/1805.02838v3.pdf
PWC	https://paperswithcode.com/paper/a-memory-network-approach-for-story-based
Repo
Framework