October 18, 2019

3346 words 16 mins read

Paper Group ANR 625

Fruit Quantity and Quality Estimation using a Robotic Vision System. Attention-based Recurrent Neural Network for Urban Vehicle Trajectory Prediction. Disturbance Grassmann Kernels for Subspace-Based Learning. Mathematics as information compression via the matching and unification of patterns. Pattern Analysis with Layered Self-Organizing Maps. SWA …

Fruit Quantity and Quality Estimation using a Robotic Vision System


Title	Fruit Quantity and Quality Estimation using a Robotic Vision System
Authors	M. Halstead, C. McCool, S. Denman, T. Perez, C. Fookes
Abstract	Accurate localisation of crop remains highly challenging in unstructured environments such as farms. Many of the developed systems still rely on the use of hand selected features for crop identification and often neglect the estimation of crop quantity and quality, which is key to assigning labor during farming processes. To alleviate these limitations we present a robotic vision system that can accurately estimate the quantity and quality of sweet pepper (Capsicum annuum L), a key horticultural crop. This system consists of three parts: detection, quality estimation, and tracking. Efficient detection is achieved using the FasterRCNN framework. Quality is then estimated in the same framework by learning a parallel layer which we show experimentally results in superior performance than treating quality as extra classes in the traditional Faster-RCNN framework. Evaluation of these two techniques outlines the improved performance of the parallel layer, where we achieve an F1 score of 77.3 for the parallel technique yet only 72.5 for the best scoring (red) of the multi-class implementation. To track the crop we present a tracking via detection approach, which uses the FasterRCNN with parallel layers, that is also a vision-only solution. This approach is cheap to implement as it only requires a camera and in experiments across 2 days we show that our proposed system can accurately estimate the number of sweet pepper present, within 4.1% of the ground truth.
Tasks
Published	2018-01-17
URL	http://arxiv.org/abs/1801.05560v1
PDF	http://arxiv.org/pdf/1801.05560v1.pdf
PWC	https://paperswithcode.com/paper/fruit-quantity-and-quality-estimation-using-a
Repo
Framework

Attention-based Recurrent Neural Network for Urban Vehicle Trajectory Prediction


Title	Attention-based Recurrent Neural Network for Urban Vehicle Trajectory Prediction
Authors	Seongjin Choi, Jiwon Kim, Hwasoo Yeo
Abstract	With the increasing deployment of diverse positioning devices and location-based services, a huge amount of spatial and temporal information has been collected and accumulated as trajectory data. Among many applications, trajectory-based location prediction is gaining increasing attention because of its potential to improve the performance of many applications in multiple domains. This research focuses on trajectory sequence prediction methods using trajectory data obtained from the vehicles in urban traffic network. As Recurrent Neural Network(RNN) model is previously proposed, we propose an improved method of Attention-based Recurrent Neural Network model(ARNN) for urban vehicle trajectory prediction. We introduce attention mechanism into urban vehicle trajectory prediction to explain the impact of network-level traffic state information. The model is evaluated using the Bluetooth data of private vehicles collected in Brisbane, Australia with 5 metrics which are widely used in the sequence modeling. The proposed ARNN model shows significant performance improvement compared to the existing RNN models considering not only the cells to be visited but also the alignment of the cells in sequence.
Tasks	Trajectory Prediction
Published	2018-12-18
URL	https://arxiv.org/abs/1812.07151v2
PDF	https://arxiv.org/pdf/1812.07151v2.pdf
PWC	https://paperswithcode.com/paper/attention-based-recurrent-neural-network-for
Repo
Framework

Disturbance Grassmann Kernels for Subspace-Based Learning


Title	Disturbance Grassmann Kernels for Subspace-Based Learning
Authors	Junyuan Hong, Huanhuan Chen, Feng Lin
Abstract	In this paper, we focus on subspace-based learning problems, where data elements are linear subspaces instead of vectors. To handle this kind of data, Grassmann kernels were proposed to measure the space structure and used with classifiers, e.g., Support Vector Machines (SVMs). However, the existing discriminative algorithms mostly ignore the instability of subspaces, which would cause the classifiers misled by disturbed instances. Thus we propose considering all potential disturbance of subspaces in learning processes to obtain more robust classifiers. Firstly, we derive the dual optimization of linear classifiers with disturbance subject to a known distribution, resulting in a new kernel, Disturbance Grassmann (DG) kernel. Secondly, we research into two kinds of disturbance, relevant to the subspace matrix and singular values of bases, with which we extend the Projection kernel on Grassmann manifolds to two new kernels. Experiments on action data indicate that the proposed kernels perform better compared to state-of-the-art subspace-based methods, even in a worse environment.
Tasks
Published	2018-02-10
URL	http://arxiv.org/abs/1802.03517v2
PDF	http://arxiv.org/pdf/1802.03517v2.pdf
PWC	https://paperswithcode.com/paper/disturbance-grassmann-kernels-for-subspace
Repo
Framework

Mathematics as information compression via the matching and unification of patterns


Title	Mathematics as information compression via the matching and unification of patterns
Authors	J Gerard Wolff
Abstract	This paper describes a novel perspective on the foundations of mathematics: how mathematics may be seen to be largely about ‘information compression via the matching and unification of patterns’ (ICMUP). ICMUP is itself a novel approach to information compression, couched in terms of non-mathematical primitives, as is necessary in any investigation of the foundations of mathematics. This new perspective on the foundations of mathematics has grown out of an extensive programme of research developing the “SP Theory of Intelligence” and its realisation in the “SP Computer Model”, a system in which a generalised version of ICMUP – the powerful concept of SP-multiple-alignment – plays a central role. These ideas may be seen to be part of a “Big Picture” comprising six areas of interest, with information compression as a unifying theme. The paper describes the close relation between mathematics and information compression, and describes examples showing how variants of ICMUP may be seen in widely-used structures and operations in mathematics. Examples are also given to show how the mathematics-related disciplines of logic and computing may be understood as ICMUP. There are many potential benefits and applications of these ideas.
Tasks
Published	2018-08-05
URL	http://arxiv.org/abs/1808.07004v2
PDF	http://arxiv.org/pdf/1808.07004v2.pdf
PWC	https://paperswithcode.com/paper/mathematics-as-information-compression-via
Repo
Framework

Pattern Analysis with Layered Self-Organizing Maps


Title	Pattern Analysis with Layered Self-Organizing Maps
Authors	David Friedlander
Abstract	This paper defines a new learning architecture, Layered Self-Organizing Maps (LSOMs), that uses the SOM and supervised-SOM learning algorithms. The architecture is validated with the MNIST database of hand-written digit images. LSOMs are similar to convolutional neural nets (covnets) in the way they sample data, but different in the way they represent features and learn. LSOMs analyze (or generate) image patches with maps of exemplars determined by the SOM learning algorithm rather than feature maps from filter-banks learned via backprop. LSOMs provide an alternative to features derived from covnets. Multi-layer LSOMs are trained bottom-up, without the use of backprop and therefore may be of interest as a model of the visual cortex. The results show organization at multiple levels. The algorithm appears to be resource efficient in learning, classifying and generating images. Although LSOMs can be used for classification, their validation accuracy for these exploratory runs was well below the state of the art. The goal of this article is to define the architecture and display the structures resulting from its application to the MNIST images.
Tasks
Published	2018-03-23
URL	http://arxiv.org/abs/1803.08996v2
PDF	http://arxiv.org/pdf/1803.08996v2.pdf
PWC	https://paperswithcode.com/paper/pattern-analysis-with-layered-self-organizing
Repo
Framework

SWAT: A System for Detecting Salient Wikipedia Entities in Texts


Title	SWAT: A System for Detecting Salient Wikipedia Entities in Texts
Authors	Marco Ponza, Paolo Ferragina, Francesco Piccinno
Abstract	We study the problem of entity salience by proposing the design and implementation of SWAT, a system that identifies the salient Wikipedia entities occurring in an input document. SWAT consists of several modules that are able to detect and classify on-the-fly Wikipedia entities as salient or not, based on a large number of syntactic, semantic and latent features properly extracted via a supervised process which has been trained over millions of examples drawn from the New York Times corpus. The validation process is performed through a large experimental assessment, eventually showing that SWAT improves known solutions over all publicly available datasets. We release SWAT via an API that we describe and comment in the paper in order to ease its use in other software.
Tasks
Published	2018-04-10
URL	https://arxiv.org/abs/1804.03580v2
PDF	https://arxiv.org/pdf/1804.03580v2.pdf
PWC	https://paperswithcode.com/paper/swat-a-system-for-detecting-salient-wikipedia
Repo
Framework

Differential Equations for Modeling Asynchronous Algorithms


Title	Differential Equations for Modeling Asynchronous Algorithms
Authors	Li He, Qi Meng, Wei Chen, Zhi-Ming Ma, Tie-Yan Liu
Abstract	Asynchronous stochastic gradient descent (ASGD) is a popular parallel optimization algorithm in machine learning. Most theoretical analysis on ASGD take a discrete view and prove upper bounds for their convergence rates. However, the discrete view has its intrinsic limitations: there is no characterization of the optimization path and the proof techniques are induction-based and thus usually complicated. Inspired by the recent successful adoptions of stochastic differential equations (SDE) to the theoretical analysis of SGD, in this paper, we study the continuous approximation of ASGD by using stochastic differential delay equations (SDDE). We introduce the approximation method and study the approximation error. Then we conduct theoretical analysis on the convergence rates of ASGD algorithm based on the continuous approximation. There are two methods: moment estimation and energy function minimization can be used to analyze the convergence rates. Moment estimation depends on the specific form of the loss function, while energy function minimization only leverages the convex property of the loss function, and does not depend on its specific form. In addition to the convergence analysis, the continuous view also helps us derive better convergence rates. All of this clearly shows the advantage of taking the continuous view in gradient descent algorithms.
Tasks
Published	2018-05-08
URL	http://arxiv.org/abs/1805.02991v1
PDF	http://arxiv.org/pdf/1805.02991v1.pdf
PWC	https://paperswithcode.com/paper/differential-equations-for-modeling
Repo
Framework

A Resource-Light Method for Cross-Lingual Semantic Textual Similarity


Title	A Resource-Light Method for Cross-Lingual Semantic Textual Similarity
Authors	Goran Glavaš, Marc Franco-Salvador, Simone Paolo Ponzetto, Paolo Rosso
Abstract	Recognizing semantically similar sentences or paragraphs across languages is beneficial for many tasks, ranging from cross-lingual information retrieval and plagiarism detection to machine translation. Recently proposed methods for predicting cross-lingual semantic similarity of short texts, however, make use of tools and resources (e.g., machine translation systems, syntactic parsers or named entity recognition) that for many languages (or language pairs) do not exist. In contrast, we propose an unsupervised and a very resource-light approach for measuring semantic similarity between texts in different languages. To operate in the bilingual (or multilingual) space, we project continuous word vectors (i.e., word embeddings) from one language to the vector space of the other language via the linear translation model. We then align words according to the similarity of their vectors in the bilingual embedding space and investigate different unsupervised measures of semantic similarity exploiting bilingual embeddings and word alignments. Requiring only a limited-size set of word translation pairs between the languages, the proposed approach is applicable to virtually any pair of languages for which there exists a sufficiently large corpus, required to learn monolingual word embeddings. Experimental results on three different datasets for measuring semantic textual similarity show that our simple resource-light approach reaches performance close to that of supervised and resource intensive methods, displaying stability across different language pairs. Furthermore, we evaluate the proposed method on two extrinsic tasks, namely extraction of parallel sentences from comparable corpora and cross lingual plagiarism detection, and show that it yields performance comparable to those of complex resource-intensive state-of-the-art models for the respective tasks.
Tasks	Cross-Lingual Semantic Textual Similarity, Information Retrieval, Machine Translation, Named Entity Recognition, Semantic Similarity, Semantic Textual Similarity, Word Embeddings
Published	2018-01-19
URL	http://arxiv.org/abs/1801.06436v1
PDF	http://arxiv.org/pdf/1801.06436v1.pdf
PWC	https://paperswithcode.com/paper/a-resource-light-method-for-cross-lingual
Repo
Framework

Size vs. Structure in Training Corpora for Word Embedding Models: Araneum Russicum Maximum and Russian National Corpus


Title	Size vs. Structure in Training Corpora for Word Embedding Models: Araneum Russicum Maximum and Russian National Corpus
Authors	Andrey Kutuzov, Maria Kunilovskaya
Abstract	In this paper, we present a distributional word embedding model trained on one of the largest available Russian corpora: Araneum Russicum Maximum (over 10 billion words crawled from the web). We compare this model to the model trained on the Russian National Corpus (RNC). The two corpora are much different in their size and compilation procedures. We test these differences by evaluating the trained models against the Russian part of the Multilingual SimLex999 semantic similarity dataset. We detect and describe numerous issues in this dataset and publish a new corrected version. Aside from the already known fact that the RNC is generally a better training corpus than web corpora, we enumerate and explain fine differences in how the models process semantic similarity task, what parts of the evaluation set are difficult for particular models and why. Additionally, the learning curves for both models are described, showing that the RNC is generally more robust as training material for this task.
Tasks	Semantic Similarity, Semantic Textual Similarity
Published	2018-01-19
URL	http://arxiv.org/abs/1801.06407v1
PDF	http://arxiv.org/pdf/1801.06407v1.pdf
PWC	https://paperswithcode.com/paper/size-vs-structure-in-training-corpora-for
Repo
Framework

The Complex Event Recognition Group


Title	The Complex Event Recognition Group
Authors	Elias Alevizos, Alexander Artikis, Nikos Katzouris, Evangelos Michelioudakis, Georgios Paliouras
Abstract	The Complex Event Recognition (CER) group is a research team, affiliated with the National Centre of Scientific Research “Demokritos” in Greece. The CER group works towards advanced and efficient methods for the recognition of complex events in a multitude of large, heterogeneous and interdependent data streams. Its research covers multiple aspects of complex event recognition, from efficient detection of patterns on event streams to handling uncertainty and noise in streams, and machine learning techniques for inferring interesting patterns. Lately, it has expanded to methods for forecasting the occurrence of events. It was founded in 2009 and currently hosts 3 senior researchers, 5 PhD students and works regularly with under-graduate students.
Tasks
Published	2018-02-12
URL	http://arxiv.org/abs/1802.04086v1
PDF	http://arxiv.org/pdf/1802.04086v1.pdf
PWC	https://paperswithcode.com/paper/the-complex-event-recognition-group
Repo
Framework

Knots in random neural networks


Title	Knots in random neural networks
Authors	Kevin K. Chen, Anthony C. Gamst, Alden K. Walker
Abstract	The weights of a neural network are typically initialized at random, and one can think of the functions produced by such a network as having been generated by a prior over some function space. Studying random networks, then, is useful for a Bayesian understanding of the network evolution in early stages of training. In particular, one can investigate why neural networks with huge numbers of parameters do not immediately overfit. We analyze the properties of random scalar-input feed-forward rectified linear unit architectures, which are random linear splines. With weights and biases sampled from certain common distributions, empirical tests show that the number of knots in the spline produced by the network is equal to the number of neurons, to very close approximation. We describe our progress towards a completely analytic explanation of this phenomenon. In particular, we show that random single-layer neural networks are equivalent to integrated random walks with variable step sizes. That each neuron produces one knot on average is equivalent to the associated integrated random walk having one zero crossing on average. We explore how properties of the integrated random walk, including the step sizes and initial conditions, affect the number of crossings. The number of knots in random neural networks can be related to the behavior of extreme learning machines, but it also establishes a prior preventing optimizers from immediately overfitting to noisy training data.
Tasks
Published	2018-11-27
URL	http://arxiv.org/abs/1811.11152v1
PDF	http://arxiv.org/pdf/1811.11152v1.pdf
PWC	https://paperswithcode.com/paper/knots-in-random-neural-networks
Repo
Framework

Toward Understanding the Impact of Staleness in Distributed Machine Learning


Title	Toward Understanding the Impact of Staleness in Distributed Machine Learning
Authors	Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric P. Xing
Abstract	Many distributed machine learning (ML) systems adopt the non-synchronous execution in order to alleviate the network communication bottleneck, resulting in stale parameters that do not reflect the latest updates. Despite much development in large-scale ML, the effects of staleness on learning are inconclusive as it is challenging to directly monitor or control staleness in complex distributed environments. In this work, we study the convergence behaviors of a wide array of ML models and algorithms under delayed updates. Our extensive experiments reveal the rich diversity of the effects of staleness on the convergence of ML algorithms and offer insights into seemingly contradictory reports in the literature. The empirical findings also inspire a new convergence analysis of stochastic gradient descent in non-convex optimization under staleness, matching the best-known convergence rate of O(1/\sqrt{T}).
Tasks
Published	2018-10-08
URL	http://arxiv.org/abs/1810.03264v1
PDF	http://arxiv.org/pdf/1810.03264v1.pdf
PWC	https://paperswithcode.com/paper/toward-understanding-the-impact-of-staleness
Repo
Framework

Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks


Title	Fully automatic detection and segmentation of abdominal aortic thrombus in post-operative CTA images using deep convolutional neural networks
Authors	Karen López-Linares, Nerea Aranjuelo, Luis Kabongo, Gregory Maclair, Nerea Lete, Mario Ceresa, Ainhoa García-Familiar, Iván Macía, Miguel A. González Ballester
Abstract	Computerized Tomography Angiography (CTA) based follow-up of Abdominal Aortic Aneurysms (AAA) treated with Endovascular Aneurysm Repair (EVAR) is essential to evaluate the progress of the patient and detect complications. In this context, accurate quantification of post-operative thrombus volume is required. However, a proper evaluation is hindered by the lack of automatic, robust and reproducible thrombus segmentation algorithms. We propose a new fully automatic approach based on Deep Convolutional Neural Networks (DCNN) for robust and reproducible thrombus region of interest detection and subsequent fine thrombus segmentation. The DetecNet detection network is adapted to perform region of interest extraction from a complete CTA and a new segmentation network architecture, based on Fully Convolutional Networks and a Holistically-Nested Edge Detection Network, is presented. These networks are trained, validated and tested in 13 post-operative CTA volumes of different patients using a 4-fold cross-validation approach to provide more robustness to the results. Our pipeline achieves a Dice score of more than 82% for post-operative thrombus segmentation and provides a mean relative volume difference between ground truth and automatic segmentation that lays within the experienced human observer variance without the need of human intervention in most common cases.
Tasks	Edge Detection
Published	2018-04-01
URL	http://arxiv.org/abs/1804.00304v1
PDF	http://arxiv.org/pdf/1804.00304v1.pdf
PWC	https://paperswithcode.com/paper/fully-automatic-detection-and-segmentation-of
Repo
Framework

Deep Crisp Boundaries: From Boundaries to Higher-level Tasks


Title	Deep Crisp Boundaries: From Boundaries to Higher-level Tasks
Authors	Yupei Wang, Xin Zhao, Yin Li, Kaiqi Huang
Abstract	Edge detection has made significant progress with the help of deep Convolutional Networks (ConvNet). These ConvNet based edge detectors have approached human level performance on standard benchmarks. We provide a systematical study of these detectors’ outputs. We show that the detection results did not accurately localize edge pixels, which can be adversarial for tasks that require crisp edge inputs. As a remedy, we propose a novel refinement architecture to address the challenging problem of learning a crisp edge detector using ConvNet. Our method leverages a top-down backward refinement pathway, and progressively increases the resolution of feature maps to generate crisp edges. Our results achieve superior performance, surpassing human accuracy when using standard criteria on BSDS500, and largely outperforming state-of-the-art methods when using more strict criteria. More importantly, we demonstrate the benefit of crisp edge maps for several important applications in computer vision, including optical flow estimation, object proposal generation and semantic segmentation.
Tasks	Edge Detection, Object Proposal Generation, Optical Flow Estimation, Semantic Segmentation
Published	2018-01-08
URL	http://arxiv.org/abs/1801.02439v3
PDF	http://arxiv.org/pdf/1801.02439v3.pdf
PWC	https://paperswithcode.com/paper/deep-crisp-boundaries-from-boundaries-to
Repo
Framework

Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization


Title	Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization
Authors	Jonas Kohler, Hadi Daneshmand, Aurelien Lucchi, Ming Zhou, Klaus Neymeyr, Thomas Hofmann
Abstract	Normalization techniques such as Batch Normalization have been applied successfully for training deep neural networks. Yet, despite its apparent empirical benefits, the reasons behind the success of Batch Normalization are mostly hypothetical. We here aim to provide a more thorough theoretical understanding from a classical optimization perspective. Our main contribution towards this goal is the identification of various problem instances in the realm of machine learning where % – under certain assumptions– Batch Normalization can provably accelerate optimization. We argue that this acceleration is due to the fact that Batch Normalization splits the optimization task into optimizing length and direction of the parameters separately. This allows gradient-based methods to leverage a favourable global structure in the loss landscape that we prove to exist in Learning Halfspace problems and neural network training with Gaussian inputs. We thereby turn Batch Normalization from an effective practical heuristic into a provably converging algorithm for these settings. Furthermore, we substantiate our analysis with empirical evidence that suggests the validity of our theoretical results in a broader context.
Tasks
Published	2018-05-27
URL	http://arxiv.org/abs/1805.10694v3
PDF	http://arxiv.org/pdf/1805.10694v3.pdf
PWC	https://paperswithcode.com/paper/exponential-convergence-rates-for-batch
Repo
Framework