July 27, 2019

2995 words 15 mins read

Paper Group ANR 571

A Survey of Structure from Motion. From Review to Rating: Exploring Dependency Measures for Text Classification. Houdini: Fooling Deep Structured Prediction Models. Exponential error rates of SDP for block models: Beyond Grothendieck’s inequality. Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast …

A Survey of Structure from Motion


Title	A Survey of Structure from Motion
Authors	Onur Ozyesil, Vladislav Voroninski, Ronen Basri, Amit Singer
Abstract	The structure from motion (SfM) problem in computer vision is the problem of recovering the three-dimensional ($3$D) structure of a stationary scene from a set of projective measurements, represented as a collection of two-dimensional ($2$D) images, via estimation of motion of the cameras corresponding to these images. In essence, SfM involves the three main stages of (1) extraction of features in images (e.g., points of interest, lines, etc.) and matching these features between images, (2) camera motion estimation (e.g., using relative pairwise camera positions estimated from the extracted features), and (3) recovery of the $3$D structure using the estimated motion and features (e.g., by minimizing the so-called reprojection error). This survey mainly focuses on relatively recent developments in the literature pertaining to stages (2) and (3). More specifically, after touching upon the early factorization-based techniques for motion and structure estimation, we provide a detailed account of some of the recent camera location estimation methods in the literature, followed by discussion of notable techniques for $3$D structure recovery. We also cover the basics of the simultaneous localization and mapping (SLAM) problem, which can be viewed as a specific case of the SfM problem. Further, our survey includes a review of the fundamentals of feature extraction and matching (i.e., stage (1) above), various recent methods for handling ambiguities in $3$D scenes, SfM techniques involving relatively uncommon camera models and image features, and popular sources of data and SfM software.
Tasks	Motion Estimation, Simultaneous Localization and Mapping
Published	2017-01-30
URL	http://arxiv.org/abs/1701.08493v2
PDF	http://arxiv.org/pdf/1701.08493v2.pdf
PWC	https://paperswithcode.com/paper/a-survey-of-structure-from-motion
Repo
Framework

From Review to Rating: Exploring Dependency Measures for Text Classification


Title	From Review to Rating: Exploring Dependency Measures for Text Classification
Authors	Samuel Cunningham-Nelson, Mahsa Baktashmotlagh, Wageeh Boles
Abstract	Various text analysis techniques exist, which attempt to uncover unstructured information from text. In this work, we explore using statistical dependence measures for textual classification, representing text as word vectors. Student satisfaction scores on a 3-point scale and their free text comments written about university subjects are used as the dataset. We have compared two textual representations: a frequency word representation and term frequency relationship to word vectors, and found that word vectors provide a greater accuracy. However, these word vectors have a large number of features which aggravates the burden of computational complexity. Thus, we explored using a non-linear dependency measure for feature selection by maximizing the dependence between the text reviews and corresponding scores. Our quantitative and qualitative analysis on a student satisfaction dataset shows that our approach achieves comparable accuracy to the full feature vector, while being an order of magnitude faster in testing. These text analysis and feature reduction techniques can be used for other textual data applications such as sentiment analysis.
Tasks	Feature Selection, Sentiment Analysis, Text Classification
Published	2017-09-04
URL	http://arxiv.org/abs/1709.00813v1
PDF	http://arxiv.org/pdf/1709.00813v1.pdf
PWC	https://paperswithcode.com/paper/from-review-to-rating-exploring-dependency
Repo
Framework

Houdini: Fooling Deep Structured Prediction Models


Title	Houdini: Fooling Deep Structured Prediction Models
Authors	Moustapha Cisse, Yossi Adi, Natalia Neverova, Joseph Keshet
Abstract	Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.
Tasks	Pose Estimation, Semantic Segmentation, Speech Recognition, Structured Prediction
Published	2017-07-17
URL	http://arxiv.org/abs/1707.05373v1
PDF	http://arxiv.org/pdf/1707.05373v1.pdf
PWC	https://paperswithcode.com/paper/houdini-fooling-deep-structured-prediction
Repo
Framework

Exponential error rates of SDP for block models: Beyond Grothendieck’s inequality


Title	Exponential error rates of SDP for block models: Beyond Grothendieck’s inequality
Authors	Yingjie Fei, Yudong Chen
Abstract	In this paper we consider the cluster estimation problem under the Stochastic Block Model. We show that the semidefinite programming (SDP) formulation for this problem achieves an error rate that decays exponentially in the signal-to-noise ratio. The error bound implies weak recovery in the sparse graph regime with bounded expected degrees, as well as exact recovery in the dense regime. An immediate corollary of our results yields error bounds under the Censored Block Model. Moreover, these error bounds are robust, continuing to hold under heterogeneous edge probabilities and a form of the so-called monotone attack. Significantly, this error rate is achieved by the SDP solution itself without any further pre- or post-processing, and improves upon existing polynomially-decaying error bounds proved using the Grothendieck\textquoteright s inequality. Our analysis has two key ingredients: (i) showing that the graph has a well-behaved spectrum, even in the sparse regime, after discounting an exponentially small number of edges, and (ii) an order-statistics argument that governs the final error rate. Both arguments highlight the implicit regularization effect of the SDP formulation.
Tasks
Published	2017-05-23
URL	http://arxiv.org/abs/1705.08391v1
PDF	http://arxiv.org/pdf/1705.08391v1.pdf
PWC	https://paperswithcode.com/paper/exponential-error-rates-of-sdp-for-block
Repo
Framework

Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images


Title	Joint Weakly and Semi-Supervised Deep Learning for Localization and Classification of Masses in Breast Ultrasound Images
Authors	Seung Yeon Shin, Soochahn Lee, Il Dong Yun, Sun Mi Kim, Kyoung Mu Lee
Abstract	We propose a framework for localization and classification of masses in breast ultrasound (BUS) images. We have experimentally found that training convolutional neural network based mass detectors with large, weakly annotated datasets presents a non-trivial problem, while overfitting may occur with those trained with small, strongly annotated datasets. To overcome these problems, we use a weakly annotated dataset together with a smaller strongly annotated dataset in a hybrid manner. We propose a systematic weakly and semi-supervised training scenario with appropriate training loss selection. Experimental results show that the proposed method can successfully localize and classify masses with less annotation effort. The results trained with only 10 strongly annotated images along with weakly annotated images were comparable to results trained from 800 strongly annotated images, with the 95% confidence interval of difference -3.00%–5.00%, in terms of the correct localization (CorLoc) measure, which is the ratio of images with intersection over union with ground truth higher than 0.5. With the same number of strongly annotated images, additional weakly annotated images can be incorporated to give a 4.5% point increase in CorLoc, from 80.00% to 84.50% (with 95% confidence intervals 76.00%–83.75% and 81.00%–88.00%). The effects of different algorithmic details and varied amount of data are presented through ablative analysis.
Tasks
Published	2017-10-10
URL	http://arxiv.org/abs/1710.03778v2
PDF	http://arxiv.org/pdf/1710.03778v2.pdf
PWC	https://paperswithcode.com/paper/joint-weakly-and-semi-supervised-deep
Repo
Framework

Exploring the Function Space of Deep-Learning Machines


Title	Exploring the Function Space of Deep-Learning Machines
Authors	Bo Li, David Saad
Abstract	The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely-connected architectures to discover a layer-wise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.
Tasks
Published	2017-08-04
URL	http://arxiv.org/abs/1708.01422v3
PDF	http://arxiv.org/pdf/1708.01422v3.pdf
PWC	https://paperswithcode.com/paper/exploring-the-function-space-of-deep-learning
Repo
Framework

Distributed Representation of Subgraphs


Title	Distributed Representation of Subgraphs
Authors	Bijaya Adhikari, Yao Zhang, Naren Ramakrishnan, B. Aditya Prakash
Abstract	Network embeddings have become very popular in learning effective feature representations of networks. Motivated by the recent successes of embeddings in natural language processing, researchers have tried to find network embeddings in order to exploit machine learning algorithms for mining tasks like node classification and edge prediction. However, most of the work focuses on finding distributed representations of nodes, which are inherently ill-suited to tasks such as community detection which are intuitively dependent on subgraphs. Here, we propose sub2vec, an unsupervised scalable algorithm to learn feature representations of arbitrary subgraphs. We provide means to characterize similarties between subgraphs and provide theoretical analysis of sub2vec and demonstrate that it preserves the so-called local proximity. We also highlight the usability of sub2vec by leveraging it for network mining tasks, like community detection. We show that sub2vec gets significant gains over state-of-the-art methods and node-embedding methods. In particular, sub2vec offers an approach to generate a richer vocabulary of features of subgraphs to support representation and reasoning.
Tasks	Community Detection, Node Classification
Published	2017-02-22
URL	http://arxiv.org/abs/1702.06921v1
PDF	http://arxiv.org/pdf/1702.06921v1.pdf
PWC	https://paperswithcode.com/paper/distributed-representation-of-subgraphs
Repo
Framework

On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent


Title	On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
Authors	Xingwen Zhang, Jeff Clune, Kenneth O. Stanley
Abstract	Because stochastic gradient descent (SGD) has shown promise optimizing neural networks with millions of parameters and few if any alternatives are known to exist, it has moved to the heart of leading approaches to reinforcement learning (RL). For that reason, the recent result from OpenAI showing that a particular kind of evolution strategy (ES) can rival the performance of SGD-based deep RL methods with large neural networks provoked surprise. This result is difficult to interpret in part because of the lingering ambiguity on how ES actually relates to SGD. The aim of this paper is to significantly reduce this ambiguity through a series of MNIST-based experiments designed to uncover their relationship. As a simple supervised problem without domain noise (unlike in most RL), MNIST makes it possible (1) to measure the correlation between gradients computed by ES and SGD and (2) then to develop an SGD-based proxy that accurately predicts the performance of different ES population sizes. These innovations give a new level of insight into the real capabilities of ES, and lead also to some unconventional means for applying ES to supervised problems that shed further light on its differences from SGD. Incorporating these lessons, the paper concludes by demonstrating that ES can achieve 99% accuracy on MNIST, a number higher than any previously published result for any evolutionary method. While not by any means suggesting that ES should substitute for SGD in supervised learning, the suite of experiments herein enables more informed decisions on the application of ES within RL and other paradigms.
Tasks
Published	2017-12-18
URL	http://arxiv.org/abs/1712.06564v1
PDF	http://arxiv.org/pdf/1712.06564v1.pdf
PWC	https://paperswithcode.com/paper/on-the-relationship-between-the-openai
Repo
Framework

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations


Title	Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations
Authors	Hongjia Li, Tianshu Wei, Ao Ren, Qi Zhu, Yanzhi Wang
Abstract	The recent breakthroughs of deep reinforcement learning (DRL) technique in Alpha Go and playing Atari have set a good example in handling large state and actions spaces of complicated control problems. The DRL technique is comprised of (i) an offline deep neural network (DNN) construction phase, which derives the correlation between each state-action pair of the system and its value function, and (ii) an online deep Q-learning phase, which adaptively derives the optimal action and updates value estimates. In this paper, we first present the general DRL framework, which can be widely utilized in many applications with different optimization objectives. This is followed by the introduction of three specific applications: the cloud computing resource allocation problem, the residential smart grid task scheduling problem, and building HVAC system optimal control problem. The effectiveness of the DRL technique in these three cyber-physical applications have been validated. Finally, this paper investigates the stochastic computing-based hardware implementations of the DRL framework, which consumes a significant improvement in area efficiency and power consumption compared with binary-based implementation counterparts.
Tasks	Q-Learning
Published	2017-10-10
URL	http://arxiv.org/abs/1710.03792v1
PDF	http://arxiv.org/pdf/1710.03792v1.pdf
PWC	https://paperswithcode.com/paper/deep-reinforcement-learning-framework
Repo
Framework

Seismic facies recognition based on prestack data using deep convolutional autoencoder


Title	Seismic facies recognition based on prestack data using deep convolutional autoencoder
Authors	Feng Qian, Miao Yin, Ming-Jun Su, Yaojun Wang, Guangmin Hu
Abstract	Prestack seismic data carries much useful information that can help us find more complex atypical reservoirs. Therefore, we are increasingly inclined to use prestack seismic data for seis- mic facies recognition. However, due to the inclusion of ex- cessive redundancy, effective feature extraction from prestack seismic data becomes critical. In this paper, we consider seis- mic facies recognition based on prestack data as an image clus- tering problem in computer vision (CV) by thinking of each prestack seismic gather as a picture. We propose a convo- lutional autoencoder (CAE) network for deep feature learn- ing from prestack seismic data, which is more effective than principal component analysis (PCA) in redundancy removing and valid information extraction. Then, using conventional classification or clustering techniques (e.g. K-means or self- organizing maps) on the extracted features, we can achieve seismic facies recognition. We applied our method to the prestack data from physical model and LZB region. The re- sult shows that our approach is superior to the conventionals.
Tasks
Published	2017-04-08
URL	http://arxiv.org/abs/1704.02446v1
PDF	http://arxiv.org/pdf/1704.02446v1.pdf
PWC	https://paperswithcode.com/paper/seismic-facies-recognition-based-on-prestack
Repo
Framework

Human and Machine Judgements for Russian Semantic Relatedness


Title	Human and Machine Judgements for Russian Semantic Relatedness
Authors	Alexander Panchenko, Dmitry Ustalov, Nikolay Arefyev, Denis Paperno, Natalia Konstantinova, Natalia Loukachevitch, Chris Biemann
Abstract	Semantic relatedness of terms represents similarity of meaning by a numerical score. On the one hand, humans easily make judgments about semantic relatedness. On the other hand, this kind of information is useful in language processing systems. While semantic relatedness has been extensively studied for English using numerous language resources, such as associative norms, human judgments, and datasets generated from lexical databases, no evaluation resources of this kind have been available for Russian to date. Our contribution addresses this problem. We present five language resources of different scale and purpose for Russian semantic relatedness, each being a list of triples (word_i, word_j, relatedness_ij). Four of them are designed for evaluation of systems for computing semantic relatedness, complementing each other in terms of the semantic relation type they represent. These benchmarks were used to organize a shared task on Russian semantic relatedness, which attracted 19 teams. We use one of the best approaches identified in this competition to generate the fifth high-coverage resource, the first open distributional thesaurus of Russian. Multiple evaluations of this thesaurus, including a large-scale crowdsourcing study involving native speakers, indicate its high accuracy.
Tasks
Published	2017-08-31
URL	http://arxiv.org/abs/1708.09702v1
PDF	http://arxiv.org/pdf/1708.09702v1.pdf
PWC	https://paperswithcode.com/paper/human-and-machine-judgements-for-russian
Repo
Framework

Low-dimensional Embeddings for Interpretable Anchor-based Topic Inference


Title	Low-dimensional Embeddings for Interpretable Anchor-based Topic Inference
Authors	Moontae Lee, David Mimno
Abstract	The anchor words algorithm performs provably efficient topic model inference by finding an approximate convex hull in a high-dimensional word co-occurrence space. However, the existing greedy algorithm often selects poor anchor words, reducing topic quality and interpretability. Rather than finding an approximate convex hull in a high-dimensional space, we propose to find an exact convex hull in a visualizable 2- or 3-dimensional space. Such low-dimensional embeddings both improve topics and clearly show users why the algorithm selects certain words.
Tasks
Published	2017-11-18
URL	http://arxiv.org/abs/1711.06826v1
PDF	http://arxiv.org/pdf/1711.06826v1.pdf
PWC	https://paperswithcode.com/paper/low-dimensional-embeddings-for-interpretable
Repo
Framework

Fast Screening Algorithm for Rotation and Scale Invariant Template Matching


Title	Fast Screening Algorithm for Rotation and Scale Invariant Template Matching
Authors	Bolin Liu, Xiao Shu, Xiaolin Wu
Abstract	This paper presents a generic pre-processor for expediting conventional template matching techniques. Instead of locating the best matched patch in the reference image to a query template via exhaustive search, the proposed algorithm rules out regions with no possible matches with minimum computational efforts. While working on simple patch features, such as mean, variance and gradient, the fast pre-screening is highly discriminative. Its computational efficiency is gained by using a novel octagonal-star-shaped template and the inclusion-exclusion principle to extract and compare patch features. Moreover, it can handle arbitrary rotation and scaling of reference images effectively. Extensive experiments demonstrate that the proposed algorithm greatly reduces the search space while never missing the best match.
Tasks
Published	2017-07-18
URL	http://arxiv.org/abs/1707.05647v2
PDF	http://arxiv.org/pdf/1707.05647v2.pdf
PWC	https://paperswithcode.com/paper/fast-screening-algorithm-for-rotation-and
Repo
Framework

Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models


Title	Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models
Authors	Katharina Kann, Hinrich Schütze
Abstract	We present a semi-supervised way of training a character-based encoder-decoder recurrent neural network for morphological reinflection, the task of generating one inflected word form from another. This is achieved by using unlabeled tokens or random strings as training data for an autoencoding task, adapting a network for morphological reinflection, and performing multi-task training. We thus use limited labeled data more effectively, obtaining up to 9.9% improvement over state-of-the-art baselines for 8 different languages.
Tasks
Published	2017-05-17
URL	http://arxiv.org/abs/1705.06106v2
PDF	http://arxiv.org/pdf/1705.06106v2.pdf
PWC	https://paperswithcode.com/paper/unlabeled-data-for-morphological-generation
Repo
Framework

X-ray Astronomical Point Sources Recognition Using Granular Binary-tree SVM


Title	X-ray Astronomical Point Sources Recognition Using Granular Binary-tree SVM
Authors	Zhixian Ma, Weitian Li, Lei Wang, Haiguang Xu, Jie Zhu
Abstract	The study on point sources in astronomical images is of special importance, since most energetic celestial objects in the Universe exhibit a point-like appearance. An approach to recognize the point sources (PS) in the X-ray astronomical images using our newly designed granular binary-tree support vector machine (GBT-SVM) classifier is proposed. First, all potential point sources are located by peak detection on the image. The image and spectral features of these potential point sources are then extracted. Finally, a classifier to recognize the true point sources is build through the extracted features. Experiments and applications of our approach on real X-ray astronomical images are demonstrated. comparisons between our approach and other SVM-based classifiers are also carried out by evaluating the precision and recall rates, which prove that our approach is better and achieves a higher accuracy of around 89%.
Tasks
Published	2017-03-07
URL	http://arxiv.org/abs/1703.02271v1
PDF	http://arxiv.org/pdf/1703.02271v1.pdf
PWC	https://paperswithcode.com/paper/x-ray-astronomical-point-sources-recognition
Repo
Framework