January 30, 2020

3097 words 15 mins read

Paper Group ANR 206

Joint 3D Localization and Classification of Space Debris using a Multispectral Rotating Point Spread Function. How do infinite width bounded norm networks look in function space?. Overcoming the Rare Word Problem for Low-Resource Language Pairs in Neural Machine Translation. KerGM: Kernelized Graph Matching. Adversarial AutoAugment. Towards a regul …

Joint 3D Localization and Classification of Space Debris using a Multispectral Rotating Point Spread Function


Title	Joint 3D Localization and Classification of Space Debris using a Multispectral Rotating Point Spread Function
Authors	Chao Wang, Grey Ballard, Robert Plemmons, Sudhakar Prasad
Abstract	We consider the problem of joint three-dimensional (3D) localization and material classification of unresolved space debris using a multispectral rotating point spread function (RPSF). The use of RPSF allows one to estimate the 3D locations of point sources from their rotated images acquired by a single 2D sensor array, since the amount of rotation of each source image about its x, y location depends on its axial distance z. Using multi-spectral images, with one RPSF per spectral band, we are able not only to localize the 3D positions of the space debris but also classify their material composition. We propose a three-stage method for achieving joint localization and classification. In Stage 1, we adopt an optimization scheme for localization in which the spectral signature of each material is assumed to be uniform, which significantly improves efficiency and yields better localization results than possible with a single spectral band. In Stage 2, we estimate the spectral signature and refine the localization result via an alternating approach. We process classification in the final stage. Both Poisson noise and Gaussian noise models are considered, and the implementation of each is discussed. Numerical tests using multispectral data from NASA show the efficiency of our three-stage approach and illustrate the improvement of point source localization and spectral classification from using multiple bands over a single band.
Tasks	Material Classification
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04749v1
PDF	https://arxiv.org/pdf/1906.04749v1.pdf
PWC	https://paperswithcode.com/paper/joint-3d-localization-and-classification-of
Repo
Framework

How do infinite width bounded norm networks look in function space?


Title	How do infinite width bounded norm networks look in function space?
Authors	Pedro Savarese, Itay Evron, Daniel Soudry, Nathan Srebro
Abstract	We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of squares of all weights in the system, except for an unregularized bias term for each unit) is bounded; or equivalently what is the minimal norm required to approximate a given function. For functions $f : \mathbb R \rightarrow \mathbb R$ and a single hidden layer, we show that the minimal network norm for representing $f$ is $\max(\int f’'(x) dx, f’(-\infty) + f’(+\infty))$, and hence the minimal norm fit for a sample is given by a linear spline interpolation.
Tasks
Published	2019-02-13
URL	http://arxiv.org/abs/1902.05040v1
PDF	http://arxiv.org/pdf/1902.05040v1.pdf
PWC	https://paperswithcode.com/paper/how-do-infinite-width-bounded-norm-networks
Repo
Framework

Overcoming the Rare Word Problem for Low-Resource Language Pairs in Neural Machine Translation


Title	Overcoming the Rare Word Problem for Low-Resource Language Pairs in Neural Machine Translation
Authors	Thi-Vinh Ngo, Thanh-Le Ha, Phuong-Thai Nguyen, Le-Minh Nguyen
Abstract	Among the six challenges of neural machine translation (NMT) coined by (Koehn and Knowles, 2017), rare-word problem is considered the most severe one, especially in translation of low-resource languages. In this paper, we propose three solutions to address the rare words in neural machine translation systems. First, we enhance source context to predict the target words by connecting directly the source embeddings to the output of the attention component in NMT. Second, we propose an algorithm to learn morphology of unknown words for English in supervised way in order to minimize the adverse effect of rare-word problem. Finally, we exploit synonymous relation from the WordNet to overcome out-of-vocabulary (OOV) problem of NMT. We evaluate our approaches on two low-resource language pairs: English-Vietnamese and Japanese-Vietnamese. In our experiments, we have achieved significant improvements of up to roughly +1.0 BLEU points in both language pairs.
Tasks	Machine Translation
Published	2019-10-07
URL	https://arxiv.org/abs/1910.03467v2
PDF	https://arxiv.org/pdf/1910.03467v2.pdf
PWC	https://paperswithcode.com/paper/overcoming-the-rare-word-problem-for-low
Repo
Framework

KerGM: Kernelized Graph Matching


Title	KerGM: Kernelized Graph Matching
Authors	Zhen Zhang, Yijian Xiang, Lingfei Wu, Bing Xue, Arye Nehorai
Abstract	Graph matching plays a central role in such fields as computer vision, pattern recognition, and bioinformatics. Graph matching problems can be cast as two types of quadratic assignment problems (QAPs): Koopmans-Beckmann’s QAP or Lawler’s QAP. In our paper, we provide a unifying view for these two problems by introducing new rules for array operations in Hilbert spaces. Consequently, Lawler’s QAP can be considered as the Koopmans-Beckmann’s alignment between two arrays in reproducing kernel Hilbert spaces (RKHS), making it possible to efficiently solve the problem without computing a huge affinity matrix. Furthermore, we develop the entropy-regularized Frank-Wolfe (EnFW) algorithm for optimizing QAPs, which has the same convergence rate as the original FW algorithm while dramatically reducing the computational burden for each outer iteration. We conduct extensive experiments to evaluate our approach, and show that our algorithm significantly outperforms the state-of-the-art in both matching accuracy and scalability.
Tasks	Graph Matching
Published	2019-11-25
URL	https://arxiv.org/abs/1911.11120v1
PDF	https://arxiv.org/pdf/1911.11120v1.pdf
PWC	https://paperswithcode.com/paper/kergm-kernelized-graph-matching-1
Repo
Framework

Adversarial AutoAugment


Title	Adversarial AutoAugment
Authors	Xinyu Zhang, Qiang Wang, Jian Zhang, Zhao Zhong
Abstract	Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks. Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy. Through finding the best policy in well-designed search space of data augmentation, AutoAugment can significantly improve validation accuracy on image classification tasks. However, this approach is not computationally practical for large-scale problems. In this paper, we develop an adversarial method to arrive at a computationally-affordable solution called Adversarial AutoAugment, which can simultaneously optimize target related object and augmentation policy search loss. The augmentation policy network attempts to increase the training loss of a target network through generating adversarial augmentation policies, while the target network can learn more robust features from harder examples to improve the generalization. In contrast to prior work, we reuse the computation in target network training for policy evaluation, and dispense with the retraining of the target network. Compared to AutoAugment, this leads to about 12x reduction in computing cost and 11x shortening in time overhead on ImageNet. We show experimental results of our approach on CIFAR-10/CIFAR-100, ImageNet, and demonstrate significant performance improvements over state-of-the-art. On CIFAR-10, we achieve a top-1 test error of 1.36%, which is the currently best performing single model. On ImageNet, we achieve a leading performance of top-1 accuracy 79.40% on ResNet-50 and 80.00% on ResNet-50-D without extra data.
Tasks	Data Augmentation, Image Classification
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11188v1
PDF	https://arxiv.org/pdf/1912.11188v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-autoaugment-1
Repo
Framework

Towards a regularity theory for ReLU networks – chain rule and global error estimates


Title	Towards a regularity theory for ReLU networks – chain rule and global error estimates
Authors	Julius Berner, Dennis Elbrächter, Philipp Grohs, Arnulf Jentzen
Abstract	Although for neural networks with locally Lipschitz continuous activation functions the classical derivative exists almost everywhere, the standard chain rule is in general not applicable. We will consider a way of introducing a derivative for neural networks that admits a chain rule, which is both rigorous and easy to work with. In addition we will present a method of converting approximation results on bounded domains to global (pointwise) estimates. This can be used to extend known neural network approximation theory to include the study of regularity properties. Of particular interest is the application to neural networks with ReLU activation function, where it contributes to the understanding of the success of deep learning methods for high-dimensional partial differential equations.
Tasks
Published	2019-05-13
URL	https://arxiv.org/abs/1905.04992v1
PDF	https://arxiv.org/pdf/1905.04992v1.pdf
PWC	https://paperswithcode.com/paper/towards-a-regularity-theory-for-relu-networks
Repo
Framework

Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks


Title	Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks
Authors	Dongqi Han, Kenji Doya, Jun Tani
Abstract	Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning. However, little effort has been spent on improving RNN architectures and on understanding the underlying neural mechanisms for performance gain. In this paper, we propose a novel, multiple-timescale, stochastic RNN for RL. Empirical results show that the network can autonomously learn to abstract sub-goals and can self-develop an action hierarchy using internal dynamics in a challenging continuous control task. Furthermore, we show that the self-developed compositionality of the network enhances faster re-learning when adapting to a new task that is a re-composition of previously learned sub-goals, than when starting from scratch. We also found that improved performance can be achieved when neural activities are subject to stochastic rather than deterministic dynamics.
Tasks	Continuous Control, Meta-Learning
Published	2019-01-29
URL	https://arxiv.org/abs/1901.10113v6
PDF	https://arxiv.org/pdf/1901.10113v6.pdf
PWC	https://paperswithcode.com/paper/emergence-of-hierarchy-via-reinforcement
Repo
Framework

Tehran Stock Exchange Prediction Using Sentiment Analysis of Online Textual Opinions


Title	Tehran Stock Exchange Prediction Using Sentiment Analysis of Online Textual Opinions
Authors	Arezoo Hatefi Ghahfarrokhi, Mehrnoush Shamsfard
Abstract	In this paper, we investigate the impact of the social media data in predicting the Tehran Stock Exchange (TSE) variables for the first time. We consider the closing price and daily return of three different stocks for this investigation. We collected our social media data from Sahamyab.com/stocktwits for about three months. To extract information from online comments, we propose a hybrid sentiment analysis approach that combines lexicon-based and learning-based methods. Since lexicons that are available for the Persian language are not practical for sentiment analysis in the stock market domain, we built a particular sentiment lexicon for this domain. After designing and calculating daily sentiment indices using the sentiment of the comments, we examine their impact on the baseline models that only use historical market data and propose new predictor models using multi regression analysis. In addition to the sentiments, we also examine the comments volume and the users’ reliabilities. We conclude that the predictability of various stocks in TSE is different depending on their attributes. Moreover, we indicate that for predicting the closing price only comments volume and for predicting the daily return both the volume and the sentiment of the comments could be useful. We demonstrate that Users’ Trust coefficients have different behaviors toward the three stocks.
Tasks	Sentiment Analysis
Published	2019-08-30
URL	https://arxiv.org/abs/1909.03792v2
PDF	https://arxiv.org/pdf/1909.03792v2.pdf
PWC	https://paperswithcode.com/paper/tehran-stock-exchange-prediction-using
Repo
Framework

Yottixel – An Image Search Engine for Large Archives of Histopathology Whole Slide Images


Title	Yottixel – An Image Search Engine for Large Archives of Histopathology Whole Slide Images
Authors	S. Kalra, C. Choi, S. Shah, L. Pantanowitz, H. R. Tizhoosh
Abstract	With the emergence of digital pathology, searching for similar images in large archives has gained considerable attention. Image retrieval can provide pathologists with unprecedented access to the evidence embodied in already diagnosed and treated cases from the past. This paper proposes a search engine specialized for digital pathology, called Yottixel, a portmanteau for “one yotta pixel,” alluding to the big-data nature of histopathology images. The most impressive characteristic of Yottixel is its ability to represent whole slide images (WSIs) in a compact manner. Yottixel can perform millions of searches in real-time with a high search accuracy and low storage profile. Yottixel uses an intelligent indexing algorithm capable of representing WSIs with a mosaic of patches by converting them into a small number of methodically extracted barcodes, called “Bunch of Barcodes” (BoB), the most prominent performance enabler of Yottixel. The performance of the prototype platform is qualitatively tested using 300 WSIs from the University of Pittsburgh Medical Center (UPMC) and 2,020 WSIs from The Cancer Genome Atlas Program (TCGA) provided by the National Cancer Institute. Both datasets amount to more than 4,000,000 patches of 1000x1000 pixels. We report three sets of experiments that show that Yottixel can accurately retrieve organs and malignancies, and its semantic ordering shows good agreement with the subjective evaluation of human observers.
Tasks	Image Retrieval
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08748v1
PDF	https://arxiv.org/pdf/1911.08748v1.pdf
PWC	https://paperswithcode.com/paper/yottixel-an-image-search-engine-for-large
Repo
Framework

Transfer of Machine Learning Fairness across Domains


Title	Transfer of Machine Learning Fairness across Domains
Authors	Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, Ed H. Chi
Abstract	If our models are used in new or unexpected cases, do we know if they will make fair predictions? Previously, researchers developed ways to debias a model for a single problem domain. However, this is often not how models are trained and used in practice. For example, labels and demographics (sensitive attributes) are often hard to observe, resulting in auxiliary or synthetic data to be used for training, and proxies of the sensitive attribute to be used for evaluation of fairness. A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs. Despite the pervasiveness of these complexities, remarkably little work in the fairness literature has theoretically examined these issues. We frame all of these settings as domain adaptation problems: how can we use what we have learned in a source domain to debias in a new target domain, without directly debiasing on the target domain as if it is a completely new problem? We offer new theoretical guarantees of improving fairness across domains, and offer a modeling approach to transfer to data-sparse target domains. We give empirical results validating the theory and showing that these modeling approaches can improve fairness metrics with less data.
Tasks	Domain Adaptation
Published	2019-06-24
URL	https://arxiv.org/abs/1906.09688v3
PDF	https://arxiv.org/pdf/1906.09688v3.pdf
PWC	https://paperswithcode.com/paper/transfer-of-machine-learning-fairness-across
Repo
Framework

Finite-Time Convergence of Continuous-Time Optimization Algorithms via Differential Inclusions


Title	Finite-Time Convergence of Continuous-Time Optimization Algorithms via Differential Inclusions
Authors	Orlando Romero, Mouhacine Benosman
Abstract	In this paper, we propose two discontinuous dynamical systems in continuous time with guaranteed prescribed finite-time local convergence to strict local minima of a given cost function. Our approach consists of exploiting a Lyapunov-based differential inequality for differential inclusions, which leads to finite-time stability and thus finite-time convergence with a provable bound on the settling time. In particular, for exact solutions to the aforementioned differential inequality, the settling-time bound is also exact, thus achieving prescribed finite-time convergence. We thus construct a class of discontinuous dynamical systems, of second order with respect to the cost function, that serve as continuous-time optimization algorithms with finite-time convergence and prescribed convergence time. Finally, we illustrate our results on the Rosenbrock function.
Tasks
Published	2019-12-18
URL	https://arxiv.org/abs/1912.08342v1
PDF	https://arxiv.org/pdf/1912.08342v1.pdf
PWC	https://paperswithcode.com/paper/finite-time-convergence-of-continuous-time
Repo
Framework

Learning Similarity Attention


Title	Learning Similarity Attention
Authors	Meng Zheng, Srikrishna Karanam, Terrence Chen, Richard J. Radke, Ziyan Wu
Abstract	We consider the problem of learning similarity functions. While there has been substantial progress in learning suitable distance metrics, these techniques in general lack decision reasoning, i.e., explaining why the input set of images is similar or dissimilar. In this work, we solve this key problem by proposing the first method to generate generic visual similarity explanations with gradient-based attention. We demonstrate that our technique is agnostic to the specific similarity model type, e.g., we show applicability to Siamese, triplet, and quadruplet models. Furthermore, we make our proposed similarity attention a principled part of the learning process, resulting in a new paradigm for learning similarity functions. We demonstrate that our learning mechanism results in more generalizable, as well as explainable, similarity models. Finally, we demonstrate the generality of our framework by means of experiments on a variety of tasks, including image retrieval, person re-identification, and low-shot semantic segmentation.
Tasks	Image Retrieval, Person Re-Identification, Semantic Segmentation
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07381v1
PDF	https://arxiv.org/pdf/1911.07381v1.pdf
PWC	https://paperswithcode.com/paper/learning-similarity-attention
Repo
Framework

Transductive Zero-Shot Hashing for Multi-Label Image Retrieval


Title	Transductive Zero-Shot Hashing for Multi-Label Image Retrieval
Authors	Qin Zou, Zheng Zhang, Ling Cao, Long Chen, Song Wang
Abstract	Hash coding has been widely used in approximate nearest neighbor search for large-scale image retrieval. Given semantic annotations such as class labels and pairwise similarities of the training data, hashing methods can learn and generate effective and compact binary codes. While some newly introduced images may contain undefined semantic labels, which we call unseen images, zeor-shot hashing techniques have been studied. However, existing zeor-shot hashing methods focus on the retrieval of single-label images, and cannot handle multi-label images. In this paper, for the first time, a novel transductive zero-shot hashing method is proposed for multi-label unseen image retrieval. In order to predict the labels of the unseen/target data, a visual-semantic bridge is built via instance-concept coherence ranking on the seen/source data. Then, pairwise similarity loss and focal quantization loss are constructed for training a hashing model using both the seen/source and unseen/target data. Extensive evaluations on three popular multi-label datasets demonstrate that, the proposed hashing method achieves significantly better results than the competing methods.
Tasks	Image Retrieval, Multi-Label Image Retrieval, Quantization
Published	2019-11-17
URL	https://arxiv.org/abs/1911.07192v1
PDF	https://arxiv.org/pdf/1911.07192v1.pdf
PWC	https://paperswithcode.com/paper/transductive-zero-shot-hashing-for-multi
Repo
Framework

Unsupervised Deep Metric Learning via Auxiliary Rotation Loss


Title	Unsupervised Deep Metric Learning via Auxiliary Rotation Loss
Authors	Xuefei Cao, Bor-Chun Chen, Ser-Nam Lim
Abstract	Deep metric learning is an important area due to its applicability to many domains such as image retrieval and person re-identification. The main drawback of such models is the necessity for labeled data. In this work, we propose to generate pseudo-labels for deep metric learning directly from clustering assignment and we introduce unsupervised deep metric learning (UDML) regularized by a self-supervision (SS) task. In particular, we propose to regularize the training process by predicting image rotations. Our method (UDML-SS) jointly learns discriminative embeddings, unsupervised clustering assignments of the embeddings, as well as a self-supervised pretext task. UDML-SS iteratively cluster embeddings using traditional clustering algorithm (e.g., k-means), and sampling training pairs based on the cluster assignment for metric learning, while optimizing self-supervised pretext task in a multi-task fashion. The role of self-supervision is to stabilize the training process and encourages the model to learn meaningful feature representations that are not distorted due to unreliable clustering assignments. The proposed method performs well on standard benchmarks for metric learning, where it outperforms current state-of-the-art approaches by a large margin and it also shows competitive performance with various metric learning loss functions.
Tasks	Image Retrieval, Metric Learning, Person Re-Identification
Published	2019-11-16
URL	https://arxiv.org/abs/1911.07072v1
PDF	https://arxiv.org/pdf/1911.07072v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-deep-metric-learning-via
Repo
Framework

Machine Reading Comprehension: a Literature Review


Title	Machine Reading Comprehension: a Literature Review
Authors	Xin Zhang, An Yang, Sujian Li, Yizhong Wang
Abstract	Machine reading comprehension aims to teach machines to understand a text like a human and is a new challenging direction in Artificial Intelligence. This article summarizes recent advances in MRC, mainly focusing on two aspects (i.e., corpus and techniques). The specific characteristics of various MRC corpus are listed and compared. The main ideas of some typical MRC techniques are also described.
Tasks	Machine Reading Comprehension, Reading Comprehension
Published	2019-06-30
URL	https://arxiv.org/abs/1907.01686v1
PDF	https://arxiv.org/pdf/1907.01686v1.pdf
PWC	https://paperswithcode.com/paper/machine-reading-comprehension-a-literature
Repo
Framework