October 17, 2019

3063 words 15 mins read

Paper Group ANR 933

Acceleration through Optimistic No-Regret Dynamics. Jigsaw Puzzle Solving Using Local Feature Co-Occurrences in Deep Neural Networks. Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings. Unsupervised Cross-lingual Transfer of Word Embedding Spaces. Balanced Distribution Adaptation for Transfer Learning …

Acceleration through Optimistic No-Regret Dynamics


Title	Acceleration through Optimistic No-Regret Dynamics
Authors	Jun-Kun Wang, Jacob Abernethy
Abstract	We consider the problem of minimizing a smooth convex function by reducing the optimization to computing the Nash equilibrium of a particular zero-sum convex-concave game. Zero-sum games can be solved using online learning dynamics, where a classical technique involves simulating two no-regret algorithms that play against each other and, after $T$ rounds, the average iterate is guaranteed to solve the original optimization problem with error decaying as $O(\log T/T)$. In this paper we show that the technique can be enhanced to a rate of $O(1/T^2)$ by extending recent work \cite{RS13,SALS15} that leverages \textit{optimistic learning} to speed up equilibrium computation. The resulting optimization algorithm derived from this analysis coincides \textit{exactly} with the well-known \NA \cite{N83a} method, and indeed the same story allows us to recover several variants of the Nesterov’s algorithm via small tweaks. We are also able to establish the accelerated linear rate for a function which is both strongly-convex and smooth. This methodology unifies a number of different iterative optimization methods: we show that the \HB algorithm is precisely the non-optimistic variant of \NA, and recent prior work already established a similar perspective on \FW \cite{AW17,ALLW18}.
Tasks
Published	2018-07-27
URL	http://arxiv.org/abs/1807.10455v3
PDF	http://arxiv.org/pdf/1807.10455v3.pdf
PWC	https://paperswithcode.com/paper/acceleration-through-optimistic-no-regret
Repo
Framework

Jigsaw Puzzle Solving Using Local Feature Co-Occurrences in Deep Neural Networks


Title	Jigsaw Puzzle Solving Using Local Feature Co-Occurrences in Deep Neural Networks
Authors	Marie-Morgane Paumard, David Picard, Hedi Tabia
Abstract	Archaeologists are in dire need of automated object reconstruction methods. Fragments reassembly is close to puzzle problems, which may be solved by computer vision algorithms. As they are often beaten on most image related tasks by deep learning algorithms, we study a classification method that can solve jigsaw puzzles. In this paper, we focus on classifying the relative position: given a couple of fragments, we compute their local relation (e.g. on top). We propose several enhancements over the state of the art in this domain, which is outperformed by our method by 25%. We propose an original dataset composed of pictures from the Metropolitan Museum of Art. We propose a greedy reconstruction method based on the predicted relative positions.
Tasks	Object Reconstruction
Published	2018-07-05
URL	http://arxiv.org/abs/1807.03155v1
PDF	http://arxiv.org/pdf/1807.03155v1.pdf
PWC	https://paperswithcode.com/paper/jigsaw-puzzle-solving-using-local-feature-co
Repo
Framework

Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings


Title	Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings
Authors	Qi Chen, Lei Wang, Yifan Wu, Guangming Wu, Zhiling Guo, Steven L. Waslander
Abstract	As an important branch of deep learning, convolutional neural network has largely improved the performance of building detection. For further accelerating the development of building detection toward automatic mapping, a benchmark dataset bears significance in fair comparisons. However, several problems still remain in the current public datasets that address this task. First, although building detection is generally considered equivalent to extracting roof outlines, most datasets directly provide building footprints as ground truths for testing and evaluation; the challenges of these benchmarks are more complicated than roof segmentation, as relief displacement leads to varying degrees of misalignment between roof outlines and footprints. On the other hand, an image dataset should feature a large quantity and high spatial resolution to effectively train a high-performance deep learning model for accurate mapping of buildings. Unfortunately, the remote sensing community still lacks proper benchmark datasets that can simultaneously satisfy these requirements. In this paper, we present a new large-scale benchmark dataset termed Aerial Imagery for Roof Segmentation (AIRS). This dataset provides a wide coverage of aerial imagery with 7.5 cm resolution and contains over 220,000 buildings. The task posed for AIRS is defined as roof segmentation. We implement several state-of-the-art deep learning methods of semantic segmentation for performance evaluation and analysis of the proposed dataset. The results can serve as the baseline for future work.
Tasks	Semantic Segmentation
Published	2018-07-25
URL	http://arxiv.org/abs/1807.09532v2
PDF	http://arxiv.org/pdf/1807.09532v2.pdf
PWC	https://paperswithcode.com/paper/aerial-imagery-for-roof-segmentation-a-large
Repo
Framework

Unsupervised Cross-lingual Transfer of Word Embedding Spaces


Title	Unsupervised Cross-lingual Transfer of Word Embedding Spaces
Authors	Ruochen Xu, Yiming Yang, Naoki Otani, Yuexin Wu
Abstract	Cross-lingual transfer of word embeddings aims to establish the semantic mappings among words in different languages by learning the transformation functions over the corresponding word embedding spaces. Successfully solving this problem would benefit many downstream tasks such as to translate text classification models from resource-rich languages (e.g. English) to low-resource languages. Supervised methods for this problem rely on the availability of cross-lingual supervision, either using parallel corpora or bilingual lexicons as the labeled data for training, which may not be available for many low resource languages. This paper proposes an unsupervised learning approach that does not require any cross-lingual labeled data. Given two monolingual word embedding spaces for any language pair, our algorithm optimizes the transformation functions in both directions simultaneously based on distributional matching as well as minimizing the back-translation losses. We use a neural network implementation to calculate the Sinkhorn distance, a well-defined distributional similarity measure, and optimize our objective through back-propagation. Our evaluation on benchmark datasets for bilingual lexicon induction and cross-lingual word similarity prediction shows stronger or competitive performance of the proposed method compared to other state-of-the-art supervised and unsupervised baseline methods over many language pairs.
Tasks	Cross-Lingual Transfer, Text Classification, Word Embeddings
Published	2018-09-10
URL	http://arxiv.org/abs/1809.03633v1
PDF	http://arxiv.org/pdf/1809.03633v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-cross-lingual-transfer-of-word
Repo
Framework

Balanced Distribution Adaptation for Transfer Learning


Title	Balanced Distribution Adaptation for Transfer Learning
Authors	Jindong Wang, Yiqiang Chen, Shuji Hao, Wenjie Feng, Zhiqi Shen
Abstract	Transfer learning has achieved promising results by leveraging knowledge from the source domain to annotate the target domain which has few or none labels. Existing methods often seek to minimize the distribution divergence between domains, such as the marginal distribution, the conditional distribution or both. However, these two distances are often treated equally in existing algorithms, which will result in poor performance in real applications. Moreover, existing methods usually assume that the dataset is balanced, which also limits their performances on imbalanced tasks that are quite common in real problems. To tackle the distribution adaptation problem, in this paper, we propose a novel transfer learning approach, named as Balanced Distribution \underline{A}daptation~(BDA), which can adaptively leverage the importance of the marginal and conditional distribution discrepancies, and several existing methods can be treated as special cases of BDA. Based on BDA, we also propose a novel Weighted Balanced Distribution Adaptation~(W-BDA) algorithm to tackle the class imbalance issue in transfer learning. W-BDA not only considers the distribution adaptation between domains but also adaptively changes the weight of each class. To evaluate the proposed methods, we conduct extensive experiments on several transfer learning tasks, which demonstrate the effectiveness of our proposed algorithms over several state-of-the-art methods.
Tasks	Transfer Learning
Published	2018-07-02
URL	http://arxiv.org/abs/1807.00516v1
PDF	http://arxiv.org/pdf/1807.00516v1.pdf
PWC	https://paperswithcode.com/paper/balanced-distribution-adaptation-for-transfer
Repo
Framework

Who Learns Better Bayesian Network Structures: Accuracy and Speed of Structure Learning Algorithms


Title	Who Learns Better Bayesian Network Structures: Accuracy and Speed of Structure Learning Algorithms
Authors	Marco Scutari, Catharina Elisabeth Graafland, José Manuel Gutiérrez
Abstract	Three classes of algorithms to learn the structure of Bayesian networks from data are common in the literature: constraint-based algorithms, which use conditional independence tests to learn the dependence structure of the data; score-based algorithms, which use goodness-of-fit scores as objective functions to maximise; and hybrid algorithms that combine both approaches. Constraint-based and score-based algorithms have been shown to learn the same structures when conditional independence and goodness of fit are both assessed using entropy and the topological ordering of the network is known (Cowell, 2001). In this paper, we investigate how these three classes of algorithms perform outside the assumptions above in terms of speed and accuracy of network reconstruction for both discrete and Gaussian Bayesian networks. We approach this question by recognising that structure learning is defined by the combination of a statistical criterion and an algorithm that determines how the criterion is applied to the data. Removing the confounding effect of different choices for the statistical criterion, we find using both simulated and real-world complex data that constraint-based algorithms are often less accurate than score-based algorithms, but are seldom faster (even at large sample sizes); and that hybrid algorithms are neither faster nor more accurate than constraint-based algorithms. This suggests that commonly held beliefs on structure learning in the literature are strongly influenced by the choice of particular statistical criteria rather than just by the properties of the algorithms themselves.
Tasks
Published	2018-05-30
URL	https://arxiv.org/abs/1805.11908v3
PDF	https://arxiv.org/pdf/1805.11908v3.pdf
PWC	https://paperswithcode.com/paper/who-learns-better-bayesian-network-structures
Repo
Framework

Attentive Semantic Role Labeling with Boundary Indicator


Title	Attentive Semantic Role Labeling with Boundary Indicator
Authors	Zhuosheng Zhang, Shexia He, Zuchao Li, Hai Zhao
Abstract	The goal of semantic role labeling (SRL) is to discover the predicate-argument structure of a sentence, which plays a critical role in deep processing of natural language. This paper introduces simple yet effective auxiliary tags for dependency-based SRL to enhance a syntax-agnostic model with multi-hop self-attention. Our syntax-agnostic model achieves competitive performance with state-of-the-art models on the CoNLL-2009 benchmarks both for English and Chinese.
Tasks	Semantic Role Labeling
Published	2018-09-08
URL	http://arxiv.org/abs/1809.02796v1
PDF	http://arxiv.org/pdf/1809.02796v1.pdf
PWC	https://paperswithcode.com/paper/attentive-semantic-role-labeling-with
Repo
Framework

Occluded object reconstruction for first responders with augmented reality glasses using conditional generative adversarial networks


Title	Occluded object reconstruction for first responders with augmented reality glasses using conditional generative adversarial networks
Authors	Kyongsik Yun, Thomas Lu, Edward Chow
Abstract	Firefighters suffer a variety of life-threatening risks, including line-of-duty deaths, injuries, and exposures to hazardous substances. Support for reducing these risks is important. We built a partially occluded object reconstruction method on augmented reality glasses for first responders. We used a deep learning based on conditional generative adversarial networks to train associations between the various images of flammable and hazardous objects and their partially occluded counterparts. Our system then reconstructed an image of a new flammable object. Finally, the reconstructed image was superimposed on the input image to provide “transparency”. The system imitates human learning about the laws of physics through experience by learning the shape of flammable objects and the flame characteristics.
Tasks	Object Reconstruction
Published	2018-04-20
URL	http://arxiv.org/abs/1805.00322v1
PDF	http://arxiv.org/pdf/1805.00322v1.pdf
PWC	https://paperswithcode.com/paper/occluded-object-reconstruction-for-first
Repo
Framework

A model of reward-modulated motor learning with parallelcortical and basal ganglia pathways


Title	A model of reward-modulated motor learning with parallelcortical and basal ganglia pathways
Authors	Ryan Pyle, Robert Rosenbaum
Abstract	Many recent studies of the motor system are divided into two distinct approaches: Those that investigate how motor responses are encoded in cortical neurons’ firing rate dynamics and those that study the learning rules by which mammals and songbirds develop reliable motor responses. Computationally, the first approach is encapsulated by reservoir computing models, which can learn intricate motor tasks and produce internal dynamics strikingly similar to those of motor cortical neurons, but rely on biologically unrealistic learning rules. The more realistic learning rules developed by the second approach are often derived for simplified, discrete tasks in contrast to the intricate dynamics that characterize real motor responses. We bridge these two approaches to develop a biologically realistic learning rule for reservoir computing. Our algorithm learns simulated motor tasks on which previous reservoir computing algorithms fail, and reproduces experimental findings including those that relate motor learning to Parkinson’s disease and its treatment.
Tasks
Published	2018-03-08
URL	http://arxiv.org/abs/1803.03304v2
PDF	http://arxiv.org/pdf/1803.03304v2.pdf
PWC	https://paperswithcode.com/paper/a-model-of-reward-modulated-motor-learning
Repo
Framework

Resembled Generative Adversarial Networks: Two Domains with Similar Attributes


Title	Resembled Generative Adversarial Networks: Two Domains with Similar Attributes
Authors	Duhyeon Bang, Hyunjung Shim
Abstract	We propose a novel algorithm, namely Resembled Generative Adversarial Networks (GAN), that generates two different domain data simultaneously where they resemble each other. Although recent GAN algorithms achieve the great success in learning the cross-domain relationship, their application is limited to domain transfers, which requires the input image. The first attempt to tackle the data generation of two domains was proposed by CoGAN. However, their solution is inherently vulnerable for various levels of domain similarities. Unlike CoGAN, our Resembled GAN implicitly induces two generators to match feature covariance from both domains, thus leading to share semantic attributes. Hence, we effectively handle a wide range of structural and semantic similarities between various two domains. Based on experimental analysis on various datasets, we verify that the proposed algorithm is effective for generating two domains with similar attributes.
Tasks
Published	2018-07-03
URL	http://arxiv.org/abs/1807.00947v1
PDF	http://arxiv.org/pdf/1807.00947v1.pdf
PWC	https://paperswithcode.com/paper/resembled-generative-adversarial-networks-two
Repo
Framework

Investigating Order Effects in Multidimensional Relevance Judgment using Query Logs


Title	Investigating Order Effects in Multidimensional Relevance Judgment using Query Logs
Authors	Sagar Uprety, Dawei Song
Abstract	There is a growing body of research which has investigated relevance judgment in IR being influenced by multiple factors or dimensions. At the same time, the Order Effects in sequential decision making have been quantitatively detected and studied in Mathematical Psychology. Combining the two phenomena, there have been some user studies carried out which investigate the Order Effects and thus incompatibility in different dimensions of relevance. In this work, we propose a methodology for carrying out such an investigation in large scale and real world data using query logs of a web search engine, and device a test to detect the presence of an irrational user behavior in relevance judgment of documents. We further validate this behavior through a Quantum Cognitive explanation of the Order and Context effects.
Tasks	Decision Making
Published	2018-07-14
URL	http://arxiv.org/abs/1807.05355v2
PDF	http://arxiv.org/pdf/1807.05355v2.pdf
PWC	https://paperswithcode.com/paper/investigating-order-effects-in
Repo
Framework

Understanding Dropout as an Optimization Trick


Title	Understanding Dropout as an Optimization Trick
Authors	Sangchul Hahn, Heeyoul Choi
Abstract	As one of standard approaches to train deep neural networks, dropout has been applied to regularize large models to avoid overfitting, and the improvement in performance by dropout has been explained as avoiding co-adaptation between nodes. However, when correlations between nodes are compared after training the networks with or without dropout, one question arises if co-adaptation avoidance explains the dropout effect completely. In this paper, we propose an additional explanation of why dropout works and propose a new technique to design better activation functions. First, we show that dropout can be explained as an optimization technique to push the input towards the saturation area of nonlinear activation function by accelerating gradient information flowing even in the saturation area in backpropagation. Based on this explanation, we propose a new technique for activation functions, {\em gradient acceleration in activation function (GAAF)}, that accelerates gradients to flow even in the saturation area. Then, input to the activation function can climb onto the saturation area which makes the network more robust because the model converges on a flat region. Experiment results support our explanation of dropout and confirm that the proposed GAAF technique improves image classification performance with expected properties.
Tasks	Image Classification
Published	2018-06-26
URL	https://arxiv.org/abs/1806.09783v3
PDF	https://arxiv.org/pdf/1806.09783v3.pdf
PWC	https://paperswithcode.com/paper/gradient-acceleration-in-activation-functions
Repo
Framework

Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter


Title	Embedded deep learning in ophthalmology: Making ophthalmic imaging smarter
Authors	Petteri Teikari, Raymond P. Najjar, Leopold Schmetterer, Dan Milea
Abstract	Deep learning has recently gained high interest in ophthalmology, due to its ability to detect clinically significant features for diagnosis and prognosis. Despite these significant advances, little is known about the ability of various deep learning systems to be embedded within ophthalmic imaging devices, allowing automated image acquisition. In this work, we will review the existing and future directions for “active acquisition” embedded deep learning, leading to as high quality images with little intervention by the human operator. In clinical practice, the improved image quality should translate into more robust deep learning-based clinical diagnostics. Embedded deep learning will be enabled by the constantly improving hardware performance with low cost. We will briefly review possible computation methods in larger clinical systems. Briefly, they can be included in a three-layer framework composed of edge, fog and cloud layers, the former being performed at a device-level. Improved edge layer performance via “active acquisition” serves as an automatic data curation operator translating to better quality data in electronic health records (EHRs), as well as on the cloud layer, for improved deep learning-based clinical data mining.
Tasks
Published	2018-10-13
URL	http://arxiv.org/abs/1810.05874v2
PDF	http://arxiv.org/pdf/1810.05874v2.pdf
PWC	https://paperswithcode.com/paper/embedded-deep-learning-in-ophthalmology
Repo
Framework

The Frontiers of Fairness in Machine Learning


Title	The Frontiers of Fairness in Machine Learning
Authors	Alexandra Chouldechova, Aaron Roth
Abstract	The last few years have seen an explosion of academic and popular interest in algorithmic fairness. Despite this interest and the volume and velocity of work that has been produced recently, the fundamental science of fairness in machine learning is still in a nascent state. In March 2018, we convened a group of experts as part of a CCC visioning workshop to assess the state of the field, and distill the most promising research directions going forward. This report summarizes the findings of that workshop. Along the way, it surveys recent theoretical work in the field and points towards promising directions for research.
Tasks
Published	2018-10-20
URL	http://arxiv.org/abs/1810.08810v1
PDF	http://arxiv.org/pdf/1810.08810v1.pdf
PWC	https://paperswithcode.com/paper/the-frontiers-of-fairness-in-machine-learning
Repo
Framework

Transfer Metric Learning: Algorithms, Applications and Outlooks


Title	Transfer Metric Learning: Algorithms, Applications and Outlooks
Authors	Yong Luo, Yonggang Wen, Ling-Yu Duan, Dacheng Tao
Abstract	Distance metric learning (DML) aims to find an appropriate way to reveal the underlying data relationship. It is critical in many machine learning, pattern recognition and data mining algorithms, and usually require large amount of label information (such as class labels or pair/triplet constraints) to achieve satisfactory performance. However, the label information may be insufficient in real-world applications due to the high-labeling cost, and DML may fail in this case. Transfer metric learning (TML) is able to mitigate this issue for DML in the domain of interest (target domain) by leveraging knowledge/information from other related domains (source domains). Although achieved a certain level of development, TML has limited success in various aspects such as selective transfer, theoretical understanding, handling complex data, big data and extreme cases. In this survey, we present a systematic review of the TML literature. In particular, we group TML into different categories according to different settings and metric transfer strategies, such as direct metric approximation, subspace approximation, distance approximation, and distribution approximation. A summarization and insightful discussion of the various TML approaches and their applications will be presented. Finally, we indicate some challenges and provide possible future directions.
Tasks	Metric Learning
Published	2018-10-09
URL	http://arxiv.org/abs/1810.03944v3
PDF	http://arxiv.org/pdf/1810.03944v3.pdf
PWC	https://paperswithcode.com/paper/transfer-metric-learning-algorithms
Repo
Framework