Paper Group AWR 162
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples. Cross-View Image Matching for Geo-localization in Urban Environments. Particle Clustering Machine: A Dynamical System Based Approach. On the State of the Art of Evaluation in Neural Language Models. Accurate Pulmonary Nodule Detection in Computed Tomography Im …
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
| Title | Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples |
| Authors | Haw-Shiuan Chang, Erik Learned-Miller, Andrew McCallum |
| Abstract | Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of mini-batch SGD, and the proximity of the correct class probability to the decision threshold. Extensive experimental results on six datasets show that our methods reliably improve accuracy in various network architectures, including additional gains on top of other popular training techniques, such as residual learning, momentum, ADAM, batch normalization, dropout, and distillation. |
| Tasks | |
| Published | 2017-04-24 |
| URL | http://arxiv.org/abs/1704.07433v4 |
| http://arxiv.org/pdf/1704.07433v4.pdf | |
| PWC | https://paperswithcode.com/paper/active-bias-training-more-accurate-neural |
| Repo | https://github.com/zhangyuwangumass/Trajectory-Reweighted-Sampler |
| Framework | pytorch |
Cross-View Image Matching for Geo-localization in Urban Environments
| Title | Cross-View Image Matching for Geo-localization in Urban Environments |
| Authors | Yicong Tian, Chen Chen, Mubarak Shah |
| Abstract | In this paper, we address the problem of cross-view image geo-localization. Specifically, we aim to estimate the GPS location of a query street view image by finding the matching images in a reference database of geo-tagged bird’s eye view images, or vice versa. To this end, we present a new framework for cross-view image geo-localization by taking advantage of the tremendous success of deep convolutional neural networks (CNNs) in image classification and object detection. First, we employ the Faster R-CNN to detect buildings in the query and reference images. Next, for each building in the query image, we retrieve the $k$ nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs. To find the correct NN for each query building, we develop an efficient multiple nearest neighbors matching method based on dominant sets. We evaluate the proposed framework on a new dataset that consists of pairs of street view and bird’s eye view images. Experimental results show that the proposed method achieves better geo-localization accuracy than other approaches and is able to generalize to images at unseen locations. |
| Tasks | Cross-View Image-to-Image Translation, Image Classification, Object Detection |
| Published | 2017-03-22 |
| URL | http://arxiv.org/abs/1703.07815v1 |
| http://arxiv.org/pdf/1703.07815v1.pdf | |
| PWC | https://paperswithcode.com/paper/cross-view-image-matching-for-geo |
| Repo | https://github.com/viibridges/crossnet |
| Framework | tf |
Particle Clustering Machine: A Dynamical System Based Approach
| Title | Particle Clustering Machine: A Dynamical System Based Approach |
| Authors | Sambarta Dasgupta, Keivan Ebrahimi, Umesh Vaidya |
| Abstract | Identification of the clusters from an unlabeled data set is one of the most important problems in Unsupervised Machine Learning. The state of the art clustering algorithms are based on either the statistical properties or the geometric properties of the data set. In this work, we propose a novel method to cluster the data points using dynamical systems theory. After constructing a gradient dynamical system using interaction potential, we prove that the asymptotic dynamics of this system will determine the cluster centers, when the dynamical system is initialized at the data points. Most of the existing heuristic-based clustering techniques suffer from a disadvantage, namely the stochastic nature of the solution. Whereas, the proposed algorithm is deterministic, and the outcome would not change over multiple runs of the proposed algorithm with the same input data. Another advantage of the proposed method is that the number of clusters, which is difficult to determine in practice, does not have to be specified in advance. Simulation results with are presented, and comparisons are made with the existing methods. |
| Tasks | |
| Published | 2017-12-30 |
| URL | http://arxiv.org/abs/1801.01017v1 |
| http://arxiv.org/pdf/1801.01017v1.pdf | |
| PWC | https://paperswithcode.com/paper/particle-clustering-machine-a-dynamical |
| Repo | https://github.com/yyll008/yyll008.github.io |
| Framework | tf |
On the State of the Art of Evaluation in Neural Language Models
| Title | On the State of the Art of Evaluation in Neural Language Models |
| Authors | Gábor Melis, Chris Dyer, Phil Blunsom |
| Abstract | Ongoing innovations in recurrent neural network architectures have provided a steady influx of apparently state-of-the-art results on language modelling benchmarks. However, these have been evaluated using differing code bases and limited computational resources, which represent uncontrolled sources of experimental variation. We reevaluate several popular architectures and regularisation methods with large-scale automatic black-box hyperparameter tuning and arrive at the somewhat surprising conclusion that standard LSTM architectures, when properly regularised, outperform more recent models. We establish a new state of the art on the Penn Treebank and Wikitext-2 corpora, as well as strong baselines on the Hutter Prize dataset. |
| Tasks | Language Modelling |
| Published | 2017-07-18 |
| URL | http://arxiv.org/abs/1707.05589v2 |
| http://arxiv.org/pdf/1707.05589v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-state-of-the-art-of-evaluation-in |
| Repo | https://github.com/deepmind/lamb |
| Framework | tf |
Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks
| Title | Accurate Pulmonary Nodule Detection in Computed Tomography Images Using Deep Convolutional Neural Networks |
| Authors | Jia Ding, Aoxue Li, Zhiqiang Hu, Liwei Wang |
| Abstract | Early detection of pulmonary cancer is the most promising way to enhance a patient’s chance for survival. Accurate pulmonary nodule detection in computed tomography (CT) images is a crucial step in diagnosing pulmonary cancer. In this paper, inspired by the successful use of deep convolutional neural networks (DCNNs) in natural image recognition, we propose a novel pulmonary nodule detection approach based on DCNNs. We first introduce a deconvolutional structure to Faster Region-based Convolutional Neural Network (Faster R-CNN) for candidate detection on axial slices. Then, a three-dimensional DCNN is presented for the subsequent false positive reduction. Experimental results of the LUng Nodule Analysis 2016 (LUNA16) Challenge demonstrate the superior detection performance of the proposed approach on nodule detection(average FROC-score of 0.891, ranking the 1st place over all submitted results). |
| Tasks | Computed Tomography (CT) |
| Published | 2017-06-14 |
| URL | http://arxiv.org/abs/1706.04303v3 |
| http://arxiv.org/pdf/1706.04303v3.pdf | |
| PWC | https://paperswithcode.com/paper/accurate-pulmonary-nodule-detection-in |
| Repo | https://github.com/SCP-173-cool/pulmonary_nodules_detection |
| Framework | tf |
Data-Driven Tree Transforms and Metrics
| Title | Data-Driven Tree Transforms and Metrics |
| Authors | Gal Mishne, Ronen Talmon, Israel Cohen, Ronald R. Coifman, Yuval Kluger |
| Abstract | We consider the analysis of high dimensional data given in the form of a matrix with columns consisting of observations and rows consisting of features. Often the data is such that the observations do not reside on a regular grid, and the given order of the features is arbitrary and does not convey a notion of locality. Therefore, traditional transforms and metrics cannot be used for data organization and analysis. In this paper, our goal is to organize the data by defining an appropriate representation and metric such that they respect the smoothness and structure underlying the data. We also aim to generalize the joint clustering of observations and features in the case the data does not fall into clear disjoint groups. For this purpose, we propose multiscale data-driven transforms and metrics based on trees. Their construction is implemented in an iterative refinement procedure that exploits the co-dependencies between features and observations. Beyond the organization of a single dataset, our approach enables us to transfer the organization learned from one dataset to another and to integrate several datasets together. We present an application to breast cancer gene expression analysis: learning metrics on the genes to cluster the tumor samples into cancer sub-types and validating the joint organization of both the genes and the samples. We demonstrate that using our approach to combine information from multiple gene expression cohorts, acquired by different profiling technologies, improves the clustering of tumor samples. |
| Tasks | |
| Published | 2017-08-18 |
| URL | http://arxiv.org/abs/1708.05768v1 |
| http://arxiv.org/pdf/1708.05768v1.pdf | |
| PWC | https://paperswithcode.com/paper/data-driven-tree-transforms-and-metrics |
| Repo | https://github.com/gmishne/pyquest |
| Framework | none |
Latent Space Oddity: on the Curvature of Deep Generative Models
| Title | Latent Space Oddity: on the Curvature of Deep Generative Models |
| Authors | Georgios Arvanitidis, Lars Kai Hansen, Søren Hauberg |
| Abstract | Deep generative models provide a systematic way to learn nonlinear data distributions, through a set of latent variables and a nonlinear “generator” function that maps latent points into the input space. The nonlinearity of the generator imply that the latent space gives a distorted view of the input space. Under mild conditions, we show that this distortion can be characterized by a stochastic Riemannian metric, and demonstrate that distances and interpolants are significantly improved under this metric. This in turn improves probability distributions, sampling algorithms and clustering in the latent space. Our geometric analysis further reveals that current generators provide poor variance estimates and we propose a new generator architecture with vastly improved variance estimates. Results are demonstrated on convolutional and fully connected variational autoencoders, but the formalism easily generalize to other deep generative models. |
| Tasks | |
| Published | 2017-10-31 |
| URL | http://arxiv.org/abs/1710.11379v2 |
| http://arxiv.org/pdf/1710.11379v2.pdf | |
| PWC | https://paperswithcode.com/paper/latent-space-oddity-on-the-curvature-of-deep |
| Repo | https://github.com/RyanPyle1/RNN-VAE |
| Framework | none |
Hyperparameter Importance Across Datasets
| Title | Hyperparameter Importance Across Datasets |
| Authors | J. N. van Rijn, F. Hutter |
| Abstract | With the advent of automated machine learning, automated hyperparameter optimization methods are by now routinely used in data mining. However, this progress is not yet matched by equal progress on automatic analyses that yield information beyond performance-optimizing hyperparameter settings. In this work, we aim to answer the following two questions: Given an algorithm, what are generally its most important hyperparameters, and what are typically good values for these? We present methodology and a framework to answer these questions based on meta-learning across many datasets. We apply this methodology using the experimental meta-data available on OpenML to determine the most important hyperparameters of support vector machines, random forests and Adaboost, and to infer priors for all their hyperparameters. The results, obtained fully automatically, provide a quantitative basis to focus efforts in both manual algorithm design and in automated hyperparameter optimization. The conducted experiments confirm that the hyperparameters selected by the proposed method are indeed the most important ones and that the obtained priors also lead to statistically significant improvements in hyperparameter optimization. |
| Tasks | Hyperparameter Optimization, Meta-Learning |
| Published | 2017-10-12 |
| URL | http://arxiv.org/abs/1710.04725v2 |
| http://arxiv.org/pdf/1710.04725v2.pdf | |
| PWC | https://paperswithcode.com/paper/hyperparameter-importance-across-datasets |
| Repo | https://github.com/janvanrijn/openml-pimp |
| Framework | none |
Multi-scale Convolutional Neural Networks for Crowd Counting
| Title | Multi-scale Convolutional Neural Networks for Crowd Counting |
| Authors | Lingke Zeng, Xiangmin Xu, Bolun Cai, Suo Qiu, Tong Zhang |
| Abstract | Crowd counting on static images is a challenging problem due to scale variations. Recently deep neural networks have been shown to be effective in this task. However, existing neural-networks-based methods often use the multi-column or multi-network model to extract the scale-relevant features, which is more complicated for optimization and computation wasting. To this end, we propose a novel multi-scale convolutional neural network (MSCNN) for single image crowd counting. Based on the multi-scale blobs, the network is able to generate scale-relevant features for higher crowd counting performances in a single-column architecture, which is both accuracy and cost effective for practical applications. Complemental results show that our method outperforms the state-of-the-art methods on both accuracy and robustness with far less number of parameters. |
| Tasks | Crowd Counting |
| Published | 2017-02-08 |
| URL | http://arxiv.org/abs/1702.02359v1 |
| http://arxiv.org/pdf/1702.02359v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-scale-convolutional-neural-networks-for-1 |
| Repo | https://github.com/Ling-Bao/mscnn |
| Framework | tf |
A scikit-based Python environment for performing multi-label classification
| Title | A scikit-based Python environment for performing multi-label classification |
| Authors | Piotr Szymański, Tomasz Kajdanowicz |
| Abstract | scikit-multilearn is a Python library for performing multi-label classification. The library is compatible with the scikit/scipy ecosystem and uses sparse matrices for all internal operations. It provides native Python implementations of popular multi-label classification methods alongside a novel framework for label space partitioning and division. It includes modern algorithm adaptation methods, network-based label space division approaches, which extracts label dependency information and multi-label embedding classifiers. It provides python wrapped access to the extensive multi-label method stack from Java libraries and makes it possible to extend deep learning single-label methods for multi-label tasks. The library allows multi-label stratification and data set management. The implementation is more efficient in problem transformation than other established libraries, has good test coverage and follows PEP8. Source code and documentation can be downloaded from http://scikit.ml and also via pip. The library follows BSD licensing scheme. |
| Tasks | Multi-Label Classification |
| Published | 2017-02-05 |
| URL | http://arxiv.org/abs/1702.01460v5 |
| http://arxiv.org/pdf/1702.01460v5.pdf | |
| PWC | https://paperswithcode.com/paper/a-scikit-based-python-environment-for |
| Repo | https://github.com/scikit-multilearn/scikit-multilearn |
| Framework | none |
SEGAN: Speech Enhancement Generative Adversarial Network
| Title | SEGAN: Speech Enhancement Generative Adversarial Network |
| Authors | Santiago Pascual, Antonio Bonafonte, Joan Serrà |
| Abstract | Current speech enhancement techniques operate on the spectral domain and/or exploit some higher-level feature. The majority of them tackle a limited number of noise conditions and rely on first-order statistics. To circumvent these issues, deep networks are being increasingly used, thanks to their ability to learn complex functions from large example sets. In this work, we propose the use of generative adversarial networks for speech enhancement. In contrast to current techniques, we operate at the waveform level, training the model end-to-end, and incorporate 28 speakers and 40 different noise conditions into the same model, such that model parameters are shared across them. We evaluate the proposed model using an independent, unseen test set with two speakers and 20 alternative noise conditions. The enhanced samples confirm the viability of the proposed model, and both objective and subjective evaluations confirm the effectiveness of it. With that, we open the exploration of generative architectures for speech enhancement, which may progressively incorporate further speech-centric design choices to improve their performance. |
| Tasks | Speech Enhancement |
| Published | 2017-03-28 |
| URL | http://arxiv.org/abs/1703.09452v3 |
| http://arxiv.org/pdf/1703.09452v3.pdf | |
| PWC | https://paperswithcode.com/paper/segan-speech-enhancement-generative |
| Repo | https://github.com/sufengniu/segan_multi |
| Framework | tf |
Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation
| Title | Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation |
| Authors | Pietro Morerio, Jacopo Cavazza, Vittorio Murino |
| Abstract | In this work, we face the problem of unsupervised domain adaptation with a novel deep learning approach which leverages on our finding that entropy minimization is induced by the optimal alignment of second order statistics between source and target domains. We formally demonstrate this hypothesis and, aiming at achieving an optimal alignment in practical cases, we adopt a more principled strategy which, differently from the current Euclidean approaches, deploys alignment along geodesics. Our pipeline can be implemented by adding to the standard classification loss (on the labeled source domain), a source-to-target regularizer that is weighted in an unsupervised and data-driven fashion. We provide extensive experiments to assess the superiority of our framework on standard domain and modality adaptation benchmarks. |
| Tasks | Domain Adaptation, Unsupervised Domain Adaptation |
| Published | 2017-11-28 |
| URL | http://arxiv.org/abs/1711.10288v1 |
| http://arxiv.org/pdf/1711.10288v1.pdf | |
| PWC | https://paperswithcode.com/paper/minimal-entropy-correlation-alignment-for |
| Repo | https://github.com/pmorerio/minimal-entropy-correlation-alignment |
| Framework | tf |
Image Crowd Counting Using Convolutional Neural Network and Markov Random Field
| Title | Image Crowd Counting Using Convolutional Neural Network and Markov Random Field |
| Authors | Kang Han, Wanggen Wan, Haiyan Yao, Li Hou |
| Abstract | In this paper, we propose a method called Convolutional Neural Network-Markov Random Field (CNN-MRF) to estimate the crowd count in a still image. We first divide the dense crowd visible image into overlapping patches and then use a deep convolutional neural network to extract features from each patch image, followed by a fully connected neural network to regress the local patch crowd count. Since the local patches have overlapping portions, the crowd count of the adjacent patches has a high correlation. We use this correlation and the Markov random field to smooth the counting results of the local patches. Experiments show that our approach significantly outperforms the state-of-the-art methods on UCF and Shanghaitech crowd counting datasets. |
| Tasks | Crowd Counting |
| Published | 2017-06-12 |
| URL | http://arxiv.org/abs/1706.03686v3 |
| http://arxiv.org/pdf/1706.03686v3.pdf | |
| PWC | https://paperswithcode.com/paper/image-crowd-counting-using-convolutional |
| Repo | https://github.com/hankong/crowd-counting |
| Framework | tf |
Axiomatic Attribution for Deep Networks
| Title | Axiomatic Attribution for Deep Networks |
| Authors | Mukund Sundararajan, Ankur Taly, Qiqi Yan |
| Abstract | We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms—Sensitivity and Implementation Invariance that attribution methods ought to satisfy. We show that they are not satisfied by most known attribution methods, which we consider to be a fundamental weakness of those methods. We use the axioms to guide the design of a new attribution method called Integrated Gradients. Our method requires no modification to the original network and is extremely simple to implement; it just needs a few calls to the standard gradient operator. We apply this method to a couple of image models, a couple of text models and a chemistry model, demonstrating its ability to debug networks, to extract rules from a network, and to enable users to engage with models better. |
| Tasks | |
| Published | 2017-03-04 |
| URL | http://arxiv.org/abs/1703.01365v2 |
| http://arxiv.org/pdf/1703.01365v2.pdf | |
| PWC | https://paperswithcode.com/paper/axiomatic-attribution-for-deep-networks |
| Repo | https://github.com/jemilc/shap |
| Framework | tf |
Identifying beneficial task relations for multi-task learning in deep neural networks
| Title | Identifying beneficial task relations for multi-task learning in deep neural networks |
| Authors | Joachim Bingel, Anders Søgaard |
| Abstract | Multi-task learning (MTL) in deep neural networks for NLP has recently received increasing interest due to some compelling benefits, including its potential to efficiently regularize models and to reduce the need for labeled data. While it has brought significant improvements in a number of NLP tasks, mixed results have been reported, and little is known about the conditions under which MTL leads to gains in NLP. This paper sheds light on the specific task relations that can lead to gains from MTL models over single-task setups. |
| Tasks | Multi-Task Learning |
| Published | 2017-02-27 |
| URL | http://arxiv.org/abs/1702.08303v1 |
| http://arxiv.org/pdf/1702.08303v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-beneficial-task-relations-for |
| Repo | https://github.com/jbingel/eacl2017_mtl |
| Framework | tf |
