Paper Group AWR 275
Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval. A Model to Search for Synthesizable Molecules. Binarized Knowledge Graph Embeddings. OpenBioLink: A benchmarking framework for large-scale biomedical link prediction. FastFusionNet: New State-of-the-Art for DAWNBench SQuAD. A Hitchhiker’s Guide to Statistical Comparison …
Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval
| Title | Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval |
| Authors | Lianli Gao, Xiaosu Zhu, Jingkuan Song, Zhou Zhao, Heng Tao Shen |
| Abstract | Product Quantization (PQ) has long been a mainstream for generating an exponentially large codebook at very low memory/time cost. Despite its success, PQ is still tricky for the decomposition of high-dimensional vector space, and the retraining of model is usually unavoidable when the code length changes. In this work, we propose a deep progressive quantization (DPQ) model, as an alternative to PQ, for large scale image retrieval. DPQ learns the quantization codes sequentially and approximates the original feature space progressively. Therefore, we can train the quantization codes with different code lengths simultaneously. Specifically, we first utilize the label information for guiding the learning of visual features, and then apply several quantization blocks to progressively approach the visual features. Each quantization block is designed to be a layer of a convolutional neural network, and the whole framework can be trained in an end-to-end manner. Experimental results on the benchmark datasets show that our model significantly outperforms the state-of-the-art for image retrieval. Our model is trained once for different code lengths and therefore requires less computation time. Additional ablation study demonstrates the effect of each component of our proposed model. Our code is released at https://github.com/cfm-uestc/DPQ. |
| Tasks | Image Retrieval, Quantization |
| Published | 2019-06-16 |
| URL | https://arxiv.org/abs/1906.06698v2 |
| https://arxiv.org/pdf/1906.06698v2.pdf | |
| PWC | https://paperswithcode.com/paper/beyond-product-quantization-deep-progressive |
| Repo | https://github.com/cfm-uestc/DPQ |
| Framework | tf |
A Model to Search for Synthesizable Molecules
| Title | A Model to Search for Synthesizable Molecules |
| Authors | John Bradshaw, Brooks Paige, Matt J. Kusner, Marwin H. S. Segler, José Miguel Hernández-Lobato |
| Abstract | Deep generative models are able to suggest new organic molecules by generating strings, trees, and graphs representing their structure. While such models allow one to generate molecules with desirable properties, they give no guarantees that the molecules can actually be synthesized in practice. We propose a new molecule generation model, mirroring a more realistic real-world process, where (a) reactants are selected, and (b) combined to form more complex molecules. More specifically, our generative model proposes a bag of initial reactants (selected from a pool of commercially-available molecules) and uses a reaction model to predict how they react together to generate new molecules. We first show that the model can generate diverse, valid and unique molecules due to the useful inductive biases of modeling reactions. Furthermore, our model allows chemists to interrogate not only the properties of the generated molecules but also the feasibility of the synthesis routes. We conclude by using our model to solve retrosynthesis problems, predicting a set of reactants that can produce a target product. |
| Tasks | |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05221v2 |
| https://arxiv.org/pdf/1906.05221v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-model-to-search-for-synthesizable-molecules |
| Repo | https://github.com/john-bradshaw/molecule-chef |
| Framework | pytorch |
Binarized Knowledge Graph Embeddings
| Title | Binarized Knowledge Graph Embeddings |
| Authors | Koki Kishimoto, Katsuhiko Hayashi, Genki Akai, Masashi Shimbo, Kazunori Komatani |
| Abstract | Tensor factorization has become an increasingly popular approach to knowledge graph completion(KGC), which is the task of automatically predicting missing facts in a knowledge graph. However, even with a simple model like CANDECOMP/PARAFAC(CP) tensor decomposition, KGC on existing knowledge graphs is impractical in resource-limited environments, as a large amount of memory is required to store parameters represented as 32-bit or 64-bit floating point numbers. This limitation is expected to become more stringent as existing knowledge graphs, which are already huge, keep steadily growing in scale. To reduce the memory requirement, we present a method for binarizing the parameters of the CP tensor decomposition by introducing a quantization function to the optimization problem. This method replaces floating point-valued parameters with binary ones after training, which drastically reduces the model size at run time. We investigate the trade-off between the quality and size of tensor factorization models for several KGC benchmark datasets. In our experiments, the proposed method successfully reduced the model size by more than an order of magnitude while maintaining the task performance. Moreover, a fast score computation technique can be developed with bitwise operations. |
| Tasks | Knowledge Graph Completion, Knowledge Graph Embeddings, Knowledge Graphs, Quantization |
| Published | 2019-02-08 |
| URL | http://arxiv.org/abs/1902.02970v1 |
| http://arxiv.org/pdf/1902.02970v1.pdf | |
| PWC | https://paperswithcode.com/paper/binarized-knowledge-graph-embeddings |
| Repo | https://github.com/KokiKishimoto/cp_decomposition |
| Framework | none |
OpenBioLink: A benchmarking framework for large-scale biomedical link prediction
| Title | OpenBioLink: A benchmarking framework for large-scale biomedical link prediction |
| Authors | Anna Breit, Simon Ott, Asan Agibetov, Matthias Samwald |
| Abstract | SUMMARY: Recently, novel machine-learning algorithms have shown potential for predicting undiscovered links in biomedical knowledge networks. However, dedicated benchmarks for measuring algorithmic progress have not yet emerged. With OpenBioLink, we introduce a large-scale, high-quality and highly challenging biomedical link prediction benchmark to transparently and reproducibly evaluate such algorithms. Furthermore, we present preliminary baseline evaluation results. AVAILABILITY AND IMPLEMENTATION: Source code, data and supplementary files are openly available at https://github.com/OpenBioLink/OpenBioLink CONTACT: matthias.samwald ((at)) meduniwien.ac.at |
| Tasks | Link Prediction |
| Published | 2019-12-10 |
| URL | https://arxiv.org/abs/1912.04616v2 |
| https://arxiv.org/pdf/1912.04616v2.pdf | |
| PWC | https://paperswithcode.com/paper/openbiolink-a-resource-and-benchmarking |
| Repo | https://github.com/OpenBioLink/OpenBioLink |
| Framework | jax |
FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
| Title | FastFusionNet: New State-of-the-Art for DAWNBench SQuAD |
| Authors | Felix Wu, Boyi Li, Lequn Wang, Ni Lao, John Blitzer, Kilian Q. Weinberger |
| Abstract | In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet. |
| Tasks | Reading Comprehension |
| Published | 2019-02-28 |
| URL | http://arxiv.org/abs/1902.11291v2 |
| http://arxiv.org/pdf/1902.11291v2.pdf | |
| PWC | https://paperswithcode.com/paper/fastfusionnet-new-state-of-the-art-for |
| Repo | https://github.com/felixgwu/FastFusionNet |
| Framework | pytorch |
A Hitchhiker’s Guide to Statistical Comparisons of Reinforcement Learning Algorithms
| Title | A Hitchhiker’s Guide to Statistical Comparisons of Reinforcement Learning Algorithms |
| Authors | Cédric Colas, Olivier Sigaud, Pierre-Yves Oudeyer |
| Abstract | Consistently checking the statistical significance of experimental results is the first mandatory step towards reproducible science. This paper presents a hitchhiker’s guide to rigorous comparisons of reinforcement learning algorithms. After introducing the concepts of statistical testing, we review the relevant statistical tests and compare them empirically in terms of false positive rate and statistical power as a function of the sample size (number of seeds) and effect size. We further investigate the robustness of these tests to violations of the most common hypotheses (normal distributions, same distributions, equal variances). Beside simulations, we compare empirical distributions obtained by running Soft-Actor Critic and Twin-Delayed Deep Deterministic Policy Gradient on Half-Cheetah. We conclude by providing guidelines and code to perform rigorous comparisons of RL algorithm performances. |
| Tasks | |
| Published | 2019-04-15 |
| URL | http://arxiv.org/abs/1904.06979v1 |
| http://arxiv.org/pdf/1904.06979v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-hitchhikers-guide-to-statistical |
| Repo | https://github.com/ccolas/rl_stats |
| Framework | none |
DisplaceNet: Recognising Displaced People from Images by Exploiting Dominance Level
| Title | DisplaceNet: Recognising Displaced People from Images by Exploiting Dominance Level |
| Authors | Grigorios Kalliatakis, Shoaib Ehsan, Maria Fasli, Klaus McDonald-Maier |
| Abstract | Every year millions of men, women and children are forced to leave their homes and seek refuge from wars, human rights violations, persecution, and natural disasters. The number of forcibly displaced people came at a record rate of 44,400 every day throughout 2017, raising the cumulative total to 68.5 million at the years end, overtaken the total population of the United Kingdom. Up to 85% of the forcibly displaced find refuge in low- and middle-income countries, calling for increased humanitarian assistance worldwide. To reduce the amount of manual labour required for human-rights-related image analysis, we introduce DisplaceNet, a novel model which infers potential displaced people from images by integrating the control level of the situation and conventional convolutional neural network (CNN) classifier into one framework for image classification. Experimental results show that DisplaceNet achieves up to 4% coverage-the proportion of a data set for which a classifier is able to produce a prediction-gain over the sole use of a CNN classifier. Our dataset, codes and trained models will be available online at https://github.com/GKalliatakis/DisplaceNet. |
| Tasks | Displaced People Recognition, Image Classification |
| Published | 2019-05-03 |
| URL | https://arxiv.org/abs/1905.02025v1 |
| https://arxiv.org/pdf/1905.02025v1.pdf | |
| PWC | https://paperswithcode.com/paper/displacenet-recognising-displaced-people-from |
| Repo | https://github.com/GKalliatakis/DisplaceNet |
| Framework | tf |
Convolutional Conditional Neural Processes
| Title | Convolutional Conditional Neural Processes |
| Authors | Jonathan Gordon, Wessel P. Bruinsma, Andrew Y. K. Foong, James Requeima, Yann Dubois, Richard E. Turner |
| Abstract | We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds data sets into an infinite-dimensional function space as opposed to a finite-dimensional vector space. To formalize this notion, we extend the theory of neural representations of sets to include functional representations, and demonstrate that any translation-equivariant embedding can be represented using a convolutional deep set. We evaluate ConvCNPs in several settings, demonstrating that they achieve state-of-the-art performance compared to existing NPs. We demonstrate that building in translation equivariance enables zero-shot generalization to challenging, out-of-domain tasks. |
| Tasks | Time Series |
| Published | 2019-10-29 |
| URL | https://arxiv.org/abs/1910.13556v3 |
| https://arxiv.org/pdf/1910.13556v3.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-conditional-neural-processes-1 |
| Repo | https://github.com/makora9143/pytorch-convcnp |
| Framework | pytorch |
Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks
| Title | Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks |
| Authors | Yuhang Li, Xin Dong, Wei Wang |
| Abstract | We propose Additive Powers-of-Two~(APoT) quantization, an efficient non-uniform quantization scheme for the bell-shaped and long-tailed distribution of weights and activations in neural networks. By constraining all quantization levels as the sum of Powers-of-Two terms, APoT quantization enjoys high computational efficiency and a good match with the distribution of weights. A simple reparameterization of the clipping function is applied to generate a better-defined gradient for learning the clipping threshold. Moreover, weight normalization is presented to refine the distribution of weights to make the training more stable and consistent. Experimental results show that our proposed method outperforms state-of-the-art methods, and is even competitive with the full-precision models, demonstrating the effectiveness of our proposed APoT quantization. For example, our 4-bit quantized ResNet-50 on ImageNet achieves 76.6% top-1 accuracy without bells and whistles; meanwhile, our model reduces 22% computational cost compared with the uniformly quantized counterpart. The code is available at https://github.com/yhhhli/APoT_Quantization. |
| Tasks | Quantization |
| Published | 2019-09-28 |
| URL | https://arxiv.org/abs/1909.13144v2 |
| https://arxiv.org/pdf/1909.13144v2.pdf | |
| PWC | https://paperswithcode.com/paper/additive-powers-of-two-quantization-a-non |
| Repo | https://github.com/yhhhli/APoT_Quantization |
| Framework | pytorch |
Fully Parameterized Quantile Function for Distributional Reinforcement Learning
| Title | Fully Parameterized Quantile Function for Distributional Reinforcement Learning |
| Authors | Derek Yang, Li Zhao, Zichuan Lin, Tao Qin, Jiang Bian, Tieyan Liu |
| Abstract | Distributional Reinforcement Learning (RL) differs from traditional RL in that, rather than the expectation of total returns, it estimates distributions and has achieved state-of-the-art performance on Atari Games. The key challenge in practical distributional RL algorithms lies in how to parameterize estimated distributions so as to better approximate the true continuous distribution. Existing distributional RL algorithms parameterize either the probability side or the return value side of the distribution function, leaving the other side uniformly fixed as in C51, QR-DQN or randomly sampled as in IQN. In this paper, we propose fully parameterized quantile function that parameterizes both the quantile fraction axis (i.e., the x-axis) and the value axis (i.e., y-axis) for distributional RL. Our algorithm contains a fraction proposal network that generates a discrete set of quantile fractions and a quantile value network that gives corresponding quantile values. The two networks are jointly trained to find the best approximation of the true distribution. Experiments on 55 Atari Games show that our algorithm significantly outperforms existing distributional RL algorithms and creates a new record for the Atari Learning Environment for non-distributed agents. |
| Tasks | Atari Games, Distributional Reinforcement Learning |
| Published | 2019-11-05 |
| URL | https://arxiv.org/abs/1911.02140v2 |
| https://arxiv.org/pdf/1911.02140v2.pdf | |
| PWC | https://paperswithcode.com/paper/fully-parameterized-quantile-function-for |
| Repo | https://github.com/ku2482/fqf-iqn-qrdqn.pytorch |
| Framework | pytorch |
Dual Variational Generation for Low-Shot Heterogeneous Face Recognition
| Title | Dual Variational Generation for Low-Shot Heterogeneous Face Recognition |
| Authors | Chaoyou Fu, Xiang Wu, Yibo Hu, Huaibo Huang, Ran He |
| Abstract | Heterogeneous Face Recognition (HFR) is a challenging issue because of the large domain discrepancy and a lack of heterogeneous data. This paper considers HFR as a dual generation problem, and proposes a novel Dual Variational Generation (DVG) framework. It generates large-scale new paired heterogeneous images with the same identity from noise, for the sake of reducing the domain gap of HFR. Specifically, we first introduce a dual variational autoencoder to represent a joint distribution of paired heterogeneous images. Then, in order to ensure the identity consistency of the generated paired heterogeneous images, we impose a distribution alignment in the latent space and a pairwise identity preserving in the image space. Moreover, the HFR network reduces the domain discrepancy by constraining the pairwise feature distances between the generated paired heterogeneous images. Extensive experiments on four HFR databases show that our method can significantly improve state-of-the-art results. The related code is available at https://github.com/BradyFU/DVG. |
| Tasks | Face Recognition, Heterogeneous Face Recognition |
| Published | 2019-03-25 |
| URL | https://arxiv.org/abs/1903.10203v3 |
| https://arxiv.org/pdf/1903.10203v3.pdf | |
| PWC | https://paperswithcode.com/paper/dual-variational-generation-for-low-shot |
| Repo | https://github.com/BradyFU/DVG |
| Framework | pytorch |
EPP: interpretable score of model predictive power
| Title | EPP: interpretable score of model predictive power |
| Authors | Alicja Gosiewska, Mateusz Bakala, Katarzyna Woznica, Maciej Zwolinski, Przemyslaw Biecek |
| Abstract | The most important part of model selection and hyperparameter tuning is the evaluation of model performance. The most popular measures, such as AUC, F1, ACC for binary classification, or RMSE, MAD for regression, or cross-entropy for multilabel classification share two common weaknesses. First is, that they are not on an interval scale. It means that the difference in performance for the two models has no direct interpretation. It makes no sense to compare such differences between datasets. Second is, that for k-fold cross-validation, the model performance is in most cases calculated as an average performance from particular folds, which neglects the information how stable is the performance for different folds. In this talk, we introduce a new EPP rating system for predictive models. We also demonstrate numerous advantages for this system, First, differences in EPP scores have probabilistic interpretation. Based on it we can assess the probability that one model will achieve better performance than another. Second, EPP scores can be directly compared between datasets. Third, they can be used for navigated hyperparameter tuning and model selection. Forth, we can create embeddings for datasets based on EPP scores. |
| Tasks | Model Selection |
| Published | 2019-08-24 |
| URL | https://arxiv.org/abs/1908.09213v1 |
| https://arxiv.org/pdf/1908.09213v1.pdf | |
| PWC | https://paperswithcode.com/paper/epp-interpretable-score-of-model-predictive |
| Repo | https://github.com/ModelOriented/DrWhy |
| Framework | none |
Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics
| Title | Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics |
| Authors | Vincent Casser, Soeren Pirk, Reza Mahjourian, Anelia Angelova |
| Abstract | We present an approach which takes advantage of both structure and semantics for unsupervised monocular learning of depth and ego-motion. More specifically, we model the motion of individual objects and learn their 3D motion vector jointly with depth and ego-motion. We obtain more accurate results, especially for challenging dynamic scenes not addressed by previous approaches. This is an extended version of Casser et al. [AAAI’19]. Code and models have been open sourced at https://sites.google.com/corp/view/struct2depth. |
| Tasks | Depth And Camera Motion, Depth Estimation, Motion Estimation |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05717v1 |
| https://arxiv.org/pdf/1906.05717v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-monocular-depth-and-ego-motion |
| Repo | https://github.com/tensorflow/models/tree/master/research/struct2depth |
| Framework | tf |
Edge-Direct Visual Odometry
| Title | Edge-Direct Visual Odometry |
| Authors | Kevin Christensen, Martial Hebert |
| Abstract | In this paper we propose an edge-direct visual odometry algorithm that efficiently utilizes edge pixels to find the relative pose that minimizes the photometric error between images. Prior work on exploiting edge pixels instead treats edges as features and employ various techniques to match edge lines or pixels, which adds unnecessary complexity. Direct methods typically operate on all pixel intensities, which proves to be highly redundant. In contrast our method builds on direct visual odometry methods naturally with minimal added computation. It is not only more efficient than direct dense methods since we iterate with a fraction of the pixels, but also more accurate. We achieve high accuracy and efficiency by extracting edges from only one image, and utilize robust Gauss-Newton to minimize the photometric error of these edge pixels. This simultaneously finds the edge pixels in the reference image, as well as the relative camera pose that minimizes the photometric error. We test various edge detectors, including learned edges, and determine that the optimal edge detector for this method is the Canny edge detection algorithm using automatic thresholding. We highlight key differences between our edge direct method and direct dense methods, in particular how higher levels of image pyramids can lead to significant aliasing effects and result in incorrect solution convergence. We show experimentally that reducing the photometric error of edge pixels also reduces the photometric error of all pixels, and we show through an ablation study the increase in accuracy obtained by optimizing edge pixels only. We evaluate our method on the RGB-D TUM benchmark on which we achieve state-of-the-art performance. |
| Tasks | Edge Detection, Visual Odometry |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.04838v1 |
| https://arxiv.org/pdf/1906.04838v1.pdf | |
| PWC | https://paperswithcode.com/paper/edge-direct-visual-odometry |
| Repo | https://github.com/kevinchristensen1/EdgeDirectVO |
| Framework | none |
Variational Prototype Replays for Continual Learning
| Title | Variational Prototype Replays for Continual Learning |
| Authors | Mengmi Zhang, Tao Wang, Joo Hwee Lim, Gabriel Kreiman, Jiashi Feng |
| Abstract | Continual learning refers to the ability to acquire and transfer knowledge without catastrophically forgetting what was previously learned. In this work, we consider \emph{few-shot} continual learning in classification tasks, and we propose a novel method, Variational Prototype Replays, that efficiently consolidates and recalls previous knowledge to avoid catastrophic forgetting. In each classification task, our method learns a set of variational prototypes with their means and variances, where embedding of the samples from the same class can be represented in a prototypical distribution and class-representative prototypes are separated apart. To alleviate catastrophic forgetting, our method replays one sample per class from previous tasks, and correspondingly matches newly predicted embeddings to their nearest class-representative prototypes stored from previous tasks. Compared with recent continual learning approaches, our method can readily adapt to new tasks with more classes without requiring the addition of new units. Furthermore, our method is more memory efficient since only class-representative prototypes with their means and variances, as well as only one sample per class from previous tasks need to be stored. Without tampering with the performance on initial tasks, our method learns novel concepts given a few training examples of each class in new tasks. |
| Tasks | Continual Learning, Metric Learning |
| Published | 2019-05-23 |
| URL | https://arxiv.org/abs/1905.09447v3 |
| https://arxiv.org/pdf/1905.09447v3.pdf | |
| PWC | https://paperswithcode.com/paper/prototype-reminding-for-continual-learning |
| Repo | https://github.com/kreimanlab/VariationalPrototypeReplaysCL |
| Framework | pytorch |
