Paper Group ANR 1355
Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks. Flash Lightens Gray Pixels. Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation. MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry. Fast acoustic scattering using convolutional neural networks. Fine-grained eval …
Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks
| Title | Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks |
| Authors | Zi-Yi Dou, Keyi Yu, Antonios Anastasopoulos |
| Abstract | Learning general representations of text is a fundamental problem for many natural language understanding (NLU) tasks. Previously, researchers have proposed to use language model pre-training and multi-task learning to learn robust representations. However, these methods can achieve sub-optimal performance in low-resource scenarios. Inspired by the recent success of optimization-based meta-learning algorithms, in this paper, we explore the model-agnostic meta-learning algorithm (MAML) and its variants for low-resource NLU tasks. We validate our methods on the GLUE benchmark and show that our proposed models can outperform several strong baselines. We further empirically demonstrate that the learned representations can be adapted to new tasks efficiently and effectively. |
| Tasks | Language Modelling, Meta-Learning, Multi-Task Learning |
| Published | 2019-08-27 |
| URL | https://arxiv.org/abs/1908.10423v1 |
| https://arxiv.org/pdf/1908.10423v1.pdf | |
| PWC | https://paperswithcode.com/paper/investigating-meta-learning-algorithms-for |
| Repo | |
| Framework | |
Flash Lightens Gray Pixels
| Title | Flash Lightens Gray Pixels |
| Authors | Yanlin Qian, Song Yan, Joni-Kristian Kämäräinen, Jiri Matas |
| Abstract | In the real world, a scene is usually cast by multiple illuminants and herein we address the problem of spatial illumination estimation. Our solution is based on detecting gray pixels with the help of flash photography. We show that flash photography significantly improves the performance of gray pixel detection without illuminant prior, training data or calibration of the flash. We also introduce a novel flash photography dataset generated from the MIT intrinsic dataset. |
| Tasks | Calibration |
| Published | 2019-02-27 |
| URL | http://arxiv.org/abs/1902.10466v1 |
| http://arxiv.org/pdf/1902.10466v1.pdf | |
| PWC | https://paperswithcode.com/paper/flash-lightens-gray-pixels |
| Repo | |
| Framework | |
Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation
| Title | Probability Map Guided Bi-directional Recurrent UNet for Pancreas Segmentation |
| Authors | Jun Li, Xiaozhu Lin, Hui Che, Hao Li, Xiaohua Qian |
| Abstract | Pancreas segmentation in medical imaging data is of great significance for clinical pancreas diagnostics and treatment. However, the large population variations in the pancreas shape and volume cause enormous segmentation difficulties, even for state-of-the-art algorithms utilizing fully-convolutional neural networks (FCNs). Specifically, pancreas segmentation suffers from the loss of spatial information in 2D methods, and the high computational cost of 3D methods. To alleviate these problems, we propose a probabilistic-map-guided bi-directional recurrent UNet (PBR-UNet) architecture, which fuses intra-slice information and inter-slice probabilistic maps into a local 3D hybrid regularization scheme, which is followed by bi-directional recurrent network optimization. The PBR-UNet method consists of an initial estimation module for efficiently extracting pixel-level probabilistic maps and a primary segmentation module for propagating hybrid information through a 2.5D U-Net architecture. Specifically, local 3D information is inferred by combining an input image with the probabilistic maps of the adjacent slices into multichannel hybrid data, and then hierarchically aggregating the hybrid information of the entire segmentation network. Besides, a bi-directional recurrent optimization mechanism is developed to update the hybrid information in both the forward and the backward directions. This allows the proposed network to make full and optimal use of the local context information. Quantitative and qualitative evaluation was performed on the NIH Pancreas-CT dataset, and our proposed PBR-UNet method achieved better segmentation results with less computational cost compared to other state-of-the-art methods. |
| Tasks | Pancreas Segmentation |
| Published | 2019-03-03 |
| URL | https://arxiv.org/abs/1903.00923v4 |
| https://arxiv.org/pdf/1903.00923v4.pdf | |
| PWC | https://paperswithcode.com/paper/probability-map-guided-bi-directional |
| Repo | |
| Framework | |
MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry
| Title | MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetry |
| Authors | Yuchao Dai, Zhidong Zhu, Zhibo Rao, Bo Li |
| Abstract | The success of existing deep-learning based multi-view stereo (MVS) approaches greatly depends on the availability of large-scale supervision in the form of dense depth maps. Such supervision, while not always possible, tends to hinder the generalization ability of the learned models in never-seen-before scenarios. In this paper, we propose the first unsupervised learning based MVS network, which learns the multi-view depth maps from the input multi-view images and does not need ground-truth 3D training data. Our network is symmetric in predicting depth maps for all views simultaneously, where we enforce cross-view consistency of multi-view depth maps during both training and testing stages. Thus, the learned multi-view depth maps naturally comply with the underlying 3D scene geometry. Besides, our network also learns the multi-view occlusion maps, which further improves the robustness of our network in handling real-world occlusions. Experimental results on multiple benchmarking datasets demonstrate the effectiveness of our network and the excellent generalization ability. |
| Tasks | |
| Published | 2019-08-30 |
| URL | https://arxiv.org/abs/1908.11526v1 |
| https://arxiv.org/pdf/1908.11526v1.pdf | |
| PWC | https://paperswithcode.com/paper/mvs2-deep-unsupervised-multi-view-stereo-with |
| Repo | |
| Framework | |
Fast acoustic scattering using convolutional neural networks
| Title | Fast acoustic scattering using convolutional neural networks |
| Authors | Ziqi Fan, Vibhav Vineet, Hannes Gamper, Nikunj Raghuvanshi |
| Abstract | Diffracted scattering and occlusion are important acoustic effects in interactive auralization and noise control applications, typically requiring expensive numerical simulation. We propose training a convolutional neural network to map from a convex scatterer’s cross-section to a 2D slice of the resulting spatial loudness distribution. We show that employing a full-resolution residual network for the resulting image-to-image regression problem yields spatially detailed loudness fields with a root-mean-squared error of less than 1 dB, at over 100x speedup compared to full wave simulation. |
| Tasks | |
| Published | 2019-10-30 |
| URL | https://arxiv.org/abs/1911.01802v3 |
| https://arxiv.org/pdf/1911.01802v3.pdf | |
| PWC | https://paperswithcode.com/paper/fast-acoustic-scattering-using-convolutional |
| Repo | |
| Framework | |
Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically-motivated Test Suite
| Title | Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically-motivated Test Suite |
| Authors | Avramidis Eleftherios, Vivien Macketanz, Arle Lommel, Hans Uszkoreit |
| Abstract | We present an alternative method of evaluating Quality Estimation systems, which is based on a linguistically-motivated Test Suite. We create a test-set consisting of 14 linguistic error categories and we gather for each of them a set of samples with both correct and erroneous translations. Then, we measure the performance of 5 Quality Estimation systems by checking their ability to distinguish between the correct and the erroneous translations. The detailed results are much more informative about the ability of each system. The fact that different Quality Estimation systems perform differently at various phenomena confirms the usefulness of the Test Suite. |
| Tasks | Machine Translation |
| Published | 2019-10-16 |
| URL | https://arxiv.org/abs/1910.07468v1 |
| https://arxiv.org/pdf/1910.07468v1.pdf | |
| PWC | https://paperswithcode.com/paper/fine-grained-evaluation-of-quality-estimation-1 |
| Repo | |
| Framework | |
Integrating Social Media into a Pan-European Flood Awareness System: A Multilingual Approach
| Title | Integrating Social Media into a Pan-European Flood Awareness System: A Multilingual Approach |
| Authors | V. Lorini, C. Castillo, F. Dottori, M. Kalas, D. Nappo, P. Salamon |
| Abstract | This paper describes a prototype system that integrates social media analysis into the European Flood Awareness System (EFAS). This integration allows the collection of social media data to be automatically triggered by flood risk warnings determined by a hydro-meteorological model. Then, we adopt a multi-lingual approach to find flood-related messages by employing two state-of-the-art methodologies: language-agnostic word embeddings and language-aligned word embeddings. Both approaches can be used to bootstrap a classifier of social media messages for a new language with little or no labeled data. Finally, we describe a method for selecting relevant and representative messages and displaying them back in the interface of EFAS. |
| Tasks | Word Embeddings |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10876v1 |
| http://arxiv.org/pdf/1904.10876v1.pdf | |
| PWC | https://paperswithcode.com/paper/integrating-social-media-into-a-pan-european |
| Repo | |
| Framework | |
Robust Reinforcement Learning in POMDPs with Incomplete and Noisy Observations
| Title | Robust Reinforcement Learning in POMDPs with Incomplete and Noisy Observations |
| Authors | Yuhui Wang, Hao He, Xiaoyang Tan |
| Abstract | In real-world scenarios, the observation data for reinforcement learning with continuous control is commonly noisy and part of it may be dynamically missing over time, which violates the assumption of many current methods developed for this. We addressed the issue within the framework of partially observable Markov Decision Process (POMDP) using a model-based method, in which the transition model is estimated from the incomplete and noisy observations using a newly proposed surrogate loss function with local approximation, while the policy and value function is learned with the help of belief imputation. For the latter purpose, a generative model is constructed and is seamlessly incorporated into the belief updating procedure of POMDP, which enables robust execution even under a significant incompleteness and noise. The effectiveness of the proposed method is verified on a collection of benchmark tasks, showing that our approach outperforms several compared methods under various challenging scenarios. |
| Tasks | Continuous Control, Imputation |
| Published | 2019-02-15 |
| URL | http://arxiv.org/abs/1902.05795v1 |
| http://arxiv.org/pdf/1902.05795v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-reinforcement-learning-in-pomdps-with |
| Repo | |
| Framework | |
Learning to Convolve: A Generalized Weight-Tying Approach
| Title | Learning to Convolve: A Generalized Weight-Tying Approach |
| Authors | Nichita Diaconu, Daniel E Worrall |
| Abstract | Recent work (Cohen & Welling, 2016) has shown that generalizations of convolutions, based on group theory, provide powerful inductive biases for learning. In these generalizations, filters are not only translated but can also be rotated, flipped, etc. However, coming up with exact models of how to rotate a 3 x 3 filter on a square pixel-grid is difficult. In this paper, we learn how to transform filters for use in the group convolution, focussing on roto-translation. For this, we learn a filter basis and all rotated versions of that filter basis. Filters are then encoded by a set of rotation invariant coefficients. To rotate a filter, we switch the basis. We demonstrate we can produce feature maps with low sensitivity to input rotations, while achieving high performance on MNIST and CIFAR-10. |
| Tasks | |
| Published | 2019-05-12 |
| URL | https://arxiv.org/abs/1905.04663v1 |
| https://arxiv.org/pdf/1905.04663v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-convolve-a-generalized-weight |
| Repo | |
| Framework | |
GRIm-RePR: Prioritising Generating Important Features for Pseudo-Rehearsal
| Title | GRIm-RePR: Prioritising Generating Important Features for Pseudo-Rehearsal |
| Authors | Craig Atkinson, Brendan McCane, Lech Szymanski, Anthony Robins |
| Abstract | Pseudo-rehearsal allows neural networks to learn a sequence of tasks without forgetting how to perform in earlier tasks. Preventing forgetting is achieved by introducing a generative network which can produce data from previously seen tasks so that it can be rehearsed along side learning the new task. This has been found to be effective in both supervised and reinforcement learning. Our current work aims to further prevent forgetting by encouraging the generator to accurately generate features important for task retention. More specifically, the generator is improved by introducing a second discriminator into the Generative Adversarial Network which learns to classify between real and fake items from the intermediate activation patterns that they produce when fed through a continual learning agent. Using Atari 2600 games, we experimentally find that improving the generator can considerably reduce catastrophic forgetting compared to the standard pseudo-rehearsal methods used in deep reinforcement learning. Furthermore, we propose normalising the Q-values taught to the long-term system as we observe this substantially reduces catastrophic forgetting by minimising the interference between tasks’ reward functions. |
| Tasks | Atari Games, Continual Learning |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.11988v1 |
| https://arxiv.org/pdf/1911.11988v1.pdf | |
| PWC | https://paperswithcode.com/paper/grim-repr-prioritising-generating-important |
| Repo | |
| Framework | |
A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection
| Title | A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection |
| Authors | Zhe Chen, Wanli Ouyang, Tongliang Liu, Dacheng Tao |
| Abstract | Deep learning-based computer vision is usually data-hungry. Many researchers attempt to augment datasets with synthesized data to improve model robustness. However, the augmentation of popular pedestrian datasets, such as Caltech and Citypersons, can be extremely challenging because real pedestrians are commonly in low quality. Due to the factors like occlusions, blurs, and low-resolution, it is significantly difficult for existing augmentation approaches, which generally synthesize data using 3D engines or generative adversarial networks (GANs), to generate realistic-looking pedestrians. Alternatively, to access much more natural-looking pedestrians, we propose to augment pedestrian detection datasets by transforming real pedestrians from the same dataset into different shapes. Accordingly, we propose the Shape Transformation-based Dataset Augmentation (STDA) framework. The proposed framework is composed of two subsequent modules, i.e. the shape-guided deformation and the environment adaptation. In the first module, we introduce a shape-guided warping field to help deform the shape of a real pedestrian into a different shape. Then, in the second stage, we propose an environment-aware blending map to better adapt the deformed pedestrians into surrounding environments, obtaining more realistic-looking pedestrians and more beneficial augmentation results for pedestrian detection. Extensive empirical studies on different pedestrian detection benchmarks show that the proposed STDA framework consistently produces much better augmentation results than other pedestrian synthesis approaches using low-quality pedestrians. By augmenting the original datasets, our proposed framework also improves the baseline pedestrian detector by up to 38% on the evaluated benchmarks, achieving state-of-the-art performance. |
| Tasks | Pedestrian Detection |
| Published | 2019-12-15 |
| URL | https://arxiv.org/abs/1912.07010v1 |
| https://arxiv.org/pdf/1912.07010v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-shape-transformation-based-dataset |
| Repo | |
| Framework | |
Policy Gradient Search: Online Planning and Expert Iteration without Search Trees
| Title | Policy Gradient Search: Online Planning and Expert Iteration without Search Trees |
| Authors | Thomas Anthony, Robert Nishihara, Philipp Moritz, Tim Salimans, John Schulman |
| Abstract | Monte Carlo Tree Search (MCTS) algorithms perform simulation-based search to improve policies online. During search, the simulation policy is adapted to explore the most promising lines of play. MCTS has been used by state-of-the-art programs for many problems, however a disadvantage to MCTS is that it estimates the values of states with Monte Carlo averages, stored in a search tree; this does not scale to games with very high branching factors. We propose an alternative simulation-based search method, Policy Gradient Search (PGS), which adapts a neural network simulation policy online via policy gradient updates, avoiding the need for a search tree. In Hex, PGS achieves comparable performance to MCTS, and an agent trained using Expert Iteration with PGS was able defeat MoHex 2.0, the strongest open-source Hex agent, in 9x9 Hex. |
| Tasks | |
| Published | 2019-04-07 |
| URL | http://arxiv.org/abs/1904.03646v1 |
| http://arxiv.org/pdf/1904.03646v1.pdf | |
| PWC | https://paperswithcode.com/paper/policy-gradient-search-online-planning-and |
| Repo | |
| Framework | |
Non-Invasive Fuhrman Grading of Clear Cell Renal Cell Carcinoma Using Computed Tomography Radiomics Features and Machine Learning
| Title | Non-Invasive Fuhrman Grading of Clear Cell Renal Cell Carcinoma Using Computed Tomography Radiomics Features and Machine Learning |
| Authors | Mostafa Nazari, Isaac Shiri, Ghasem Hajianfar, Niki Oveisi, Hamid Abdollahi, Mohammad Reza Deevband, Mehrdad Oveisi |
| Abstract | Purpose: To identify optimal classification methods for computed tomography (CT) radiomics-based preoperative prediction of clear cells renal cell carcinoma (ccRCC) grade. Methods and material: Seventy one ccRCC patients were included in the study. Three image preprocessing techniques (Laplacian of Gaussian, wavelet filter, and discretization of the intensity values) were applied on tumor volumes. In total, 2530 radiomics features (tumor shape and size, intensity statistics, and texture) were extracted from each segmented tumor volume. Univariate analysis was performed to assess the association of each feature with the histological condition. In the case of multivariate analysis, the following was implemented: three feature selection including the least absolute shrinkage and selection operator (LASSO), students t-test and minimum Redundancy Maximum Relevance (mRMR) algorithms. These selected features were then used to construct three classification models (SVM, random forest, and logistic regression) to discriminate the high from low-grade ccRCC at nephrectomy. Lastly, multivariate model performance was evaluated on the bootstrapped validation cohort using the area under receiver operating characteristic curve (AUC). Results: Univariate analysis demonstrated that among different image sets, 128 bin discretized images have statistically significant different (q-value < 0.05) texture parameters with a mean of AUC 0.74 (q-value < 0.05). The three ML-based classifier shows proficient discrimination of the high from low-grade ccRCC. The AUC was 0.78 in logistic regression, 0.62 in random forest, and 0.83 in SVM model, respectively. Conclusion: Radiomics features can be a useful and promising non-invasive method for preoperative evaluation of ccRCC Fuhrman grades. Key words: RCC, Radiomics, Machine Learning, Computed Tomography |
| Tasks | Computed Tomography (CT), Feature Selection |
| Published | 2019-09-26 |
| URL | https://arxiv.org/abs/1909.12286v1 |
| https://arxiv.org/pdf/1909.12286v1.pdf | |
| PWC | https://paperswithcode.com/paper/non-invasive-fuhrman-grading-of-clear-cell |
| Repo | |
| Framework | |
A Variational Autoencoder for Probabilistic Non-Negative Matrix Factorisation
| Title | A Variational Autoencoder for Probabilistic Non-Negative Matrix Factorisation |
| Authors | Steven Squires, Adam Prügel Bennett, Mahesan Niranjan |
| Abstract | We introduce and demonstrate the variational autoencoder (VAE) for probabilistic non-negative matrix factorisation (PAE-NMF). We design a network which can perform non-negative matrix factorisation (NMF) and add in aspects of a VAE to make the coefficients of the latent space probabilistic. By restricting the weights in the final layer of the network to be non-negative and using the non-negative Weibull distribution we produce a probabilistic form of NMF which allows us to generate new data and find a probability distribution that effectively links the latent and input variables. We demonstrate the effectiveness of PAE-NMF on three heterogeneous datasets: images, financial time series and genomic. |
| Tasks | Time Series |
| Published | 2019-06-13 |
| URL | https://arxiv.org/abs/1906.05912v1 |
| https://arxiv.org/pdf/1906.05912v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-variational-autoencoder-for-probabilistic-1 |
| Repo | |
| Framework | |
Spatially-Coupled Neural Network Architectures
| Title | Spatially-Coupled Neural Network Architectures |
| Authors | Arman Hasanzadeh, Nagaraj T. Janakiraman, Vamsi K. Amalladinne, Krishna R. Narayanan |
| Abstract | In this work, we leverage advances in sparse coding techniques to reduce the number of trainable parameters in a fully connected neural network. While most of the works in literature impose $\ell_1$ regularization, DropOut or DropConnect techniques to induce sparsity, our scheme considers feature importance as a criterion to allocate the trainable parameters (resources) efficiently in the network. Even though sparsity is ensured, $\ell_1$ regularization requires training on all the resources in a deep neural network. The DropOut/DropConnect techniques reduce the number of trainable parameters in the training stage by dropping a random collection of neurons/edges in the hidden layers. However, both these techniques do not pay heed to the underlying structure in the data when dropping the neurons/edges. Moreover, these frameworks require a storage space equivalent to the number of parameters in a fully connected neural network. We address the above issues with a more structured architecture inspired from spatially-coupled sparse constructions. The proposed architecture is shown to have a performance akin to a conventional fully connected neural network with dropouts, and yet achieving a $94%$ reduction in the training parameters. Extensive simulations are presented and the performance of the proposed scheme is compared against traditional neural network architectures. |
| Tasks | Feature Importance |
| Published | 2019-07-03 |
| URL | https://arxiv.org/abs/1907.02051v1 |
| https://arxiv.org/pdf/1907.02051v1.pdf | |
| PWC | https://paperswithcode.com/paper/spatially-coupled-neural-network |
| Repo | |
| Framework | |
