Paper Group ANR 850
Geometry of the Hough transforms with applications to synthetic data. On Dropout and Nuclear Norm Regularization. Accelerating Training using Tensor Decomposition. Weighted Sampling for Combined Model Selection and Hyperparameter Tuning. Sequential Dynamic Resource Allocation for Epidemic Control. Latent-Space Laplacian Pyramids for Adversarial Rep …
Geometry of the Hough transforms with applications to synthetic data
| Title | Geometry of the Hough transforms with applications to synthetic data |
| Authors | Mauro C. Beltrametti, Cristina Campi, Anna Maria Massone, Maria-Laura Torrente |
| Abstract | In the framework of the Hough transform technique to detect curves in images, we provide a bound for the number of Hough transforms to be considered for a successful optimization of the accumulator function in the recognition algorithm. Such a bound is consequence of geometrical arguments. We also show the robustness of the results when applied to synthetic datasets strongly perturbed by noise. An algebraic approach, discussed in the appendix, leads to a better bound of theoretical interest in the exact case. |
| Tasks | |
| Published | 2019-04-04 |
| URL | http://arxiv.org/abs/1904.02587v1 |
| http://arxiv.org/pdf/1904.02587v1.pdf | |
| PWC | https://paperswithcode.com/paper/geometry-of-the-hough-transforms-with |
| Repo | |
| Framework | |
On Dropout and Nuclear Norm Regularization
| Title | On Dropout and Nuclear Norm Regularization |
| Authors | Poorya Mianjy, Raman Arora |
| Abstract | We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regularizer is the squared nuclear norm of the network map, and (c) for a sufficiently large dropout rate, we characterize the global optima of the dropout objective. We validate our theoretical findings with empirical results. |
| Tasks | |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.11887v1 |
| https://arxiv.org/pdf/1905.11887v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-dropout-and-nuclear-norm-regularization |
| Repo | |
| Framework | |
Accelerating Training using Tensor Decomposition
| Title | Accelerating Training using Tensor Decomposition |
| Authors | Mostafa Elhoushi, Ye Henry Tian, Zihao Chen, Farhan Shafiq, Joey Yiwei Li |
| Abstract | Tensor decomposition is one of the well-known approaches to reduce the latency time and number of parameters of a pre-trained model. However, in this paper, we propose an approach to use tensor decomposition to reduce training time of training a model from scratch. In our approach, we train the model from scratch (i.e., randomly initialized weights) with its original architecture for a small number of epochs, then the model is decomposed, and then continue training the decomposed model till the end. There is an optional step in our approach to convert the decomposed architecture back to the original architecture. We present results of using this approach on both CIFAR10 and Imagenet datasets, and show that there can be upto 2x speed up in training time with accuracy drop of upto 1.5% only, and in other cases no accuracy drop. This training acceleration approach is independent of hardware and is expected to have similar speed ups on both CPU and GPU platforms. |
| Tasks | |
| Published | 2019-09-10 |
| URL | https://arxiv.org/abs/1909.05675v1 |
| https://arxiv.org/pdf/1909.05675v1.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-training-using-tensor |
| Repo | |
| Framework | |
Weighted Sampling for Combined Model Selection and Hyperparameter Tuning
| Title | Weighted Sampling for Combined Model Selection and Hyperparameter Tuning |
| Authors | Dimitrios Sarigiannis, Thomas Parnell, Haris Pozidis |
| Abstract | The combined algorithm selection and hyperparameter tuning (CASH) problem is characterized by large hierarchical hyperparameter spaces. Model-free hyperparameter tuning methods can explore such large spaces efficiently since they are highly parallelizable across multiple machines. When no prior knowledge or meta-data exists to boost their performance, these methods commonly sample random configurations following a uniform distribution. In this work, we propose a novel sampling distribution as an alternative to uniform sampling and prove theoretically that it has a better chance of finding the best configuration in a worst-case setting. In order to compare competing methods rigorously in an experimental setting, one must perform statistical hypothesis testing. We show that there is little-to-no agreement in the automated machine learning literature regarding which methods should be used. We contrast this disparity with the methods recommended by the broader statistics literature, and identify a suitable approach. We then select three popular model-free solutions to CASH and evaluate their performance, with uniform sampling as well as the proposed sampling scheme, across 67 datasets from the OpenML platform. We investigate the trade-off between exploration and exploitation across the three algorithms, and verify empirically that the proposed sampling distribution improves performance in all cases. |
| Tasks | Model Selection |
| Published | 2019-09-16 |
| URL | https://arxiv.org/abs/1909.07140v3 |
| https://arxiv.org/pdf/1909.07140v3.pdf | |
| PWC | https://paperswithcode.com/paper/weighted-sampling-for-combined-model |
| Repo | |
| Framework | |
Sequential Dynamic Resource Allocation for Epidemic Control
| Title | Sequential Dynamic Resource Allocation for Epidemic Control |
| Authors | Mathilde Fekom, Nicolas Vayatis, Argyris Kalogeratos |
| Abstract | Under the Dynamic Resource Allocation (DRA) model, an administrator has the mission to allocate dynamically a limited budget of resources to the nodes of a network in order to reduce a diffusion process (DP) (e.g. an epidemic). The standard DRA assumes that the administrator has constantly full information and instantaneous access to the entire network. Towards bringing such strategies closer to real-life constraints, we first present the Restricted DRA model extension where, at each intervention round, the access is restricted to only a fraction of the network nodes, called sample. Then, inspired by sequential selection problems such as the well-known Secretary Problem, we propose the Sequential DRA (SDRA) model. Our model introduces a sequential aspect in the decision process over the sample of each round, offering a completely new perspective to the dynamic DP control. Finally, we incorporate several sequential selection algorithms to SDRA control strategies and compare their performance in SIS epidemic simulations. |
| Tasks | |
| Published | 2019-09-20 |
| URL | https://arxiv.org/abs/1909.09678v1 |
| https://arxiv.org/pdf/1909.09678v1.pdf | |
| PWC | https://paperswithcode.com/paper/190909678 |
| Repo | |
| Framework | |
Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds
| Title | Latent-Space Laplacian Pyramids for Adversarial Representation Learning with 3D Point Clouds |
| Authors | Vage Egiazarian, Savva Ignatyev, Alexey Artemov, Oleg Voynov, Andrey Kravchenko, Youyi Zheng, Luiz Velho, Evgeny Burnaev |
| Abstract | Constructing high-quality generative models for 3D shapes is a fundamental task in computer vision with diverse applications in geometry processing, engineering, and design. Despite the recent progress in deep generative modelling, synthesis of finely detailed 3D surfaces, such as high-resolution point clouds, from scratch has not been achieved with existing approaches. In this work, we propose to employ the latent-space Laplacian pyramid representation within a hierarchical generative model for 3D point clouds. We combine the recently proposed latent-space GAN and Laplacian GAN architectures to form a multi-scale model capable of generating 3D point clouds at increasing levels of detail. Our evaluation demonstrates that our model outperforms the existing generative models for 3D point clouds. |
| Tasks | Generating 3D Point Clouds, Representation Learning |
| Published | 2019-12-13 |
| URL | https://arxiv.org/abs/1912.06466v1 |
| https://arxiv.org/pdf/1912.06466v1.pdf | |
| PWC | https://paperswithcode.com/paper/latent-space-laplacian-pyramids-for |
| Repo | |
| Framework | |
Time Masking: Leveraging Temporal Information in Spoken Dialogue Systems
| Title | Time Masking: Leveraging Temporal Information in Spoken Dialogue Systems |
| Authors | Rylan Conway, Lambert Mathias |
| Abstract | In a spoken dialogue system, dialogue state tracker (DST) components track the state of the conversation by updating a distribution of values associated with each of the slots being tracked for the current user turn, using the interactions until then. Much of the previous work has relied on modeling the natural order of the conversation, using distance based offsets as an approximation of time. In this work, we hypothesize that leveraging the wall-clock temporal difference between turns is crucial for finer-grained control of dialogue scenarios. We develop a novel approach that applies a {\it time mask}, based on the wall-clock time difference, to the associated slot embeddings and empirically demonstrate that our proposed approach outperforms existing approaches that leverage distance offsets, on both an internal benchmark dataset as well as DSTC2. |
| Tasks | Spoken Dialogue Systems |
| Published | 2019-07-25 |
| URL | https://arxiv.org/abs/1907.11315v1 |
| https://arxiv.org/pdf/1907.11315v1.pdf | |
| PWC | https://paperswithcode.com/paper/time-masking-leveraging-temporal-information |
| Repo | |
| Framework | |
Closed-loop Model Selection for Kernel-based Models using Bayesian Optimization
| Title | Closed-loop Model Selection for Kernel-based Models using Bayesian Optimization |
| Authors | Thomas Beckers, Somil Bansal, Claire J. Tomlin, Sandra Hirche |
| Abstract | Kernel-based nonparametric models have become very attractive for model-based control approaches for nonlinear systems. However, the selection of the kernel and its hyperparameters strongly influences the quality of the learned model. Classically, these hyperparameters are optimized to minimize the prediction error of the model but this process totally neglects its later usage in the control loop. In this work, we present a framework to optimize the kernel and hyperparameters of a kernel-based model directly with respect to the closed-loop performance of the model. Our framework uses Bayesian optimization to iteratively refine the kernel-based model using the observed performance on the actual system until a desired performance is achieved. We demonstrate the proposed approach in a simulation and on a 3-DoF robotic arm. |
| Tasks | Model Selection |
| Published | 2019-09-12 |
| URL | https://arxiv.org/abs/1909.05699v1 |
| https://arxiv.org/pdf/1909.05699v1.pdf | |
| PWC | https://paperswithcode.com/paper/closed-loop-model-selection-for-kernel-based |
| Repo | |
| Framework | |
Incrementalizing RASA’s Open-Source Natural Language Understanding Pipeline
| Title | Incrementalizing RASA’s Open-Source Natural Language Understanding Pipeline |
| Authors | Andrew Rafla, Casey Kennington |
| Abstract | As spoken dialogue systems and chatbots are gaining more widespread adoption, commercial and open-sourced services for natural language understanding are emerging. In this paper, we explain how we altered the open-source RASA natural language understanding pipeline to process incrementally (i.e., word-by-word), following the incremental unit framework proposed by Schlangen and Skantze. To do so, we altered existing RASA components to process incrementally, and added an update-incremental intent recognition model as a component to RASA. Our evaluations on the Snips dataset show that our changes allow RASA to function as an effective incremental natural language understanding service. |
| Tasks | Spoken Dialogue Systems |
| Published | 2019-07-11 |
| URL | https://arxiv.org/abs/1907.05403v1 |
| https://arxiv.org/pdf/1907.05403v1.pdf | |
| PWC | https://paperswithcode.com/paper/incrementalizing-rasas-open-source-natural |
| Repo | |
| Framework | |
Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks
| Title | Technical Considerations for Semantic Segmentation in MRI using Convolutional Neural Networks |
| Authors | Arjun D. Desai, Garry E. Gold, Brian A. Hargreaves, Akshay S. Chaudhari |
| Abstract | High-fidelity semantic segmentation of magnetic resonance volumes is critical for estimating tissue morphometry and relaxation parameters in both clinical and research applications. While manual segmentation is accepted as the gold-standard, recent advances in deep learning and convolutional neural networks (CNNs) have shown promise for efficient automatic segmentation of soft tissues. However, due to the stochastic nature of deep learning and the multitude of hyperparameters in training networks, predicting network behavior is challenging. In this paper, we quantify the impact of three factors associated with CNN segmentation performance: network architecture, training loss functions, and training data characteristics. We evaluate the impact of these variations on the segmentation of femoral cartilage and propose potential modifications to CNN architectures and training protocols to train these models with confidence. |
| Tasks | Semantic Segmentation |
| Published | 2019-02-05 |
| URL | http://arxiv.org/abs/1902.01977v1 |
| http://arxiv.org/pdf/1902.01977v1.pdf | |
| PWC | https://paperswithcode.com/paper/technical-considerations-for-semantic |
| Repo | |
| Framework | |
Meta-Learning via Learned Loss
| Title | Meta-Learning via Learned Loss |
| Authors | Sarah Bechtle, Artem Molchanov, Yevgen Chebotar, Edward Grefenstette, Ludovic Righetti, Gaurav Sukhatme, Franziska Meier |
| Abstract | Typically, loss functions, regularization mechanisms and other important aspects of training parametric models are chosen heuristically from a limited set of options. In this paper, we take the first step towards automating this process, with the view of producing models which train faster and more robustly. Concretely, we present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for meta-training such loss functions, targeted at maximizing the performance of the model trained under them. The loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional information at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time. |
| Tasks | Meta-Learning |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05374v3 |
| https://arxiv.org/pdf/1906.05374v3.pdf | |
| PWC | https://paperswithcode.com/paper/meta-learning-via-learned-loss |
| Repo | |
| Framework | |
Hybrid Machine Learning Forecasts for the FIFA Women’s World Cup 2019
| Title | Hybrid Machine Learning Forecasts for the FIFA Women’s World Cup 2019 |
| Authors | Andreas Groll, Christophe Ley, Gunther Schauberger, Hans Van Eetvelde, Achim Zeileis |
| Abstract | In this work, we combine two different ranking methods together with several other predictors in a joint random forest approach for the scores of soccer matches. The first ranking method is based on the bookmaker consensus, the second ranking method estimates adequate ability parameters that reflect the current strength of the teams best. The proposed combined approach is then applied to the data from the two previous FIFA Women’s World Cups 2011 and 2015. Finally, based on the resulting estimates, the FIFA Women’s World Cup 2019 is simulated repeatedly and winning probabilities are obtained for all teams. The model clearly favors the defending champion USA before the host France. |
| Tasks | |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.01131v1 |
| https://arxiv.org/pdf/1906.01131v1.pdf | |
| PWC | https://paperswithcode.com/paper/hybrid-machine-learning-forecasts-for-the |
| Repo | |
| Framework | |
Subspace Clustering of Very Sparse High-Dimensional Data
| Title | Subspace Clustering of Very Sparse High-Dimensional Data |
| Authors | Hankui Peng, Nicos Pavlidis, Idris Eckley, Ioannis Tsalamanis |
| Abstract | In this paper we consider the problem of clustering collections of very short texts using subspace clustering. This problem arises in many applications such as product categorisation, fraud detection, and sentiment analysis. The main challenge lies in the fact that the vectorial representation of short texts is both high-dimensional, due to the large number of unique terms in the corpus, and extremely sparse, as each text contains a very small number of words with no repetition. We propose a new, simple subspace clustering algorithm that relies on linear algebra to cluster such datasets. Experimental results on identifying product categories from product names obtained from the US Amazon website indicate that the algorithm can be competitive against state-of-the-art clustering algorithms. |
| Tasks | Fraud Detection, Sentiment Analysis |
| Published | 2019-01-25 |
| URL | http://arxiv.org/abs/1901.09108v1 |
| http://arxiv.org/pdf/1901.09108v1.pdf | |
| PWC | https://paperswithcode.com/paper/subspace-clustering-of-very-sparse-high |
| Repo | |
| Framework | |
Adversarially Trained Convolutional Neural Networks for Semantic Segmentation of Ischaemic Stroke Lesion using Multisequence Magnetic Resonance Imaging
| Title | Adversarially Trained Convolutional Neural Networks for Semantic Segmentation of Ischaemic Stroke Lesion using Multisequence Magnetic Resonance Imaging |
| Authors | Rachana Sathish, Ronnie Rajan, Anusha Vupputuri, Nirmalya Ghosh, Debdoot Sheet |
| Abstract | Ischaemic stroke is a medical condition caused by occlusion of blood supply to the brain tissue thus forming a lesion. A lesion is zoned into a core associated with irreversible necrosis typically located at the center of the lesion, while reversible hypoxic changes in the outer regions of the lesion are termed as the penumbra. Early estimation of core and penumbra in ischaemic stroke is crucial for timely intervention with thrombolytic therapy to reverse the damage and restore normalcy. Multisequence magnetic resonance imaging (MRI) is commonly employed for clinical diagnosis. However, a sequence singly has not been found to be sufficiently able to differentiate between core and penumbra, while a combination of sequences is required to determine the extent of the damage. The challenge, however, is that with an increase in the number of sequences, it cognitively taxes the clinician to discover symptomatic biomarkers in these images. In this paper, we present a data-driven fully automated method for estimation of core and penumbra in ischaemic lesions using diffusion-weighted imaging (DWI) and perfusion-weighted imaging (PWI) sequence maps of MRI. The method employs recent developments in convolutional neural networks (CNN) for semantic segmentation in medical images. In the absence of availability of a large amount of labeled data, the CNN is trained using an adversarial approach employing cross-entropy as a segmentation loss along with losses aggregated from three discriminators of which two employ relativistic visual Turing test. This method is experimentally validated on the ISLES-2015 dataset through three-fold cross-validation to obtain with an average Dice score of 0.82 and 0.73 for segmentation of penumbra and core respectively. |
| Tasks | Semantic Segmentation |
| Published | 2019-08-03 |
| URL | https://arxiv.org/abs/1908.01176v1 |
| https://arxiv.org/pdf/1908.01176v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarially-trained-convolutional-neural |
| Repo | |
| Framework | |
Accurate and Fast reconstruction of Porous Media from Extremely Limited Information Using Conditional Generative Adversarial Network
| Title | Accurate and Fast reconstruction of Porous Media from Extremely Limited Information Using Conditional Generative Adversarial Network |
| Authors | Junxi Feng, Xiaohai He, Qizhi Teng, Chao Ren, Honggang Chen, Yang Li |
| Abstract | Porous media are ubiquitous in both nature and engineering applications, thus their modelling and understanding is of vital importance. In contrast to direct acquisition of three-dimensional (3D) images of such medium, obtaining its sub-region (s) like two-dimensional (2D) images or several small areas could be much feasible. Therefore, reconstructing whole images from the limited information is a primary technique in such cases. Specially, in practice the given data cannot generally be determined by users and may be incomplete or partially informed, thus making existing reconstruction methods inaccurate or even ineffective. To overcome this shortcoming, in this study we proposed a deep learning-based framework for reconstructing full image from its much smaller sub-area(s). Particularly, conditional generative adversarial network (CGAN) is utilized to learn the mapping between input (partial image) and output (full image). To preserve the reconstruction accuracy, two simple but effective objective functions are proposed and then coupled with the other two functions to jointly constrain the training procedure. Due to the inherent essence of this ill-posed problem, a Gaussian noise is introduced for producing reconstruction diversity, thus allowing for providing multiple candidate outputs. Extensively tested on a variety of porous materials and demonstrated by both visual inspection and quantitative comparison, the method is shown to be accurate, stable yet fast ($\sim0.08s$ for a $128 \times 128$ image reconstruction). We highlight that the proposed approach can be readily extended, such as incorporating any user-define conditional data and an arbitrary number of object functions into reconstruction, and being coupled with other reconstruction methods. |
| Tasks | Image Reconstruction |
| Published | 2019-04-04 |
| URL | http://arxiv.org/abs/1905.02135v1 |
| http://arxiv.org/pdf/1905.02135v1.pdf | |
| PWC | https://paperswithcode.com/paper/190502135 |
| Repo | |
| Framework | |
