Paper Group ANR 19
Deep Distribution Regression. Universal Inference Using the Split Likelihood Ratio Test. Normalizing flows for novelty detection in industrial time series data. Natural Vocabulary Emerges from Free-Form Annotations. Neural Shape Parsers for Constructive Solid Geometry. Local Differential Privacy in Decentralized Optimization. Identifiability of Gau …
Deep Distribution Regression
| Title | Deep Distribution Regression |
| Authors | Rui Li, Howard D. Bondell, Brian J. Reich |
| Abstract | Due to their flexibility and predictive performance, machine-learning based regression methods have become an important tool for predictive modeling and forecasting. However, most methods focus on estimating the conditional mean or specific quantiles of the target quantity and do not provide the full conditional distribution, which contains uncertainty information that might be crucial for decision making. In this article, we provide a general solution by transforming a conditional distribution estimation problem into a constrained multi-class classification problem, in which tools such as deep neural networks. We propose a novel joint binary cross-entropy loss function to accomplish this goal. We demonstrate its performance in various simulation studies comparing to state-of-the-art competing methods. Additionally, our method shows improved accuracy in a probabilistic solar energy forecasting problem. |
| Tasks | Decision Making |
| Published | 2019-03-14 |
| URL | http://arxiv.org/abs/1903.06023v1 |
| http://arxiv.org/pdf/1903.06023v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-distribution-regression |
| Repo | |
| Framework | |
Universal Inference Using the Split Likelihood Ratio Test
| Title | Universal Inference Using the Split Likelihood Ratio Test |
| Authors | Larry Wasserman, Aaditya Ramdas, Sivaraman Balakrishnan |
| Abstract | We propose a general method for constructing hypothesis tests and confidence sets that have finite sample guarantees without regularity conditions. We refer to such procedures as universal.'' The method is very simple and is based on a modified version of the usual likelihood ratio statistic, that we call the split likelihood ratio test’’ (split LRT). The method is especially appealing for irregular statistical models. Canonical examples include mixture models and models that arise in shape-constrained inference. %mixture models and shape-constrained models are just two examples. Constructing tests and confidence sets for such models is notoriously difficult. Typical inference methods, like the likelihood ratio test, are not useful in these cases because they have intractable limiting distributions. In contrast, the method we suggest works for any parametric model and also for some nonparametric models. The split LRT can also be used with profile likelihoods to deal with nuisance parameters, and it can also be run sequentially to yield anytime-valid $p$-values and confidence sequences. |
| Tasks | |
| Published | 2019-12-24 |
| URL | https://arxiv.org/abs/1912.11436v2 |
| https://arxiv.org/pdf/1912.11436v2.pdf | |
| PWC | https://paperswithcode.com/paper/universal-inference-using-the-split |
| Repo | |
| Framework | |
Normalizing flows for novelty detection in industrial time series data
| Title | Normalizing flows for novelty detection in industrial time series data |
| Authors | Maximilian Schmidt, Marko Simic |
| Abstract | Flow-based deep generative models learn data distributions by transforming a simple base distribution into a complex distribution via a set of invertible transformations. Due to the invertibility, such models can score unseen data samples by computing their exact likelihood under the learned distribution. This makes flow-based models a perfect tool for novelty detection, an anomaly detection technique where unseen data samples are classified as normal or abnormal by scoring them against a learned model of normal data. We show that normalizing flows can be used as novelty detectors in time series. Two flow-based models, Masked Autoregressive Flows and Free-form Jacobian of Reversible Dynamics restricted by autoregressive MADE networks, are tested on synthetic data and motor current data from an industrial machine and achieve good results, outperforming a conventional novelty detection method, the Local Outlier Factor. |
| Tasks | Anomaly Detection, Time Series |
| Published | 2019-06-17 |
| URL | https://arxiv.org/abs/1906.06904v1 |
| https://arxiv.org/pdf/1906.06904v1.pdf | |
| PWC | https://paperswithcode.com/paper/normalizing-flows-for-novelty-detection-in |
| Repo | |
| Framework | |
Natural Vocabulary Emerges from Free-Form Annotations
| Title | Natural Vocabulary Emerges from Free-Form Annotations |
| Authors | Jordi Pont-Tuset, Michael Gygli, Vittorio Ferrari |
| Abstract | We propose an approach for annotating object classes using free-form text written by undirected and untrained annotators. Free-form labeling is natural for annotators, they intuitively provide very specific and exhaustive labels, and no training stage is necessary. We first collect 729 labels on 15k images using 124 different annotators. Then we automatically enrich the structure of these free-form annotations by discovering a natural vocabulary of 4020 classes within them. This vocabulary represents the natural distribution of objects well and is learned directly from data, instead of being an educated guess done before collecting any labels. Hence, the natural vocabulary emerges from a large mass of free-form annotations. To do so, we (i) map the raw input strings to entities in an ontology of physical objects (which gives them an unambiguous meaning); and (ii) leverage inter-annotator co-occurrences, as well as biases and knowledge specific to individual annotators. Finally, we also automatically extract natural vocabularies of reduced size that have high object coverage while remaining specific. These reduced vocabularies represent the natural distribution of objects much better than commonly used predefined vocabularies. Moreover, they feature more uniform sample distribution over classes. |
| Tasks | |
| Published | 2019-06-04 |
| URL | https://arxiv.org/abs/1906.01542v1 |
| https://arxiv.org/pdf/1906.01542v1.pdf | |
| PWC | https://paperswithcode.com/paper/natural-vocabulary-emerges-from-free-form |
| Repo | |
| Framework | |
Neural Shape Parsers for Constructive Solid Geometry
| Title | Neural Shape Parsers for Constructive Solid Geometry |
| Authors | Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, Subhransu Maji |
| Abstract | Constructive Solid Geometry (CSG) is a geometric modeling technique that defines complex shapes by recursively applying boolean operations on primitives such as spheres and cylinders. We present CSGNe, a deep network architecture that takes as input a 2D or 3D shape and outputs a CSG program that models it. Parsing shapes into CSG programs is desirable as it yields a compact and interpretable generative model. However, the task is challenging since the space of primitives and their combinations can be prohibitively large. CSGNe uses a convolutional encoder and recurrent decoder based on deep networks to map shapes to modeling instructions in a feed-forward manner and is significantly faster than bottom-up approaches. We investigate two architectures for this task — a vanilla encoder (CNN) - decoder (RNN) and another architecture that augments the encoder with an explicit memory module based on the program execution stack. The stack augmentation improves the reconstruction quality of the generated shape and learning efficiency. Our approach is also more effective as a shape primitive detector compared to a state-of-the-art object detector. Finally, we demonstrate CSGNet can be trained on novel datasets without program annotations through policy gradient techniques. |
| Tasks | |
| Published | 2019-12-22 |
| URL | https://arxiv.org/abs/1912.11393v1 |
| https://arxiv.org/pdf/1912.11393v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-shape-parsers-for-constructive-solid |
| Repo | |
| Framework | |
Local Differential Privacy in Decentralized Optimization
| Title | Local Differential Privacy in Decentralized Optimization |
| Authors | Hanshen Xiao, Yu Ye, Srinivas Devadas |
| Abstract | Privacy concerns with sensitive data are receiving increasing attention. In this paper, we study local differential privacy (LDP) in interactive decentralized optimization. By constructing random local aggregators, we propose a framework to amplify LDP by a constant. We take Alternating Direction Method of Multipliers (ADMM), and decentralized gradient descent as two concrete examples, where experiments support our theory. In an asymptotic view, we address the following question: Under LDP, is it possible to design a distributed private minimizer for arbitrary closed convex constraints with utility loss not explicitly dependent on dimensionality? As an affiliated result, we also show that with merely linear secret sharing, information theoretic privacy is achievable for bounded colluding agents. |
| Tasks | |
| Published | 2019-02-16 |
| URL | https://arxiv.org/abs/1902.06101v2 |
| https://arxiv.org/pdf/1902.06101v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-privacy-preserving-decentralized |
| Repo | |
| Framework | |
Identifiability of Gaussian Structural Equation Models with Homogeneous and Heterogeneous Error Variances
| Title | Identifiability of Gaussian Structural Equation Models with Homogeneous and Heterogeneous Error Variances |
| Authors | Gunwoong Park, Younghwan Kim |
| Abstract | In this work, we consider the identifiability assumption of Gaussian linear structural equation models (SEMs) in which each variable is determined by a linear function of its parents plus normally distributed error. It has been shown that linear Gaussian structural equation models are fully identifiable if all error variances are the same or known. Hence, this work proves the identifiability of Gaussian SEMs with both homogeneous and heterogeneous unknown error variances. Our new identifiability assumption exploits not only error variances, but edge weights; hence, it is strictly milder than prior work on the identifiability result. We further provide a structure learning algorithm that is statistically consistent and computationally feasible, based on our new assumption. The proposed algorithm assumes that all relevant variables are observed, while it does not assume causal minimality and faithfulness. We verify our theoretical findings through simulations and real multivariate data, and compare our algorithm to state-of-the-art PC, GES and GDS algorithms. |
| Tasks | |
| Published | 2019-01-29 |
| URL | https://arxiv.org/abs/1901.10134v3 |
| https://arxiv.org/pdf/1901.10134v3.pdf | |
| PWC | https://paperswithcode.com/paper/identifiability-of-gaussian-structural-2 |
| Repo | |
| Framework | |
Unsupervised and Generic Short-Term Anticipation of Human Body Motions
| Title | Unsupervised and Generic Short-Term Anticipation of Human Body Motions |
| Authors | Kristina Enes, Hassan Errami, Moritz Wolter, Tim Krake, Bernhard Eberhardt, Andreas Weber, Jörg Zimmermann |
| Abstract | Various neural network based methods are capable of anticipating human body motions from data for a short period of time. What these methods lack are the interpretability and explainability of the network and its results. We propose to use Dynamic Mode Decomposition with delays to represent and anticipate human body motions. Exploring the influence of the number of delays on the reconstruction and prediction of various motion classes, we show that the anticipation errors in our results are comparable or even better for very short anticipation times ($<0.4$ sec) to a recurrent neural network based method. We perceive our method as a first step towards the interpretability of the results by representing human body motions as linear combinations of ``factors’'. In addition, compared to the neural network based methods large training times are not needed. Actually, our methods do not even regress to any other motions than the one to be anticipated and hence is of a generic nature. | |
| Tasks | |
| Published | 2019-12-13 |
| URL | https://arxiv.org/abs/1912.06688v1 |
| https://arxiv.org/pdf/1912.06688v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-and-generic-short-term |
| Repo | |
| Framework | |
Streaming Adaptation of Deep Forecasting Models using Adaptive Recurrent Units
| Title | Streaming Adaptation of Deep Forecasting Models using Adaptive Recurrent Units |
| Authors | Prathamesh Deshpande, Sunita Sarawagi |
| Abstract | We present ARU, an Adaptive Recurrent Unit for streaming adaptation of deep globally trained time-series forecasting models. The ARU combines the advantages of learning complex data transformations across multiple time series from deep global models, with per-series localization offered by closed-form linear models. Unlike existing methods of adaptation that are either memory-intensive or non-responsive after training, ARUs require only fixed sized state and adapt to streaming data via an easy RNN-like update operation. The core principle driving ARU is simple — maintain sufficient statistics of conditional Gaussian distributions and use them to compute local parameters in closed form. Our contribution is in embedding such local linear models in globally trained deep models while allowing end-to-end training on the one hand, and easy RNN-like updates on the other. Across several datasets we show that ARU is more effective than recently proposed local adaptation methods that tax the global network to compute local parameters. |
| Tasks | Time Series, Time Series Forecasting |
| Published | 2019-06-24 |
| URL | https://arxiv.org/abs/1906.09926v2 |
| https://arxiv.org/pdf/1906.09926v2.pdf | |
| PWC | https://paperswithcode.com/paper/streaming-adaptation-of-deep-forecasting |
| Repo | |
| Framework | |
Verification Code Recognition Based on Active and Deep Learning
| Title | Verification Code Recognition Based on Active and Deep Learning |
| Authors | Dongliang Xu, Bailing Wang, XiaoJiang Du, Xiaoyan Zhu, zhitao Guan, Xiaoyan Yu, Jingyu Liu |
| Abstract | A verification code is an automated test method used to distinguish between humans and computers. Humans can easily identify verification codes, whereas machines cannot. With the development of convolutional neural networks, automatically recognizing a verification code is now possible for machines. However, the advantages of convolutional neural networks depend on the data used by the training classifier, particularly the size of the training set. Therefore, identifying a verification code using a convolutional neural network is difficult when training data are insufficient. This study proposes an active and deep learning strategy to obtain new training data on a special verification code set without manual intervention. A feature learning model for a scene with less training data is presented in this work, and the verification code is identified by the designed convolutional neural network. Experiments show that the method can considerably improve the recognition accuracy of a neural network when the amount of initial training data is small. |
| Tasks | |
| Published | 2019-02-12 |
| URL | http://arxiv.org/abs/1902.04401v1 |
| http://arxiv.org/pdf/1902.04401v1.pdf | |
| PWC | https://paperswithcode.com/paper/verification-code-recognition-based-on-active |
| Repo | |
| Framework | |
A Deep Neural Information Fusion Architecture for Textual Network Embeddings
| Title | A Deep Neural Information Fusion Architecture for Textual Network Embeddings |
| Authors | Zenan Xu, Qinliang Su, Xiaojun Quan, Weijia Zhang |
| Abstract | Textual network embeddings aim to learn a low-dimensional representation for every node in the network so that both the structural and textual information from the networks can be well preserved in the representations. Traditionally, the structural and textual embeddings were learned by models that rarely take the mutual influences between them into account. In this paper, a deep neural architecture is proposed to effectively fuse the two kinds of informations into one representation. The novelties of the proposed architecture are manifested in the aspects of a newly defined objective function, the complementary information fusion method for structural and textual features, and the mutual gate mechanism for textual feature extraction. Experimental results show that the proposed model outperforms the comparing methods on all three datasets. |
| Tasks | |
| Published | 2019-08-29 |
| URL | https://arxiv.org/abs/1908.11057v1 |
| https://arxiv.org/pdf/1908.11057v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-neural-information-fusion-architecture |
| Repo | |
| Framework | |
Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings
| Title | Brain2Char: A Deep Architecture for Decoding Text from Brain Recordings |
| Authors | Pengfei Sun, Gopala K. Anumanchipalli, Edward F. Chang |
| Abstract | Decoding language representations directly from the brain can enable new Brain-Computer Interfaces (BCI) for high bandwidth human-human and human-machine communication. Clinically, such technologies can restore communication in people with neurological conditions affecting their ability to speak. In this study, we propose a novel deep network architecture Brain2Char, for directly decoding text (specifically character sequences) from direct brain recordings (called Electrocorticography, ECoG). Brain2Char framework combines state-of-the-art deep learning modules — 3D Inception layers for multiband spatiotemporal feature extraction from neural data and bidirectional recurrent layers, dilated convolution layers followed by language model weighted beam search to decode character sequences, optimizing a connectionist temporal classification (CTC) loss. Additionally, given the highly non-linear transformations that underlie the conversion of cortical function to character sequences, we perform regularizations on the network’s latent representations motivated by insights into cortical encoding of speech production and artifactual aspects specific to ECoG data acquisition. To do this, we impose auxiliary losses on latent representations for articulatory movements, speech acoustics and session specific non-linearities. In 3 participants tested here, Brain2Char achieves 10.6%, 8.5% and 7.0% Word Error Rates (WER) respectively on vocabulary sizes ranging from 1200 to 1900 words. Brain2Char also performs well when 2 participants silently mimed sentences. These results set a new state-of-the-art on decoding text from brain and demonstrate the potential of Brain2Char as a high-performance communication BCI. |
| Tasks | Language Modelling |
| Published | 2019-09-03 |
| URL | https://arxiv.org/abs/1909.01401v1 |
| https://arxiv.org/pdf/1909.01401v1.pdf | |
| PWC | https://paperswithcode.com/paper/brain2char-a-deep-architecture-for-decoding |
| Repo | |
| Framework | |
A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC
| Title | A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC |
| Authors | Tianyi Li, Mai Xu, Ren Yang, Xiaoming Tao |
| Abstract | High efficiency video coding (HEVC) has brought outperforming efficiency for video compression. To reduce the compression artifacts of HEVC, we propose a DenseNet based approach as the in-loop filter of HEVC, which leverages multiple adjacent frames to enhance the quality of each encoded frame. Specifically, the higher-quality frames are found by a reference frame selector (RFS). Then, a deep neural network for multi-frame in-loop filter (named MIF-Net) is developed to enhance the quality of each encoded frame by utilizing the spatial information of this frame and the temporal information of its neighboring higher-quality frames. The MIF-Net is built on the recently developed DenseNet, benefiting from the improved generalization capacity and computational efficiency. Finally, experimental results verify the effectiveness of our multi-frame in-loop filter, outperforming the HM baseline and other state-of-the-art approaches. |
| Tasks | Video Compression |
| Published | 2019-03-05 |
| URL | http://arxiv.org/abs/1903.01648v1 |
| http://arxiv.org/pdf/1903.01648v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-densenet-based-approach-for-multi-frame-in |
| Repo | |
| Framework | |
Variable Selection with Random Survival Forest and Bayesian Additive Regression Tree for Survival Data
| Title | Variable Selection with Random Survival Forest and Bayesian Additive Regression Tree for Survival Data |
| Authors | Satabdi Saha, Duchwan Ryu, Nader Ebrahimi |
| Abstract | In this paper we utilize a survival analysis methodology incorporating Bayesian additive regression trees to account for nonlinear and additive covariate effects. We compare the performance of Bayesian additive regression trees, Cox proportional hazards and random survival forests models for censored survival data, using simulation studies and survival analysis for breast cancer with U.S. SEER database for the year 2005. In simulation studies, we compare the three models across varying sample sizes and censoring rates on the basis of bias and prediction accuracy. In survival analysis for breast cancer, we retrospectively analyze a subset of 1500 patients having invasive ductal carcinoma that is a common form of breast cancer mostly affecting older woman. Predictive potential of the three models are then compared using some widely used performance assessment measures in survival literature. |
| Tasks | Survival Analysis |
| Published | 2019-10-04 |
| URL | https://arxiv.org/abs/1910.02160v2 |
| https://arxiv.org/pdf/1910.02160v2.pdf | |
| PWC | https://paperswithcode.com/paper/variable-selection-with-random-survival |
| Repo | |
| Framework | |
Reasoning in Highly Reactive Environments
| Title | Reasoning in Highly Reactive Environments |
| Authors | Francesco Pacenza |
| Abstract | The aim of my Ph.D. thesis concerns Reasoning in Highly Reactive Environments. As reasoning in highly reactive environments, we identify the setting in which a knowledge-based agent, with given goals, is deployed in an environment subject to repeated, sudden and possibly unknown changes. This is for instance the typical setting in which, e.g., artificial agents for video-games (the so called “bots”), cleaning robots, bomb clearing robots, and so on are deployed. In all these settings one can follow the classical approach in which the operations of the agent are distinguished in “sensing” the environment with proper interface devices, “thinking”, and then behaving accordingly using proper actuators. In order to operate in an highly reactive environment, an artificial agent needs to be: 1. Responsive -> The agent must be able to react repeatedly and in a reasonable amount of time; 2. Elastic -> The agent must stay reactive also under varying workload; 3. Resilient -> The agent must stay responsive also in case of internal failure or failure of one of the programmed actions in the environment. Nowadays, thanks to new technologies in the field of Artificial Intelligence, it is already technically possible to create AI agents that are able to operate in reactive environments. Nevertheless, several issues stay unsolved, and are subject of ongoing research. |
| Tasks | |
| Published | 2019-09-18 |
| URL | https://arxiv.org/abs/1909.08260v1 |
| https://arxiv.org/pdf/1909.08260v1.pdf | |
| PWC | https://paperswithcode.com/paper/reasoning-in-highly-reactive-environments |
| Repo | |
| Framework | |
