Paper Group ANR 1461
MLRG Deep Curvature. On power chi expansions of $f$-divergences. HoughNet: neural network architecture for vanishing points detection. TzK: Flow-Based Conditional Generative Model. An Automated Engineering Assistant: Learning Parsers for Technical Drawings. Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild. Machine learning in he …
MLRG Deep Curvature
| Title | MLRG Deep Curvature |
| Authors | Diego Granziol, Xingchen Wan, Timur Garipov, Dmitry Vetrov, Stephen Roberts |
| Abstract | We present MLRG Deep Curvature suite, a PyTorch-based, open-source package for analysis and visualisation of neural network curvature and loss landscape. Despite of providing rich information into properties of neural network and useful for a various designed tasks, curvature information is still not made sufficient use for various reasons, and our method aims to bridge this gap. We present a primer, including its main practical desiderata and common misconceptions, of \textit{Lanczos algorithm}, the theoretical backbone of our package, and present a series of examples based on synthetic toy examples and realistic modern neural networks tested on CIFAR datasets, and show the superiority of our package against existing competing approaches for the similar purposes. |
| Tasks | |
| Published | 2019-12-20 |
| URL | https://arxiv.org/abs/1912.09656v1 |
| https://arxiv.org/pdf/1912.09656v1.pdf | |
| PWC | https://paperswithcode.com/paper/mlrg-deep-curvature |
| Repo | |
| Framework | |
On power chi expansions of $f$-divergences
| Title | On power chi expansions of $f$-divergences |
| Authors | Frank Nielsen, Gaëtan Hadjeres |
| Abstract | We consider both finite and infinite power chi expansions of $f$-divergences derived from Taylor’s expansions of smooth generators, and elaborate on cases where these expansions yield closed-form formula, bounded approximations, or analytic divergence series expressions of $f$-divergences. |
| Tasks | |
| Published | 2019-03-14 |
| URL | http://arxiv.org/abs/1903.05818v1 |
| http://arxiv.org/pdf/1903.05818v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-power-chi-expansions-of-f-divergences |
| Repo | |
| Framework | |
HoughNet: neural network architecture for vanishing points detection
| Title | HoughNet: neural network architecture for vanishing points detection |
| Authors | Alexander Sheshkus, Anastasia Ingacheva, Vladimir Arlazarov, Dmitry Nikolaev |
| Abstract | In this paper we introduce a novel neural network architecture based on Fast Hough Transform layer. The layer of this type allows our neural network to accumulate features from linear areas across the entire image instead of local areas. We demonstrate its potential by solving the problem of vanishing points detection in the images of documents. Such problem occurs when dealing with camera shots of the documents in uncontrolled conditions. In this case, the document image can suffer several specific distortions including projective transform. To train our model, we use MIDV-500 dataset and provide testing results. The strong generalization ability of the suggested method is proven with its applying to a completely different ICDAR 2011 dewarping contest. In previously published papers considering these dataset authors measured the quality of vanishing point detection by counting correctly recognized words with open OCR engine Tesseract. To compare with them, we reproduce this experiment and show that our method outperforms the state-of-the-art result. |
| Tasks | Optical Character Recognition |
| Published | 2019-09-09 |
| URL | https://arxiv.org/abs/1909.03812v2 |
| https://arxiv.org/pdf/1909.03812v2.pdf | |
| PWC | https://paperswithcode.com/paper/houghnet-neural-network-architecture-for |
| Repo | |
| Framework | |
TzK: Flow-Based Conditional Generative Model
| Title | TzK: Flow-Based Conditional Generative Model |
| Authors | Micha Livne, David Fleet |
| Abstract | We formulate a new class of conditional generative models based on probability flows. Trained with maximum likelihood, it provides efficient inference and sampling from class-conditionals or the joint distribution, and does not require a priori knowledge of the number of classes or the relationships between classes. This allows one to train generative models from multiple, heterogeneous datasets, while retaining strong prior models over subsets of the data (e.g., from a single dataset, class label, or attribute). In this paper, in addition to end-to-end learning, we show how one can learn a single model from multiple datasets with a relatively weak Glow architecture, and then extend it by conditioning on different knowledge types (e.g., a single dataset). This yields log likelihood comparable to state-of-the-art, compelling samples from conditional priors. |
| Tasks | |
| Published | 2019-02-05 |
| URL | http://arxiv.org/abs/1902.01893v4 |
| http://arxiv.org/pdf/1902.01893v4.pdf | |
| PWC | https://paperswithcode.com/paper/tzk-flow-based-conditional-generative-model |
| Repo | |
| Framework | |
An Automated Engineering Assistant: Learning Parsers for Technical Drawings
| Title | An Automated Engineering Assistant: Learning Parsers for Technical Drawings |
| Authors | Dries Van Daele, Nicholas Decleyre, Herman Dubois, Wannes Meert |
| Abstract | From a set of technical drawings and expert knowledge, we automatically learn a parser to interpret such a drawing. This enables automatic reasoning and learning on top of a large database of technical drawings. In this work, we develop a similarity based search algorithm to help engineers and designers find or complete designs more easily and flexibly. This is part of an ongoing effort to build an automated engineering assistant. The proposed methods make use of both neural methods to learn to interpret images, and symbolic methods to learn to interpret the structure in the technical drawing and incorporate expert knowledge. |
| Tasks | |
| Published | 2019-09-18 |
| URL | https://arxiv.org/abs/1909.08552v1 |
| https://arxiv.org/pdf/1909.08552v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-automated-engineering-assistant-learning |
| Repo | |
| Framework | |
Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
| Title | Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild |
| Authors | Xin Chen, Lingxi Xie, Jun Wu, Qi Tian |
| Abstract | With the rapid development of neural architecture search (NAS), researchers found powerful network architectures for a wide range of vision tasks. However, it remains unclear if the searched architecture can transfer across different types of tasks as manually designed ones did. This paper puts forward this problem, referred to as NAS in the wild, which explores the possibility of finding the optimal architecture in a proxy dataset and then deploying it to mostly unseen scenarios. We instantiate this setting using a currently popular algorithm named differentiable architecture search (DARTS), which often suffers unsatisfying performance while being transferred across different tasks. We argue that the accuracy drop originates from the formulation that uses a super-network for search but a sub-network for re-training. The different properties of these stages have resulted in a significant optimization gap, and consequently, the architectural parameters “over-fit” the super-network. To alleviate the gap, we present a progressive method that gradually increases the network depth during the search stage, which leads to the Progressive DARTS (P-DARTS) algorithm. With a reduced search cost (7 hours on a single GPU), P-DARTS achieves improved performance on both the proxy dataset (CIFAR10) and a few target problems (ImageNet classification, COCO detection and three ReID benchmarks). Our code is available at \url{https://github.com/chenxin061/pdarts}. |
| Tasks | Neural Architecture Search |
| Published | 2019-12-23 |
| URL | https://arxiv.org/abs/1912.10952v2 |
| https://arxiv.org/pdf/1912.10952v2.pdf | |
| PWC | https://paperswithcode.com/paper/progressive-darts-bridging-the-optimization |
| Repo | |
| Framework | |
Machine learning in healthcare – a system’s perspective
| Title | Machine learning in healthcare – a system’s perspective |
| Authors | Awais Ashfaq, Slawomir Nowaczyk |
| Abstract | A consequence of the fragmented and siloed healthcare landscape is that patient care (and data) is split along multitude of different facilities and computer systems and enabling interoperability between these systems is hard. The lack interoperability not only hinders continuity of care and burdens providers, but also hinders effective application of Machine Learning (ML) algorithms. Thus, most current ML algorithms, designed to understand patient care and facilitate clinical decision-support, are trained on limited datasets. This approach is analogous to the Newtonian paradigm of Reductionism in which a system is broken down into elementary components and a description of the whole is formed by understanding those components individually. A key limitation of the reductionist approach is that it ignores the component-component interactions and dynamics within the system which are often of prime significance in understanding the overall behaviour of complex adaptive systems (CAS). Healthcare is a CAS. Though the application of ML on health data have shown incremental improvements for clinical decision support, ML has a much a broader potential to restructure care delivery as a whole and maximize care value. However, this ML potential remains largely untapped: primarily due to functional limitations of Electronic Health Records (EHR) and the inability to see the healthcare system as a whole. This viewpoint (i) articulates the healthcare as a complex system which has a biological and an organizational perspective, (ii) motivates with examples, the need of a system’s approach when addressing healthcare challenges via ML and, (iii) emphasizes to unleash EHR functionality - while duly respecting all ethical and legal concerns - to reap full benefits of ML. |
| Tasks | |
| Published | 2019-09-14 |
| URL | https://arxiv.org/abs/1909.07370v2 |
| https://arxiv.org/pdf/1909.07370v2.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-in-healthcare-a-systems |
| Repo | |
| Framework | |
Modeling Human Annotation Errors to Design Bias-Aware Systems for Social Stream Processing
| Title | Modeling Human Annotation Errors to Design Bias-Aware Systems for Social Stream Processing |
| Authors | Rahul Pandey, Carlos Castillo, Hemant Purohit |
| Abstract | High-quality human annotations are necessary to create effective machine learning systems for social media. Low-quality human annotations indirectly contribute to the creation of inaccurate or biased learning systems. We show that human annotation quality is dependent on the ordering of instances shown to annotators (referred as ‘annotation schedule’), and can be improved by local changes in the instance ordering provided to the annotators, yielding a more accurate annotation of the data stream for efficient real-time social media analytics. We propose an error-mitigating active learning algorithm that is robust with respect to some cases of human errors when deciding an annotation schedule. We validate the human error model and evaluate the proposed algorithm against strong baselines by experimenting on classification tasks of relevant social media posts during crises. According to these experiments, considering the order in which data instances are presented to human annotators leads to both an increase in accuracy for machine learning and awareness toward some potential biases in human learning that may affect the automated classifier. |
| Tasks | Active Learning |
| Published | 2019-07-16 |
| URL | https://arxiv.org/abs/1907.07228v1 |
| https://arxiv.org/pdf/1907.07228v1.pdf | |
| PWC | https://paperswithcode.com/paper/modeling-human-annotation-errors-to-design |
| Repo | |
| Framework | |
Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models
| Title | Privacy for Rescue: A New Testimony Why Privacy is Vulnerable In Deep Models |
| Authors | Ruiyuan Gao, Ming Dun, Hailong Yang, Zhongzhi Luan, Depei Qian |
| Abstract | The huge computation demand of deep learning models and limited computation resources on the edge devices calls for the cooperation between edge device and cloud service by splitting the deep models into two halves. However, transferring the intermediates results from the partial models between edge device and cloud service makes the user privacy vulnerable since the attacker can intercept the intermediate results and extract privacy information from them. Existing research works rely on metrics that are either impractical or insufficient to measure the effectiveness of privacy protection methods in the above scenario, especially from the aspect of a single user. In this paper, we first present a formal definition of the privacy protection problem in the edge-cloud system running DNN models. Then, we analyze the-state-of-the-art methods and point out the drawbacks of their methods, especially the evaluation metrics such as the Mutual Information (MI). In addition, we perform several experiments to demonstrate that although existing methods perform well under MI, they are not effective enough to protect the privacy of a single user. To address the drawbacks of the evaluation metrics, we propose two new metrics that are more accurate to measure the effectiveness of privacy protection methods. Finally, we highlight several potential research directions to encourage future efforts addressing the privacy protection problem. |
| Tasks | |
| Published | 2019-12-31 |
| URL | https://arxiv.org/abs/2001.00493v1 |
| https://arxiv.org/pdf/2001.00493v1.pdf | |
| PWC | https://paperswithcode.com/paper/privacy-for-rescue-a-new-testimony-why |
| Repo | |
| Framework | |
No Pressure! Addressing the Problem of Local Minima in Manifold Learning Algorithms
| Title | No Pressure! Addressing the Problem of Local Minima in Manifold Learning Algorithms |
| Authors | Max Vladymyrov |
| Abstract | Nonlinear embedding manifold learning methods provide invaluable visual insights into the structure of high-dimensional data. However, due to a complicated nonconvex objective function, these methods can easily get stuck in local minima and their embedding quality can be poor. We propose a natural extension to several manifold learning methods aimed at identifying pressured points, i.e. points stuck in poor local minima and have poor embedding quality. We show that the objective function can be decreased by temporarily allowing these points to make use of an extra dimension in the embedding space. Our method is able to improve the objective function value of existing methods even after they get stuck in a poor local minimum. |
| Tasks | |
| Published | 2019-06-26 |
| URL | https://arxiv.org/abs/1906.11389v2 |
| https://arxiv.org/pdf/1906.11389v2.pdf | |
| PWC | https://paperswithcode.com/paper/no-pressure-addressing-the-problem-of-local |
| Repo | |
| Framework | |
Popularity Prediction on Social Platforms with Coupled Graph Neural Networks
| Title | Popularity Prediction on Social Platforms with Coupled Graph Neural Networks |
| Authors | Qi Cao, Huawei Shen, Jinhua Gao, Bingzheng Wei, Xueqi Cheng |
| Abstract | Predicting the popularity of online content on social platforms is an important task for both researchers and practitioners. Previous methods mainly leverage demographics, temporal and structural patterns of early adopters for popularity prediction. However, most existing methods are less effective to precisely capture the cascading effect in information diffusion, in which early adopters try to activate potential users along the underlying network. In this paper, we consider the problem of network-aware popularity prediction, leveraging both early adopters and social networks for popularity prediction. We propose to capture the cascading effect explicitly, modeling the activation state of a target user given the activation state and influence of his/her neighbors. To achieve this goal, we propose a novel method, namely CoupledGNN, which uses two coupled graph neural networks to capture the interplay between node activation states and the spread of influence. By stacking graph neural network layers, our proposed method naturally captures the cascading effect along the network in a successive manner. Experiments conducted on both synthetic and real-world Sina Weibo datasets demonstrate that our method significantly outperforms the state-of-the-art methods for popularity prediction. |
| Tasks | |
| Published | 2019-06-21 |
| URL | https://arxiv.org/abs/1906.09032v2 |
| https://arxiv.org/pdf/1906.09032v2.pdf | |
| PWC | https://paperswithcode.com/paper/coupled-graph-neural-networks-for-predicting |
| Repo | |
| Framework | |
Online monitoring for safe pedestrian-vehicle interactions
| Title | Online monitoring for safe pedestrian-vehicle interactions |
| Authors | Peter Du, Zhe Huang, Tianqi Liu, Ke Xu, Qichao Gao, Hussein Sibai, Katherine Driggs-Campbell, Sayan Mitra |
| Abstract | As autonomous systems begin to operate amongst humans, methods for safe interaction must be investigated. We consider an example of a small autonomous vehicle in a pedestrian zone that must safely maneuver around people in a free-form fashion. We investigate two key questions: How can we effectively integrate pedestrian intent estimation into our autonomous stack. Can we develop an online monitoring framework to give formal guarantees on the safety of such human-robot interactions. We present a pedestrian intent estimation framework that can accurately predict future pedestrian trajectories given multiple possible goal locations. We integrate this into a reachability-based online monitoring scheme that formally assesses the safety of these interactions with nearly real-time performance (approximately 0.3 seconds). These techniques are integrated on a test vehicle with a complete in-house autonomous stack, demonstrating effective and safe interaction in real-world experiments. |
| Tasks | |
| Published | 2019-10-12 |
| URL | https://arxiv.org/abs/1910.05599v1 |
| https://arxiv.org/pdf/1910.05599v1.pdf | |
| PWC | https://paperswithcode.com/paper/online-monitoring-for-safe-pedestrian-vehicle |
| Repo | |
| Framework | |
Beyond DAGs: Modeling Causal Feedback with Fuzzy Cognitive Maps
| Title | Beyond DAGs: Modeling Causal Feedback with Fuzzy Cognitive Maps |
| Authors | Osonde Osoba, Bart Kosko |
| Abstract | Fuzzy cognitive maps (FCMs) model feedback causal relations in interwoven webs of causality and policy variables. FCMs are fuzzy signed directed graphs that allow degrees of causal influence and event occurrence. Such causal models can simulate a wide range of policy scenarios and decision processes. Their directed loops or cycles directly model causal feedback. Their nonlinear dynamics permit forward-chaining inference from input causes and policy options to output effects. Users can add detailed dynamics and feedback links directly to the causal model or infer them with statistical learning laws. Users can fuse or combine FCMs from multiple experts by weighting and adding the underlying fuzzy edge matrices and do so recursively if needed. The combined FCM tends to better represent domain knowledge as the expert sample size increases if the expert sample approximates a random sample. Many causal models use more restrictive directed acyclic graphs (DAGs) and Bayesian probabilities. DAGs do not model causal feedback because they do not contain closed loops. Combining DAGs also tends to produce cycles and thus tends not to produce a new DAG. Combining DAGs tends to produce a FCM. FCM causal influence is also transitive whereas probabilistic causal influence is not transitive in general. Overall: FCMs trade the numerical precision of probabilistic DAGs for pattern prediction, faster and scalable computation, ease of combination, and richer feedback representation. We show how FCMs can apply to problems of public support for insurgency and terrorism and to US-China conflict relations in Graham Allison’s Thucydides-trap framework. The appendix gives the textual justification of the Thucydides-trap FCM. It also extends our earlier theorem [Osoba-Kosko2017] to a more general result that shows the transitive and total causal influence that upstream concept nodes exert on downstream nodes. |
| Tasks | |
| Published | 2019-06-26 |
| URL | https://arxiv.org/abs/1906.11247v2 |
| https://arxiv.org/pdf/1906.11247v2.pdf | |
| PWC | https://paperswithcode.com/paper/beyond-dags-modeling-causal-feedback-with |
| Repo | |
| Framework | |
MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network
| Title | MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network |
| Authors | Hao Wang, Tong Xu, Qi Liu, Defu Lian, Enhong Chen, Dongfang Du, Han Wu, Wen Su |
| Abstract | Recently, the Network Representation Learning (NRL) techniques, which represent graph structure via low-dimension vectors to support social-oriented application, have attracted wide attention. Though large efforts have been made, they may fail to describe the multiple aspects of similarity between social users, as only a single vector for one unique aspect has been represented for each node. To that end, in this paper, we propose a novel end-to-end framework named MCNE to learn multiple conditional network representations, so that various preferences for multiple behaviors could be fully captured. Specifically, we first design a binary mask layer to divide the single vector as conditional embeddings for multiple behaviors. Then, we introduce the attention network to model interaction relationship among multiple preferences, and further utilize the adapted message sending and receiving operation of graph neural network, so that multi-aspect preference information from high-order neighbors will be captured. Finally, we utilize Bayesian Personalized Ranking loss function to learn the preference similarity on each behavior, and jointly learn multiple conditional node embeddings via multi-task learning framework. Extensive experiments on public datasets validate that our MCNE framework could significantly outperform several state-of-the-art baselines, and further support the visualization and transfer learning tasks with excellent interpretability and robustness. |
| Tasks | Multi-Task Learning, Representation Learning, Transfer Learning |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1905.11013v1 |
| https://arxiv.org/pdf/1905.11013v1.pdf | |
| PWC | https://paperswithcode.com/paper/mcne-an-end-to-end-framework-for-learning |
| Repo | |
| Framework | |
On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternatives
| Title | On the Optimality of Gaussian Kernel Based Nonparametric Tests against Smooth Alternatives |
| Authors | Tong Li, Ming Yuan |
| Abstract | Nonparametric tests via kernel embedding of distributions have witnessed a great deal of practical successes in recent years. However, statistical properties of these tests are largely unknown beyond consistency against a fixed alternative. To fill in this void, we study here the asymptotic properties of goodness-of-fit, homogeneity and independence tests using Gaussian kernels, arguably the most popular and successful among such tests. Our results provide theoretical justifications for this common practice by showing that tests using Gaussian kernel with an appropriately chosen scaling parameter are minimax optimal against smooth alternatives in all three settings. In addition, our analysis also pinpoints the importance of choosing a diverging scaling parameter when using Gaussian kernels and suggests a data-driven choice of the scaling parameter that yields tests optimal, up to an iterated logarithmic factor, over a wide range of smooth alternatives. Numerical experiments are also presented to further demonstrate the practical merits of the methodology. |
| Tasks | |
| Published | 2019-09-07 |
| URL | https://arxiv.org/abs/1909.03302v1 |
| https://arxiv.org/pdf/1909.03302v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-optimality-of-gaussian-kernel-based |
| Repo | |
| Framework | |
