Paper Group AWR 392
Traffic map prediction using UNet based deep convolutional neural network. OmniNet: A unified architecture for multi-modal multi-task learning. Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro. nnU-Net: Breaking the Spell on Successful Medical Image Segmentation. Real numbers, data science and chaos: How to fit a …
Traffic map prediction using UNet based deep convolutional neural network
| Title | Traffic map prediction using UNet based deep convolutional neural network |
| Authors | Sungbin Choi |
| Abstract | This paper describes our UNet based deep convolutional neural network approach on the Traffic4cast challenge 2019. Challenges task is to predict future traffic flow volume, heading and speed on high resolution whole city map. We used UNet based deep convolutional neural network to train predictive model for the short term traffic forecast. On each convolution block, layers are densely connected with subsequent layers like a DenseNet. Trained and evaluated on the real world data set collected from three distinct cities in the world, our method achieved best performance in this challenge. |
| Tasks | |
| Published | 2019-11-25 |
| URL | https://arxiv.org/abs/1912.05288v1 |
| https://arxiv.org/pdf/1912.05288v1.pdf | |
| PWC | https://paperswithcode.com/paper/traffic-map-prediction-using-unet-based-deep |
| Repo | https://github.com/SungbinChoi/traffic4cast2019 |
| Framework | tf |
OmniNet: A unified architecture for multi-modal multi-task learning
| Title | OmniNet: A unified architecture for multi-modal multi-task learning |
| Authors | Subhojeet Pramanik, Priyanka Agrawal, Aman Hussain |
| Abstract | Transformer is a popularly used neural network architecture, especially for language understanding. We introduce an extended and unified architecture which can be used for tasks involving a variety of modalities like image, text, videos, etc. We propose a spatio-temporal cache mechanism that enables learning spatial dimension of the input in addition to the hidden states corresponding to the temporal input sequence. The proposed architecture further enables a single model to support tasks with multiple input modalities as well as asynchronous multi-task learning, thus we refer to it as OmniNet. For example, a single instance of OmniNet can concurrently learn to perform the tasks of part-of-speech tagging, image captioning, visual question answering and video activity recognition. We demonstrate that training these four tasks together results in about three times compressed model while retaining the performance in comparison to training them individually. We also show that using this neural network pre-trained on some modalities assists in learning an unseen task. This illustrates the generalization capacity of the self-attention mechanism on the spatio-temporal cache present in OmniNet. |
| Tasks | Image Captioning, Multi-Task Learning, Part-Of-Speech Tagging, Question Answering, Visual Question Answering |
| Published | 2019-07-17 |
| URL | https://arxiv.org/abs/1907.07804v1 |
| https://arxiv.org/pdf/1907.07804v1.pdf | |
| PWC | https://paperswithcode.com/paper/omninet-a-unified-architecture-for-multi |
| Repo | https://github.com/subho406/OmniNet |
| Framework | pytorch |
Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro
| Title | Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro |
| Authors | Du Phan, Neeraj Pradhan, Martin Jankowiak |
| Abstract | NumPyro is a lightweight library that provides an alternate NumPy backend to the Pyro probabilistic programming language with the same modeling interface, language primitives and effect handling abstractions. Effect handlers allow Pyro’s modeling API to be extended to NumPyro despite its being built atop a fundamentally different JAX-based functional backend. In this work, we demonstrate the power of composing Pyro’s effect handlers with the program transformations that enable hardware acceleration, automatic differentiation, and vectorization in JAX. In particular, NumPyro provides an iterative formulation of the No-U-Turn Sampler (NUTS) that can be end-to-end JIT compiled, yielding an implementation that is much faster than existing alternatives in both the small and large dataset regimes. |
| Tasks | Probabilistic Programming |
| Published | 2019-12-24 |
| URL | https://arxiv.org/abs/1912.11554v1 |
| https://arxiv.org/pdf/1912.11554v1.pdf | |
| PWC | https://paperswithcode.com/paper/composable-effects-for-flexible-and |
| Repo | https://github.com/pyro-ppl/numpyro |
| Framework | jax |
nnU-Net: Breaking the Spell on Successful Medical Image Segmentation
| Title | nnU-Net: Breaking the Spell on Successful Medical Image Segmentation |
| Authors | Fabian Isensee, Jens Petersen, Simon A. A. Kohl, Paul F. Jäger, Klaus H. Maier-Hein |
| Abstract | Fueled by the diversity of datasets, semantic segmentation is a popular subfield in medical image analysis with a vast number of new methods being proposed each year. This ever-growing jungle of methodologies, however, becomes increasingly impenetrable. At the same time, many proposed methods fail to generalize beyond the experiments they were demonstrated on, thus hampering the process of developing a segmentation algorithm on a new dataset. Here we present nnU-Net (‘no-new-Net’), a framework that automatically adapts itself to any given new dataset. While this process was completely human-driven so far, we make a first attempt to automate necessary adaptations such as preprocessing, the exact patch size, batch size, and inference settings based on the properties of a given dataset. Remarkably, nnU-Net strips away the architectural bells and whistles that are typically proposed in the literature and relies on just a simple U-Net architecture embedded in a robust training scheme. Out of the box, nnU-Net achieves state of the art performance on six well-established segmentation challenges. Source code is available at https://github.com/MIC-DKFZ/nnunet. |
| Tasks | Medical Image Segmentation, Semantic Segmentation |
| Published | 2019-04-17 |
| URL | http://arxiv.org/abs/1904.08128v1 |
| http://arxiv.org/pdf/1904.08128v1.pdf | |
| PWC | https://paperswithcode.com/paper/nnu-net-breaking-the-spell-on-successful |
| Repo | https://github.com/NeuroAI-HD/HD-GLIO-AUTO |
| Framework | none |
Real numbers, data science and chaos: How to fit any dataset with a single parameter
| Title | Real numbers, data science and chaos: How to fit any dataset with a single parameter |
| Authors | Laurent Boué |
| Abstract | We show how any dataset of any modality (time-series, images, sound…) can be approximated by a well-behaved (continuous, differentiable…) scalar function with a single real-valued parameter. Building upon elementary concepts from chaos theory, we adopt a pedagogical approach demonstrating how to adjust this parameter in order to achieve arbitrary precision fit to all samples of the data. Targeting an audience of data scientists with a taste for the curious and unusual, the results presented here expand on previous similar observations regarding expressiveness power and generalization of machine learning models. |
| Tasks | Time Series |
| Published | 2019-04-28 |
| URL | http://arxiv.org/abs/1904.12320v1 |
| http://arxiv.org/pdf/1904.12320v1.pdf | |
| PWC | https://paperswithcode.com/paper/real-numbers-data-science-and-chaos-how-to |
| Repo | https://github.com/Ranlot/single-parameter-fit |
| Framework | none |
Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting
| Title | Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting |
| Authors | Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, Eric Horvitz, Stefano Ermon |
| Abstract | A learned generative model often produces biased statistics relative to the underlying data distribution. A standard technique to correct this bias is importance sampling, where samples from the model are weighted by the likelihood ratio under model and true distributions. When the likelihood ratio is unknown, it can be estimated by training a probabilistic classifier to distinguish samples from the two distributions. We employ this likelihood-free importance weighting method to correct for the bias in generative models. We find that this technique consistently improves standard goodness-of-fit metrics for evaluating the sample quality of state-of-the-art deep generative models, suggesting reduced bias. Finally, we demonstrate its utility on representative applications in a) data augmentation for classification using generative adversarial networks, and b) model-based policy evaluation using off-policy data. |
| Tasks | Data Augmentation |
| Published | 2019-06-23 |
| URL | https://arxiv.org/abs/1906.09531v2 |
| https://arxiv.org/pdf/1906.09531v2.pdf | |
| PWC | https://paperswithcode.com/paper/bias-correction-of-learned-generative-models |
| Repo | https://github.com/aditya-grover/bias-correction-generative |
| Framework | pytorch |
EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras
| Title | EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras |
| Authors | Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos |
| Abstract | Dynamic obstacle avoidance on quadrotors requires low latency. A class of sensors that are particularly suitable for such scenarios are event cameras. In this paper, we present a deep learning – based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation. Our approach uses a series of shallow neural networks for estimating both the ego-motion and the motion of independently moving objects. The networks are trained in simulation and directly transfer to the real world without any fine-tuning or retraining. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with obstacles of different shapes and sizes, achieving an overall success rate of 70% including objects of unknown shape and a low light testing scenario. To our knowledge, this is the first deep learning – based solution to the problem of dynamic obstacle avoidance using event cameras on a quadrotor. Finally, we also extend our work to the pursuit task by merely reversing the control policy, proving that our navigation stack can cater to different scenarios. |
| Tasks | Motion Estimation |
| Published | 2019-06-07 |
| URL | https://arxiv.org/abs/1906.02919v3 |
| https://arxiv.org/pdf/1906.02919v3.pdf | |
| PWC | https://paperswithcode.com/paper/evdodge-embodied-ai-for-high-speed-dodging-on |
| Repo | https://github.com/prgumd/EVDodge |
| Framework | none |
Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes
| Title | Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes |
| Authors | Matt Jordan, Justin Lewis, Alexandros G. Dimakis |
| Abstract | We propose a novel method for computing exact pointwise robustness of deep neural networks for all convex $\ell_p$ norms. Our algorithm, GeoCert, finds the largest $\ell_p$ ball centered at an input point $x_0$, within which the output class of a given neural network with ReLU nonlinearities remains unchanged. We relate the problem of computing pointwise robustness of these networks to that of computing the maximum norm ball with a fixed center that can be contained in a non-convex polytope. This is a challenging problem in general, however we show that there exists an efficient algorithm to compute this for polyhedral complices. Further we show that piecewise linear neural networks partition the input space into a polyhedral complex. Our algorithm has the ability to almost immediately output a nontrivial lower bound to the pointwise robustness which is iteratively improved until it ultimately becomes tight. We empirically show that our approach generates distance lower bounds that are tighter compared to prior work, under moderate time constraints. |
| Tasks | |
| Published | 2019-03-20 |
| URL | https://arxiv.org/abs/1903.08778v2 |
| https://arxiv.org/pdf/1903.08778v2.pdf | |
| PWC | https://paperswithcode.com/paper/provable-certificates-for-adversarial |
| Repo | https://github.com/revbucket/geometric-certificates |
| Framework | pytorch |
Bridging the domain gap in cross-lingual document classification
| Title | Bridging the domain gap in cross-lingual document classification |
| Authors | Guokun Lai, Barlas Oguz, Yiming Yang, Veselin Stoyanov |
| Abstract | The scarcity of labeled training data often prohibits the internationalization of NLP models to multiple languages. Recent developments in cross-lingual understanding (XLU) has made progress in this area, trying to bridge the language barrier using language universal representations. However, even if the language problem was resolved, models trained in one language would not transfer to another language perfectly due to the natural domain drift across languages and cultures. We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language. We combine state-of-the-art cross-lingual methods with recently proposed methods for weakly supervised learning such as unsupervised pre-training and unsupervised data augmentation to simultaneously close both the language gap and the domain gap in XLU. We show that addressing the domain gap is crucial. We improve over strong baselines and achieve a new state-of-the-art for cross-lingual document classification. |
| Tasks | Cross-Lingual Document Classification, Data Augmentation, Document Classification |
| Published | 2019-09-16 |
| URL | https://arxiv.org/abs/1909.07009v2 |
| https://arxiv.org/pdf/1909.07009v2.pdf | |
| PWC | https://paperswithcode.com/paper/bridging-the-domain-gap-in-cross-lingual |
| Repo | https://github.com/laiguokun/xlu-data |
| Framework | none |
Creative GANs for generating poems, lyrics, and metaphors
| Title | Creative GANs for generating poems, lyrics, and metaphors |
| Authors | Asir Saeed, Suzana Ilić, Eva Zangerle |
| Abstract | Generative models for text have substantially contributed to tasks like machine translation and language modeling, using maximum likelihood optimization (MLE). However, for creative text generation, where multiple outputs are possible and originality and uniqueness are encouraged, MLE falls short. Methods optimized for MLE lead to outputs that can be generic, repetitive and incoherent. In this work, we use a Generative Adversarial Network framework to alleviate this problem. We evaluate our framework on poetry, lyrics and metaphor datasets, each with widely different characteristics, and report better performance of our objective function over other generative models. |
| Tasks | Language Modelling, Machine Translation, Text Generation |
| Published | 2019-09-20 |
| URL | https://arxiv.org/abs/1909.09534v1 |
| https://arxiv.org/pdf/1909.09534v1.pdf | |
| PWC | https://paperswithcode.com/paper/creative-gans-for-generating-poems-lyrics-and |
| Repo | https://github.com/Machine-Learning-Tokyo/Poetry-GAN |
| Framework | none |
Symbolic Graph Embedding using Frequent Pattern Mining
| Title | Symbolic Graph Embedding using Frequent Pattern Mining |
| Authors | Blaz Škrlj, Jan Kralj, Nada Lavrač |
| Abstract | Relational data mining is becoming ubiquitous in many fields of study. It offers insights into behaviour of complex, real-world systems which cannot be modeled directly using propositional learning. We propose Symbolic Graph Embedding (SGE), an algorithm aimed to learn symbolic node representations. Built on the ideas from the field of inductive logic programming, SGE first samples a given node’s neighborhood and interprets it as a transaction database, which is used for frequent pattern mining to identify logical conjuncts of items that co-occur frequently in a given context. Such patterns are in this work used as features to represent individual nodes, yielding interpretable, symbolic node embeddings. The proposed SGE approach on a venue classification task outperforms shallow node embedding methods such as DeepWalk, and performs similarly to metapath2vec, a black-box representation learner that can exploit node and edge types in a given graph. The proposed SGE approach performs especially well when small amounts of data are used for learning, scales to graphs with millions of nodes and edges, and can be run on an of-the-shelf laptop. |
| Tasks | Graph Embedding |
| Published | 2019-10-29 |
| URL | https://arxiv.org/abs/1910.13314v1 |
| https://arxiv.org/pdf/1910.13314v1.pdf | |
| PWC | https://paperswithcode.com/paper/191013314 |
| Repo | https://github.com/SkBlaz/SGE |
| Framework | none |
Fair Resource Allocation in Federated Learning
| Title | Fair Resource Allocation in Federated Learning |
| Authors | Tian Li, Maziar Sanjabi, Ahmad Beirami, Virginia Smith |
| Abstract | Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency. |
| Tasks | |
| Published | 2019-05-25 |
| URL | https://arxiv.org/abs/1905.10497v2 |
| https://arxiv.org/pdf/1905.10497v2.pdf | |
| PWC | https://paperswithcode.com/paper/190510497 |
| Repo | https://github.com/litian96/fair_flearn |
| Framework | none |
Nonlinear generalization of the monotone single index model
| Title | Nonlinear generalization of the monotone single index model |
| Authors | Zeljko Kereta, Timo Klock, Valeriya Naumova |
| Abstract | Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as $g(<a,x>)$, where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that uses a localized proxy of the geodesic metric. We present theoretical guarantees for estimation of local index vectors and out-of-sample prediction, and demonstrate the performance of our method with experiments on synthetic and real-world data sets, comparing it with state-of-the-art methods. |
| Tasks | |
| Published | 2019-02-24 |
| URL | https://arxiv.org/abs/1902.09024v2 |
| https://arxiv.org/pdf/1902.09024v2.pdf | |
| PWC | https://paperswithcode.com/paper/nonlinear-generalization-of-the-single-index |
| Repo | https://github.com/soply/local_sim_experiments |
| Framework | none |
s-LWSR: Super Lightweight Super-Resolution Network
| Title | s-LWSR: Super Lightweight Super-Resolution Network |
| Authors | Biao Li, Jiabin Liu, Bo Wang, Zhiquan Qi, Yong Shi |
| Abstract | Deep learning (DL) architectures for superresolution (SR) normally contain tremendous parameters, which has been regarded as the crucial advantage for obtaining satisfying performance. However, with the widespread use of mobile phones for taking and retouching photos, this character greatly hampers the deployment of DL-SR models on the mobile devices. To address this problem, in this paper, we propose a super lightweight SR network: s-LWSR. There are mainly three contributions in our work. Firstly, in order to efficiently abstract features from the low resolution image, we build an information pool to mix multi-level information from the first half part of the pipeline. Accordingly, the information pool feeds the second half part with the combination of hierarchical features from the previous layers. Secondly, we employ a compression module to further decrease the size of parameters. Intensive analysis confirms its capacity of trade-off between model complexity and accuracy. Thirdly, by revealing the specific role of activation in deep models, we remove several activation layers in our SR model to retain more information for performance improvement. Extensive experiments show that our s-LWSR, with limited parameters and operations, can achieve similar performance to other cumbersome DL-SR methods. |
| Tasks | Super-Resolution |
| Published | 2019-09-24 |
| URL | https://arxiv.org/abs/1909.10774v1 |
| https://arxiv.org/pdf/1909.10774v1.pdf | |
| PWC | https://paperswithcode.com/paper/s-lwsr-super-lightweight-super-resolution |
| Repo | https://github.com/Sudo-Biao/s-LWSR |
| Framework | pytorch |
A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling
| Title | A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling |
| Authors | Qingrong Xia, Zhenghua Li, Min Zhang |
| Abstract | Semantic role labeling (SRL) aims to identify the predicate-argument structure of a sentence. Inspired by the strong correlation between syntax and semantics, previous works pay much attention to improve SRL performance on exploiting syntactic knowledge, achieving significant results. Pipeline methods based on automatic syntactic trees and multi-task learning (MTL) approaches using standard syntactic trees are two common research orientations. In this paper, we adopt a simple unified span-based model for both span-based and word-based Chinese SRL as a strong baseline. Besides, we present a MTL framework that includes the basic SRL module and a dependency parser module. Different from the commonly used hard parameter sharing strategy in MTL, the main idea is to extract implicit syntactic representations from the dependency parser as external inputs for the basic SRL model. Experiments on the benchmarks of Chinese Proposition Bank 1.0 and CoNLL-2009 Chinese datasets show that our proposed framework can effectively improve the performance over the strong baselines. With the external BERT representations, our framework achieves new state-of-the-art 87.54 and 88.5 F1 scores on the two test data of the two benchmarks, respectively. In-depth analysis are conducted to gain more insights on the proposed framework and the effectiveness of syntax. |
| Tasks | Multi-Task Learning, Semantic Role Labeling |
| Published | 2019-11-12 |
| URL | https://arxiv.org/abs/1911.04641v1 |
| https://arxiv.org/pdf/1911.04641v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-syntax-aware-multi-task-learning-framework-1 |
| Repo | https://github.com/KiroSummer/A_Syntax-aware_MTL_Framework_for_Chinese_SRL |
| Framework | pytorch |
