Paper Group ANR 1128
Automatically Generating Macro Research Reports from a Piece of News. PipeMare: Asynchronous Pipeline Parallel DNN Training. Inferring the Optimal Policy using Markov Chain Monte Carlo. Walsh-Hadamard Variational Inference for Bayesian Deep Learning. How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?. LO-Net: Deep …
Automatically Generating Macro Research Reports from a Piece of News
| Title | Automatically Generating Macro Research Reports from a Piece of News |
| Authors | Wenxin Hu, Xiaofeng Zhang, Gang Yang |
| Abstract | Automatically generating macro research reports from economic news is an important yet challenging task. As we all know, it requires the macro analysts to write such reports within a short period of time after the important economic news are released. This motivates our work, i.e., using AI techniques to save manual cost. The goal of the proposed system is to generate macro research reports as the draft for macro analysts. Essentially, the core challenge is the long text generation issue. To address this issue, we propose a novel deep learning technique based approach which includes two components, i.e., outline generation and macro research report generation.For the model performance evaluation, we first crawl a large news-to-report dataset and then evaluate our approach on this dataset, and the generated reports are given for the subjective evaluation. |
| Tasks | Text Generation |
| Published | 2019-11-21 |
| URL | https://arxiv.org/abs/1911.09572v1 |
| https://arxiv.org/pdf/1911.09572v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatically-generating-macro-research |
| Repo | |
| Framework | |
PipeMare: Asynchronous Pipeline Parallel DNN Training
| Title | PipeMare: Asynchronous Pipeline Parallel DNN Training |
| Authors | Bowen Yang, Jian Zhang, Jonathan Li, Christopher Ré, Christopher R. Aberger, Christopher De Sa |
| Abstract | Pipeline parallelism (PP) when training neural networks enables larger models to be partitioned spatially, leading to both lower network communication and overall higher hardware utilization. Unfortunately, to preserve the statistical efficiency of sequential training, existing PP techniques sacrifice hardware efficiency by decreasing pipeline utilization or incurring extra memory costs. In this paper, we investigate to what extent these sacrifices are necessary. We devise PipeMare, a simple yet robust training method that tolerates asynchronous updates during PP execution without sacrificing utilization or memory, which allows efficient use of fine-grained pipeline parallelism. Concretely, when tested on ResNet and Transformer networks, asynchrony enables PipeMare to use up to $2.7\times$ less memory or get $4.3\times$ higher pipeline utilization, with similar model quality, when compared to state-of-the-art synchronous PP training techniques. |
| Tasks | |
| Published | 2019-10-09 |
| URL | https://arxiv.org/abs/1910.05124v2 |
| https://arxiv.org/pdf/1910.05124v2.pdf | |
| PWC | https://paperswithcode.com/paper/pipemare-asynchronous-pipeline-parallel-dnn |
| Repo | |
| Framework | |
Inferring the Optimal Policy using Markov Chain Monte Carlo
| Title | Inferring the Optimal Policy using Markov Chain Monte Carlo |
| Authors | Brandon Trabucco, Albert Qu, Simon Li, Ganeshkumar Ashokavardhanan |
| Abstract | This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises many real world systems such as playing video games, simulated control tasks, and real robot locomotion. Existing methods for estimating the optimal stochastic control policy rely on high variance estimates of the policy descent. However, these methods are not guaranteed to find the optimal stochastic policy, and the high variance gradient estimates make convergence unstable. In order to resolve these problems, we propose a technique using Markov Chain Monte Carlo to generate samples from the posterior distribution of the parameters conditioned on being optimal. Our method provably converges to the globally optimal stochastic policy, and empirically similar variance compared to the policy gradient. |
| Tasks | |
| Published | 2019-11-16 |
| URL | https://arxiv.org/abs/1912.02714v1 |
| https://arxiv.org/pdf/1912.02714v1.pdf | |
| PWC | https://paperswithcode.com/paper/inferring-the-optimal-policy-using-markov |
| Repo | |
| Framework | |
Walsh-Hadamard Variational Inference for Bayesian Deep Learning
| Title | Walsh-Hadamard Variational Inference for Bayesian Deep Learning |
| Authors | Simone Rossi, Sebastien Marmin, Maurizio Filippone |
| Abstract | Over-parameterized models, such as DeepNets and ConvNets, form a class of models that are routinely adopted in a wide variety of applications, and for which Bayesian inference is desirable but extremely challenging. Variational inference offers the tools to tackle this challenge in a scalable way and with some degree of flexibility on the approximation, but for over-parameterized models this is challenging due to the over-regularization property of the variational objective. Inspired by the literature on kernel methods, and in particular on structured approximations of distributions of random matrices, this paper proposes Walsh-Hadamard Variational Inference (WHVI), which uses Walsh-Hadamard-based factorization strategies to reduce the parameterization and accelerate computations, thus avoiding over-regularization issues with the variational objective. Extensive theoretical and empirical analyses demonstrate that WHVI yields considerable speedups and model reductions compared to other techniques to carry out approximate inference for over-parameterized models, and ultimately show how advances in kernel methods can be translated into advances in approximate Bayesian inference. |
| Tasks | Bayesian Inference |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1905.11248v1 |
| https://arxiv.org/pdf/1905.11248v1.pdf | |
| PWC | https://paperswithcode.com/paper/walsh-hadamard-variational-inference-for |
| Repo | |
| Framework | |
How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?
| Title | How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages? |
| Authors | Hila Gonen, Yova Kementchedjhieva, Yoav Goldberg |
| Abstract | Many natural languages assign grammatical gender also to inanimate nouns in the language. In such languages, words that relate to the gender-marked nouns are inflected to agree with the noun’s gender. We show that this affects the word representations of inanimate nouns, resulting in nouns with the same gender being closer to each other than nouns with different gender. While “embedding debiasing” methods fail to remove the effect, we demonstrate that a careful application of methods that neutralize grammatical gender signals from the words’ context when training word embeddings is effective in removing it. Fixing the grammatical gender bias yields a positive effect on the quality of the resulting word embeddings, both in monolingual and cross-lingual settings. We note that successfully removing gender signals, while achievable, is not trivial to do and that a language-specific morphological analyzer, together with careful usage of it, are essential for achieving good results. |
| Tasks | Word Embeddings |
| Published | 2019-10-30 |
| URL | https://arxiv.org/abs/1910.14161v1 |
| https://arxiv.org/pdf/1910.14161v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-does-grammatical-gender-affect-noun-1 |
| Repo | |
| Framework | |
LO-Net: Deep Real-time Lidar Odometry
| Title | LO-Net: Deep Real-time Lidar Odometry |
| Authors | Qing Li, Shaoyang Chen, Cheng Wang, Xin Li, Chenglu Wen, Ming Cheng, Jonathan Li |
| Abstract | We present a novel deep convolutional network pipeline, LO-Net, for real-time lidar odometry estimation. Unlike most existing lidar odometry (LO) estimations that go through individually designed feature selection, feature matching, and pose estimation pipeline, LO-Net can be trained in an end-to-end manner. With a new mask-weighted geometric constraint loss, LO-Net can effectively learn feature representation for LO estimation, and can implicitly exploit the sequential dependencies and dynamics in the data. We also design a scan-to-map module, which uses the geometric and semantic information learned in LO-Net, to improve the estimation accuracy. Experiments on benchmark datasets demonstrate that LO-Net outperforms existing learning based approaches and has similar accuracy with the state-of-the-art geometry-based approach, LOAM. |
| Tasks | Feature Selection, Pose Estimation |
| Published | 2019-04-17 |
| URL | https://arxiv.org/abs/1904.08242v2 |
| https://arxiv.org/pdf/1904.08242v2.pdf | |
| PWC | https://paperswithcode.com/paper/190408242 |
| Repo | |
| Framework | |
Testing Neural Program Analyzers
| Title | Testing Neural Program Analyzers |
| Authors | Md Rafiqul Islam Rabin, Ke Wang, Mohammad Amin Alipour |
| Abstract | Deep neural networks have been increasingly used in software engineering and program analysis tasks. They usually take a program and make some predictions about it, e.g., bug prediction. We call these models neural program analyzers. The reliability of neural programs can impact the reliability of the encompassing analyses. In this paper, we describe our ongoing efforts to develop effective techniques for testing neural programs. We discuss the challenges involved in developing such tools and our future plans. In our preliminary experiment on a neural model recently proposed in the literature, we found that the model is very brittle, and simple perturbations in the input can cause the model to make mistakes in its prediction. |
| Tasks | |
| Published | 2019-08-25 |
| URL | https://arxiv.org/abs/1908.10711v2 |
| https://arxiv.org/pdf/1908.10711v2.pdf | |
| PWC | https://paperswithcode.com/paper/testing-neural-programs |
| Repo | |
| Framework | |
Open Compound Domain Adaptation
| Title | Open Compound Domain Adaptation |
| Authors | Ziwei Liu, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong |
| Abstract | A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e.g., sunny weather) for achieving high performance on the test data in a target domain (e.g., rainy weather). Whether the target contains a single homogeneous domain or multiple heterogeneous domains, existing works always assume that there exist clear distinctions between the domains, which is often not true in practice (e.g., changes in weather). We study an open compound domain adaptation (OCDA) problem, in which the target is a compound of multiple homogeneous domains without domain labels, reflecting realistic data collection from mixed and novel situations. We propose a new approach based on two technical insights into OCDA: 1) a curriculum domain adaptation strategy to bootstrap generalization across domains in a data-driven self-organizing fashion and 2) a memory module to increase the model’s agility towards novel domains. Our experiments on digit classification, facial expression recognition, semantic segmentation, and reinforcement learning demonstrate the effectiveness of our approach. |
| Tasks | Domain Adaptation, Facial Expression Recognition, Semantic Segmentation |
| Published | 2019-09-08 |
| URL | https://arxiv.org/abs/1909.03403v2 |
| https://arxiv.org/pdf/1909.03403v2.pdf | |
| PWC | https://paperswithcode.com/paper/compound-domain-adaptation-in-an-open-world |
| Repo | |
| Framework | |
Exploring OpenStreetMap Availability for Driving Environment Understanding
| Title | Exploring OpenStreetMap Availability for Driving Environment Understanding |
| Authors | Yang Zheng, Izzat H. Izzat, John H. L. Hansen |
| Abstract | With the great achievement of artificial intelligence, vehicle technologies have advanced significantly from human centric driving towards fully automated driving. An intelligent vehicle should be able to understand the driver’s perception of the environment as well as controlling behavior of the vehicle. Since high digital map information has been available to provide rich environmental context about static roads, buildings and traffic infrastructures, it would be worthwhile to explore map data capability for driving task understanding. Alternative to commercial used maps, the OpenStreetMap (OSM) data is a free open dataset, which makes it unique for the exploration research. This study is focused on two tasks that leverage OSM for driving environment understanding. First, driving scenario attributes are retrieved from OSM elements, which are combined with vehicle dynamic signals for the driving event recognition. Utilizing steering angle changes and based on a Bi-directional Recurrent Neural Network (Bi-RNN), a driving sequence is segmented and classified as lane-keeping, lane-change-left, lane-change-right, turn-left, and turn-right events. Second, for autonomous driving perception, OSM data can be used to render virtual street views, represented as prior knowledge to fuse with vision/laser systems for road semantic segmentation. Five different types of road masks are generated from OSM, images, and Lidar points, and fused to characterize the drivable space at the driver’s perspective. An alternative data-driven approach is based on a Fully Convolutional Network (FCN), OSM availability for deep learning methods are discussed to reveal potential usage on compensating street view images and automatic road semantic annotation. |
| Tasks | Autonomous Driving, Semantic Segmentation |
| Published | 2019-03-11 |
| URL | http://arxiv.org/abs/1903.04084v1 |
| http://arxiv.org/pdf/1903.04084v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-openstreetmap-availability-for |
| Repo | |
| Framework | |
A Fine-Grained Facial Expression Database for End-to-End Multi-Pose Facial Expression Recognition
| Title | A Fine-Grained Facial Expression Database for End-to-End Multi-Pose Facial Expression Recognition |
| Authors | Wenxuan Wang, Qiang Sun, Tao Chen, Chenjie Cao, Ziqi Zheng, Guoqiang Xu, Han Qiu, Yanwei Fu |
| Abstract | The recent research of facial expression recognition has made a lot of progress due to the development of deep learning technologies, but some typical challenging problems such as the variety of rich facial expressions and poses are still not resolved. To solve these problems, we develop a new Facial Expression Recognition (FER) framework by involving the facial poses into our image synthesizing and classification process. There are two major novelties in this work. First, we create a new facial expression dataset of more than 200k images with 119 persons, 4 poses and 54 expressions. To our knowledge this is the first dataset to label faces with subtle emotion changes for expression recognition purpose. It is also the first dataset that is large enough to validate the FER task on unbalanced poses, expressions, and zero-shot subject IDs. Second, we propose a facial pose generative adversarial network (FaPE-GAN) to synthesize new facial expression images to augment the data set for training purpose, and then learn a LightCNN based Fa-Net model for expression classification. Finally, we advocate four novel learning tasks on this dataset. The experimental results well validate the effectiveness of the proposed approach. |
| Tasks | Facial Expression Recognition |
| Published | 2019-07-25 |
| URL | https://arxiv.org/abs/1907.10838v1 |
| https://arxiv.org/pdf/1907.10838v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-fine-grained-facial-expression-database-for |
| Repo | |
| Framework | |
Integrating neural networks into the blind deblurring framework to compete with the end-to-end learning-based methods
| Title | Integrating neural networks into the blind deblurring framework to compete with the end-to-end learning-based methods |
| Authors | Junde Wu, Xiaoguang Di, Jiehao Huang, Yu Zhang |
| Abstract | Recently, end-to-end learning-based methods based on deep neural network (DNN) have been proven effective for blind deblurring. Without human-made assumptions and numerical algorithms, they are able to restore images with fewer artifacts and better perceptual quality. However, in practice, we also find some of their drawbacks. Without the theoretical guidance, these methods can not perform well when the motion is complex and sometimes generate unreasonable results. In this paper, for overcoming these drawbacks, we integrate deep convolution neural networks into conventional deblurring framework. Specifically, we build Stacked Estimation Residual Net (SEN) to estimate the motion flow map and Recurrent Prior Generative and Adversarial Net (RP-GAN) to learn the implicit image prior in the optimization model. Comparing with state-of-the-art end-to-end learning-based methods, our method restores reasonable details and shows better generalization ability. |
| Tasks | Deblurring |
| Published | 2019-03-07 |
| URL | https://arxiv.org/abs/1903.02731v2 |
| https://arxiv.org/pdf/1903.02731v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-deep-neural-networks-in-blind |
| Repo | |
| Framework | |
FASTER Recurrent Networks for Efficient Video Classification
| Title | FASTER Recurrent Networks for Efficient Video Classification |
| Authors | Linchao Zhu, Laura Sevilla-Lara, Du Tran, Matt Feiszli, Yi Yang, Heng Wang |
| Abstract | Typical video classification methods often divide a video into short clips, do inference on each clip independently, then aggregate the clip-level predictions to generate the video-level results. However, processing visually similar clips independently ignores the temporal structure of the video sequence, and increases the computational cost at inference time. In this paper, we propose a novel framework named FASTER, i.e., Feature Aggregation for Spatio-TEmporal Redundancy. FASTER aims to leverage the redundancy between neighboring clips and reduce the computational cost by learning to aggregate the predictions from models of different complexities. The FASTER framework can integrate high quality representations from expensive models to capture subtle motion information and lightweight representations from cheap models to cover scene changes in the video. A new recurrent network (i.e., FAST-GRU) is designed to aggregate the mixture of different representations. Compared with existing approaches, FASTER can reduce the FLOPs by over 10x? while maintaining the state-of-the-art accuracy across popular datasets, such as Kinetics, UCF-101 and HMDB-51. |
| Tasks | Action Classification, Action Recognition In Videos, Video Classification |
| Published | 2019-06-10 |
| URL | https://arxiv.org/abs/1906.04226v2 |
| https://arxiv.org/pdf/1906.04226v2.pdf | |
| PWC | https://paperswithcode.com/paper/faster-recurrent-networks-for-video |
| Repo | |
| Framework | |
Machine-Assisted Map Editing
| Title | Machine-Assisted Map Editing |
| Authors | Favyen Bastani, Songtao He, Sofiane Abbar, Mohammad Alizadeh, Hari Balakrishnan, Sanjay Chawla, Sam Madden |
| Abstract | Mapping road networks today is labor-intensive. As a result, road maps have poor coverage outside urban centers in many countries. Systems to automatically infer road network graphs from aerial imagery and GPS trajectories have been proposed to improve coverage of road maps. However, because of high error rates, these systems have not been adopted by mapping communities. We propose machine-assisted map editing, where automatic map inference is integrated into existing, human-centric map editing workflows. To realize this, we build Machine-Assisted iD (MAiD), where we extend the web-based OpenStreetMap editor, iD, with machine-assistance functionality. We complement MAiD with a novel approach for inferring road topology from aerial imagery that combines the speed of prior segmentation approaches with the accuracy of prior iterative graph construction methods. We design MAiD to tackle the addition of major, arterial roads in regions where existing maps have poor coverage, and the incremental improvement of coverage in regions where major roads are already mapped. We conduct two user studies and find that, when participants are given a fixed time to map roads, they are able to add as much as 3.5x more roads with MAiD. |
| Tasks | graph construction |
| Published | 2019-06-17 |
| URL | https://arxiv.org/abs/1906.07138v1 |
| https://arxiv.org/pdf/1906.07138v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-assisted-map-editing |
| Repo | |
| Framework | |
Credit Scoring for Micro-Loans
| Title | Credit Scoring for Micro-Loans |
| Authors | Nikolay Dubina, Dasom Kang, Alex Suh |
| Abstract | Credit Scores are ubiquitous and instrumental for loan providers and regulators. In this paper we showcase how micro-loan credit system can be developed in real setting. We show what challenges arise and discuss solutions. Particularly, we are concerned about model interpretability and data quality. In the final section, we introduce semi-supervised algorithm that aids model development and evaluate its performance |
| Tasks | |
| Published | 2019-05-10 |
| URL | https://arxiv.org/abs/1905.03946v1 |
| https://arxiv.org/pdf/1905.03946v1.pdf | |
| PWC | https://paperswithcode.com/paper/credit-scoring-for-micro-loans |
| Repo | |
| Framework | |
Prediction of rare feature combinations in population synthesis: Application of deep generative modelling
| Title | Prediction of rare feature combinations in population synthesis: Application of deep generative modelling |
| Authors | Sergio Garrido, Stanislav S. Borysov, Francisco C. Pereira, Jeppe Rich |
| Abstract | In population synthesis applications, when considering populations with many attributes, a fundamental problem is the estimation of rare combinations of feature attributes. Unsurprisingly, it is notably more difficult to reliably representthe sparser regions of such multivariate distributions and in particular combinations of attributes which are absent from the original sample. In the literature this is commonly known as sampling zeros for which no systematic solution has been proposed so far. In this paper, two machine learning algorithms, from the family of deep generative models,are proposed for the problem of population synthesis and with particular attention to the problem of sampling zeros. Specifically, we introduce the Wasserstein Generative Adversarial Network (WGAN) and the Variational Autoencoder(VAE), and adapt these algorithms for a large-scale population synthesis application. The models are implemented on a Danish travel survey with a feature-space of more than 60 variables. The models are validated in a cross-validation scheme and a set of new metrics for the evaluation of the sampling-zero problem is proposed. Results show how these models are able to recover sampling zeros while keeping the estimation of truly impossible combinations, the structural zeros, at a comparatively low level. Particularly, for a low dimensional experiment, the VAE, the marginal sampler and the fully random sampler generate 5%, 21% and 26%, respectively, more structural zeros per sampling zero generated by the WGAN, while for a high dimensional case, these figures escalate to 44%, 2217% and 170440%, respectively. This research directly supports the development of agent-based systems and in particular cases where detailed socio-economic or geographical representations are required. |
| Tasks | |
| Published | 2019-09-17 |
| URL | https://arxiv.org/abs/1909.07689v1 |
| https://arxiv.org/pdf/1909.07689v1.pdf | |
| PWC | https://paperswithcode.com/paper/prediction-of-rare-feature-combinations-in |
| Repo | |
| Framework | |
