Paper Group ANR 698
Singular Value Decomposition and Neural Networks. Synthetic QA Corpora Generation with Roundtrip Consistency. Simulation-based reinforcement learning for real-world autonomous driving. Learning to Train with Synthetic Humans. Alpha Discovery Neural Network based on Prior Knowledge. Automatic Differentiation for Adjoint Stencil Loops. A Segmentation …
Singular Value Decomposition and Neural Networks
| Title | Singular Value Decomposition and Neural Networks |
| Authors | Bernhard Bermeitinger, Tomas Hrycej, Siegfried Handschuh |
| Abstract | Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks—it is their linear analogy. Besides of this insight, it can be used as a good initial guess for the network parameters, leading to substantially better optimization results. |
| Tasks | |
| Published | 2019-06-27 |
| URL | https://arxiv.org/abs/1906.11755v1 |
| https://arxiv.org/pdf/1906.11755v1.pdf | |
| PWC | https://paperswithcode.com/paper/singular-value-decomposition-and-neural |
| Repo | |
| Framework | |
Synthetic QA Corpora Generation with Roundtrip Consistency
| Title | Synthetic QA Corpora Generation with Roundtrip Consistency |
| Authors | Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, Michael Collins |
| Abstract | We introduce a novel method of generating synthetic question answering corpora by combining models of question generation and answer extraction, and by filtering the results to ensure roundtrip consistency. By pretraining on the resulting corpora we obtain significant improvements on SQuAD2 and NQ, establishing a new state-of-the-art on the latter. Our synthetic data generation models, for both question generation and answer extraction, can be fully reproduced by finetuning a publicly available BERT model on the extractive subsets of SQuAD2 and NQ. We also describe a more powerful variant that does full sequence-to-sequence pretraining for question generation, obtaining exact match and F1 at less than 0.1% and 0.4% from human performance on SQuAD2. |
| Tasks | Question Answering, Question Generation, Synthetic Data Generation |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05416v1 |
| https://arxiv.org/pdf/1906.05416v1.pdf | |
| PWC | https://paperswithcode.com/paper/synthetic-qa-corpora-generation-with |
| Repo | |
| Framework | |
Simulation-based reinforcement learning for real-world autonomous driving
| Title | Simulation-based reinforcement learning for real-world autonomous driving |
| Authors | Błażej Osiński, Adam Jakubowski, Piotr Miłoś, Paweł Zięcina, Christopher Galias, Silviu Homoceanu, Henryk Michalewski |
| Abstract | We use reinforcement learning in simulation to obtain a driving system controlling a full-size real-world vehicle. The driving policy takes RGB images from a single camera and their semantic segmentation as input. We use mostly synthetic data, with labelled real-world data appearing only in the training of the segmentation network. Using reinforcement learning in simulation and synthetic data is motivated by lowering costs and engineering effort. In real-world experiments we confirm that we achieved successful sim-to-real policy transfer. Based on the extensive evaluation, we analyze how design decisions about perception, control, and training impact the real-world performance. |
| Tasks | Autonomous Driving, Semantic Segmentation |
| Published | 2019-11-29 |
| URL | https://arxiv.org/abs/1911.12905v3 |
| https://arxiv.org/pdf/1911.12905v3.pdf | |
| PWC | https://paperswithcode.com/paper/simulation-based-reinforcement-learning-for |
| Repo | |
| Framework | |
Learning to Train with Synthetic Humans
| Title | Learning to Train with Synthetic Humans |
| Authors | David T. Hoffmann, Dimitrios Tzionas, Micheal J. Black, Siyu Tang |
| Abstract | Neural networks need big annotated datasets for training. However, manual annotation can be too expensive or even unfeasible for certain tasks, like multi-person 2D pose estimation with severe occlusions. A remedy for this is synthetic data with perfect ground truth. Here we explore two variations of synthetic data for this challenging problem; a dataset with purely synthetic humans and a real dataset augmented with synthetic humans. We then study which approach better generalizes to real data, as well as the influence of virtual humans in the training loss. Using the augmented dataset, without considering synthetic humans in the loss, leads to the best results. We observe that not all synthetic samples are equally informative for training, while the informative samples are different for each training stage. To exploit this observation, we employ an adversarial student-teacher framework; the teacher improves the student by providing the hardest samples for its current state as a challenge. Experiments show that the student-teacher framework outperforms normal training on the purely synthetic dataset. |
| Tasks | Pose Estimation |
| Published | 2019-08-02 |
| URL | https://arxiv.org/abs/1908.00967v1 |
| https://arxiv.org/pdf/1908.00967v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-train-with-synthetic-humans |
| Repo | |
| Framework | |
Alpha Discovery Neural Network based on Prior Knowledge
| Title | Alpha Discovery Neural Network based on Prior Knowledge |
| Authors | Jie Fang, Zhikang Xia, Xiang Liu, Shutao Xia, Jianwu Lin, Yong Jiang |
| Abstract | In financial automatic feature construction task, genetic programming (GP) is the state-of-the-art technique. It employs reverse polish expression to represent features and then simulate the evolution process. However, with the development of deep learning, more choices to design this algorithm are available. This paper proposes Alpha Discovery Neural Network (ADNN), equipped with different kinds of feature extractors to construct diversified financial technical factors based on prior knowledge. The experiment result shows that both fully-connected network and recurrent network are good at extracting information from financial time series, but convolution network structure can not effectively extract this information. ADNN effectively enrich the current factor pool because in all cases, ADNN can construct more informative and diversified features than GP. Moreover, features constructed by ADNN can always improve original strategy return, Sharpe ratio and max draw-down. |
| Tasks | Time Series |
| Published | 2019-12-26 |
| URL | https://arxiv.org/abs/1912.11761v4 |
| https://arxiv.org/pdf/1912.11761v4.pdf | |
| PWC | https://paperswithcode.com/paper/alpha-discovery-neural-network-based-on-prior |
| Repo | |
| Framework | |
Automatic Differentiation for Adjoint Stencil Loops
| Title | Automatic Differentiation for Adjoint Stencil Loops |
| Authors | Jan Hückelheim, Navjot Kukreja, Sri Hari Krishna Narayanan, Fabio Luporini, Gerard Gorman, Paul Hovland |
| Abstract | Stencil loops are a common motif in computations including convolutional neural networks, structured-mesh solvers for partial differential equations, and image processing. Stencil loops are easy to parallelise, and their fast execution is aided by compilers, libraries, and domain-specific languages. Reverse-mode automatic differentiation, also known as algorithmic differentiation, autodiff, adjoint differentiation, or back-propagation, is sometimes used to obtain gradients of programs that contain stencil loops. Unfortunately, conventional automatic differentiation results in a memory access pattern that is not stencil-like and not easily parallelisable. In this paper we present a novel combination of automatic differentiation and loop transformations that preserves the structure and memory access pattern of stencil loops, while computing fully consistent derivatives. The generated loops can be parallelised and optimised for performance in the same way and using the same tools as the original computation. We have implemented this new technique in the Python tool PerforAD, which we release with this paper along with test cases derived from seismic imaging and computational fluid dynamics applications. |
| Tasks | |
| Published | 2019-07-05 |
| URL | https://arxiv.org/abs/1907.02818v1 |
| https://arxiv.org/pdf/1907.02818v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-differentiation-for-adjoint-stencil |
| Repo | |
| Framework | |
A Segmentation-Oriented Inter-Class Transfer Method: Application to Retinal Vessel Segmentation
| Title | A Segmentation-Oriented Inter-Class Transfer Method: Application to Retinal Vessel Segmentation |
| Authors | Chengzhi Shi, Jihong Liu, Dali Chen |
| Abstract | Retinal vessel segmentation, as a principal nonintrusive diagnose method for ophthalmology diseases or diabetics, suffers from data scarcity due to requiring pixel-wise labels. In this paper, we proposed a convenient patch-based two-stage transfer method. First, based on the information bottleneck theory, we insert one dimensionality-reduced layer for task-specific feature space. Next, the semi-supervised clustering is conducted to select instances, from different sources databases, possessing similarities in the feature space. Surprisingly, we empirically demonstrate that images from different classes possessing similarities contribute to better performance than some same-class instances. The proposed framework achieved an accuracy of 97%, 96.8%, and 96.77% on DRIVE, STARE, and HRF respectively, outperforming current methods and independent human observers (DRIVE (96.37%) and STARE (93.39%)). |
| Tasks | Retinal Vessel Segmentation |
| Published | 2019-06-20 |
| URL | https://arxiv.org/abs/1906.08501v1 |
| https://arxiv.org/pdf/1906.08501v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-segmentation-oriented-inter-class-transfer |
| Repo | |
| Framework | |
A Distributed Neural Network Architecture for Robust Non-Linear Spatio-Temporal Prediction
| Title | A Distributed Neural Network Architecture for Robust Non-Linear Spatio-Temporal Prediction |
| Authors | Matthias Karlbauer, Sebastian Otte, Hendrik P. A. Lensch, Thomas Scholten, Volker Wulfmeyer, Martin V. Butz |
| Abstract | We introduce a distributed spatio-temporal artificial neural network architecture (DISTANA). It encodes mesh nodes using recurrent, neural prediction kernels (PKs), while neural transition kernels (TKs) transfer information between neighboring PKs, together modeling and predicting spatio-temporal time series dynamics. As a consequence, DISTANA assumes that generally applicable causes, which may be locally modified, generate the observed data. DISTANA learns in a parallel, spatially distributed manner, scales to large problem spaces, is capable of approximating complex dynamics, and is particularly robust to overfitting when compared to other competitive ANN models. Moreover, it is applicable to heterogeneously structured meshes. |
| Tasks | Time Series |
| Published | 2019-12-23 |
| URL | https://arxiv.org/abs/1912.11141v1 |
| https://arxiv.org/pdf/1912.11141v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-distributed-neural-network-architecture-for |
| Repo | |
| Framework | |
Path-Based Contextualization of Knowledge Graphs for Textual Entailment
| Title | Path-Based Contextualization of Knowledge Graphs for Textual Entailment |
| Authors | Kshitij Fadnis, Kartik Talamadupula, Pavan Kapanipathi, Haque Ishfaq, Salim Roukos, Achille Fokoue |
| Abstract | In this paper, we introduce the problem of knowledge graph contextualization – that is, given a specific NLP task, the problem of extracting meaningful and relevant sub-graphs from a given knowledge graph. The task in the case of this paper is the textual entailment problem, and the context is a relevant sub-graph for an instance of the textual entailment problem – where given two sentences p and h, the entailment relationship between them has to be predicted automatically. We base our methodology on finding paths in a cost-customized external knowledge graph, and building the most relevant sub-graph that connects p and h. We show that our path selection mechanism to generate sub-graphs not only reduces noise, but also retrieves meaningful information from large knowledge graphs. Our evaluation shows that using information on entities as well as the relationships between them improves on the performance of purely text-based systems. |
| Tasks | Knowledge Graphs, Natural Language Inference |
| Published | 2019-11-05 |
| URL | https://arxiv.org/abs/1911.02085v2 |
| https://arxiv.org/pdf/1911.02085v2.pdf | |
| PWC | https://paperswithcode.com/paper/heuristics-for-interpretable-knowledge-graph |
| Repo | |
| Framework | |
Mixture of Pre-processing Experts Model for Noise Robust Deep Learning on Resource Constrained Platforms
| Title | Mixture of Pre-processing Experts Model for Noise Robust Deep Learning on Resource Constrained Platforms |
| Authors | Taesik Na, Minah Lee, Burhan A. Mudassar, Priyabrata Saha, Jong Hwan Ko, Saibal Mukhopadhyay |
| Abstract | Deep learning on an edge device requires energy efficient operation due to ever diminishing power budget. Intentional low quality data during the data acquisition for longer battery life, and natural noise from the low cost sensor degrade the quality of target output which hinders adoption of deep learning on an edge device. To overcome these problems, we propose simple yet efficient mixture of pre-processing experts (MoPE) model to handle various image distortions including low resolution and noisy images. We also propose to use adversarially trained auto encoder as a pre-processing expert for the noisy images. We evaluate our proposed method for various machine learning tasks including object detection on MS-COCO 2014 dataset, multiple object tracking problem on MOT-Challenge dataset, and human activity classification on UCF 101 dataset. Experimental results show that the proposed method achieves better detection, tracking and activity classification accuracies under noise without sacrificing accuracies for the clean images. The overheads of our proposed MoPE are 0.67% and 0.17% in terms of memory and computation compared to the baseline object detection network. |
| Tasks | Multiple Object Tracking, Object Detection, Object Tracking |
| Published | 2019-04-29 |
| URL | http://arxiv.org/abs/1904.12426v1 |
| http://arxiv.org/pdf/1904.12426v1.pdf | |
| PWC | https://paperswithcode.com/paper/mixture-of-pre-processing-experts-model-for-1 |
| Repo | |
| Framework | |
A Grid-based Approach for Convexity Analysis of a Density-based Cluster
| Title | A Grid-based Approach for Convexity Analysis of a Density-based Cluster |
| Authors | Sayyed-Ahmad Naghavi-Nozad, Seyed-Mojtaba Banaei, Mohsen Saberi |
| Abstract | This paper presents a novel geometrical approach to investigate the convexity of a density-based cluster. Our approach is grid-based and we are about to calibrate the value space of the cluster. However, the cluster objects are coming from an infinite distribution, their number is finite, and thus, the regarding shape will not be sharp. Therefore, we establish the precision of the grid properly in a way that, the reliable approximate boundaries of the cluster are founded. After that, regarding the simple notion of convex sets and midpoint convexity, we investigate whether or not the density-based cluster is convex. Moreover, our experiments on synthetic datasets demonstrate the desirable performance of our method. |
| Tasks | |
| Published | 2019-10-03 |
| URL | https://arxiv.org/abs/1910.01492v1 |
| https://arxiv.org/pdf/1910.01492v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-grid-based-approach-for-convexity-analysis |
| Repo | |
| Framework | |
ASSD: Attentive Single Shot Multibox Detector
| Title | ASSD: Attentive Single Shot Multibox Detector |
| Authors | Jingru Yi, Pengxiang Wu, Dimitris N. Metaxas |
| Abstract | This paper proposes a new deep neural network for object detection. The proposed network, termed ASSD, builds feature relations in the spatial space of the feature map. With the global relation information, ASSD learns to highlight useful regions on the feature maps while suppressing the irrelevant information, thereby providing reliable guidance for object detection. Compared to methods that rely on complicated CNN layers to refine the feature maps, ASSD is simple in design and is computationally efficient. Experimental results show that ASSD competes favorably with the state-of-the-arts, including SSD, DSSD, FSSD and RetinaNet. |
| Tasks | Object Detection |
| Published | 2019-09-27 |
| URL | https://arxiv.org/abs/1909.12456v1 |
| https://arxiv.org/pdf/1909.12456v1.pdf | |
| PWC | https://paperswithcode.com/paper/assd-attentive-single-shot-multibox-detector |
| Repo | |
| Framework | |
Toward Runtime-Throttleable Neural Networks
| Title | Toward Runtime-Throttleable Neural Networks |
| Authors | Jesse Hostetler |
| Abstract | As deep neural network (NN) methods have matured, there has been increasing interest in deploying NN solutions to “edge computing” platforms such as mobile phones or embedded controllers. These platforms are often resource-constrained, especially in energy storage and power, but state-of-the-art NN architectures are designed with little regard for resource use. Existing techniques for reducing the resource footprint of NN models produce static models that occupy a single point in the trade-space between performance and resource use. This paper presents an approach to creating runtime-throttleable NNs that can adaptively balance performance and resource use in response to a control signal. Throttleable networks allow intelligent resource management, for example by allocating fewer resources in “easy” conditions or when battery power is low. We describe a generic formulation of throttling via block-level gating, apply it to create throttleable versions of several standard CNN architectures, and demonstrate that our approach allows smooth performance throttling over a wide range of operating points in image classification and object detection tasks, with only a small loss in peak accuracy. |
| Tasks | Image Classification, Object Detection |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.13179v1 |
| https://arxiv.org/pdf/1905.13179v1.pdf | |
| PWC | https://paperswithcode.com/paper/toward-runtime-throttleable-neural-networks |
| Repo | |
| Framework | |
Koopman Representations of Dynamic Systems with Control
| Title | Koopman Representations of Dynamic Systems with Control |
| Authors | Craig Bakker, Steven Rosenthal, Kathleen E. Nowak |
| Abstract | The design and analysis of optimal control policies for dynamical systems can be complicated by nonlinear dependence in the state variables. Koopman operators have been used to simplify the analysis of dynamical systems by mapping the flow of the system onto a space of observables where the dynamics are linear (and possibly infinte). This paper focuses on the development of consistent Koopman representations for controlled dynamical system. We introduce the concept of dynamical consistency for Koopman representations and analyze several existing and proposed representations deriving necessary constraints on the dynamical system, observables, and Koopman operators. Our main result is a hybrid formulation which independently and jointly observes the state and control inputs. This formulation admits a relatively large space of dynamical systems compared to earlier formulations while keeping the Koopman operator independent of the state and control inputs. More generally, this work provides an analysis framework to evaluate and rank proposed simplifications to the general Koopman representation for controlled dynamical systems. |
| Tasks | |
| Published | 2019-08-06 |
| URL | https://arxiv.org/abs/1908.02233v1 |
| https://arxiv.org/pdf/1908.02233v1.pdf | |
| PWC | https://paperswithcode.com/paper/koopman-representations-of-dynamic-systems |
| Repo | |
| Framework | |
Visual Evaluation of Generative Adversarial Networks for Time Series Data
| Title | Visual Evaluation of Generative Adversarial Networks for Time Series Data |
| Authors | Hiba Arnout, Johannes Kehrer, Johanna Bronner, Thomas Runkler |
| Abstract | A crucial factor to trust Machine Learning (ML) algorithm decisions is a good representation of its application field by the training dataset. This is particularly true when parts of the training data have been artificially generated to overcome common training problems such as lack of data or imbalanced dataset. Over the last few years, Generative Adversarial Networks (GANs) have shown remarkable results in generating realistic data. However, this ML approach lacks an objective function to evaluate the quality of the generated data. Numerous GAN applications focus on generating image data mostly because they can be easily evaluated by a human eye. Less efforts have been made to generate time series data. Assessing their quality is more complicated, particularly for technical data. In this paper, we propose a human-centered approach supporting a ML or domain expert to accomplish this task using Visual Analytics (VA) techniques. The presented approach consists of two views, namely a GAN Iteration View showing similarity metrics between real and generated data over the iterations of the generation process and a Detailed Comparative View equipped with different time series visualizations such as TimeHistograms, to compare the generated data at different iteration steps. Starting from the GAN Iteration View, the user can choose suitable iteration steps for detailed inspection. We evaluate our approach with a usage scenario that enabled an efficient comparison of two different GAN models. |
| Tasks | Time Series |
| Published | 2019-12-23 |
| URL | https://arxiv.org/abs/2001.00062v1 |
| https://arxiv.org/pdf/2001.00062v1.pdf | |
| PWC | https://paperswithcode.com/paper/visual-evaluation-of-generative-adversarial |
| Repo | |
| Framework | |
