Paper Group ANR 254
Asymptotics of Wide Networks from Feynman Diagrams. Transparency in Maintenance of Recruitment Chatbots. Variational Integrator Networks for Physically Structured Embeddings. Multi-label Classification for Automatic Tag Prediction in the Context of Programming Challenges. Inserting Videos into Videos. Probabilistic Neural Architecture Search. A Fam …
Asymptotics of Wide Networks from Feynman Diagrams
| Title | Asymptotics of Wide Networks from Feynman Diagrams |
| Authors | Ethan Dyer, Guy Gur-Ari |
| Abstract | Understanding the asymptotic behavior of wide networks is of considerable interest. In this work, we present a general method for analyzing this large width behavior. The method is an adaptation of Feynman diagrams, a standard tool for computing multivariate Gaussian integrals. We apply our method to study training dynamics, improving existing bounds and deriving new results on wide network evolution during stochastic gradient descent. Going beyond the strict large width limit, we present closed-form expressions for higher-order terms governing wide network training, and test these predictions empirically. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11304v1 |
| https://arxiv.org/pdf/1909.11304v1.pdf | |
| PWC | https://paperswithcode.com/paper/asymptotics-of-wide-networks-from-feynman |
| Repo | |
| Framework | |
Transparency in Maintenance of Recruitment Chatbots
| Title | Transparency in Maintenance of Recruitment Chatbots |
| Authors | Kit Kuksenok, Nina Praß |
| Abstract | We report on experiences with implementing conversational agents in the recruitment domain based on a machine learning (ML) system. Recruitment chatbots mediate communication between job-seekers and recruiters by exposing ML data to recruiter teams. Errors are difficult to understand, communicate, and resolve because they may span and combine UX, ML, and software issues. In an effort to improve organizational and technical transparency, we came to rely on a key contact role. Though effective for design and development, the centralization of this role poses challenges for transparency in sustained maintenance of this kind of ML-based mediating system. |
| Tasks | |
| Published | 2019-05-09 |
| URL | https://arxiv.org/abs/1905.03640v1 |
| https://arxiv.org/pdf/1905.03640v1.pdf | |
| PWC | https://paperswithcode.com/paper/transparency-in-maintenance-of-recruitment |
| Repo | |
| Framework | |
Variational Integrator Networks for Physically Structured Embeddings
| Title | Variational Integrator Networks for Physically Structured Embeddings |
| Authors | Steindor Saemundsson, Alexander Terenin, Katja Hofmann, Marc Peter Deisenroth |
| Abstract | Learning workable representations of dynamical systems is becoming an increasingly important problem in a number of application areas. By leveraging recent work connecting deep neural networks to systems of differential equations, we propose \emph{variational integrator networks}, a class of neural network architectures designed to preserve the geometric structure of physical systems. This class of network architectures facilitates accurate long-term prediction, interpretability, and data-efficient learning, while still remaining highly flexible and capable of modeling complex behavior. We demonstrate that they can accurately learn dynamical systems from both noisy observations in phase space and from image pixels within which the unknown dynamics are embedded. |
| Tasks | |
| Published | 2019-10-21 |
| URL | https://arxiv.org/abs/1910.09349v2 |
| https://arxiv.org/pdf/1910.09349v2.pdf | |
| PWC | https://paperswithcode.com/paper/variational-integrator-networks-for |
| Repo | |
| Framework | |
Multi-label Classification for Automatic Tag Prediction in the Context of Programming Challenges
| Title | Multi-label Classification for Automatic Tag Prediction in the Context of Programming Challenges |
| Authors | Bianca Iancu, Gabriele Mazzola, Kyriakos Psarakis, Panagiotis Soilis |
| Abstract | One of the best ways for developers to test and improve their skills in a fun and challenging way are programming challenges, offered by a plethora of websites. For the inexperienced ones, some of the problems might appear too challenging, requiring some suggestions to implement a solution. On the other hand, tagging problems can be a tedious task for problem creators. In this paper, we focus on automating the task of tagging a programming challenge description using machine and deep learning methods. We observe that the deep learning methods implemented outperform well-known IR approaches such as tf-idf, thus providing a starting point for further research on the task. |
| Tasks | Multi-Label Classification |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.12224v1 |
| https://arxiv.org/pdf/1911.12224v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-label-classification-for-automatic-tag |
| Repo | |
| Framework | |
Inserting Videos into Videos
| Title | Inserting Videos into Videos |
| Authors | Donghoon Lee, Tomas Pfister, Ming-Hsuan Yang |
| Abstract | In this paper, we introduce a new problem of manipulating a given video by inserting other videos into it. Our main task is, given an object video and a scene video, to insert the object video at a user-specified location in the scene video so that the resulting video looks realistic. We aim to handle different object motions and complex backgrounds without expensive segmentation annotations. As it is difficult to collect training pairs for this problem, we synthesize fake training pairs that can provide helpful supervisory signals when training a neural network with unpaired real data. The proposed network architecture can take both real and fake pairs as input and perform both supervised and unsupervised training in an adversarial learning scheme. To synthesize a realistic video, the network renders each frame based on the current input and previous frames. Within this framework, we observe that injecting noise into previous frames while generating the current frame stabilizes training. We conduct experiments on real-world videos in object tracking and person re-identification benchmark datasets. Experimental results demonstrate that the proposed algorithm is able to synthesize long sequences of realistic videos with a given object video inserted. |
| Tasks | Object Tracking, Person Re-Identification |
| Published | 2019-03-15 |
| URL | http://arxiv.org/abs/1903.06571v1 |
| http://arxiv.org/pdf/1903.06571v1.pdf | |
| PWC | https://paperswithcode.com/paper/inserting-videos-into-videos |
| Repo | |
| Framework | |
Probabilistic Neural Architecture Search
| Title | Probabilistic Neural Architecture Search |
| Authors | Francesco Paolo Casale, Jonathan Gordon, Nicolo Fusi |
| Abstract | In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complexity or high memory usage. In this work, we propose a Probabilistic approach to neural ARchitecture SEarCh (PARSEC) that drastically reduces memory requirements while maintaining state-of-the-art computational complexity, making it possible to directly search over more complex architectures and larger datasets. Our approach only requires as much memory as is needed to train a single architecture from our search space. This is due to a memory-efficient sampling procedure wherein we learn a probability distribution over high-performing neural network architectures. Importantly, this framework enables us to transfer the distribution of architectures learnt on smaller problems to larger ones, further reducing the computational cost. We showcase the advantages of our approach in applications to CIFAR-10 and ImageNet, where our approach outperforms methods with double its computational cost and matches the performance of methods with costs that are three orders of magnitude larger. |
| Tasks | Neural Architecture Search |
| Published | 2019-02-13 |
| URL | http://arxiv.org/abs/1902.05116v1 |
| http://arxiv.org/pdf/1902.05116v1.pdf | |
| PWC | https://paperswithcode.com/paper/probabilistic-neural-architecture-search |
| Repo | |
| Framework | |
A Family of Exact Goodness-of-Fit Tests for High-Dimensional Discrete Distributions
| Title | A Family of Exact Goodness-of-Fit Tests for High-Dimensional Discrete Distributions |
| Authors | Feras A. Saad, Cameron E. Freer, Nathanael L. Ackerman, Vikash K. Mansinghka |
| Abstract | The objective of goodness-of-fit testing is to assess whether a dataset of observations is likely to have been drawn from a candidate probability distribution. This paper presents a rank-based family of goodness-of-fit tests that is specialized to discrete distributions on high-dimensional domains. The test is readily implemented using a simulation-based, linear-time procedure. The testing procedure can be customized by the practitioner using knowledge of the underlying data domain. Unlike most existing test statistics, the proposed test statistic is distribution-free and its exact (non-asymptotic) sampling distribution is known in closed form. We establish consistency of the test against all alternatives by showing that the test statistic is distributed as a discrete uniform if and only if the samples were drawn from the candidate distribution. We illustrate its efficacy for assessing the sample quality of approximate sampling algorithms over combinatorially large spaces with intractable probabilities, including random partitions in Dirichlet process mixture models and random lattices in Ising models. |
| Tasks | |
| Published | 2019-02-26 |
| URL | http://arxiv.org/abs/1902.10142v1 |
| http://arxiv.org/pdf/1902.10142v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-family-of-exact-goodness-of-fit-tests-for |
| Repo | |
| Framework | |
HyperStream: a Workflow Engine for Streaming Data
| Title | HyperStream: a Workflow Engine for Streaming Data |
| Authors | Tom Diethe, Meelis Kull, Niall Twomey, Kacper Sokol, Hao Song, Miquel Perello-Nieto, Emma Tonkin, Peter Flach |
| Abstract | This paper describes HyperStream, a large-scale, flexible and robust software package, written in the Python language, for processing streaming data with workflow creation capabilities. HyperStream overcomes the limitations of other computational engines and provides high-level interfaces to execute complex nesting, fusion, and prediction both in online and offline forms in streaming environments. HyperStream is a general purpose tool that is well-suited for the design, development, and deployment of Machine Learning algorithms and predictive models in a wide space of sequential predictive problems. Source code, installation instructions, examples, and documentation can be found at: https://github.com/IRC-SPHERE/HyperStream. |
| Tasks | |
| Published | 2019-08-07 |
| URL | https://arxiv.org/abs/1908.02858v1 |
| https://arxiv.org/pdf/1908.02858v1.pdf | |
| PWC | https://paperswithcode.com/paper/hyperstream-a-workflow-engine-for-streaming |
| Repo | |
| Framework | |
Orderless Recurrent Models for Multi-label Classification
| Title | Orderless Recurrent Models for Multi-label Classification |
| Authors | Vacit Oguz Yazici, Abel Gonzalez-Garcia, Arnau Ramisa, Bartlomiej Twardowski, Joost van de Weijer |
| Abstract | Recurrent neural networks (RNN) are popular for many computer vision tasks, including multi-label classification. Since RNNs produce sequential outputs, labels need to be ordered for the multi-label classification task. Current approaches sort labels according to their frequency, typically ordering them in either rare-first or frequent-first. These imposed orderings do not take into account that the natural order to generate the labels can change for each image, e.g.\ first the dominant object before summing up the smaller objects in the image. Therefore, in this paper, we propose ways to dynamically order the ground truth labels with the predicted label sequence. This allows for the faster training of more optimal LSTM models for multi-label classification. Analysis evidences that our method does not suffer from duplicate generation, something which is common for other models. Furthermore, it outperforms other CNN-RNN models, and we show that a standard architecture of an image encoder and language decoder trained with our proposed loss obtains the state-of-the-art results on the challenging MS-COCO, WIDER Attribute and PA-100K and competitive results on NUS-WIDE. |
| Tasks | Multi-Label Classification |
| Published | 2019-11-22 |
| URL | https://arxiv.org/abs/1911.09996v3 |
| https://arxiv.org/pdf/1911.09996v3.pdf | |
| PWC | https://paperswithcode.com/paper/orderless-recurrent-models-for-multi-label |
| Repo | |
| Framework | |
Learning Multi-Party Turn-Taking Models from Dialogue Logs
| Title | Learning Multi-Party Turn-Taking Models from Dialogue Logs |
| Authors | Maira Gatti de Bayser, Paulo Cavalin, Claudio Pinhanez, Bianca Zadrozny |
| Abstract | This paper investigates the application of machine learning (ML) techniques to enable intelligent systems to learn multi-party turn-taking models from dialogue logs. The specific ML task consists of determining who speaks next, after each utterance of a dialogue, given who has spoken and what was said in the previous utterances. With this goal, this paper presents comparisons of the accuracy of different ML techniques such as Maximum Likelihood Estimation (MLE), Support Vector Machines (SVM), and Convolutional Neural Networks (CNN) architectures, with and without utterance data. We present three corpora: the first with dialogues from an American TV situated comedy (chit-chat), the second with logs from a financial advice multi-bot system and the third with a corpus created from the Multi-Domain Wizard-of-Oz dataset (both are topic-oriented). The results show: (i) the size of the corpus has a very positive impact on the accuracy for the content-based deep learning approaches and those models perform best in the larger datasets; and (ii) if the dialogue dataset is small and topic-oriented (but with few topics), it is sufficient to use an agent-only MLE or SVM models, although slightly higher accuracies can be achieved with the use of the content of the utterances with a CNN model. |
| Tasks | |
| Published | 2019-07-03 |
| URL | https://arxiv.org/abs/1907.02090v1 |
| https://arxiv.org/pdf/1907.02090v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-multi-party-turn-taking-models-from |
| Repo | |
| Framework | |
Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery
| Title | Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery |
| Authors | Samuel Kim, Peter Lu, Srijon Mukherjee, Michael Gilbert, Li Jing, Vladimir Ceperic, Marin Soljacic |
| Abstract | Symbolic regression is a powerful technique that can discover analytical equations that describe data, which can lead to explainable models and generalizability outside of the training data set. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing tasks, but are often seen as black-box models that are difficult to interpret and typically extrapolate poorly. Here we use a neural network-based architecture for symbolic regression that we call the Sequential Equation Learner (SEQL) network and integrate it with other deep learning architectures such that the whole system can be trained end-to-end through backpropagation. To demonstrate the power of such systems, we study their performance on several substantially different tasks. First, we show that the neural network can perform symbolic regression and learn the form of several functions. Next, we present an MNIST arithmetic task where a separate part of the neural network extracts the digits. Finally, we demonstrate prediction of dynamical systems where an unknown parameter is extracted through an encoder. We find that the EQL-based architecture can extrapolate quite well outside of the training data set compared to a standard neural network-based architecture, paving the way for deep learning to be applied in scientific exploration and discovery. |
| Tasks | |
| Published | 2019-12-10 |
| URL | https://arxiv.org/abs/1912.04825v1 |
| https://arxiv.org/pdf/1912.04825v1.pdf | |
| PWC | https://paperswithcode.com/paper/integration-of-neural-network-based-symbolic |
| Repo | |
| Framework | |
Physics-informed semantic inpainting: Application to geostatistical modeling
| Title | Physics-informed semantic inpainting: Application to geostatistical modeling |
| Authors | Qiang Zheng, Lingzao Zeng, Zhendan Cao, George Em Karniadakis |
| Abstract | A fundamental problem in geostatistical modeling is to infer the heterogeneous geological field based on limited measurements and some prior spatial statistics. Semantic inpainting, a technique for image processing using deep generative models, has been recently applied for this purpose, demonstrating its effectiveness in dealing with complex spatial patterns. However, the original semantic inpainting framework incorporates only information from direct measurements, while in geostatistics indirect measurements are often plentiful. To overcome this limitation, here we propose a physics-informed semantic inpainting framework, employing the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) and jointly incorporating the direct and indirect measurements by exploiting the underlying physical laws. Our simulation results for a high-dimensional problem with 512 dimensions show that in the new method, the physical conservation laws are satisfied and contribute in enhancing the inpainting performance compared to using only the direct measurements. |
| Tasks | |
| Published | 2019-09-19 |
| URL | https://arxiv.org/abs/1909.09459v2 |
| https://arxiv.org/pdf/1909.09459v2.pdf | |
| PWC | https://paperswithcode.com/paper/physics-informed-semantic-inpainting |
| Repo | |
| Framework | |
Weakly Supervised Tracklet Person Re-Identification by Deep Feature-wise Mutual Learning
| Title | Weakly Supervised Tracklet Person Re-Identification by Deep Feature-wise Mutual Learning |
| Authors | Zhirui Chen, Jianheng Li, Wei-Shi Zheng |
| Abstract | The scalability problem caused by the difficulty in annotating Person Re-identification(Re-ID) datasets has become a crucial bottleneck in the development of Re-ID.To address this problem, many unsupervised Re-ID methods have recently been proposed.Nevertheless, most of these models require transfer from another auxiliary fully supervised dataset, which is still expensive to obtain.In this work, we propose a Re-ID model based on Weakly Supervised Tracklets(WST) data from various camera views, which can be inexpensively acquired by combining the fragmented tracklets of the same person in the same camera view over a period of time.We formulate our weakly supervised tracklets Re-ID model by a novel method, named deep feature-wise mutual learning(DFML), which consists of Mutual Learning on Feature Extractors (MLFE) and Mutual Learning on Feature Classifiers (MLFC).We propose MLFE by leveraging two feature extractors to learn from each other to extract more robust and discriminative features.On the other hand, we propose MLFC by adapting discriminative features from various camera views to each classifier. Extensive experiments demonstrate the superiority of our proposed DFML over the state-of-the-art unsupervised models and even some supervised models on three Re-ID benchmark datasets. |
| Tasks | Person Re-Identification |
| Published | 2019-10-31 |
| URL | https://arxiv.org/abs/1910.14333v1 |
| https://arxiv.org/pdf/1910.14333v1.pdf | |
| PWC | https://paperswithcode.com/paper/weakly-supervised-tracklet-person-re |
| Repo | |
| Framework | |
Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties
| Title | Classified Regression for Bayesian Optimization: Robot Learning with Unknown Penalties |
| Authors | Alonso Marco, Dominik Baumann, Philipp Hennig, Sebastian Trimpe |
| Abstract | Learning robot controllers by minimizing a black-box objective cost using Bayesian optimization (BO) can be time-consuming and challenging. It is very often the case that some roll-outs result in failure behaviors, causing premature experiment detention. In such cases, the designer is forced to decide on heuristic cost penalties because the acquired data is often scarce, or not comparable with that of the stable policies. To overcome this, we propose a Bayesian model that captures exactly what we know about the cost of unstable controllers prior to data collection: Nothing, except that it should be a somewhat large number. The resulting Bayesian model, approximated with a Gaussian process, predicts high cost values in regions where failures are likely to occur. In this way, the model guides the BO exploration toward regions of stability. We demonstrate the benefits of the proposed model in several illustrative and statistical synthetic benchmarks, and also in experiments on a real robotic platform. In addition, we propose and experimentally validate a new BO method to account for unknown constraints. Such method is an extension of Max-Value Entropy Search, a recent information-theoretic method, to solve unconstrained global optimization problems. |
| Tasks | |
| Published | 2019-07-24 |
| URL | https://arxiv.org/abs/1907.10383v1 |
| https://arxiv.org/pdf/1907.10383v1.pdf | |
| PWC | https://paperswithcode.com/paper/classified-regression-for-bayesian |
| Repo | |
| Framework | |
Learning Disentangled Representation for Robust Person Re-identification
| Title | Learning Disentangled Representation for Robust Person Re-identification |
| Authors | Chanho Eom, Bumsub Ham |
| Abstract | We address the problem of person re-identification (reID), that is, retrieving person images from a large dataset, given a query image of the person of interest. A key challenge is to learn person representations robust to intra-class variations, as different persons can have the same attribute and the same person’s appearance looks different with viewpoint changes. Recent reID methods focus on learning discriminative features but robust to only a particular factor of variations (e.g., human pose), which requires corresponding supervisory signals (e.g., pose annotations). To tackle this problem, we propose to disentangle identity-related and -unrelated features from person images. Identity-related features contain information useful for specifying a particular person (e.g., clothing), while identity-unrelated ones hold other factors (e.g., human pose, scale changes). To this end, we introduce a new generative adversarial network, dubbed \emph{identity shuffle GAN} (IS-GAN), that factorizes these features using identification labels without any auxiliary information. We also propose an identity-shuffling technique to regularize the disentangled features. Experimental results demonstrate the effectiveness of IS-GAN, significantly outperforming the state of the art on standard reID benchmarks including the Market-1501, CUHK03 and DukeMTMC-reID. Our code and models are available online: https://cvlab-yonsei.github.io/projects/ISGAN/. |
| Tasks | Person Re-Identification |
| Published | 2019-10-26 |
| URL | https://arxiv.org/abs/1910.12003v2 |
| https://arxiv.org/pdf/1910.12003v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-disentangled-representation-for |
| Repo | |
| Framework | |
