Paper Group NANR 247
Statestream: A toolbox to explore layerwise-parallel deep neural networks. CodeSLAM â Learning a Compact, Optimisable Representation for Dense Visual SLAM. Random Decision Syntax Trees at SemEval-2018 Task 3: LSTMs and Sentiment Scores for Irony Detection. Adding the Third Dimension to Spatial Relation Detection in 2D Images. Stance-In-Depth Deep …
Statestream: A toolbox to explore layerwise-parallel deep neural networks
| Title | Statestream: A toolbox to explore layerwise-parallel deep neural networks |
| Authors | Volker Fischer |
| Abstract | Building deep neural networks to control autonomous agents which have to interact in real-time with the physical world, such as robots or automotive vehicles, requires a seamless integration of time into a network’s architecture. The central question of this work is, how the temporal nature of reality should be reflected in the execution of a deep neural network and its components. Most artificial deep neural networks are partitioned into a directed graph of connected modules or layers and the layers themselves consist of elemental building blocks, such as single units. For most deep neural networks, all units of a layer are processed synchronously and in parallel, but layers themselves are processed in a sequential manner. In contrast, all elements of a biological neural network are processed in parallel. In this paper, we define a class of networks between these two extreme cases. These networks are executed in a streaming or synchronous layerwise-parallel manner, unlocking the layers of such networks for parallel processing. Compared to the standard layerwise-sequential deep networks, these new layerwise-parallel networks show a fundamentally different temporal behavior and flow of information, especially for networks with skip or recurrent connections. We argue that layerwise-parallel deep networks are better suited for future challenges of deep neural network design, such as large functional modularized and/or recurrent architectures as well as networks allocating different network capacities dependent on current stimulus and/or task complexity. We layout basic properties and discuss major challenges for layerwise-parallel networks. Additionally, we provide a toolbox to design, train, evaluate, and online-interact with layerwise-parallel networks. |
| Tasks | |
| Published | 2018-01-01 |
| URL | https://openreview.net/forum?id=SkfNU2e0Z |
| https://openreview.net/pdf?id=SkfNU2e0Z | |
| PWC | https://paperswithcode.com/paper/statestream-a-toolbox-to-explore-layerwise |
| Repo | |
| Framework | |
CodeSLAM â Learning a Compact, Optimisable Representation for Dense Visual SLAM
| Title | CodeSLAM â Learning a Compact, Optimisable Representation for Dense Visual SLAM |
| Authors | Michael Bloesch, Jan Czarnowski, Ronald Clark, Stefan Leutenegger, Andrew J. Davison |
| Abstract | The representation of geometry in real-time 3D perception systems continues to be a critical research issue. Dense maps capture complete surface shape and can be augmented with semantic labels, but their high dimensionality makes them computationally costly to store and process, and unsuitable for rigorous probabilistic inference. Sparse feature-based representations avoid these problems, but capture only partial scene information and are mainly useful for localisation only. We present a new compact but dense representation of scene geometry which is conditioned on the intensity data from a single image and generated from a code consisting of a small number of parameters. We are inspired by work both on learned depth from images, and auto-encoders. Our approach is suitable for use in a keyframe-based monocular dense SLAM system: While each keyframe with a code can produce a depth map, the code can be optimised efficiently jointly with pose variables and together with the codes of overlapping keyframes to attain global consistency. Conditioning the depth map on the image allows the code to only represent aspects of the local geometry which cannot directly be predicted from the image. We explain how to learn our code representation, and demonstrate its advantageous properties in monocular SLAM. |
| Tasks | |
| Published | 2018-06-01 |
| URL | http://openaccess.thecvf.com/content_cvpr_2018/html/Bloesch_CodeSLAM_--_Learning_CVPR_2018_paper.html |
| http://openaccess.thecvf.com/content_cvpr_2018/papers/Bloesch_CodeSLAM_--_Learning_CVPR_2018_paper.pdf | |
| PWC | https://paperswithcode.com/paper/codeslam-a-learning-a-compact-optimisable |
| Repo | |
| Framework | |
Random Decision Syntax Trees at SemEval-2018 Task 3: LSTMs and Sentiment Scores for Irony Detection
| Title | Random Decision Syntax Trees at SemEval-2018 Task 3: LSTMs and Sentiment Scores for Irony Detection |
| Authors | Aidan San |
| Abstract | We propose a Long Short Term Memory Neural Network model for irony detection in tweets in this paper. Our model is trained using word embeddings and emoji embeddings. We show that adding sentiment scores to our model improves the F1 score of our baseline LSTM by approximately .012, and therefore show that high-level features can be used to improve word embeddings in certain Natural Language Processing applications. Our model ranks 24/43 for binary classification and 5/31 for multiclass classification. We make our model easily accessible to the research community. |
| Tasks | Chatbot, Sarcasm Detection, Sentiment Analysis, Word Embeddings |
| Published | 2018-06-01 |
| URL | https://www.aclweb.org/anthology/S18-1091/ |
| https://www.aclweb.org/anthology/S18-1091 | |
| PWC | https://paperswithcode.com/paper/random-decision-syntax-trees-at-semeval-2018 |
| Repo | |
| Framework | |
Adding the Third Dimension to Spatial Relation Detection in 2D Images
| Title | Adding the Third Dimension to Spatial Relation Detection in 2D Images |
| Authors | Br Birmingham, on, Adrian Muscat, Anja Belz |
| Abstract | Detection of spatial relations between objects in images is currently a popular subject in image description research. A range of different language and geometric object features have been used in this context, but methods have not so far used explicit information about the third dimension (depth), except when manually added to annotations. The lack of such information hampers detection of spatial relations that are inherently 3D. In this paper, we use a fully automatic method for creating a depth map of an image and derive several different object-level depth features from it which we add to an existing feature set to test the effect on spatial relation detection. We show that performance increases are obtained from adding depth features in all scenarios tested. |
| Tasks | Text Generation |
| Published | 2018-11-01 |
| URL | https://www.aclweb.org/anthology/W18-6517/ |
| https://www.aclweb.org/anthology/W18-6517 | |
| PWC | https://paperswithcode.com/paper/adding-the-third-dimension-to-spatial |
| Repo | |
| Framework | |
Stance-In-Depth Deep Neural Approach to Stance Classification
| Title | Stance-In-Depth Deep Neural Approach to Stance Classification |
| Authors | Gayathri Rajendran, Bhadrachalam Chitturi, Prabaharan Poornachandran |
| Abstract | Understanding the user intention from text is a problem of growing interest. The social media like Twitter, Facebook etc. extract user intention to analyze the behaviour of a user which in turn is employed for bot recognition, satire detection, fake news detection etc.. The process of identifying stance of a user from the text is called stance detection. This article compares the headline and body pair of a news article and classifies the pair as related or unrelated. The related pair is further classified into agree, disagree, discuss. We call related as detailed classification and unrelated as broad classification. We employ deep neural nets for feature extraction and stance classification. RNN models and its extensions showed significant variations in the classification of detailed class. Bidirectional LSTM model achieved the best accuracy for broad as well as detailed classification |
| Tasks | Fake News Detection, Stance Detection |
| Published | 2018-06-08 |
| URL | https://www.sciencedirect.com/science/article/pii/S1877050918308640 |
| https://ac.els-cdn.com/S1877050918308640/1-s2.0-S1877050918308640-main.pdf?_tid=556a1792-f871-4058-97de-e3cc8bc4ed63&acdnat=1551979346_642c1f818cbfdeb437938a5298dc364f | |
| PWC | https://paperswithcode.com/paper/stance-in-depth-deep-neural-approach-to |
| Repo | |
| Framework | |
Joint Learning of Sense and Word Embeddings
| Title | Joint Learning of Sense and Word Embeddings |
| Authors | Mohammed Alsuhaibani, Danushka Bollegala |
| Abstract | |
| Tasks | Dependency Parsing, Sentiment Analysis, Text Classification, Word Embeddings, Word Sense Disambiguation |
| Published | 2018-05-01 |
| URL | https://www.aclweb.org/anthology/L18-1033/ |
| https://www.aclweb.org/anthology/L18-1033 | |
| PWC | https://paperswithcode.com/paper/joint-learning-of-sense-and-word-embeddings |
| Repo | |
| Framework | |
Bayesian Adversarial Learning
| Title | Bayesian Adversarial Learning |
| Authors | Nanyang Ye, Zhanxing Zhu |
| Abstract | Deep neural networks have been known to be vulnerable to adversarial attacks, raising lots of security concerns in the practical deployment. Popular defensive approaches can be formulated as a (distributionally) robust optimization problem, which minimizes a ``point estimate’’ of worst-case loss derived from either per-datum perturbation or adversary data-generating distribution within certain pre-defined constraints. This point estimate ignores potential test adversaries that are beyond the pre-defined constraints. The model robustness might deteriorate sharply in the scenario of stronger test adversarial data. In this work, a novel robust training framework is proposed to alleviate this issue, Bayesian Robust Learning, in which a distribution is put on the adversarial data-generating distribution to account for the uncertainty of the adversarial data-generating process. The uncertainty directly helps to consider the potential adversaries that are stronger than the point estimate in the cases of distributionally robust optimization. The uncertainty of model parameters is also incorporated to accommodate the full Bayesian framework. We design a scalable Markov Chain Monte Carlo sampling strategy to obtain the posterior distribution over model parameters. Various experiments are conducted to verify the superiority of BAL over existing adversarial training methods. The code for BAL is available at \url{https://tinyurl.com/ycxsaewr }. | |
| Tasks | |
| Published | 2018-12-01 |
| URL | http://papers.nips.cc/paper/7921-bayesian-adversarial-learning |
| http://papers.nips.cc/paper/7921-bayesian-adversarial-learning.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-adversarial-learning |
| Repo | |
| Framework | |
stagNet: An Attentive Semantic RNN for Group Activity Recognition
| Title | stagNet: An Attentive Semantic RNN for Group Activity Recognition |
| Authors | Mengshi Qi, Jie Qin, Annan Li, Yunhong Wang, Jiebo Luo, Luc Van Gool |
| Abstract | Group activity recognition plays a fundamental role in a variety of applications, e.g. sports video analysis and intelligent surveillance. How to model the spatio-temporal contextual information in a scene still remains a crucial yet challenging issue. We propose a novel attentive semantic recurrent neural network (RNN), namely stagNet, for understanding group activities in videos, based on the spatio-temporal attention and semantic graph. A semantic graph is explicitly modeled to describe the spatial context of the whole scene, which is further integrated with the temporal factor via structural-RNN. Benefiting from the ‘factor sharing’ and ‘message passing’ mechanisms, our model is able to extract discriminative spatio-temporal features and to capture inter-group relationships. Moreover, we adopt a spatio-temporal attention model to attend to key persons/frames for improved performance. Two widely-used datasets are employed for performance evaluation, and the extensive results demonstrate the superiority of our method. |
| Tasks | Activity Recognition, Group Activity Recognition |
| Published | 2018-09-01 |
| URL | http://openaccess.thecvf.com/content_ECCV_2018/html/Mengshi_Qi_stagNet_An_Attentive_ECCV_2018_paper.html |
| http://openaccess.thecvf.com/content_ECCV_2018/papers/Mengshi_Qi_stagNet_An_Attentive_ECCV_2018_paper.pdf | |
| PWC | https://paperswithcode.com/paper/stagnet-an-attentive-semantic-rnn-for-group |
| Repo | |
| Framework | |
Convolutional Sequence Modeling Revisited
| Title | Convolutional Sequence Modeling Revisited |
| Authors | Shaojie Bai, J. Zico Kolter, Vladlen Koltun |
| Abstract | This paper revisits the problem of sequence modeling using convolutional architectures. Although both convolutional and recurrent architectures have a long history in sequence prediction, the current “default” mindset in much of the deep learning community is that generic sequence modeling is best handled using recurrent networks. The goal of this paper is to question this assumption. Specifically, we consider a simple generic temporal convolution network (TCN), which adopts features from modern ConvNet architectures such as a dilations and residual connections. We show that on a variety of sequence modeling tasks, including many frequently used as benchmarks for evaluating recurrent networks, the TCN outperforms baseline RNN methods (LSTMs, GRUs, and vanilla RNNs) and sometimes even highly specialized approaches. We further show that the potential “infinite memory” advantage that RNNs have over TCNs is largely absent in practice: TCNs indeed exhibit longer effective history sizes than their recurrent counterparts. As a whole, we argue that it may be time to (re)consider ConvNets as the default “go to” architecture for sequence modeling. |
| Tasks | Language Modelling, Time Series |
| Published | 2018-01-01 |
| URL | https://openreview.net/forum?id=rk8wKk-R- |
| https://openreview.net/pdf?id=rk8wKk-R- | |
| PWC | https://paperswithcode.com/paper/convolutional-sequence-modeling-revisited |
| Repo | |
| Framework | |
Language Models Learn POS First
| Title | Language Models Learn POS First |
| Authors | Naomi Saphra, Adam Lopez |
| Abstract | A glut of recent research shows that language models capture linguistic structure. Such work answers the question of whether a model represents linguistic structure. But how and when are these structures acquired? Rather than treating the training process itself as a black box, we investigate how representations of linguistic structure are learned over time. In particular, we demonstrate that different aspects of linguistic structure are learned at different rates, with part of speech tagging acquired early and global topic information learned continuously. |
| Tasks | Language Modelling, Part-Of-Speech Tagging |
| Published | 2018-11-01 |
| URL | https://www.aclweb.org/anthology/W18-5438/ |
| https://www.aclweb.org/anthology/W18-5438 | |
| PWC | https://paperswithcode.com/paper/language-models-learn-pos-first |
| Repo | |
| Framework | |
Chi-square Generative Adversarial Network
| Title | Chi-square Generative Adversarial Network |
| Authors | Chenyang Tao, Liqun Chen, Ricardo Henao, Jianfeng Feng, Lawrence Carin Duke |
| Abstract | To assess the difference between real and synthetic data, Generative Adversarial Networks (GANs) are trained using a distribution discrepancy measure. Three widely employed measures are information-theoretic divergences, integral probability metrics, and Hilbert space discrepancy metrics. We elucidate the theoretical connections between these three popular GAN training criteria and propose a novel procedure, called $\chi^2$ (Chi-square) GAN, that is conceptually simple, stable at training and resistant to mode collapse. Our procedure naturally generalizes to address the problem of simultaneous matching of multiple distributions. Further, we propose a resampling strategy that significantly improves sample quality, by repurposing the trained critic function via an importance weighting mechanism. Experiments show that the proposed procedure improves stability and convergence, and yields state-of-art results on a wide range of generative modeling tasks. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2369 |
| http://proceedings.mlr.press/v80/tao18b/tao18b.pdf | |
| PWC | https://paperswithcode.com/paper/chi-square-generative-adversarial-network |
| Repo | |
| Framework | |
Streaming Principal Component Analysis in Noisy Setting
| Title | Streaming Principal Component Analysis in Noisy Setting |
| Authors | Teodor Vanislavov Marinov, Poorya Mianjy, Raman Arora |
| Abstract | We study streaming algorithms for principal component analysis (PCA) in noisy settings. We present computationally efficient algorithms with sub-linear regret bounds for PCA in the presence of noise, missing data, and gross outliers. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2459 |
| http://proceedings.mlr.press/v80/marinov18a/marinov18a.pdf | |
| PWC | https://paperswithcode.com/paper/streaming-principal-component-analysis-in |
| Repo | |
| Framework | |
Partial Optimality and Fast Lower Bounds for Weighted Correlation Clustering
| Title | Partial Optimality and Fast Lower Bounds for Weighted Correlation Clustering |
| Authors | Jan-Hendrik Lange, Andreas Karrenbauer, Bjoern Andres |
| Abstract | Weighted correlation clustering is hard to solve and hard to approximate for general graphs. Its applications in network analysis and computer vision call for efficient algorithms. To this end, we make three contributions: We establish partial optimality conditions that can be checked efficiently, and doing so recursively solves the problem for series-parallel graphs to optimality, in linear time. We exploit the packing dual of the problem to compute a heuristic, but non-trivial lower bound faster than that of a canonical linear program relaxation. We introduce a re-weighting with the dual solution by which efficient local search algorithms converge to better feasible solutions. The effectiveness of our methods is demonstrated empirically on a number of benchmark instances. |
| Tasks | |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2206 |
| http://proceedings.mlr.press/v80/lange18a/lange18a.pdf | |
| PWC | https://paperswithcode.com/paper/partial-optimality-and-fast-lower-bounds-for |
| Repo | |
| Framework | |
Hierarchical Deep Generative Models for Multi-Rate Multivariate Time Series
| Title | Hierarchical Deep Generative Models for Multi-Rate Multivariate Time Series |
| Authors | Zhengping Che, Sanjay Purushotham, Guangyu Li, Bo Jiang, Yan Liu |
| Abstract | Multi-Rate Multivariate Time Series (MR-MTS) are the multivariate time series observations which come with various sampling rates and encode multiple temporal dependencies. State-space models such as Kalman filters and deep learning models such as deep Markov models are mainly designed for time series data with the same sampling rate and cannot capture all the dependencies present in the MR-MTS data. To address this challenge, we propose the Multi-Rate Hierarchical Deep Markov Model (MR-HDMM), a novel deep generative model which uses the latent hierarchical structure with a learnable switch mechanism to capture the temporal dependencies of MR-MTS. Experimental results on two real-world datasets demonstrate that our MR-HDMM model outperforms the existing state-of-the-art deep learning and state-space models on forecasting and interpolation tasks. In addition, the latent hierarchies in our model provide a way to show and interpret the multiple temporal dependencies. |
| Tasks | Time Series |
| Published | 2018-07-01 |
| URL | https://icml.cc/Conferences/2018/Schedule?showEvent=2108 |
| http://proceedings.mlr.press/v80/che18a/che18a.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-deep-generative-models-for-multi |
| Repo | |
| Framework | |
Deep Boosting of Diverse Experts
| Title | Deep Boosting of Diverse Experts |
| Authors | Wei Zhang, Qiuyu Chen, Jun Yu, Jianping Fan |
| Abstract | In this paper, a deep boosting algorithm is developed to learn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs (base experts) with diverse capabilities, e.g., these base deep CNNs are sequentially trained to recognize a set of object classes in an easy-to-hard way according to their learning complexities. Our experimental results have demonstrated that our deep boosting algorithm can significantly improve the accuracy rates on large-scale visual recognition. |
| Tasks | Object Recognition |
| Published | 2018-01-01 |
| URL | https://openreview.net/forum?id=B16_iGWCW |
| https://openreview.net/pdf?id=B16_iGWCW | |
| PWC | https://paperswithcode.com/paper/deep-boosting-of-diverse-experts |
| Repo | |
| Framework | |
