Paper Group NAWR 37
Deep Multi-State Dynamic Recurrent Neural Networks Operating on Wavelet Based Neural Features for Robust Brain Machine Interfaces. An Empirical Study on Pre-trained Embeddings and Language Models for Bot Detection. Annotating with Pros and Cons of Technologies in Computer Science Papers. Input-Output Equivalence of Unitary and Contractive RNNs. Sel …
Deep Multi-State Dynamic Recurrent Neural Networks Operating on Wavelet Based Neural Features for Robust Brain Machine Interfaces
| Title | Deep Multi-State Dynamic Recurrent Neural Networks Operating on Wavelet Based Neural Features for Robust Brain Machine Interfaces |
| Authors | Benyamin Allahgholizadeh Haghi, Spencer Kellis, Sahil Shah, Maitreyi Ashok, Luke Bashford, Daniel Kramer, Brian Lee, Charles Liu, Richard Andersen, Azita Emami |
| Abstract | We present a new deep multi-state Dynamic Recurrent Neural Network (DRNN) architecture for Brain Machine Interface (BMI) applications. Our DRNN is used to predict Cartesian representation of a computer cursor movement kinematics from open-loop neural data recorded from the posterior parietal cortex (PPC) of a human subject in a BMI system. We design the algorithm to achieve a reasonable trade-off between performance and robustness, and we constrain memory usage in favor of future hardware implementation. We feed the predictions of the network back to the input to improve prediction performance and robustness. We apply a scheduled sampling approach to the model in order to solve a statistical distribution mismatch between the ground truth and predictions. Additionally, we configure a small DRNN to operate with a short history of input, reducing the required buffering of input data and number of memory accesses. This configuration lowers the expected power consumption in a neural network accelerator. Operating on wavelet-based neural features, we show that the average performance of DRNN surpasses other state-of-the-art methods in the literature on both single- and multi-day data recorded over 43 days. Results show that multi-state DRNN has the potential to model the nonlinear relationships between the neural data and kinematics for robust BMIs. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9594-deep-multi-state-dynamic-recurrent-neural-networks-operating-on-wavelet-based-neural-features-for-robust-brain-machine-interfaces |
| http://papers.nips.cc/paper/9594-deep-multi-state-dynamic-recurrent-neural-networks-operating-on-wavelet-based-neural-features-for-robust-brain-machine-interfaces.pdf | |
| PWC | https://paperswithcode.com/paper/deep-multi-state-dynamic-recurrent-neural |
| Repo | https://github.com/BenyaminHaghi/DRNN-NeurIPS2019 |
| Framework | pytorch |
An Empirical Study on Pre-trained Embeddings and Language Models for Bot Detection
| Title | An Empirical Study on Pre-trained Embeddings and Language Models for Bot Detection |
| Authors | Andres Garcia-Silva, Cristian Berrio, Jos{'e} Manuel G{'o}mez-P{'e}rez |
| Abstract | Fine-tuning pre-trained language models has significantly advanced the state of art in a wide range of NLP downstream tasks. Usually, such language models are learned from large and well-formed text corpora from e.g. encyclopedic resources, books or news. However, a significant amount of the text to be analyzed nowadays is Web data, often from social media. In this paper we consider the research question: How do standard pre-trained language models generalize and capture the peculiarities of rather short, informal and frequently automatically generated text found in social media? To answer this question, we focus on bot detection in Twitter as our evaluation task and test the performance of fine-tuning approaches based on language models against popular neural architectures such as LSTM and CNN combined with pre-trained and contextualized embeddings. Our results also show strong performance variations among the different language model approaches, which suggest further research. |
| Tasks | Language Modelling |
| Published | 2019-08-01 |
| URL | https://www.aclweb.org/anthology/W19-4317/ |
| https://www.aclweb.org/anthology/W19-4317 | |
| PWC | https://paperswithcode.com/paper/an-empirical-study-on-pre-trained-embeddings |
| Repo | https://github.com/cberrioa/An-Empirical-study-on-Pre-trained-Embeddings-and-Language-Models-for-Bot-Detection |
| Framework | none |
Annotating with Pros and Cons of Technologies in Computer Science Papers
| Title | Annotating with Pros and Cons of Technologies in Computer Science Papers |
| Authors | Hono Shirai, Naoya Inoue, Jun Suzuki, Kentaro Inui |
| Abstract | This paper explores a task for extracting a technological expression and its pros/cons from computer science papers. We report ongoing efforts on an annotated corpus of pros/cons and an analysis of the nature of the automatic extraction task. Specifically, we show how to adapt the targeted sentiment analysis task for pros/cons extraction in computer science papers and conduct an annotation study. In order to identify the challenges of the automatic extraction task, we construct a strong baseline model and conduct an error analysis. The experiments show that pros/cons can be consistently annotated by several annotators, and that the task is challenging due to domain-specific knowledge. The annotated dataset is made publicly available for research purposes. |
| Tasks | Sentiment Analysis |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/W19-2605/ |
| https://www.aclweb.org/anthology/W19-2605 | |
| PWC | https://paperswithcode.com/paper/annotating-with-pros-and-cons-of-technologies |
| Repo | https://github.com/cl-tohoku/scientific-paper-pros-cons |
| Framework | none |
Input-Output Equivalence of Unitary and Contractive RNNs
| Title | Input-Output Equivalence of Unitary and Contractive RNNs |
| Authors | Melikasadat Emami, Mojtaba Sahraee Ardakan, Sundeep Rangan, Alyson K. Fletcher |
| Abstract | Unitary recurrent neural networks (URNNs) have been proposed as a method to overcome the vanishing and exploding gradient problem in modeling data with long-term dependencies. A basic question is how restrictive is the unitary constraint on the possible input-output mappings of such a network? This works shows that for any contractive RNN with ReLU activations, there is a URNN with at most twice the number of hidden states and the identical input-output mapping. Hence, with ReLU activations, URNNs are as expressive as general RNNs. In contrast, for certain smooth activations, it is shown that the input-output mapping of an RNN cannot be matched with a URNN, even with an arbitrary number of states. The theoretical results are supported by experiments on modeling of slowly-varying dynamical systems. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9671-input-output-equivalence-of-unitary-and-contractive-rnns |
| http://papers.nips.cc/paper/9671-input-output-equivalence-of-unitary-and-contractive-rnns.pdf | |
| PWC | https://paperswithcode.com/paper/input-output-equivalence-of-unitary-and-1 |
| Repo | https://github.com/melikaemami/URNN |
| Framework | tf |
Self-Guided Network for Fast Image Denoising
| Title | Self-Guided Network for Fast Image Denoising |
| Authors | Shuhang Gu, Yawei Li, Luc Van Gool, Radu Timofte |
| Abstract | During the past years, tremendous advances in image restoration tasks have been achieved using highly complex neural networks. Despite their good restoration performance, the heavy computational burden hinders the deployment of these networks on constrained devices, e.g. smart phones and consumer electronic products. To tackle this problem, we propose a self-guided network (SGN), which adopts a top-down self-guidance architecture to better exploit image multi-scale information. SGN directly generates multi-resolution inputs with the shuffling operation. Large-scale contextual information extracted at low resolution is gradually propagated into the higher resolution sub-networks to guide the feature extraction processes at these scales. Such a self-guidance strategy enables SGN to efficiently incorporate multi-scale information and extract good local features to recover noisy images. We validate the effectiveness of SGN through extensive experiments. The experimental results demonstrate that SGN greatly improves the memory and runtime efficiency over state-of-the-art efficient methods, without trading off PSNR accuracy. |
| Tasks | Denoising, Image Denoising, Image Restoration |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Gu_Self-Guided_Network_for_Fast_Image_Denoising_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Gu_Self-Guided_Network_for_Fast_Image_Denoising_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/self-guided-network-for-fast-image-denoising |
| Repo | https://github.com/ShuhangGu/SGN_ICCV2019 |
| Framework | pytorch |
Fast Image Restoration With Multi-Bin Trainable Linear Units
| Title | Fast Image Restoration With Multi-Bin Trainable Linear Units |
| Authors | Shuhang Gu, Wen Li, Luc Van Gool, Radu Timofte |
| Abstract | Tremendous advances in image restoration tasks such as denoising and super-resolution have been achieved using neural networks. Such approaches generally employ very deep architectures, large number of parameters, large receptive fields and high nonlinear modeling capacity. In order to obtain efficient and fast image restoration networks one should improve upon the above mentioned requirements. In this paper we propose a novel activation function, the multi-bin trainable linear unit (MTLU), for increasing the nonlinear modeling capacity together with lighter and shallower networks. We validate the proposed fast image restoration networks for image denoising (FDnet) and super-resolution (FSRnet) on standard benchmarks. We achieve large improvements in both memory and runtime over current state-of-the-art for comparable or better PSNR accuracies. |
| Tasks | Denoising, Image Denoising, Image Restoration, Super-Resolution |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Gu_Fast_Image_Restoration_With_Multi-Bin_Trainable_Linear_Units_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Gu_Fast_Image_Restoration_With_Multi-Bin_Trainable_Linear_Units_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/fast-image-restoration-with-multi-bin |
| Repo | https://github.com/ShuhangGu/MTLU_ICCV2019 |
| Framework | pytorch |
Solving Interpretable Kernel Dimensionality Reduction
| Title | Solving Interpretable Kernel Dimensionality Reduction |
| Authors | Chieh Wu, Jared Miller, Yale Chang, Mario Sznaier, Jennifer Dy |
| Abstract | Kernel dimensionality reduction (KDR) algorithms find a low dimensional representation of the original data by optimizing kernel dependency measures that are capable of capturing nonlinear relationships. The standard strategy is to first map the data into a high dimensional feature space using kernels prior to a projection onto a low dimensional space. While KDR methods can be easily solved by keeping the most dominant eigenvectors of the kernel matrix, its features are no longer easy to interpret. Alternatively, Interpretable KDR (IKDR) is different in that it projects onto a subspace \textit{before} the kernel feature mapping, therefore, the projection matrix can indicate how the original features linearly combine to form the new features. Unfortunately, the IKDR objective requires a non-convex manifold optimization that is difficult to solve and can no longer be solved by eigendecomposition. Recently, an efficient iterative spectral (eigendecomposition) method (ISM) has been proposed for this objective in the context of alternative clustering. However, ISM only provides theoretical guarantees for the Gaussian kernel. This greatly constrains ISM’s usage since any kernel method using ISM is now limited to a single kernel. This work extends the theoretical guarantees of ISM to an entire family of kernels, thereby empowering ISM to solve any kernel method of the same objective. In identifying this family, we prove that each kernel within the family has a surrogate $\Phi$ matrix and the optimal projection is formed by its most dominant eigenvectors. With this extension, we establish how a wide range of IKDR applications across different learning paradigms can be solved by ISM. To support reproducible results, the source code is made publicly available on \url{https://github.com/ANONYMIZED}. |
| Tasks | Dimensionality Reduction |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9005-solving-interpretable-kernel-dimensionality-reduction |
| http://papers.nips.cc/paper/9005-solving-interpretable-kernel-dimensionality-reduction.pdf | |
| PWC | https://paperswithcode.com/paper/solving-interpretable-kernel-dimensionality |
| Repo | https://github.com/chieh-neu/ISM_supervised_DR |
| Framework | none |
Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization
| Title | Fine-Grained Segmentation Networks: Self-Supervised Segmentation for Improved Long-Term Visual Localization |
| Authors | Mans Larsson, Erik Stenborg, Carl Toft, Lars Hammarstrand, Torsten Sattler, Fredrik Kahl |
| Abstract | Long-term visual localization is the problem of estimating the camera pose of a given query image in a scene whose appearance changes over time. It is an important problem in practice that is, for example, encountered in autonomous driving. In order to gain robustness to such changes, long-term localization approaches often use segmantic segmentations as an invariant scene representation, as the semantic meaning of each scene part should not be affected by seasonal and other changes. However, these representations are typically not very discriminative due to the very limited number of available classes. In this paper, we propose a novel neural network, the Fine-Grained Segmentation Network (FGSN), that can be used to provide image segmentations with a larger number of labels and can be trained in a self-supervised fashion. In addition, we show how FGSNs can be trained to output consistent labels across seasonal changes. We show through extensive experiments that integrating the fine-grained segmentations produced by our FGSNs into existing localization algorithms leads to substantial improvements in localization performance. |
| Tasks | Autonomous Driving, Visual Localization |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Larsson_Fine-Grained_Segmentation_Networks_Self-Supervised_Segmentation_for_Improved_Long-Term_Visual_Localization_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Larsson_Fine-Grained_Segmentation_Networks_Self-Supervised_Segmentation_for_Improved_Long-Term_Visual_Localization_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/fine-grained-segmentation-networks-self-1 |
| Repo | https://github.com/maunzzz/fine-grained-segmentation-networks |
| Framework | pytorch |
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
| Title | Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders |
| Authors | Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, Andrew McCallum |
| Abstract | We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predicts each word in an input sentence conditioned on the rest of the sentence. During training we use dynamic programming to consider all possible binary trees over the sentence, and for inference we use the CKY algorithm to extract the highest scoring parse. DIORA outperforms previously reported results for unsupervised binary constituency parsing on the benchmark WSJ dataset. |
| Tasks | Constituency Grammar Induction, Constituency Parsing |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/N19-1116/ |
| https://www.aclweb.org/anthology/N19-1116 | |
| PWC | https://paperswithcode.com/paper/unsupervised-latent-tree-induction-with-deep-1 |
| Repo | https://github.com/iesl/diora |
| Framework | pytorch |
Fully Convolutional Geometric Features
| Title | Fully Convolutional Geometric Features |
| Authors | Christopher Choy, Jaesik Park, Vladlen Koltun |
| Abstract | Extracting geometric features from 3D scans or point clouds is the first step in applications such as registration, reconstruction, and tracking. State-of-the-art methods require computing low-level features as input or extracting patch-based features with limited receptive field. In this work, we present fully-convolutional geometric features, computed in a single pass by a 3D fully-convolutional network. We also present new metric learning losses that dramatically improve performance. Fully-convolutional geometric features are compact, capture broad spatial context, and scale to large scenes. We experimentally validate our approach on both indoor and outdoor datasets. Fully-convolutional geometric features achieve state-of-the-art accuracy without requiring prepossessing, are compact (32 dimensions), and are 600 times faster than the most accurate prior method. |
| Tasks | 3D Feature Matching, 3D Point Cloud Matching, 3D Shape Representation, Metric Learning |
| Published | 2019-10-27 |
| URL | https://github.com/chrischoy/FCGF |
| https://node1.chrischoy.org/data/publications/fcgf/fcgf.pdf | |
| PWC | https://paperswithcode.com/paper/fully-convolutional-geometric-features |
| Repo | https://github.com/chrischoy/FCGF |
| Framework | pytorch |
Mutually Regressive Point Processes
| Title | Mutually Regressive Point Processes |
| Authors | Ifigeneia Apostolopoulou, Scott Linderman, Kyle Miller, Artur Dubrawski |
| Abstract | Many real-world data represent sequences of interdependent events unfolding over time. They can be modeled naturally as realizations of a point process. Despite many potential applications, existing point process models are limited in their ability to capture complex patterns of interaction. Hawkes processes admit many efficient inference algorithms, but are limited to mutually excitatory effects. Non- linear Hawkes processes allow for more complex influence patterns, but for their estimation it is typically necessary to resort to discrete-time approximations that may yield poor generative models. In this paper, we introduce the first general class of Bayesian point process models extended with a nonlinear component that allows both excitatory and inhibitory relationships in continuous time. We derive a fully Bayesian inference algorithm for these processes using Polya-Gamma augmentation and Poisson thinning. We evaluate the proposed model on single and multi-neuronal spike train recordings. Results demonstrate that the proposed model, unlike existing point process models, can generate biologically-plausible spike trains, while still achieving competitive predictive likelihoods. |
| Tasks | Bayesian Inference, Point Processes |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8755-mutually-regressive-point-processes |
| http://papers.nips.cc/paper/8755-mutually-regressive-point-processes.pdf | |
| PWC | https://paperswithcode.com/paper/mutually-regressive-point-processes |
| Repo | https://github.com/ifiaposto/Mutually-Regressive-Point-Processes |
| Framework | none |
MinScIE: Citation-centered Open Information Extraction
| Title | MinScIE: Citation-centered Open Information Extraction |
| Authors | Anne Lauscher, Yide Song, Kiril Gashteovski |
| Abstract | Acknowledging the importance of citations in scientific literature, in this work we present MinScIE, an Open Information Extraction system which provides structured knowledge enriched with semantic information about citations. By comparing our system to it’s original core, MinIE, we show that our approach improves extraction precision by 3 percentage points. |
| Tasks | Open Information Extraction |
| Published | 2019-06-01 |
| URL | https://ub-madoc.bib.uni-mannheim.de/49216/1/_JCDL19Demo__MinScIE%20%284%29.pdf |
| https://ub-madoc.bib.uni-mannheim.de/49216/1/_JCDL19Demo__MinScIE%20%284%29.pdf | |
| PWC | https://paperswithcode.com/paper/minscie-citation-centered-open-information |
| Repo | https://github.com/gkiril/MinSCIE |
| Framework | none |
Modeling Dynamic Functional Connectivity with Latent Factor Gaussian Processes
| Title | Modeling Dynamic Functional Connectivity with Latent Factor Gaussian Processes |
| Authors | Lingge Li, Dustin Pluta, Babak Shahbaba, Norbert Fortin, Hernando Ombao, Pierre Baldi |
| Abstract | Dynamic functional connectivity, as measured by the time-varying covariance of neurological signals, is believed to play an important role in many aspects of cognition. While many methods have been proposed, reliably establishing the presence and characteristics of brain connectivity is challenging due to the high dimensionality and noisiness of neuroimaging data. We present a latent factor Gaussian process model which addresses these challenges by learning a parsimonious representation of connectivity dynamics. The proposed model naturally allows for inference and visualization of the time-varying connectivity. As an illustration of the scientific utility of the model, application to a data set of rat local field potential activity recorded during a complex non-spatial memory task provides evidence of stimuli differentiation. |
| Tasks | Gaussian Processes |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9036-modeling-dynamic-functional-connectivity-with-latent-factor-gaussian-processes |
| http://papers.nips.cc/paper/9036-modeling-dynamic-functional-connectivity-with-latent-factor-gaussian-processes.pdf | |
| PWC | https://paperswithcode.com/paper/modeling-dynamic-functional-connectivity-with |
| Repo | https://github.com/modestbayes/LFGP_NeurIPS |
| Framework | none |
DTWNet: a Dynamic Time Warping Network
| Title | DTWNet: a Dynamic Time Warping Network |
| Authors | Xingyu Cai, Tingyang Xu, Jinfeng Yi, Junzhou Huang, Sanguthevar Rajasekaran |
| Abstract | Dynamic Time Warping (DTW) is widely used as a similarity measure in various domains. Due to its invariance against warping in the time axis, DTW provides more meaningful discrepancy measurements between two signals than other dis- tance measures. In this paper, we propose a novel component in an artificial neural network. In contrast to the previous successful usage of DTW as a loss function, the proposed framework leverages DTW to obtain a better feature extraction. For the first time, the DTW loss is theoretically analyzed, and a stochastic backpropogation scheme is proposed to improve the accuracy and efficiency of the DTW learning. We also demonstrate that the proposed framework can be used as a data analysis tool to perform data decomposition. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9338-dtwnet-a-dynamic-time-warping-network |
| http://papers.nips.cc/paper/9338-dtwnet-a-dynamic-time-warping-network.pdf | |
| PWC | https://paperswithcode.com/paper/dtwnet-a-dynamic-time-warping-network |
| Repo | https://github.com/TideDancer/DTWNet |
| Framework | pytorch |
Knowledge Extraction with No Observable Data
| Title | Knowledge Extraction with No Observable Data |
| Authors | Jaemin Yoo, Minyong Cho, Taebum Kim, U Kang |
| Abstract | Knowledge distillation is to transfer the knowledge of a large neural network into a smaller one and has been shown to be effective especially when the amount of training data is limited or the size of the student model is very small. To transfer the knowledge, it is essential to observe the data that have been used to train the network since its knowledge is concentrated on a narrow manifold rather than the whole input space. However, the data are not accessible in many cases due to the privacy or confidentiality issues in medical, industrial, and military domains. To the best of our knowledge, there has been no approach that distills the knowledge of a neural network when no data are observable. In this work, we propose KegNet (Knowledge Extraction with Generative Networks), a novel approach to extract the knowledge of a trained deep neural network and to generate artificial data points that replace the missing training data in knowledge distillation. Experiments show that KegNet outperforms all baselines for data-free knowledge distillation. We provide the source code of our paper in https://github.com/snudatalab/KegNet. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8538-knowledge-extraction-with-no-observable-data |
| http://papers.nips.cc/paper/8538-knowledge-extraction-with-no-observable-data.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-extraction-with-no-observable-data |
| Repo | https://github.com/snudatalab/KegNet |
| Framework | pytorch |
