October 20, 2019

3050 words 15 mins read

Paper Group AWR 278

LAPRAN: A Scalable Laplacian Pyramid Reconstructive Adversarial Network for Flexible Compressive Sensing Reconstruction. Learning sentence embeddings using Recursive Networks. Aggregated Momentum: Stability Through Passive Damping. PU-Net: Point Cloud Upsampling Network. Skeleton-based Gesture Recognition Using Several Fully Connected Layers with P …

LAPRAN: A Scalable Laplacian Pyramid Reconstructive Adversarial Network for Flexible Compressive Sensing Reconstruction


Title	LAPRAN: A Scalable Laplacian Pyramid Reconstructive Adversarial Network for Flexible Compressive Sensing Reconstruction
Authors	Kai Xu, Zhikang Zhang, Fengbo Ren
Abstract	This paper addresses the single-image compressive sensing (CS) and reconstruction problem. We propose a scalable Laplacian pyramid reconstructive adversarial network (LAPRAN) that enables high-fidelity, flexible and fast CS images reconstruction. LAPRAN progressively reconstructs an image following the concept of Laplacian pyramid through multiple stages of reconstructive adversarial networks (RANs). At each pyramid level, CS measurements are fused with a contextual latent vector to generate a high-frequency image residual. Consequently, LAPRAN can produce hierarchies of reconstructed images and each with an incremental resolution and improved quality. The scalable pyramid structure of LAPRAN enables high-fidelity CS reconstruction with a flexible resolution that is adaptive to a wide range of compression ratios (CRs), which is infeasible with existing methods. Experimental results on multiple public datasets show that LAPRAN offers an average 7.47dB and 5.98dB PSNR, and an average 57.93% and 33.20% SSIM improvement compared to model-based and data-driven baselines, respectively.
Tasks	Compressive Sensing
Published	2018-07-24
URL	http://arxiv.org/abs/1807.09388v3
PDF	http://arxiv.org/pdf/1807.09388v3.pdf
PWC	https://paperswithcode.com/paper/lapran-a-scalable-laplacian-pyramid
Repo	https://github.com/PSCLab-ASU/LAPRAN-PyTorch
Framework	pytorch

Learning sentence embeddings using Recursive Networks


Title	Learning sentence embeddings using Recursive Networks
Authors	Anson Bastos
Abstract	Learning sentence vectors that generalise well is a challenging task. In this paper we compare three methods of learning phrase embeddings: 1) Using LSTMs, 2) using recursive nets, 3) A variant of the method 2 using the POS information of the phrase. We train our models on dictionary definitions of words to obtain a reverse dictionary application similar to Felix et al. [1]. To see if our embeddings can be transferred to a new task we also train and test on the rotten tomatoes dataset [2]. We train keeping the sentence embeddings fixed as well as with fine tuning.
Tasks	Sentence Embeddings
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08353v1
PDF	http://arxiv.org/pdf/1805.08353v1.pdf
PWC	https://paperswithcode.com/paper/learning-sentence-embeddings-using-recursive
Repo	https://github.com/dyashkir/todo-ideas-links
Framework	none

Aggregated Momentum: Stability Through Passive Damping


Title	Aggregated Momentum: Stability Through Passive Damping
Authors	James Lucas, Shengyang Sun, Richard Zemel, Roger Grosse
Abstract	Momentum is a simple and widely used trick which allows gradient-based optimizers to pick up speed along low curvature directions. Its performance depends crucially on a damping coefficient $\beta$. Large $\beta$ values can potentially deliver much larger speedups, but are prone to oscillations and instability; hence one typically resorts to small values such as 0.5 or 0.9. We propose Aggregated Momentum (AggMo), a variant of momentum which combines multiple velocity vectors with different $\beta$ parameters. AggMo is trivial to implement, but significantly dampens oscillations, enabling it to remain stable even for aggressive $\beta$ values such as 0.999. We reinterpret Nesterov’s accelerated gradient descent as a special case of AggMo and analyze rates of convergence for quadratic objectives. Empirically, we find that AggMo is a suitable drop-in replacement for other momentum methods, and frequently delivers faster convergence.
Tasks
Published	2018-04-01
URL	http://arxiv.org/abs/1804.00325v3
PDF	http://arxiv.org/pdf/1804.00325v3.pdf
PWC	https://paperswithcode.com/paper/aggregated-momentum-stability-through-passive
Repo	https://github.com/AtheMathmo/AggMo
Framework	pytorch

PU-Net: Point Cloud Upsampling Network


Title	PU-Net: Point Cloud Upsampling Network
Authors	Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, Pheng-Ann Heng
Abstract	Learning and analyzing 3D point clouds with deep networks is challenging due to the sparseness and irregularity of the data. In this paper, we present a data-driven point cloud upsampling technique. The key idea is to learn multi-level features per point and expand the point set via a multi-branch convolution unit implicitly in feature space. The expanded feature is then split to a multitude of features, which are then reconstructed to an upsampled point set. Our network is applied at a patch-level, with a joint loss function that encourages the upsampled points to remain on the underlying surface with a uniform distribution. We conduct various experiments using synthesis and scan data to evaluate our method and demonstrate its superiority over some baseline methods and an optimization-based method. Results show that our upsampled points have better uniformity and are located closer to the underlying surfaces.
Tasks	Point Cloud Super Resolution
Published	2018-01-21
URL	http://arxiv.org/abs/1801.06761v2
PDF	http://arxiv.org/pdf/1801.06761v2.pdf
PWC	https://paperswithcode.com/paper/pu-net-point-cloud-upsampling-network
Repo	https://github.com/yulequan/PU-Net
Framework	tf

Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module


Title	Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module
Authors	Chenyang Li, Xin Zhang, Lufan Liao, Lianwen Jin, Weixin Yang
Abstract	The skeleton based gesture recognition is gaining more popularity due to its wide possible applications. The key issues are how to extract discriminative features and how to design the classification model. In this paper, we first leverage a robust feature descriptor, path signature (PS), and propose three PS features to explicitly represent the spatial and temporal motion characteristics, i.e., spatial PS (S_PS), temporal PS (T_PS) and temporal spatial PS (T_S_PS). Considering the significance of fine hand movements in the gesture, we propose an “attention on hand” (AOH) principle to define joint pairs for the S_PS and select single joint for the T_PS. In addition, the dyadic method is employed to extract the T_PS and T_S_PS features that encode global and local temporal dynamics in the motion. Secondly, without the recurrent strategy, the classification model still faces challenges on temporal variation among different sequences. We propose a new temporal transformer module (TTM) that can match the sequence key frames by learning the temporal shifting parameter for each input. This is a learning-based module that can be included into standard neural network architecture. Finally, we design a multi-stream fully connected layer based network to treat spatial and temporal features separately and fused them together for the final result. We have tested our method on three benchmark gesture datasets, i.e., ChaLearn 2016, ChaLearn 2013 and MSRC-12. Experimental results demonstrate that we achieve the state-of-the-art performance on skeleton-based gesture recognition with high computational efficiency.
Tasks	Gesture Recognition
Published	2018-11-17
URL	http://arxiv.org/abs/1811.07081v2
PDF	http://arxiv.org/pdf/1811.07081v2.pdf
PWC	https://paperswithcode.com/paper/skeleton-based-gesture-recognition-using
Repo	https://github.com/LiChenyang-Github/Temporal-Transformer-Module
Framework	tf

EBIC: an evolutionary-based parallel biclustering algorithm for pattern discover


Title	EBIC: an evolutionary-based parallel biclustering algorithm for pattern discover
Authors	Patryk Orzechowski, Moshe Sipper, Xiuzhen Huang, Jason H. Moore
Abstract	In this paper a novel biclustering algorithm based on artificial intelligence (AI) is introduced. The method called EBIC aims to detect biologically meaningful, order-preserving patterns in complex data. The proposed algorithm is probably the first one capable of discovering with accuracy exceeding 50% multiple complex patterns in real gene expression datasets. It is also one of the very few biclustering methods designed for parallel environments with multiple graphics processing units (GPUs). We demonstrate that EBIC outperforms state-of-the-art biclustering methods, in terms of recovery and relevance, on both synthetic and genetic datasets. EBIC also yields results over 12 times faster than the most accurate reference algorithms. The proposed algorithm is anticipated to be added to the repertoire of unsupervised machine learning algorithms for the analysis of datasets, including those from large-scale genomic studies.
Tasks
Published	2018-01-09
URL	http://arxiv.org/abs/1801.03039v2
PDF	http://arxiv.org/pdf/1801.03039v2.pdf
PWC	https://paperswithcode.com/paper/ebic-an-evolutionary-based-parallel
Repo	https://github.com/EpistasisLab/ebic
Framework	none

BioSentVec: creating sentence embeddings for biomedical texts


Title	BioSentVec: creating sentence embeddings for biomedical texts
Authors	Qingyu Chen, Yifan Peng, Zhiyong Lu
Abstract	Sentence embeddings have become an essential part of today’s natural language processing (NLP) systems, especially together advanced deep learning methods. Although pre-trained sentence encoders are available in the general domain, none exists for biomedical texts to date. In this work, we introduce BioSentVec: the first open set of sentence embeddings trained with over 30 million documents from both scholarly articles in PubMed and clinical notes in the MIMIC-III Clinical Database. We evaluate BioSentVec embeddings in two sentence pair similarity tasks in different text genres. Our benchmarking results demonstrate that the BioSentVec embeddings can better capture sentence semantics compared to the other competitive alternatives and achieve state-of-the-art performance in both tasks. We expect BioSentVec to facilitate the research and development in biomedical text mining and to complement the existing resources in biomedical word embeddings. BioSentVec is publicly available at https://github.com/ncbi-nlp/BioSentVec
Tasks	Sentence Embeddings, Sentence Embeddings For Biomedical Texts, Word Embeddings
Published	2018-10-22
URL	https://arxiv.org/abs/1810.09302v6
PDF	https://arxiv.org/pdf/1810.09302v6.pdf
PWC	https://paperswithcode.com/paper/biosentvec-creating-sentence-embeddings-for
Repo	https://github.com/ncbi-nlp/BioSentVec
Framework	none

Developing a Portable Natural Language Processing Based Phenotyping System


Title	Developing a Portable Natural Language Processing Based Phenotyping System
Authors	Himanshu Sharma, Chengsheng Mao, Yizhen Zhang, Haleh Vatani, Liang Yao, Yizhen Zhong, Luke Rasmussen, Guoqian Jiang, Jyotishman Pathak, Yuan Luo
Abstract	This paper presents a portable phenotyping system that is capable of integrating both rule-based and statistical machine learning based approaches. Our system utilizes UMLS to extract clinically relevant features from the unstructured text and then facilitates portability across different institutions and data systems by incorporating OHDSI’s OMOP Common Data Model (CDM) to standardize necessary data elements. Our system can also store the key components of rule-based systems (e.g., regular expression matches) in the format of OMOP CDM, thus enabling the reuse, adaptation and extension of many existing rule-based clinical NLP systems. We experimented with our system on the corpus from i2b2’s Obesity Challenge as a pilot study. Our system facilitates portable phenotyping of obesity and its 15 comorbidities based on the unstructured patient discharge summaries, while achieving a performance that often ranked among the top 10 of the challenge participants. This standardization enables a consistent application of numerous rule-based and machine learning based classification techniques downstream.
Tasks
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06638v1
PDF	http://arxiv.org/pdf/1807.06638v1.pdf
PWC	https://paperswithcode.com/paper/developing-a-portable-natural-language
Repo	https://github.com/mocherson/portableNLP
Framework	none

Unsupervised Semantic-based Aggregation of Deep Convolutional Features


Title	Unsupervised Semantic-based Aggregation of Deep Convolutional Features
Authors	Jian Xu, Chunheng Wang, Chengzuo Qi, Cunzhao Shi, Baihua Xiao
Abstract	In this paper, we propose a simple but effective semantic-based aggregation (SBA) method. The proposed SBA utilizes the discriminative filters of deep convolutional layers as semantic detectors. Moreover, we propose the effective unsupervised strategy to select some semantic detectors to generate the “probabilistic proposals”, which highlight certain discriminative pattern of objects and suppress the noise of background. The final global SBA representation could then be acquired by aggregating the regional representations weighted by the selected “probabilistic proposals” corresponding to various semantic content. Our unsupervised SBA is easy to generalize and achieves excellent performance on various tasks. We conduct comprehensive experiments and show that our unsupervised SBA outperforms the state-of-the-art unsupervised and supervised aggregation methods on image retrieval, place recognition and cloud classification.
Tasks	Image Retrieval
Published	2018-04-03
URL	http://arxiv.org/abs/1804.01422v1
PDF	http://arxiv.org/pdf/1804.01422v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-semantic-based-aggregation-of
Repo	https://github.com/XJhaoren/PWA
Framework	none

BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation


Title	BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation
Authors	Dayiheng Liu, Jie Fu, Qian Qu, Jiancheng Lv
Abstract	Incorporating prior knowledge like lexical constraints into the model’s output to generate meaningful and coherent sentences has many applications in dialogue system, machine translation, image captioning, etc. However, existing RNN-based models incrementally generate sentences from left to right via beam search, which makes it difficult to directly introduce lexical constraints into the generated sentences. In this paper, we propose a new algorithmic framework, dubbed BFGAN, to address this challenge. Specifically, we employ a backward generator and a forward generator to generate lexically constrained sentences together, and use a discriminator to guide the joint training of two generators by assigning them reward signals. Due to the difficulty of BFGAN training, we propose several training techniques to make the training process more stable and efficient. Our extensive experiments on two large-scale datasets with human evaluation demonstrate that BFGAN has significant improvements over previous methods.
Tasks	Image Captioning, Machine Translation
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08097v2
PDF	http://arxiv.org/pdf/1806.08097v2.pdf
PWC	https://paperswithcode.com/paper/bfgan-backward-and-forward-generative
Repo	https://github.com/dayihengliu/BFGAN
Framework	none

Specification-Driven Multi-Perspective Predictive Business Process Monitoring (Extended Version)


Title	Specification-Driven Multi-Perspective Predictive Business Process Monitoring (Extended Version)
Authors	Ario Santoso
Abstract	Predictive analysis in business process monitoring aims at forecasting the future information of a running business process. The prediction is typically made based on the model extracted from historical process execution logs (event logs). In practice, different business domains might require different kinds of predictions. Hence, it is important to have a means for properly specifying the desired prediction tasks, and a mechanism to deal with these various prediction tasks. Although there have been many studies in this area, they mostly focus on a specific prediction task. This work introduces a language for specifying the desired prediction tasks, and this language allows us to express various kinds of prediction tasks. This work also presents a mechanism for automatically creating the corresponding prediction model based on the given specification. Thus, different from previous studies, our approach enables us to deal with various kinds of prediction tasks based on the given specification. A prototype implementing our approach has been developed and experiments using a real-life event log have been conducted.
Tasks
Published	2018-04-02
URL	http://arxiv.org/abs/1804.00617v2
PDF	http://arxiv.org/pdf/1804.00617v2.pdf
PWC	https://paperswithcode.com/paper/specification-driven-multi-perspective
Repo	https://github.com/ariosantoso/Specification-Driven-Predictive-Process-Monitoring
Framework	none

Mimic and Classify : A meta-algorithm for Conditional Independence Testing


Title	Mimic and Classify : A meta-algorithm for Conditional Independence Testing
Authors	Rajat Sen, Karthikeyan Shanmugam, Himanshu Asnani, Arman Rahimzamani, Sreeram Kannan
Abstract	Given independent samples generated from the joint distribution $p(\mathbf{x},\mathbf{y},\mathbf{z})$, we study the problem of Conditional Independence (CI-Testing), i.e., whether the joint equals the CI distribution $p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})= p(\mathbf{z}) p(\mathbf{y}\mathbf{z})p(\mathbf{x}\mathbf{z})$ or not. We cast this problem under the purview of the proposed, provable meta-algorithm, “Mimic and Classify”, which is realized in two-steps: (a) Mimic the CI distribution close enough to recover the support, and (b) Classify to distinguish the joint and the CI distribution. Thus, as long as we have a good generative model and a good classifier, we potentially have a sound CI Tester. With this modular paradigm, CI Testing becomes amiable to be handled by state-of-the-art, both generative and classification methods from the modern advances in Deep Learning, which in general can handle issues related to curse of dimensionality and operation in small sample regime. We show intensive numerical experiments on synthetic and real datasets where new mimic methods such conditional GANs, Regression with Neural Nets, outperform the current best CI Testing performance in the literature. Our theoretical results provide analysis on the estimation of null distribution as well as allow for general measures, i.e., when either some of the random variables are discrete and some are continuous or when one or more of them are discrete-continuous mixtures.
Tasks
Published	2018-06-25
URL	http://arxiv.org/abs/1806.09708v1
PDF	http://arxiv.org/pdf/1806.09708v1.pdf
PWC	https://paperswithcode.com/paper/mimic-and-classify-a-meta-algorithm-for
Repo	https://github.com/rajatsen91/mimic_classify
Framework	pytorch

Deep Learning: A Critical Appraisal


Title	Deep Learning: A Critical Appraisal
Authors	Gary Marcus
Abstract	Although deep learning has historical roots going back decades, neither the term “deep learning” nor the approach was popular just over five years ago, when the field was reignited by papers such as Krizhevsky, Sutskever and Hinton’s now classic (2012) deep network model of Imagenet. What has the field discovered in the five subsequent years? Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.
Tasks	Speech Recognition
Published	2018-01-02
URL	http://arxiv.org/abs/1801.00631v1
PDF	http://arxiv.org/pdf/1801.00631v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-a-critical-appraisal
Repo	https://github.com/astoycos/Mini_Project2
Framework	tf

A Neural Compositional Paradigm for Image Captioning


Title	A Neural Compositional Paradigm for Image Captioning
Authors	Bo Dai, Sanja Fidler, Dahua Lin
Abstract	Mainstream captioning models often follow a sequential structure to generate captions, leading to issues such as introduction of irrelevant semantics, lack of diversity in the generated captions, and inadequate generalization performance. In this paper, we present an alternative paradigm for image captioning, which factorizes the captioning procedure into two stages: (1) extracting an explicit semantic representation from the given image; and (2) constructing the caption based on a recursive compositional procedure in a bottom-up manner. Compared to conventional ones, our paradigm better preserves the semantic content through an explicit factorization of semantics and syntax. By using the compositional generation procedure, caption construction follows a recursive structure, which naturally fits the properties of human language. Moreover, the proposed compositional procedure requires less data to train, generalizes better, and yields more diverse captions.
Tasks	Image Captioning
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09630v1
PDF	http://arxiv.org/pdf/1810.09630v1.pdf
PWC	https://paperswithcode.com/paper/a-neural-compositional-paradigm-for-image
Repo	https://github.com/ajaysub110/A-Neural-Compositional-Paradigm-for-Image-Captioning
Framework	pytorch

An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution


Title	An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
Authors	Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, Jason Yosinski
Abstract	Few ideas have enjoyed as large an impact on deep learning as convolution. For any problem involving pixels or spatial representations, common intuition holds that convolutional neural networks may be appropriate. In this paper we show a striking counterexample to this intuition via the seemingly trivial coordinate transform problem, which simply requires learning a mapping between coordinates in (x,y) Cartesian space and one-hot pixel space. Although convolutional networks would seem appropriate for this task, we show that they fail spectacularly. We demonstrate and carefully analyze the failure first on a toy problem, at which point a simple fix becomes obvious. We call this solution CoordConv, which works by giving convolution access to its own input coordinates through the use of extra coordinate channels. Without sacrificing the computational and parametric efficiency of ordinary convolution, CoordConv allows networks to learn either complete translation invariance or varying degrees of translation dependence, as required by the end task. CoordConv solves the coordinate transform problem with perfect generalization and 150 times faster with 10–100 times fewer parameters than convolution. This stark contrast raises the question: to what extent has this inability of convolution persisted insidiously inside other tasks, subtly hampering performance from within? A complete answer to this question will require further investigation, but we show preliminary evidence that swapping convolution for CoordConv can improve models on a diverse set of tasks. Using CoordConv in a GAN produced less mode collapse as the transform between high-level spatial latents and pixels becomes easier to learn. A Faster R-CNN detection model trained on MNIST showed 24% better IOU when using CoordConv, and in the RL domain agents playing Atari games benefit significantly from the use of CoordConv layers.
Tasks	Atari Games, Image Classification
Published	2018-07-09
URL	http://arxiv.org/abs/1807.03247v2
PDF	http://arxiv.org/pdf/1807.03247v2.pdf
PWC	https://paperswithcode.com/paper/an-intriguing-failing-of-convolutional-neural
Repo	https://github.com/felixriese/CNN-SoilTextureClassification
Framework	tf