January 30, 2020

3445 words 17 mins read

Paper Group ANR 231

Relational Reasoning Network (RRN) for Anatomical Landmarking. Detection of Advanced Malware by Machine Learning Techniques. Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translation. Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimation. The Kernel Spatial Scan Statistic. Insect pest image detection and r …

Relational Reasoning Network (RRN) for Anatomical Landmarking


Title	Relational Reasoning Network (RRN) for Anatomical Landmarking
Authors	Neslisah Torosdagli, Mary McIntosh, Denise K. Liberton, Payal Verma, Murat Sincan, Wade W. Han, Janice S. Lee, Ulas Bagci
Abstract	Accurately identifying anatomical landmarks is a crucial step in deformation analysis and surgical planning for craniomaxillofacial (CMF) bones. Available methods require segmentation of the object of interest for precise landmarking. Unlike those, our purpose in this study is to perform anatomical landmarking using the inherent relation of CMF bones without explicitly segmenting them. We propose a new deep network architecture, called relational reasoning network (RRN), to accurately learn the local and the global relations of the landmarks. Specifically, we are interested in learning landmarks in CMF region: mandible, maxilla, and nasal bones. The proposed RRN works in an end-to-end manner, utilizing learned relations of the landmarks based on dense-block units and without the need for segmentation. For a given a few landmarks as input, the proposed system accurately and efficiently localizes the remaining landmarks on the aforementioned bones. For a comprehensive evaluation of RRN, we used cone-beam computed tomography (CBCT) scans of 250 patients. The proposed system identifies the landmark locations very accurately even when there are severe pathologies or deformations in the bones. The proposed RRN has also revealed unique relationships among the landmarks that help us infer several reasoning about informativeness of the landmark points. RRN is invariant to order of landmarks and it allowed us to discover the optimal configurations (number and location) for landmarks to be localized within the object of interest (mandible) or nearby objects (maxilla and nasal). To the best of our knowledge, this is the first of its kind algorithm finding anatomical relations of the objects using deep learning.
Tasks	Relational Reasoning
Published	2019-04-08
URL	http://arxiv.org/abs/1904.04354v1
PDF	http://arxiv.org/pdf/1904.04354v1.pdf
PWC	https://paperswithcode.com/paper/relational-reasoning-network-rrn-for
Repo
Framework

Detection of Advanced Malware by Machine Learning Techniques


Title	Detection of Advanced Malware by Machine Learning Techniques
Authors	Sanjay Sharma, C. Rama Krishna, Sanjay K. Sahay
Abstract	In today’s digital world most of the anti-malware tools are signature based which is ineffective to detect advanced unknown malware viz. metamorphic malware. In this paper, we study the frequency of opcode occurrence to detect unknown malware by using machine learning technique. For the purpose, we have used kaggle Microsoft malware classification challenge dataset. The top 20 features obtained from fisher score, information gain, gain ratio, chi-square and symmetric uncertainty feature selection methods are compared. We also studied multiple classifier available in WEKA GUI based machine learning tool and found that five of them (Random Forest, LMT, NBT, J48 Graft and REPTree) detect malware with almost 100% accuracy.
Tasks	Feature Selection, Malware Classification
Published	2019-03-07
URL	http://arxiv.org/abs/1903.02966v1
PDF	http://arxiv.org/pdf/1903.02966v1.pdf
PWC	https://paperswithcode.com/paper/detection-of-advanced-malware-by-machine
Repo
Framework

Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translation


Title	Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translation
Authors	Jetic Gū, Hassan S. Shavarani, Anoop Sarkar
Abstract	Neural machine translation (NMT) systems require large amounts of high quality in-domain parallel corpora for training. State-of-the-art NMT systems still face challenges related to out-of-vocabulary words and dealing with low-resource language pairs. In this paper, we propose and compare several models for fusion of bilingual lexicons with an end-to-end trained sequence-to-sequence model for machine translation. The result is a fusion model with two information sources for the decoder: a neural conditional language model and a bilingual lexicon. This fusion model learns how to combine both sources of information in order to produce higher quality translation output. Our experiments show that our proposed models work well in relatively low-resource scenarios, and also effectively reduce the parameter size and training cost for NMT without sacrificing performance.
Tasks	Language Modelling, Machine Translation
Published	2019-09-17
URL	https://arxiv.org/abs/1909.07907v1
PDF	https://arxiv.org/pdf/1909.07907v1.pdf
PWC	https://paperswithcode.com/paper/pointer-based-fusion-of-bilingual-lexicons
Repo
Framework

Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimation


Title	Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimation
Authors	Melkior Ornik, Ufuk Topcu
Abstract	This paper proposes a formal approach to learning and planning for agents operating in a priori unknown, time-varying environments. The proposed method computes the maximally likely model of the environment, given the observations about the environment made by an agent earlier in the system run and assuming knowledge of a bound on the maximal rate of change of system dynamics. Such an approach generalizes the estimation method commonly used in learning algorithms for unknown Markov decision processes with time-invariant transition probabilities, but is also able to quickly and correctly identify the system dynamics following a change. Based on the proposed method, we generalize the exploration bonuses used in learning for time-invariant Markov decision processes by introducing a notion of uncertainty in a learned time-varying model, and develop a control policy for time-varying Markov decision processes based on the exploitation and exploration trade-off. We demonstrate the proposed methods on four numerical examples: a patrolling task with a change in system dynamics, a two-state MDP with periodically changing outcomes of actions, a wind flow estimation task, and a multi-arm bandit problem with periodically changing probabilities of different rewards.
Tasks
Published	2019-11-29
URL	https://arxiv.org/abs/1911.12976v1
PDF	https://arxiv.org/pdf/1911.12976v1.pdf
PWC	https://paperswithcode.com/paper/learning-and-planning-for-time-varying-mdps
Repo
Framework

The Kernel Spatial Scan Statistic


Title	The Kernel Spatial Scan Statistic
Authors	Mingxuan Han, Michael Matheny, Jeff M. Phillips
Abstract	Kulldorff’s (1997) seminal paper on spatial scan statistics (SSS) has led to many methods considering different regions of interest, different statistical models, and different approximations while also having numerous applications in epidemiology, environmental monitoring, and homeland security. SSS provides a way to rigorously test for the existence of an anomaly and provide statistical guarantees as to how “anomalous” that anomaly is. However, these methods rely on defining specific regions where the spatial information a point contributes is limited to binary 0 or 1, of either inside or outside the region, while in reality anomalies will tend to follow smooth distributions with decaying density further from an epicenter. In this work, we propose a method that addresses this shortcoming through a continuous scan statistic that generalizes SSS by allowing the point contribution to be defined by a kernel. We provide extensive experimental and theoretical results that shows our methods can be computed efficiently while providing high statistical power for detecting anomalous regions.
Tasks	Epidemiology
Published	2019-06-13
URL	https://arxiv.org/abs/1906.09381v2
PDF	https://arxiv.org/pdf/1906.09381v2.pdf
PWC	https://paperswithcode.com/paper/the-kernel-spatial-scan-statistic
Repo
Framework

Insect pest image detection and recognition based on bio-inspired methods


Title	Insect pest image detection and recognition based on bio-inspired methods
Authors	Loris Nanni, Gianluca Maguolo, Fabio Pancino
Abstract	Insect pests recognition is necessary for crop protection in many areas of the world. In this paper we propose an automatic classifier based on the fusion between saliency methods and convolutional neural networks. Saliency methods are famous image processing algorithms that highlight the most relevant pixels of an image. In this paper, we use three different saliency methods as image preprocessing and create three different images for every saliency method. Hence, we create 3x3=9 new images for every original image to train different convolutional neural networks. We evaluate the performance of every preprocessing/network couple and we also evaluate the performance of their ensemble. We test our approach on both a small dataset and the large IP102 dataset. Our best ensembles reaches the state of the art accuracy on both the smaller dataset (92.43%) and the IP102 dataset (61.93%), approaching the performance of human experts on the smaller one. Besides, we share our MATLAB code at: https://github.com/LorisNanni/.
Tasks
Published	2019-10-01
URL	https://arxiv.org/abs/1910.00296v3
PDF	https://arxiv.org/pdf/1910.00296v3.pdf
PWC	https://paperswithcode.com/paper/research-on-insect-pest-image-detection-and
Repo
Framework

Error Analysis for Vietnamese Dependency Parsing


Title	Error Analysis for Vietnamese Dependency Parsing
Authors	Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
Abstract	Dependency parsing is needed in different applications of natural language processing. In this paper, we present a thorough error analysis for dependency parsing for the Vietnamese language, using two state-of-the-art parsers: MSTParser and MaltParser. The error analysis results provide us insights in order to improve the performance of dependency parsing for the Vietnamese language.
Tasks	Dependency Parsing
Published	2019-11-09
URL	https://arxiv.org/abs/1911.03724v1
PDF	https://arxiv.org/pdf/1911.03724v1.pdf
PWC	https://paperswithcode.com/paper/error-analysis-for-vietnamese-dependency
Repo
Framework

Language Features Matter: Effective Language Representations for Vision-Language Tasks


Title	Language Features Matter: Effective Language Representations for Vision-Language Tasks
Authors	Andrea Burns, Reuben Tan, Kate Saenko, Stan Sclaroff, Bryan A. Plummer
Abstract	Shouldn’t language and vision features be treated equally in vision-language (VL) tasks? Many VL approaches treat the language component as an afterthought, using simple language models that are either built upon fixed word embeddings trained on text-only data or are learned from scratch. We believe that language features deserve more attention, and conduct experiments which compare different word embeddings, language models, and embedding augmentation steps on five common VL tasks: image-sentence retrieval, image captioning, visual question answering, phrase grounding, and text-to-clip retrieval. Our experiments provide some striking results; an average embedding language model outperforms an LSTM on retrieval-style tasks; state-of-the-art representations such as BERT perform relatively poorly on vision-language tasks. From this comprehensive set of experiments we propose a set of best practices for incorporating the language component of VL tasks. To further elevate language features, we also show that knowledge in vision-language problems can be transferred across tasks to gain performance with multi-task training. This multi-task training is applied to a new Graph Oriented Vision-Language Embedding (GrOVLE), which we adapt from Word2Vec using WordNet and an original visual-language graph built from Visual Genome, providing a ready-to-use vision-language embedding: http://ai.bu.edu/grovle.
Tasks	Image Captioning, Language Modelling, Phrase Grounding, Question Answering, Visual Question Answering, Word Embeddings
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06327v1
PDF	https://arxiv.org/pdf/1908.06327v1.pdf
PWC	https://paperswithcode.com/paper/language-features-matter-effective-language
Repo
Framework

Adaptive Locality Preserving Regression


Title	Adaptive Locality Preserving Regression
Authors	Jie Wen, Zuofeng Zhong, Zheng Zhang, Lunke Fei, Zhihui Lai, Runze Chen
Abstract	This paper proposes a novel discriminative regression method, called adaptive locality preserving regression (ALPR) for classification. In particular, ALPR aims to learn a more flexible and discriminative projection that not only preserves the intrinsic structure of data, but also possesses the properties of feature selection and interpretability. To this end, we introduce a target learning technique to adaptively learn a more discriminative and flexible target matrix rather than the pre-defined strict zero-one label matrix for regression. Then a locality preserving constraint regularized by the adaptive learned weights is further introduced to guide the projection learning, which is beneficial to learn a more discriminative projection and avoid overfitting. Moreover, we replace the conventional `Frobenius norm’ with the special l21 norm to constrain the projection, which enables the method to adaptively select the most important features from the original high-dimensional data for feature extraction. In this way, the negative influence of the redundant features and noises residing in the original data can be greatly eliminated. Besides, the proposed method has good interpretability for features owing to the row-sparsity property of the l21 norm. Extensive experiments conducted on the synthetic database with manifold structure and many real-world databases prove the effectiveness of the proposed method. \|
Tasks	Feature Selection
Published	2019-01-03
URL	http://arxiv.org/abs/1901.00563v1
PDF	http://arxiv.org/pdf/1901.00563v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-locality-preserving-regression
Repo
Framework

Deep Learning Sentiment Analysis of Amazon.com Reviews and Ratings


Title	Deep Learning Sentiment Analysis of Amazon.com Reviews and Ratings
Authors	Nishit Shrestha, Fatma Nasoz
Abstract	Our study employs sentiment analysis to evaluate the compatibility of Amazon.com reviews with their corresponding ratings. Sentiment analysis is the task of identifying and classifying the sentiment expressed in a piece of text as being positive or negative. On e-commerce websites such as Amazon.com, consumers can submit their reviews along with a specific polarity rating. In some instances, there is a mismatch between the review and the rating. To identify the reviews with mismatched ratings we performed sentiment analysis using deep learning on Amazon.com product review data. Product reviews were converted to vectors using paragraph vector, which then was used to train a recurrent neural network with gated recurrent unit. Our model incorporated both semantic relationship of review text and product information. We also developed a web service application that predicts the rating score for a submitted review using the trained model and if there is a mismatch between predicted rating score and submitted rating score, it provides feedback to the reviewer.
Tasks	Sentiment Analysis
Published	2019-04-04
URL	http://arxiv.org/abs/1904.04096v1
PDF	http://arxiv.org/pdf/1904.04096v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-sentiment-analysis-of-amazoncom
Repo
Framework

When a Period Is Not a Full Stop: Light Curve Structure Reveals Fundamental Parameters of Cepheid and RR Lyrae Stars


Title	When a Period Is Not a Full Stop: Light Curve Structure Reveals Fundamental Parameters of Cepheid and RR Lyrae Stars
Authors	Earl P. Bellinger, Shashi M. Kanbur, Anupam Bhardwaj, Marcella Marconi
Abstract	The period of pulsation and the structure of the light curve for Cepheid and RR Lyrae variables depend on the fundamental parameters of the star: mass, radius, luminosity, and effective temperature. Here we train artificial neural networks on theoretical pulsation models to predict the fundamental parameters of these stars based on their period and light curve structure. We find significant improvements to estimates of these parameters made using light curve structure and period over estimates made using only the period. Given that the models are able to reproduce most observables, we find that the fundamental parameters of these stars can be estimated up to 60% more accurately when light curve structure is taken into consideration. We quantify which aspects of light curve structure are most important in determining fundamental parameters, and find for example that the second Fourier amplitude component of RR Lyrae light curves is even more important than period in determining the effective temperature of the star. We apply this analysis to observations of hundreds Cepheids in the Large Magellanic Cloud and thousands of RR Lyrae in the Magellanic Clouds and Galactic bulge to produce catalogs of estimated masses, radii, luminosities, and other parameters of these stars. As an example application, we estimate Wesenheit indices and use those to derive distance moduli to the Magellanic Clouds of $\mu_{\text{LMC},\text{CEP}} = 18.688 \pm 0.093$, $\mu_{\text{LMC},\text{RRL}} = 18.52 \pm 0.14$, and $\mu_{\text{SMC},\text{RRL}} = 18.88 \pm 0.17$ mag.
Tasks
Published	2019-11-25
URL	https://arxiv.org/abs/1911.11767v1
PDF	https://arxiv.org/pdf/1911.11767v1.pdf
PWC	https://paperswithcode.com/paper/when-a-period-is-not-a-full-stop-light-curve
Repo
Framework

Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation


Title	Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentation
Authors	Liyuan Pan, Yuchao Dai, Miaomiao Liu, Fatih Porikli, Quan Pan
Abstract	Stereo videos for the dynamic scenes often show unpleasant blurred effects due to the camera motion and the multiple moving objects with large depth variations. Given consecutive blurred stereo video frames, we aim to recover the latent clean images, estimate the 3D scene flow and segment the multiple moving objects. These three tasks have been previously addressed separately, which fail to exploit the internal connections among these tasks and cannot achieve optimality. In this paper, we propose to jointly solve these three tasks in a unified framework by exploiting their intrinsic connections. To this end, we represent the dynamic scenes with the piece-wise planar model, which exploits the local structure of the scene and expresses various dynamic scenes. Under our model, these three tasks are naturally connected and expressed as the parameter estimation of 3D scene structure and camera motion (structure and motion for the dynamic scenes). By exploiting the blur model constraint, the moving objects and the 3D scene structure, we reach an energy minimization formulation for joint deblurring, scene flow and segmentation. We evaluate our approach extensively on both synthetic datasets and publicly available real datasets with fast-moving objects, camera motion, uncontrolled lighting conditions and shadows. Experimental results demonstrate that our method can achieve significant improvement in stereo video deblurring, scene flow estimation and moving object segmentation, over state-of-the-art methods.
Tasks	Deblurring, Scene Flow Estimation, Semantic Segmentation
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02442v1
PDF	https://arxiv.org/pdf/1910.02442v1.pdf
PWC	https://paperswithcode.com/paper/joint-stereo-video-deblurring-scene-flow
Repo
Framework

Time Series Simulation by Conditional Generative Adversarial Net


Title	Time Series Simulation by Conditional Generative Adversarial Net
Authors	Rao Fu, Jie Chen, Shutian Zeng, Yiping Zhuang, Agus Sudjianto
Abstract	Generative Adversarial Net (GAN) has been proven to be a powerful machine learning tool in image data analysis and generation. In this paper, we propose to use Conditional Generative Adversarial Net (CGAN) to learn and simulate time series data. The conditions can be both categorical and continuous variables containing different kinds of auxiliary information. Our simulation studies show that CGAN is able to learn different kinds of normal and heavy tail distributions, as well as dependent structures of different time series and it can further generate conditional predictive distributions consistent with the training data distributions. We also provide an in-depth discussion on the rationale of GAN and the neural network as hierarchical splines to draw a clear connection with the existing statistical method for distribution generation. In practice, CGAN has a wide range of applications in the market risk and counterparty risk analysis: it can be applied to learn the historical data and generate scenarios for the calculation of Value-at-Risk (VaR) and Expected Shortfall (ES) and predict the movement of the market risk factors. We present a real data analysis including a backtesting to demonstrate CGAN is able to outperform the Historic Simulation, a popular method in market risk analysis for the calculation of VaR. CGAN can also be applied in the economic time series modeling and forecasting, and an example of hypothetical shock analysis for economic models and the generation of potential CCAR scenarios by CGAN is given at the end of the paper.
Tasks	Time Series
Published	2019-04-25
URL	http://arxiv.org/abs/1904.11419v1
PDF	http://arxiv.org/pdf/1904.11419v1.pdf
PWC	https://paperswithcode.com/paper/time-series-simulation-by-conditional
Repo
Framework

An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms


Title	An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms
Authors	Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li
Abstract	Deep Learning (DL) has recently achieved tremendous success. A variety of DL frameworks and platforms play a key role to catalyze such progress. However, the differences in architecture designs and implementations of existing frameworks and platforms bring new challenges for DL software development and deployment. Till now, there is no study on how various mainstream frameworks and platforms influence both DL software development and deployment in practice. To fill this gap, we take the first step towards understanding how the most widely-used DL frameworks and platforms support the DL software development and deployment. We conduct a systematic study on these frameworks and platforms by using two types of DNN architectures and three popular datasets. (1) For development process, we investigate the prediction accuracy under the same runtime training configuration or same model weights/biases. We also study the adversarial robustness of trained models by leveraging the existing adversarial attack techniques. The experimental results show that the computing differences across frameworks could result in an obvious prediction accuracy decline, which should draw the attention of DL developers. (2) For deployment process, we investigate the prediction accuracy and performance (refers to time cost and memory consumption) when the trained models are migrated/quantized from PC to real mobile devices and web browsers. The DL platform study unveils that the migration and quantization still suffer from compatibility and reliability issues. Meanwhile, we find several DL software bugs by using the results as a benchmark. We further validate the results through bug confirmation from stakeholders and industrial positive feedback to highlight the implications of our study. Through our study, we summarize practical guidelines, identify challenges and pinpoint new research directions.
Tasks	Adversarial Attack, Quantization
Published	2019-09-15
URL	https://arxiv.org/abs/1909.06727v1
PDF	https://arxiv.org/pdf/1909.06727v1.pdf
PWC	https://paperswithcode.com/paper/an-empirical-study-towards-characterizing
Repo
Framework

WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding


Title	WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding
Authors	Shuyu Lin, Ronald Clark, Robert Birke, Niki Trigoni, Stephen Roberts
Abstract	Variational Auto-encoders (VAEs) have been very successful as methods for forming compressed latent representations of complex, often high-dimensional, data. In this paper, we derive an alternative variational lower bound from the one common in VAEs, which aims to minimize aggregate information loss. Using our lower bound as the objective function for an auto-encoder enables us to place a prior on the bulk statistics, corresponding to an aggregate posterior for the entire dataset, as opposed to a single sample posterior as in the original VAE. This alternative form of prior constraint allows individual posteriors more flexibility to preserve necessary information for good reconstruction quality. We further derive an analytic approximation to our lower bound, leading to an efficient learning algorithm - WiSE-ALE. Through various examples, we demonstrate that WiSE-ALE can reach excellent reconstruction quality in comparison to other state-of-the-art VAE models, while still retaining the ability to learn a smooth, compact representation.
Tasks
Published	2019-02-16
URL	http://arxiv.org/abs/1902.06160v3
PDF	http://arxiv.org/pdf/1902.06160v3.pdf
PWC	https://paperswithcode.com/paper/wise-vae-wide-sample-estimator-vae
Repo
Framework