January 28, 2020

3104 words 15 mins read

Paper Group ANR 775

CompenNet++: End-to-end Full Projector Compensation. Black-box constructions for exchangeable sequences of random multisets. HapPenIng: Happen, Predict, Infer – Event Series Completion in a Knowledge Graph. NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data. PRNet: Self-Supervised Learning for …

CompenNet++: End-to-end Full Projector Compensation


Title	CompenNet++: End-to-end Full Projector Compensation
Authors	Bingyao Huang, Haibin Ling
Abstract	Full projector compensation aims to modify a projector input image such that it can compensate for both geometric and photometric disturbance of the projection surface. Traditional methods usually solve the two parts separately, although they are known to correlate with each other. In this paper, we propose the first end-to-end solution, named CompenNet++, to solve the two problems jointly. Our work non-trivially extends CompenNet, which was recently proposed for photometric compensation with promising performance. First, we propose a novel geometric correction subnet, which is designed with a cascaded coarse-to-fine structure to learn the sampling grid directly from photometric sampling images. Second, by concatenating the geometric correction subset with CompenNet, CompenNet++ accomplishes full projector compensation and is end-to-end trainable. Third, after training, we significantly simplify both geometric and photometric compensation parts, and hence largely improves the running time efficiency. Moreover, we construct the first setup-independent full compensation benchmark to facilitate the study on this topic. In our thorough experiments, our method shows clear advantages over previous arts with promising compensation quality and meanwhile being practically convenient.
Tasks
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06246v1
PDF	https://arxiv.org/pdf/1908.06246v1.pdf
PWC	https://paperswithcode.com/paper/compennet-end-to-end-full-projector
Repo
Framework

Black-box constructions for exchangeable sequences of random multisets


Title	Black-box constructions for exchangeable sequences of random multisets
Authors	Creighton Heaukulani, Daniel M. Roy
Abstract	We develop constructions for exchangeable sequences of point processes that are rendered conditionally-i.i.d. negative binomial processes by a (possibly unknown) random measure called the base measure. Negative binomial processes are useful in Bayesian nonparametrics as models for random multisets, and in applications we are often interested in cases when the base measure itself is difficult to construct (for example when it has countably infinite support). While a finitary construction for an important case (corresponding to a beta process base measure) has appeared in the literature, our constructions generalize to any random base measure, requiring only an exchangeable sequence of Bernoulli processes rendered conditionally-i.i.d. by the same underlying random base measure. Because finitary constructions for such Bernoulli processes are known for several different classes of random base measures–including generalizations of the beta process and hierarchies thereof–our results immediately provide constructions for negative binomial processes with a random base measure from any member of these classes.
Tasks	Point Processes
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06349v1
PDF	https://arxiv.org/pdf/1908.06349v1.pdf
PWC	https://paperswithcode.com/paper/black-box-constructions-for-exchangeable
Repo
Framework

HapPenIng: Happen, Predict, Infer – Event Series Completion in a Knowledge Graph


Title	HapPenIng: Happen, Predict, Infer – Event Series Completion in a Knowledge Graph
Authors	Simon Gottschalk, Elena Demidova
Abstract	Event series, such as the Wimbledon Championships and the US presidential elections, represent important happenings in key societal areas including sports, culture and politics. However, semantic reference sources, such as Wikidata, DBpedia and EventKG knowledge graphs, provide only an incomplete event series representation. In this paper we target the problem of event series completion in a knowledge graph. We address two tasks: 1) prediction of sub-event relations, and 2) inference of real-world events that happened as a part of event series and are missing in the knowledge graph. To address these problems, our proposed supervised HapPenIng approach leverages structural features of event series. HapPenIng does not require any external knowledge - the characteristics making it unique in the context of event inference. Our experimental evaluation demonstrates that HapPenIng outperforms the baselines by 44 and 52 percentage points in terms of precision for the sub-event prediction and the inference tasks, correspondingly.
Tasks	Knowledge Graphs
Published	2019-09-12
URL	https://arxiv.org/abs/1909.06219v1
PDF	https://arxiv.org/pdf/1909.06219v1.pdf
PWC	https://paperswithcode.com/paper/happening-happen-predict-infer-event-series
Repo
Framework

NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data


Title	NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data
Authors	Yifan Sun, Linan Zhang, Hayden Schaeffer
Abstract	We propose a neural network based approach for extracting models from dynamic data using ordinary and partial differential equations. In particular, given a time-series or spatio-temporal dataset, we seek to identify an accurate governing system which respects the intrinsic differential structure. The unknown governing model is parameterized by using both (shallow) multilayer perceptrons and nonlinear differential terms, in order to incorporate relevant correlations between spatio-temporal samples. We demonstrate the approach on several examples where the data is sampled from various dynamical systems and give a comparison to recurrent networks and other data-discovery methods. In addition, we show that for MNIST and Fashion MNIST, our approach lowers the parameter cost as compared to other deep neural networks.
Tasks	Image Classification, Time Series
Published	2019-08-08
URL	https://arxiv.org/abs/1908.03190v1
PDF	https://arxiv.org/pdf/1908.03190v1.pdf
PWC	https://paperswithcode.com/paper/neupde-neural-network-based-ordinary-and
Repo
Framework

PRNet: Self-Supervised Learning for Partial-to-Partial Registration


Title	PRNet: Self-Supervised Learning for Partial-to-Partial Registration
Authors	Yue Wang, Justin M. Solomon
Abstract	We present a simple, flexible, and general framework titled Partial Registration Network (PRNet), for partial-to-partial point cloud registration. Inspired by recently-proposed learning-based methods for registration, we use deep networks to tackle non-convexity of the alignment and partial correspondence problems. While previous learning-based methods assume the entire shape is visible, PRNet is suitable for partial-to-partial registration, outperforming PointNetLK, DCP, and non-learning methods on synthetic data. PRNet is self-supervised, jointly learning an appropriate geometric representation, a keypoint detector that finds points in common between partial views, and keypoint-to-keypoint correspondences. We show PRNet predicts keypoints and correspondences consistently across views and objects. Furthermore, the learned representation is transferable to classification.
Tasks	Point Cloud Registration
Published	2019-10-27
URL	https://arxiv.org/abs/1910.12240v2
PDF	https://arxiv.org/pdf/1910.12240v2.pdf
PWC	https://paperswithcode.com/paper/prnet-self-supervised-learning-for-partial-to
Repo
Framework

Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging


Title	Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging
Authors	John R. Zech, Jessica Zosa Forde, Michael L. Littman
Abstract	We reproduced the results of CheXNet with fixed hyperparameters and 50 different random seeds to identify 14 finding in chest radiographs (x-rays). Because CheXNet fine-tunes a pre-trained DenseNet, the random seed affects the ordering of the batches of training data but not the initialized model weights. We found substantial variability in predictions for the same radiograph across model runs (mean ln[(maximum probability)/(minimum probability)] 2.45, coefficient of variation 0.543). This individual radiograph-level variability was not fully reflected in the variability of AUC on a large test set. Averaging predictions from 10 models reduced variability by nearly 70% (mean coefficient of variation from 0.543 to 0.169, t-test 15.96, p-value < 0.0001). We encourage researchers to be aware of the potential variability of CNNs and ensemble predictions from multiple models to minimize the effect this variability may have on the care of individual patients when these models are deployed clinically.
Tasks
Published	2019-12-08
URL	https://arxiv.org/abs/1912.03606v1
PDF	https://arxiv.org/pdf/1912.03606v1.pdf
PWC	https://paperswithcode.com/paper/individual-predictions-matter-assessing-the
Repo
Framework

Classification via an Embedded Approach


Title	Classification via an Embedded Approach
Authors	Jose de Jesus Rubio, Francisco Jacob Avila, Adolfo Melendez, Juan Manuel Stein, Jesus Alberto Meda, Carlos Aguilar
Abstract	This paper presents the results of an automated volatile organic compound (VOC) classification process implemented by embedding a machine learning algorithm into an Arduino Uno board. An electronic nose prototype is constructed to detect VOCs from three different fruits. The electronic nose is constructed using an array of five tin dioxide (SnO2) gas sensors, an Arduino Uno board used as a data acquisition section, as well as an intelligent classification module by embedding an approach function which receives data signals from the electronic nose. For the intelligent classification module, a training algorithm is also implemented to create the base of a portable, automated, fast-response, and economical electronic nose device. This solution proposes a portable system to identify and classify VOCs without using a personal computer (PC). Results show an acceptable precision for the embedded approach in comparison with the performance of a toolbox used in a PC. This constitutes an embedded solution able to recognize VOCs in a reliable way to create application products for a wide variety of industries, which are able to classify data acquired by an electronic nose, as VOCs. With this proposed and implemented algorithm, a precision of 99% for classification was achieved into the embedded solution.
Tasks
Published	2019-05-15
URL	https://arxiv.org/abs/1905.06431v1
PDF	https://arxiv.org/pdf/1905.06431v1.pdf
PWC	https://paperswithcode.com/paper/classification-via-an-embedded-approach
Repo
Framework

Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition


Title	Language Model Bootstrapping Using Neural Machine Translation For Conversational Speech Recognition
Authors	Surabhi Punjabi, Harish Arsikere, Sri Garimella
Abstract	Building conversational speech recognition systems for new languages is constrained by the availability of utterances that capture user-device interactions. Data collection is both expensive and limited by the speed of manual transcription. In order to address this, we advocate the use of neural machine translation as a data augmentation technique for bootstrapping language models. Machine translation (MT) offers a systematic way of incorporating collections from mature, resource-rich conversational systems that may be available for a different language. However, ingesting raw translations from a general purpose MT system may not be effective owing to the presence of named entities, intra sentential code-switching and the domain mismatch between the conversational data being translated and the parallel text used for MT training. To circumvent this, we explore the following domain adaptation techniques: (a) sentence embedding based data selection for MT training, (b) model finetuning, and (c) rescoring and filtering translated hypotheses. Using Hindi as the experimental testbed, we translate US English utterances to supplement the transcribed collections. We observe a relative word error rate reduction of 7.8-15.6%, depending on the bootstrapping phase. Fine grained analysis reveals that translation particularly aids the interaction scenarios which are underrepresented in the transcribed data.
Tasks	Data Augmentation, Domain Adaptation, Language Modelling, Machine Translation, Sentence Embedding, Speech Recognition
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00958v1
PDF	https://arxiv.org/pdf/1912.00958v1.pdf
PWC	https://paperswithcode.com/paper/language-model-bootstrapping-using-neural
Repo
Framework

Structured Knowledge Distillation for Dense Prediction


Title	Structured Knowledge Distillation for Dense Prediction
Authors	Yifan Liu, Changyong Shun, Jingdong Wang, Chunhua Shen
Abstract	In this paper, we consider transferring the structure information from large networks to small ones for dense prediction tasks. Previous knowledge distillation strategies used for dense prediction tasks often directly borrow the distillation scheme for image classification and perform knowledge distillation for each pixel separately, leading to sub-optimal performance. Here we propose to distill structured knowledge from large networks to small networks, taking into account the fact that dense prediction is a structured prediction problem. Specifically, we study two structured distillation schemes: i)pair-wise distillation that distills the pairwise similarities by building a static graph, and ii)holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated by extensive experiments on three dense prediction tasks: semantic segmentation, depth estimation, and object detection.
Tasks	Depth Estimation, Image Classification, Object Detection, Scene Parsing, Semantic Segmentation, Structured Prediction
Published	2019-03-11
URL	https://arxiv.org/abs/1903.04197v5
PDF	https://arxiv.org/pdf/1903.04197v5.pdf
PWC	https://paperswithcode.com/paper/structured-knowledge-distillation-for
Repo
Framework

Effective Sentence Scoring Method using Bidirectional Language Model for Speech Recognition


Title	Effective Sentence Scoring Method using Bidirectional Language Model for Speech Recognition
Authors	Joongbo Shin, Yoonhyung Lee, Kyomin Jung
Abstract	In automatic speech recognition, many studies have shown performance improvements using language models (LMs). Recent studies have tried to use bidirectional LMs (biLMs) instead of conventional unidirectional LMs (uniLMs) for rescoring the $N$-best list decoded from the acoustic model. In spite of their theoretical benefits, the biLMs have not given notable improvements compared to the uniLMs in their experiments. This is because their biLMs do not consider the interaction between the two directions. In this paper, we propose a novel sentence scoring method considering the interaction between the past and the future words on the biLM. Our experimental results on the LibriSpeech corpus show that the biLM with the proposed sentence scoring outperforms the uniLM for the $N$-best list rescoring, consistently and significantly in all experimental conditions. The analysis of WERs by word position demonstrates that the biLM is more robust than the uniLM especially when a recognized sentence is short or a misrecognized word is at the beginning of the sentence.
Tasks	Language Modelling, Speech Recognition
Published	2019-05-16
URL	https://arxiv.org/abs/1905.06655v1
PDF	https://arxiv.org/pdf/1905.06655v1.pdf
PWC	https://paperswithcode.com/paper/effective-sentence-scoring-method-using
Repo
Framework

GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection


Title	GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection
Authors	Quoc Phong Nguyen, Kar Wai Lim, Dinil Mon Divakaran, Kian Hsiang Low, Mun Choon Chan
Abstract	This paper looks into the problem of detecting network anomalies by analyzing NetFlow records. While many previous works have used statistical models and machine learning techniques in a supervised way, such solutions have the limitations that they require large amount of labeled data for training and are unlikely to detect zero-day attacks. Existing anomaly detection solutions also do not provide an easy way to explain or identify attacks in the anomalous traffic. To address these limitations, we develop and present GEE, a framework for detecting and explaining anomalies in network traffic. GEE comprises of two components: (i) Variational Autoencoder (VAE) - an unsupervised deep-learning technique for detecting anomalies, and (ii) a gradient-based fingerprinting technique for explaining anomalies. Evaluation of GEE on the recent UGR dataset demonstrates that our approach is effective in detecting different anomalies as well as identifying fingerprints that are good representations of these various attacks.
Tasks	Anomaly Detection
Published	2019-03-15
URL	http://arxiv.org/abs/1903.06661v1
PDF	http://arxiv.org/pdf/1903.06661v1.pdf
PWC	https://paperswithcode.com/paper/gee-a-gradient-based-explainable-variational
Repo
Framework

Leveraging Pre-trained Checkpoints for Sequence Generation Tasks


Title	Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
Authors	Sascha Rothe, Shashi Narayan, Aliaksei Severyn
Abstract	Unsupervised pre-training of large neural models has recently revolutionized Natural Language Processing. Warm-starting from the publicly released checkpoints, NLP practitioners have pushed the state-of-the-art on multiple benchmarks while saving significant amounts of compute time. So far the focus has been mainly on the Natural Language Understanding tasks. In this paper, we present an extensive empirical study on the utility of initializing large Transformer-based sequence-to-sequence models with the publicly available pre-trained BERT and GPT-2 checkpoints for sequence generation. We have run over 300 experiments spending thousands of TPU hours to find the recipe that works best and demonstrate that it results in new state-of-the-art results on Machine Translation, Summarization, Sentence Splitting and Sentence Fusion.
Tasks	Machine Translation
Published	2019-07-29
URL	https://arxiv.org/abs/1907.12461v1
PDF	https://arxiv.org/pdf/1907.12461v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-pre-trained-checkpoints-for
Repo
Framework

The Kalai-Smorodinski solution for many-objective Bayesian optimization


Title	The Kalai-Smorodinski solution for many-objective Bayesian optimization
Authors	Mickaël Binois, Victor Picheny, Patrick Taillandier, Abderrahmane Habbal
Abstract	An ongoing aim of research in multiobjective Bayesian optimization is to extend its applicability to a large number of objectives. While coping with a limited budget of evaluations, recovering the set of optimal compromise solutions generally requires numerous observations and is less interpretable since this set tends to grow larger with the number of objectives. We thus propose to focus on a specific solution originating from game theory, the Kalai-Smorodinsky solution, which possesses attractive properties. In particular, it ensures equal marginal gains over all objectives. We further make it insensitive to a monotonic transformation of the objectives by considering the objectives in the copula space. A novel tailored algorithm is proposed to search for the solution, in the form of a Bayesian optimization algorithm: sequential sampling decisions are made based on acquisition functions that derive from an instrumental Gaussian process prior. Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the Rpackage GPGame available on CRAN at https://cran.r-project.org/package=GPGame.
Tasks
Published	2019-02-18
URL	https://arxiv.org/abs/1902.06565v2
PDF	https://arxiv.org/pdf/1902.06565v2.pdf
PWC	https://paperswithcode.com/paper/the-kalai-smorodinski-solution-for-many
Repo
Framework

Detection of Alzheimers Disease from MRI using Convolutional Neural Networks, Exploring Transfer Learning And BellCNN


Title	Detection of Alzheimers Disease from MRI using Convolutional Neural Networks, Exploring Transfer Learning And BellCNN
Authors	GuruRaj Awate
Abstract	There is a need for automatic diagnosis of certain diseases from medical images that could help medical practitioners for further assessment towards treating the illness. Alzheimers disease is a good example of a disease that is often misdiagnosed. Alzheimers disease (Hear after referred to as AD), is caused by atrophy of certain brain regions and by brain cell death and is the leading cause of dementia and memory loss [1]. MRI scans reveal this information but atrophied regions are different for different individuals which makes the diagnosis a bit more trickier and often gets misdiagnosed [1, 13]. We believe that our approach to this particular problem would improve the assessment quality by pre-flagging the images which are more likely to have AD. We propose two solutions to this; one with transfer learning [9] and other by BellCNN [14], a custom made Convolutional Neural Network (Hear after referred to as CNN). Advantages and disadvantages of each approach will also be discussed in their respective sections. The dataset used for this project is provided by Open Access Series of Imaging Studies (Hear after referred to as OASIS) [2, 3, 4], which contains over 400 subjects, 100 of whom have mild to severe dementia. The dataset has labeled these subjects by two standards of diagnosis; MiniMental State Examination (Hear after referred to as MMSE) and Clinical Dementia Rating (Hear after referred to as CDR). These are some of the general tools and concepts which are prerequisites to our solution; CNN [5, 6], Neural Networks [10] (Hear after referred to as NN), Anaconda bundle for python, Regression, Tensorflow [7]. Keywords: Alzheimers Disease, Convolutional Neural Network, BellCNN, Image Recognition, Machine Learning, MRI, OASIS, Tensorflow
Tasks	Transfer Learning
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10231v1
PDF	http://arxiv.org/pdf/1901.10231v1.pdf
PWC	https://paperswithcode.com/paper/detection-of-alzheimers-disease-from-mri
Repo
Framework

Collaborative Distillation for Top-N Recommendation


Title	Collaborative Distillation for Top-N Recommendation
Authors	Jae-woong Lee, Minjin Choi, Jongwuk Lee, Hyunjung Shim
Abstract	Knowledge distillation (KD) is a well-known method to reduce inference latency by compressing a cumbersome teacher model to a small student model. Despite the success of KD in the classification task, applying KD to recommender models is challenging due to the sparsity of positive feedback, the ambiguity of missing feedback, and the ranking problem associated with the top-N recommendation. To address the issues, we propose a new KD model for the collaborative filtering approach, namely collaborative distillation (CD). Specifically, (1) we reformulate a loss function to deal with the ambiguity of missing feedback. (2) We exploit probabilistic rank-aware sampling for the top-N recommendation. (3) To train the proposed model effectively, we develop two training strategies for the student model, called the teacher- and the student-guided training methods, selecting the most useful feedback from the teacher model. Via experimental results, we demonstrate that the proposed model outperforms the state-of-the-art method by 2.7-33.2% and 2.7-29.1% in hit rate (HR) and normalized discounted cumulative gain (NDCG), respectively. Moreover, the proposed model achieves the performance comparable to the teacher model.
Tasks
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05276v1
PDF	https://arxiv.org/pdf/1911.05276v1.pdf
PWC	https://paperswithcode.com/paper/collaborative-distillation-for-top-n
Repo
Framework