January 25, 2020

3076 words 15 mins read

Paper Group ANR 1689

Knowledge-Induced Learning with Adaptive Sampling Variational Autoencoders for Open Set Fault Diagnostics. Kernel-Based Ensemble Learning in Python. Combining Q&A Pair Quality and Question Relevance Features on Community-based Question Retrieval. Computation Reallocation for Object Detection. Deep Learning for Automated Classification and Character …

Knowledge-Induced Learning with Adaptive Sampling Variational Autoencoders for Open Set Fault Diagnostics


Title	Knowledge-Induced Learning with Adaptive Sampling Variational Autoencoders for Open Set Fault Diagnostics
Authors	Manuel Arias Chao, Bryan T. Adey, Olga Fink
Abstract	The recent increase in the availability of system condition monitoring data has lead to increases in the use of data-driven approaches for fault diagnostics. The accuracy of the fault detection and classification using these approaches is generally good when abundant labelled data on healthy and faulty system conditions exists and the diagnosis problem is formulated as a supervised learning task, i.e. supervised fault diagnosis. It is, however, relatively common in real situations that only a small fraction of the system condition monitoring data are labeled as healthy and the rest is unlabeled due to the uncertainty of the number and type of faults that may occur. In this case, supervised fault diagnosis performs poorly. Fault diagnosis with an unknown number and nature of faults is an open set learning problem where the knowledge of the faulty system is incomplete during training and the number and extent of the faults, of different types, can evolve during testing. In this paper, we propose to formulate the open set diagnostics problem as a semi-supervised learning problem and we demonstrate how it can be solved using a knowledge-induced learning approach with adaptive sampling variational autoencoders (KIL-AdaVAE) in combination with a one-class classifier. The fault detection and segmentation capability of the proposed method is demonstrated on a simulated case study using the Advanced Geared Turbofan 30000 (AGTF30) dynamical model under real flight conditions and induced faults of 17 fault types. The performance of the method is compared to the different learning strategies (supervised learning, supervised learning with embedding and semi-supervised learning) and deep learning algorithms. The results demonstrate that the proposed method is able to significantly outperform all other tested methods in terms of fault detection and fault segmentation.
Tasks	Fault Detection, One-class classifier, Open Set Learning
Published	2019-12-28
URL	https://arxiv.org/abs/1912.12502v1
PDF	https://arxiv.org/pdf/1912.12502v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-induced-learning-with-adaptive
Repo
Framework

Kernel-Based Ensemble Learning in Python


Title	Kernel-Based Ensemble Learning in Python
Authors	Benjamin Guedj, Bhargav Srinivasa Desikan
Abstract	We propose a new supervised learning algorithm, for classification and regression problems where two or more preliminary predictors are available. We introduce \texttt{KernelCobra}, a non-linear learning strategy for combining an arbitrary number of initial predictors. \texttt{KernelCobra} builds on the COBRA algorithm introduced by \citet{biau2016cobra}, which combined estimators based on a notion of proximity of predictions on the training data. While the COBRA algorithm used a binary threshold to declare which training data were close and to be used, we generalize this idea by using a kernel to better encapsulate the proximity information. Such a smoothing kernel provides more representative weights to each of the training points which are used to build the aggregate and final predictor, and \texttt{KernelCobra} systematically outperforms the COBRA algorithm. While COBRA is intended for regression, \texttt{KernelCobra} deals with classification and regression. \texttt{KernelCobra} is included as part of the open source Python package \texttt{Pycobra} (0.2.4 and onward), introduced by \citet{guedj2018pycobra}. Numerical experiments assess the performance (in terms of pure prediction and computational complexity) of \texttt{KernelCobra} on real-life and synthetic datasets.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08311v1
PDF	https://arxiv.org/pdf/1912.08311v1.pdf
PWC	https://paperswithcode.com/paper/kernel-based-ensemble-learning-in-python
Repo
Framework

Combining Q&A Pair Quality and Question Relevance Features on Community-based Question Retrieval


Title	Combining Q&A Pair Quality and Question Relevance Features on Community-based Question Retrieval
Authors	Dong Li, Lin Li
Abstract	The Q&A community has become an important way for people to access knowledge and information from the Internet. However, the existing translation based on models does not consider the query specific semantics when assigning weights to query terms in question retrieval. So we improve the term weighting model based on the traditional topic translation model and further considering the quality characteristics of question and answer pairs, this paper proposes a communitybased question retrieval method that combines question and answer on quality and question relevance (T2LM+). We have also proposed a question retrieval method based on convolutional neural networks. The results show that Compared with the relatively advanced methods, the two methods proposed in this paper increase MAP by 4.91% and 6.31%.
Tasks
Published	2019-07-03
URL	https://arxiv.org/abs/1907.02031v1
PDF	https://arxiv.org/pdf/1907.02031v1.pdf
PWC	https://paperswithcode.com/paper/combining-qa-pair-quality-and-question
Repo
Framework

Computation Reallocation for Object Detection


Title	Computation Reallocation for Object Detection
Authors	Feng Liang, Chen Lin, Ronghao Guo, Ming Sun, Wei Wu, Junjie Yan, Wanli Ouyang
Abstract	The allocation of computation resources in the backbone is a crucial issue in object detection. However, classification allocation pattern is usually adopted directly to object detector, which is proved to be sub-optimal. In order to reallocate the engaged computation resources in a more efficient way, we present CR-NAS (Computation Reallocation Neural Architecture Search) that can learn computation reallocation strategies across different feature resolution and spatial position diectly on the target detection dataset. A two-level reallocation space is proposed for both stage and spatial reallocation. A novel hierarchical search procedure is adopted to cope with the complex search space. We apply CR-NAS to multiple backbones and achieve consistent improvements. Our CR-ResNet50 and CR-MobileNetV2 outperforms the baseline by 1.9% and 1.7% COCO AP respectively without any additional computation budget. The models discovered by CR-NAS can be equiped to other powerful detection neck/head and be easily transferred to other dataset, e.g. PASCAL VOC, and other vision tasks, e.g. instance segmentation. Our CR-NAS can be used as a plugin to improve the performance of various networks, which is demanding.
Tasks	Instance Segmentation, Neural Architecture Search, Object Detection, Semantic Segmentation
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11234v1
PDF	https://arxiv.org/pdf/1912.11234v1.pdf
PWC	https://paperswithcode.com/paper/computation-reallocation-for-object-detection-1
Repo
Framework

Deep Learning for Automated Classification and Characterization of Amorphous Materials


Title	Deep Learning for Automated Classification and Characterization of Amorphous Materials
Authors	Kirk Swanson, Shubhendu Trivedi, Joshua Lequieu, Kyle Swanson, Risi Kondor
Abstract	It is difficult to quantify structure-property relationships and to identify structural features of complex materials. The characterization of amorphous materials is especially challenging because their lack of long-range order makes it difficult to define structural metrics. In this work, we apply deep learning algorithms to accurately classify amorphous materials and characterize their structural features. Specifically, we show that convolutional neural networks and message passing neural networks can classify two-dimensional liquids and liquid-cooled glasses from molecular dynamics simulations with greater than 0.98 AUC, with no a priori assumptions about local particle relationships, even when the liquids and glasses are prepared at the same inherent structure energy. Furthermore, we demonstrate that message passing neural networks surpass convolutional neural networks in this context in both accuracy and interpretability. We extract a clear interpretation of how message passing neural networks evaluate liquid and glass structures by using a self-attention mechanism. Using this interpretation, we derive three novel structural metrics that accurately characterize glass formation. The methods presented here provide us with a procedure to identify important structural features in materials that could be missed by standard techniques and give us a unique insight into how these neural networks process data.
Tasks
Published	2019-09-10
URL	https://arxiv.org/abs/1909.04648v1
PDF	https://arxiv.org/pdf/1909.04648v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-automated-classification
Repo
Framework

Variational Diffusion Autoencoders with Random Walk Sampling


Title	Variational Diffusion Autoencoders with Random Walk Sampling
Authors	Henry Li, Ofir Lindenbaum, Xiuyuan Cheng, Alexander Cloninger
Abstract	Variational inference (VI) methods and especially variational autoencoders (VAEs) specify scalable generative models that enjoy an intuitive connection to manifold learning — with many default priors the posterior/likelihood pair $q(zx)$/$p(xz)$ can be viewed as an approximate homeomorphism (and its inverse) between the data manifold and a latent Euclidean space. However, these approximations are well-documented to become degenerate in training. Unless the subjective prior is carefully chosen, the topologies of the prior and data distributions often will not match. Conversely, diffusion maps (DM) automatically $\textit{infer}$ the data topology and enjoy a rigorous connection to manifold learning, but do not scale easily or provide the inverse homeomorphism. In this paper, we propose $\textbf{a)}$ a principled measure for recognizing the mismatch between data and latent distributions and $\textbf{b)}$ a method that combines the advantages of variational inference and diffusion maps to learn a homeomorphic generative model. The measure, the$ \textit{locally bi-Lipschitz property}$, is a sufficient condition for a homeomorphism and easy to compute and interpret. The method, the $\textit{variational diffusion autoencoder}$ (VDAE), is a novel generative algorithm that first infers the topology of the data distribution, then models a diffusion random walk over the data. To achieve efficient computation in VDAEs, we use stochastic versions of both variational inference and manifold learning optimization. We prove approximation theoretic results for the dimension dependence of VDAEs, and that locally isotropic sampling in the latent space results in a random walk over the reconstructed manifold. Finally, we demonstrate our method on various real and synthetic datasets, and show that it exhibits performance superior to other generative models.
Tasks
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12724v3
PDF	https://arxiv.org/pdf/1905.12724v3.pdf
PWC	https://paperswithcode.com/paper/diffusion-variational-autoencoders-1
Repo
Framework

Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task


Title	Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task
Authors	Gustavo Aguilar, Fahad AlGhamdi, Victor Soto, Mona Diab, Julia Hirschberg, Thamar Solorio
Abstract	In the third shared task of the Computational Approaches to Linguistic Code-Switching (CALCS) workshop, we focus on Named Entity Recognition (NER) on code-switched social-media data. We divide the shared task into two competitions based on the English-Spanish (ENG-SPA) and Modern Standard Arabic-Egyptian (MSA-EGY) language pairs. We use Twitter data and 9 entity types to establish a new dataset for code-switched NER benchmarks. In addition to the CS phenomenon, the diversity of the entities and the social media challenges make the task considerably hard to process. As a result, the best scores of the competitions are 63.76% and 71.61% for ENG-SPA and MSA-EGY, respectively. We present the scores of 9 participants and discuss the most common challenges among submissions.
Tasks	Named Entity Recognition
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04138v1
PDF	https://arxiv.org/pdf/1906.04138v1.pdf
PWC	https://paperswithcode.com/paper/named-entity-recognition-on-code-switched-2
Repo
Framework

Hypernetwork functional image representation


Title	Hypernetwork functional image representation
Authors	Sylwester Klocek, Łukasz Maziarka, Maciej Wołczyk, Jacek Tabor, Jakub Nowak, Marek Śmieja
Abstract	Motivated by the human way of memorizing images we introduce their functional representation, where an image is represented by a neural network. For this purpose, we construct a hypernetwork which takes an image and returns weights to the target network, which maps point from the plane (representing positions of the pixel) into its corresponding color in the image. Since the obtained representation is continuous, one can easily inspect the image at various resolutions and perform on it arbitrary continuous operations. Moreover, by inspecting interpolations we show that such representation has some properties characteristic to generative models. To evaluate the proposed mechanism experimentally, we apply it to image super-resolution problem. Despite using a single model for various scaling factors, we obtained results comparable to existing super-resolution methods.
Tasks	Image Super-Resolution, Super-Resolution
Published	2019-02-27
URL	https://arxiv.org/abs/1902.10404v3
PDF	https://arxiv.org/pdf/1902.10404v3.pdf
PWC	https://paperswithcode.com/paper/multi-task-hypernetworks
Repo
Framework

Variable Rate Deep Image Compression With a Conditional Autoencoder


Title	Variable Rate Deep Image Compression With a Conditional Autoencoder
Authors	Yoojin Choi, Mostafa El-Khamy, Jungwon Lee
Abstract	In this paper, we propose a novel variable-rate learned image compression framework with a conditional autoencoder. Previous learning-based image compression methods mostly require training separate networks for different compression rates so they can yield compressed images of varying quality. In contrast, we train and deploy only one variable-rate image compression network implemented with a conditional autoencoder. We provide two rate control parameters, i.e., the Lagrange multiplier and the quantization bin size, which are given as conditioning variables to the network. Coarse rate adaptation to a target is performed by changing the Lagrange multiplier, while the rate can be further fine-tuned by adjusting the bin size used in quantizing the encoded representation. Our experimental results show that the proposed scheme provides a better rate-distortion trade-off than the traditional variable-rate image compression codecs such as JPEG2000 and BPG. Our model also shows comparable and sometimes better performance than the state-of-the-art learned image compression models that deploy multiple networks trained for varying rates.
Tasks	Image Compression, Quantization
Published	2019-09-11
URL	https://arxiv.org/abs/1909.04802v1
PDF	https://arxiv.org/pdf/1909.04802v1.pdf
PWC	https://paperswithcode.com/paper/variable-rate-deep-image-compression-with-a
Repo
Framework

word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement


Title	word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement
Authors	Aliakbar Panahi, Seyran Saeedi, Tom Arodz
Abstract	Deep learning natural language processing models often use vector word embeddings, such as word2vec or GloVe, to represent words. A discrete sequence of words can be much more easily integrated with downstream neural layers if it is represented as a sequence of continuous vectors. Also, semantic relationships between words, learned from a text corpus, can be encoded in the relative configurations of the embedding vectors. However, storing and accessing embedding vectors for all words in a dictionary requires large amount of space, and may stain systems with limited GPU memory. Here, we used approaches inspired by quantum computing to propose two related methods, {\em word2ket} and {\em word2ketXS}, for storing word embedding matrix during training and inference in a highly efficient way. Our approach achieves a hundred-fold or more reduction in the space required to store the embeddings with almost no relative drop in accuracy in practical natural language processing tasks.
Tasks	Word Embeddings
Published	2019-11-12
URL	https://arxiv.org/abs/1911.04975v3
PDF	https://arxiv.org/pdf/1911.04975v3.pdf
PWC	https://paperswithcode.com/paper/word2ket-space-efficient-word-embeddings-1
Repo
Framework

Deep Neural Network and Data Augmentation Methodology for off-axis iris segmentation in wearable headsets


Title	Deep Neural Network and Data Augmentation Methodology for off-axis iris segmentation in wearable headsets
Authors	Viktor Varkarakis, Shabab Bazrafkan, Peter Corcoran
Abstract	A data augmentation methodology is presented and applied to generate a large dataset of off-axis iris regions and train a low-complexity deep neural network. Although of low complexity the resulting network achieves a high level of accuracy in iris region segmentation for challenging off-axis eye-patches. Interestingly, this network is also shown to achieve high levels of performance for regular, frontal, segmentation of iris regions, comparing favorably with state-of-the-art techniques of significantly higher complexity. Due to its lower complexity, this network is well suited for deployment in embedded applications such as augmented and mixed reality headsets.
Tasks	Data Augmentation, Iris Segmentation
Published	2019-03-01
URL	http://arxiv.org/abs/1903.00389v1
PDF	http://arxiv.org/pdf/1903.00389v1.pdf
PWC	https://paperswithcode.com/paper/deep-neural-network-and-data-augmentation
Repo
Framework

A Novel GAN-based Fault Diagnosis Approach for Imbalanced Industrial Time Series


Title	A Novel GAN-based Fault Diagnosis Approach for Imbalanced Industrial Time Series
Authors	Wenqian Jiang, Cheng Cheng, Beitong Zhou, Guijun Ma, Ye Yuan
Abstract	This paper proposes a novel fault diagnosis approach based on generative adversarial networks (GAN) for imbalanced industrial time series where normal samples are much larger than failure cases. We combine a well-designed feature extractor with GAN to help train the whole network. Aimed at obtaining data distribution and hidden pattern in both original distinguishing features and latent space, the encoder-decoder-encoder three-sub-network is employed in GAN, based on Deep Convolution Generative Adversarial Networks (DCGAN) but without Tanh activation layer and only trained on normal samples. In order to verify the validity and feasibility of our approach, we test it on rolling bearing data from Case Western Reserve University and further verify it on data collected from our laboratory. The results show that our proposed approach can achieve excellent performance in detecting faulty by outputting much larger evaluation scores.
Tasks	Time Series
Published	2019-04-01
URL	http://arxiv.org/abs/1904.00575v1
PDF	http://arxiv.org/pdf/1904.00575v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-gan-based-fault-diagnosis-approach
Repo
Framework

Generalized Speedy Q-learning


Title	Generalized Speedy Q-learning
Authors	Indu John, Chandramouli Kamanchi, Shalabh Bhatnagar
Abstract	In this paper, we derive a generalization of the Speedy Q-learning (SQL) algorithm that was proposed in the Reinforcement Learning (RL) literature to handle slow convergence of Watkins’ Q-learning. In most RL algorithms such as Q-learning, the Bellman equation and the Bellman operator play an important role. It is possible to generalize the Bellman operator using the technique of successive relaxation. We use the generalized Bellman operator to derive a simple and efficient family of algorithms called Generalized Speedy Q-learning (GSQL-w) and analyze its finite time performance. We show that GSQL-w has an improved finite time performance bound compared to SQL for the case when the relaxation parameter w is greater than 1. This improvement is a consequence of the contraction factor of the generalized Bellman operator being less than that of the standard Bellman operator. Numerical experiments are provided to demonstrate the empirical performance of the GSQL-w algorithm.
Tasks	Q-Learning
Published	2019-11-01
URL	https://arxiv.org/abs/1911.00397v2
PDF	https://arxiv.org/pdf/1911.00397v2.pdf
PWC	https://paperswithcode.com/paper/generalized-speedy-q-learning
Repo
Framework

Single-shot 3D shape reconstruction using deep convolutional neural networks


Title	Single-shot 3D shape reconstruction using deep convolutional neural networks
Authors	Hieu Nguyen, Hui Li, Qiang Qiu, Yuzeng Wang, Zhaoyang Wang
Abstract	A robust single-shot 3D shape reconstruction technique integrating the fringe projection profilometry (FPP) technique with the deep convolutional neural networks (CNNs) is proposed in this letter. The input of the proposed technique is a single FPP image, and the training and validation data sets are prepared by using the conventional multi-frequency FPP technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D images to its corresponding 3D shape. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.
Tasks
Published	2019-09-17
URL	https://arxiv.org/abs/1909.07766v1
PDF	https://arxiv.org/pdf/1909.07766v1.pdf
PWC	https://paperswithcode.com/paper/single-shot-3d-shape-reconstruction-using
Repo
Framework

Working Memory Graphs


Title	Working Memory Graphs
Authors	Ricky Loynd, Roland Fernandez, Asli Celikyilmaz, Adith Swaminathan, Matthew Hausknecht
Abstract	Transformers have increasingly outperformed gated RNNs in obtaining new state-of-the-art results on supervised tasks involving text sequences. Inspired by this trend, we study the question of how Transformer-based models can improve the performance of sequential decision-making agents. We present the Working Memory Graph (WMG), an agent that employs multi-head self-attention to reason over a dynamic set of vectors representing observed and recurrent state. We evaluate WMG in three environments featuring factored observation spaces: a Pathfinding environment that requires complex reasoning over past observations, BabyAI gridworld levels that involve text instructions, and Sokoban which emphasizes future planning. We find that the combination of WMG’s Transformer-based architecture with factored observation spaces leads to significant gains in learning efficiency compared to other architectures across all tasks. Our results imply that for environments where it is possible to factorize environment observations, WMG’s Transformer-based architecture can dramatically boost sample efficiency.
Tasks	Decision Making
Published	2019-11-17
URL	https://arxiv.org/abs/1911.07141v2
PDF	https://arxiv.org/pdf/1911.07141v2.pdf
PWC	https://paperswithcode.com/paper/working-memory-graphs
Repo
Framework