January 29, 2020

3385 words 16 mins read

Paper Group ANR 489

Efficient 3D Fully Convolutional Networks for Pulmonary Lobe Segmentation in CT Images. Local block-wise self attention for normal organ segmentation. Spatiotemporal Emotion Recognition using Deep CNN Based on EEG during Music Listening. Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout. Individualized Multilayer Tensor Le …

Efficient 3D Fully Convolutional Networks for Pulmonary Lobe Segmentation in CT Images


Title	Efficient 3D Fully Convolutional Networks for Pulmonary Lobe Segmentation in CT Images
Authors	Hoileong Lee, Tahreema Matin, Fergus Gleeson, Vicente Grau
Abstract	The human lung is a complex respiratory organ, consisting of five distinct anatomic compartments called lobes. Accurate and automatic segmentation of these pulmonary lobes from computed tomography (CT) images is of clinical importance for lung disease assessment and treatment planning. However, this task is challenging due to ambiguous lobar boundaries, anatomical variations and pathological deformations. In this paper, we propose a high-resolution and efficient 3D fully convolutional network to automatically segment the lobes. We refer to the network as Pulmonary Lobe Segmentation Network (PLS-Net), which is designed to efficiently exploit 3D spatial and contextual information from high-resolution volumetric CT images for effective volume-to-volume learning and inference. The PLS-Net is based on an asymmetric encoder-decoder architecture with three novel components: (i) 3D depthwise separable convolutions to improve the network efficiency by factorising each regular 3D convolution into two simpler operations; (ii) dilated residual dense blocks to efficiently expand the receptive field of the network and aggregate multi-scale contextual information for segmentation; and (iii) input reinforcement at each downsampled resolution to compensate for the loss of spatial information due to convolutional and downsampling operations. We evaluated the proposed PLS-Net on a multi-institutional dataset that consists of 210 CT images acquired from patients with a wide range of lung abnormalities. Experimental results show that our PLS-Net achieves state-of-the-art performance with better computational efficiency. Further experiments confirm the effectiveness of each novel component of the PLS-Net.
Tasks	Computed Tomography (CT)
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07474v1
PDF	https://arxiv.org/pdf/1909.07474v1.pdf
PWC	https://paperswithcode.com/paper/efficient-3d-fully-convolutional-networks-for
Repo
Framework

Local block-wise self attention for normal organ segmentation


Title	Local block-wise self attention for normal organ segmentation
Authors	Jue Jiang, Elguindi Sharif, Hyemin Um, Sean Berry, Harini Veeraraghavan
Abstract	We developed a new and computationally simple local block-wise self attention based normal structures segmentation approach applied to head and neck computed tomography (CT) images. Our method uses the insight that normal organs exhibit regularity in their spatial location and inter-relation within images, which can be leveraged to simplify the computations required to aggregate feature information. We accomplish this by using local self attention blocks that pass information between each other to derive the attention map. We show that adding additional attention layers increases the contextual field and captures focused attention from relevant structures. We developed our approach using U-net and compared it against multiple state-of-the-art self attention methods. All models were trained on 48 internal headneck CT scans and tested on 48 CT scans from the external public domain database of computational anatomy dataset. Our method achieved the highest Dice similarity coefficient segmentation accuracy of 0.85$\pm$0.04, 0.86$\pm$0.04 for left and right parotid glands, 0.79$\pm$0.07 and 0.77$\pm$0.05 for left and right submandibular glands, 0.93$\pm$0.01 for mandible and 0.88$\pm$0.02 for the brain stem with the lowest increase of 66.7% computing time per image and 0.15% increase in model parameters compared with standard U-net. The best state-of-the-art method called point-wise spatial attention, achieved \textcolor{black}{comparable accuracy but with 516.7% increase in computing time and 8.14% increase in parameters compared with standard U-net.} Finally, we performed ablation tests and studied the impact of attention block size, overlap of the attention blocks, additional attention layers, and attention block placement on segmentation performance.
Tasks	Computed Tomography (CT)
Published	2019-09-11
URL	https://arxiv.org/abs/1909.05054v1
PDF	https://arxiv.org/pdf/1909.05054v1.pdf
PWC	https://paperswithcode.com/paper/local-block-wise-self-attention-for-normal
Repo
Framework

Spatiotemporal Emotion Recognition using Deep CNN Based on EEG during Music Listening


Title	Spatiotemporal Emotion Recognition using Deep CNN Based on EEG during Music Listening
Authors	Panayu Keelawat, Nattapong Thammasan, Masayuki Numao, Boonserm Kijsirikul
Abstract	Emotion recognition based on EEG has become an active research area. As one of the machine learning models, CNN has been utilized to solve diverse problems including issues in this domain. In this work, a study of CNN and its spatiotemporal feature extraction has been conducted in order to explore capabilities of the model in varied window sizes and electrode orders. Our investigation was conducted in subject-independent fashion. Results have shown that temporal information in distinct window sizes significantly affects recognition performance in both 10-fold and leave-one-subject-out cross validation. Spatial information from varying electrode order has modicum effect on classification. SVM classifier depending on spatiotemporal knowledge on the same dataset was previously employed and compared to these empirical results. Even though CNN and SVM have a homologous trend in window size effect, CNN outperformed SVM using leave-one-subject-out cross validation. This could be caused by different extracted features in the elicitation process.
Tasks	EEG, Emotion Recognition
Published	2019-10-22
URL	https://arxiv.org/abs/1910.09719v1
PDF	https://arxiv.org/pdf/1910.09719v1.pdf
PWC	https://paperswithcode.com/paper/spatiotemporal-emotion-recognition-using-deep
Repo
Framework

Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout


Title	Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout
Authors	Isidro Cortes-Ciriano, Andreas Bender
Abstract	While the use of deep learning in drug discovery is gaining increasing attention, the lack of methods to compute reliable errors in prediction for Neural Networks prevents their application to guide decision making in domains where identifying unreliable predictions is essential, e.g. precision medicine. Here, we present a framework to compute reliable errors in prediction for Neural Networks using Test-Time Dropout and Conformal Prediction. Specifically, the algorithm consists of training a single Neural Network using dropout, and then applying it N times to both the validation and test sets, also employing dropout in this step. Therefore, for each instance in the validation and test sets an ensemble of predictions were generated. The residuals and absolute errors in prediction for the validation set were then used to compute prediction errors for test set instances using Conformal Prediction. We show using 24 bioactivity data sets from ChEMBL 23 that dropout Conformal Predictors are valid (i.e., the fraction of instances whose true value lies within the predicted interval strongly correlates with the confidence level) and efficient, as the predicted confidence intervals span a narrower set of values than those computed with Conformal Predictors generated using Random Forest (RF) models. Lastly, we show in retrospective virtual screening experiments that dropout and RF-based Conformal Predictors lead to comparable retrieval rates of active compounds. Overall, we propose a computationally efficient framework (as only N extra forward passes are required in addition to training a single network) to harness Test-Time Dropout and the Conformal Prediction framework, and to thereby generate reliable prediction errors for deep Neural Networks.
Tasks	Decision Making, Drug Discovery
Published	2019-04-12
URL	http://arxiv.org/abs/1904.06330v1
PDF	http://arxiv.org/pdf/1904.06330v1.pdf
PWC	https://paperswithcode.com/paper/reliable-prediction-errors-for-deep-neural
Repo
Framework

Individualized Multilayer Tensor Learning with An Application in Imaging Analysis


Title	Individualized Multilayer Tensor Learning with An Application in Imaging Analysis
Authors	Xiwei Tang, Xuan Bi, Annie Qu
Abstract	This work is motivated by multimodality breast cancer imaging data, which is quite challenging in that the signals of discrete tumor-associated microvesicles (TMVs) are randomly distributed with heterogeneous patterns. This imposes a significant challenge for conventional imaging regression and dimension reduction models assuming a homogeneous feature structure. We develop an innovative multilayer tensor learning method to incorporate heterogeneity to a higher-order tensor decomposition and predict disease status effectively through utilizing subject-wise imaging features and multimodality information. Specifically, we construct a multilayer decomposition which leverages an individualized imaging layer in addition to a modality-specific tensor structure. One major advantage of our approach is that we are able to efficiently capture the heterogeneous spatial features of signals that are not characterized by a population structure as well as integrating multimodality information simultaneously. To achieve scalable computing, we develop a new bi-level block improvement algorithm. In theory, we investigate both the algorithm convergence property, tensor signal recovery error bound and asymptotic consistency for prediction model estimation. We also apply the proposed method for simulated and human breast cancer imaging data. Numerical results demonstrate that the proposed method outperforms other existing competing methods.
Tasks	Dimensionality Reduction
Published	2019-03-21
URL	http://arxiv.org/abs/1903.08871v1
PDF	http://arxiv.org/pdf/1903.08871v1.pdf
PWC	https://paperswithcode.com/paper/individualized-multilayer-tensor-learning
Repo
Framework

Virtual Representations for Iterative IoT Deployment


Title	Virtual Representations for Iterative IoT Deployment
Authors	Sebastian R. Bader, Maria Maleshkova
Abstract	A central vision of the Internet of Things is the representation of the physical world in a consistent virtual environment. Especially in the context of smart factories the connection of the different, heterogeneous production modules through a digital shop floor promises faster conversion rates, data-driven maintenance or automated machine configurations for use cases, which have not been known at design time. Nevertheless, these scenarios demand IoT representations of all participating machines and components, which requires high installation efforts and hardware adjustments. We propose an incremental process for bringing the shop floor closer to the IoT vision. Currently the majority of systems, components or parts are not yet connected with the internet and might not even provide the possibility to be technically equipped with sensors. However, those could be essential parts for a realistic digital shop floor representation. We, therefore, propose Virtual Representations, which are capable of independently calculating a physical object’s condition by dynamically collecting and interpreting already available data through RESTful Web APIs. The internal logic of such Virtual Representations are further adjustable at runtime, since changes to its respective physical object, its environment or updates to the resource itself should not cause any downtime.
Tasks
Published	2019-03-02
URL	http://arxiv.org/abs/1903.00718v1
PDF	http://arxiv.org/pdf/1903.00718v1.pdf
PWC	https://paperswithcode.com/paper/virtual-representations-for-iterative-iot
Repo
Framework

Dynamic Cell Structure via Recursive-Recurrent Neural Networks


Title	Dynamic Cell Structure via Recursive-Recurrent Neural Networks
Authors	Xin Qian, Matthew Kennedy, Diego Klabjan
Abstract	In a recurrent setting, conventional approaches to neural architecture search find and fix a general model for all data samples and time steps. We propose a novel algorithm that can dynamically search for the structure of cells in a recurrent neural network model. Based on a combination of recurrent and recursive neural networks, our algorithm is able to construct customized cell structures for each data sample and time step, allowing for a more efficient architecture search than existing models. Experiments on three common datasets show that the algorithm discovers high-performance cell architectures and achieves better prediction accuracy compared to the GRU structure for language modelling and sentiment analysis.
Tasks	Language Modelling, Neural Architecture Search, Sentiment Analysis
Published	2019-05-25
URL	https://arxiv.org/abs/1905.10540v1
PDF	https://arxiv.org/pdf/1905.10540v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-cell-structure-via-recursive
Repo
Framework

MVP-Net: Multi-view FPN with Position-aware Attention for Deep Universal Lesion Detection


Title	MVP-Net: Multi-view FPN with Position-aware Attention for Deep Universal Lesion Detection
Authors	Zihao Li, Shu Zhang, Junge Zhang, Kaiqi Huang, Yizhou Wang, Yizhou Yu
Abstract	Universal lesion detection (ULD) on computed tomography (CT) images is an important but underdeveloped problem. Recently, deep learning-based approaches have been proposed for ULD, aiming to learn representative features from annotated CT data. However, the hunger for data of deep learning models and the scarcity of medical annotation hinders these approaches to advance further. In this paper, we propose to incorporate domain knowledge in clinical practice into the model design of universal lesion detectors. Specifically, as radiologists tend to inspect multiple windows for an accurate diagnosis, we explicitly model this process and propose a multi-view feature pyramid network (FPN), where multi-view features are extracted from images rendered with varied window widths and window levels; to effectively combine this multi-view information, we further propose a position-aware attention module. With the proposed model design, the data-hunger problem is relieved as the learning task is made easier with the correctly induced clinical practice prior. We show promising results with the proposed model, achieving an absolute gain of $\mathbf{5.65%}$ (in the sensitivity of FPs@4.0) over the previous state-of-the-art on the NIH DeepLesion dataset.
Tasks	Computed Tomography (CT)
Published	2019-09-10
URL	https://arxiv.org/abs/1909.04247v3
PDF	https://arxiv.org/pdf/1909.04247v3.pdf
PWC	https://paperswithcode.com/paper/mvp-net-multi-view-fpn-with-position-aware
Repo
Framework

Integrating Data and Image Domain Deep Learning for Limited Angle Tomography using Consensus Equilibrium


Title	Integrating Data and Image Domain Deep Learning for Limited Angle Tomography using Consensus Equilibrium
Authors	Muhammad Usman Ghani, W. Clem Karl
Abstract	Computed Tomography (CT) is a non-invasive imaging modality with applications ranging from healthcare to security. It reconstructs cross-sectional images of an object using a collection of projection data collected at different angles. Conventional methods, such as FBP, require that the projection data be uniformly acquired over the complete angular range. In some applications, it is not possible to acquire such data. Security is one such domain where non-rotational scanning configurations are being developed which violate the complete data assumption. Conventional methods produce images from such data that are filled with artifacts. The recent success of deep learning (DL) methods has inspired researchers to post-process these artifact laden images using deep neural networks (DNNs). This approach has seen limited success on real CT problems. Another approach has been to pre-process the incomplete data using DNNs aiming to avoid the creation of artifacts altogether. Due to imperfections in the learning process, this approach can still leave perceptible residual artifacts. In this work, we aim to combine the power of deep learning in both the data and image domains through a two-step process based on the consensus equilibrium (CE) framework. Specifically, we use conditional generative adversarial networks (cGANs) in both the data and the image domain for enhanced performance and efficient computation and combine them through a consensus process. We demonstrate the effectiveness of our approach on a real security CT dataset for a challenging 90 degree limited-angle problem. The same framework can be applied to other limited data problems arising in applications such as electron microscopy, non-destructive evaluation, and medical imaging.
Tasks	Computed Tomography (CT)
Published	2019-08-31
URL	https://arxiv.org/abs/1909.00240v1
PDF	https://arxiv.org/pdf/1909.00240v1.pdf
PWC	https://paperswithcode.com/paper/integrating-data-and-image-domain-deep
Repo
Framework

3D Anchor-Free Lesion Detector on Computed Tomography Scans


Title	3D Anchor-Free Lesion Detector on Computed Tomography Scans
Authors	Ning Zhang, Dechun Wang, Xinzi Sun, Pengfei Zhang, Chenxi Zhang, Yu Cao, Benyuan Liu
Abstract	Lesions are injuries and abnormal tissues in the human body. Detecting lesions in 3D Computed Tomography (CT) scans can be time-consuming even for very experienced physicians and radiologists. In recent years, CNN based lesion detectors have demonstrated huge potentials. Most of current state-of-the-art lesion detectors employ anchors to enumerate all possible bounding boxes with respect to the dataset in process. This anchor mechanism greatly improves the detection performance while also constraining the generalization ability of detectors. In this paper, we propose an anchor-free lesion detector. The anchor mechanism is removed and lesions are formalized as single keypoints. By doing so, we witness a considerable performance gain in terms of both accuracy and inference speed compared with the anchor-based baseline
Tasks	Computed Tomography (CT)
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11324v1
PDF	https://arxiv.org/pdf/1908.11324v1.pdf
PWC	https://paperswithcode.com/paper/3d-anchor-free-lesion-detector-on-computed
Repo
Framework

Perturbation theory approach to study the latent space degeneracy of Variational Autoencoders


Title	Perturbation theory approach to study the latent space degeneracy of Variational Autoencoders
Authors	Helena Andrés-Terré, Pietro Lió
Abstract	The use of Variational Autoencoders in different Machine Learning tasks has drastically increased in the last years. They have been developed as denoising, clustering and generative tools, highlighting a large potential in a wide range of fields. Their embeddings are able to extract relevant information from highly dimensional inputs, but the converged models can differ significantly and lead to degeneracy on the latent space. We leverage the relation between theoretical physics and machine learning to explain this behaviour, and introduce a new approach to correct for degeneration by using perturbation theory. The re-formulation of the embedding as multi-dimensional generative distribution, allows mapping to a new set of functions and their corresponding energy spectrum. We optimise for a perturbed Hamiltonian, with an additional energy potential that is related to the unobserved topology of the data. Our results show the potential of a new theoretical approach that can be used to interpret the latent space and generative nature of unsupervised learning, while the energy landscapes defined by the perturbations can be further used for modelling and dynamical purposes.
Tasks	Denoising
Published	2019-07-10
URL	https://arxiv.org/abs/1907.05267v1
PDF	https://arxiv.org/pdf/1907.05267v1.pdf
PWC	https://paperswithcode.com/paper/perturbation-theory-approach-to-study-the
Repo
Framework

Active Anomaly Detection for time-domain discoveries


Title	Active Anomaly Detection for time-domain discoveries
Authors	Emille E. O. Ishida, Matwey V. Kornilov, Konstantin L. Malanchev, Maria V. Pruzhinskaya, Alina A. Volnova, Vladimir S. Korolev, Florian Mondon, Sreevarsha Sreejith, Anastasia Malancheva, Shubhomoy Das
Abstract	We present the first application of adaptive machine learning to the identification of anomalies in a data set of non-periodic astronomical light curves. The method follows an active learning strategy where highly informative objects are selected to be labelled. This new information is subsequently used to improve the machine learning model, allowing its accuracy to evolve with the addition of every new classification. For the case of anomaly detection, the algorithm aims to maximize the number of real anomalies presented to the expert by slightly modifying the decision boundary of a traditional isolation forest in each iteration. As a proof of concept, we apply the Active Anomaly Discovery (AAD) algorithm to light curves from the Open Supernova Catalog and compare its results to those of a static Isolation Forest (IF). For both methods, we visually inspected objects within 2% highest anomaly scores. We show that AAD was able to identify 80% more true anomalies than IF. This result is the first evidence that AAD algorithms can play a central role in the search for new physics in the era of large scale sky surveys.
Tasks	Active Learning, Anomaly Detection
Published	2019-09-29
URL	https://arxiv.org/abs/1909.13260v1
PDF	https://arxiv.org/pdf/1909.13260v1.pdf
PWC	https://paperswithcode.com/paper/active-anomaly-detection-for-time-domain
Repo
Framework

Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads


Title	Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads
Authors	Gyanendra Sharma, Richard J Radke
Abstract	In this paper, we present a multi-user interaction interface for a large immersive space that supports simultaneous screen interactions by combining (1) user input via personal smartphones and Bluetooth microphones, (2) spatial tracking via an overhead array of Kinect sensors, and (3) WebSocket interfaces to a webpage running on the large screen. Users are automatically, dynamically assigned personal and shared screen sub-spaces based on their tracked location with respect to the screen, and use a webpage on their personal smartphone for touchpad-type input. We report user experiments using our interaction framework that involve image selection and placement tasks, with the ultimate goal of realizing display-wall environments as viable, interactive workspaces with natural multimodal interfaces.
Tasks
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11751v1
PDF	https://arxiv.org/pdf/1911.11751v1.pdf
PWC	https://paperswithcode.com/paper/multi-person-spatial-interaction-in-a-large
Repo
Framework

Advantage Amplification in Slowly Evolving Latent-State Environments


Title	Advantage Amplification in Slowly Evolving Latent-State Environments
Authors	Martin Mladenov, Ofer Meshi, Jayden Ooi, Dale Schuurmans, Craig Boutilier
Abstract	Latent-state environments with long horizons, such as those faced by recommender systems, pose significant challenges for reinforcement learning (RL). In this work, we identify and analyze several key hurdles for RL in such environments, including belief state error and small action advantage. We develop a general principle of advantage amplification that can overcome these hurdles through the use of temporal abstraction. We propose several aggregation methods and prove they induce amplification in certain settings. We also bound the loss in optimality incurred by our methods in environments where latent state evolves slowly and demonstrate their performance empirically in a stylized user-modeling task.
Tasks	Recommendation Systems
Published	2019-05-29
URL	https://arxiv.org/abs/1905.13559v1
PDF	https://arxiv.org/pdf/1905.13559v1.pdf
PWC	https://paperswithcode.com/paper/advantage-amplification-in-slowly-evolving
Repo
Framework

Data-driven discovery of free-form governing differential equations


Title	Data-driven discovery of free-form governing differential equations
Authors	Steven Atkinson, Waad Subber, Liping Wang, Genghis Khan, Philippe Hawi, Roger Ghanem
Abstract	We present a method of discovering governing differential equations from data without the need to specify a priori the terms to appear in the equation. The input to our method is a dataset (or ensemble of datasets) corresponding to a particular solution (or ensemble of particular solutions) of a differential equation. The output is a human-readable differential equation with parameters calibrated to the individual particular solutions provided. The key to our method is to learn differentiable models of the data that subsequently serve as inputs to a genetic programming algorithm in which graphs specify computation over arbitrary compositions of functions, parameters, and (potentially differential) operators on functions. Differential operators are composed and evaluated using recursive application of automatic differentiation, allowing our algorithm to explore arbitrary compositions of operators without the need for human intervention. We also demonstrate an active learning process to identify and remedy deficiencies in the proposed governing equations.
Tasks	Active Learning
Published	2019-09-27
URL	https://arxiv.org/abs/1910.05117v2
PDF	https://arxiv.org/pdf/1910.05117v2.pdf
PWC	https://paperswithcode.com/paper/data-driven-discovery-of-free-form-governing
Repo
Framework