Paper Group ANR 553
Flexible Mining of Prefix Sequences from Time-Series Traces. Multi-modal segmentation with missing MR sequences using pre-trained fusion networks. Affine Self Convolution. Analyzing the Structure of Attention in a Transformer Language Model. Outcome-Driven Clustering of Acute Coronary Syndrome Patients using Multi-Task Neural Network with Attention …
Flexible Mining of Prefix Sequences from Time-Series Traces
| Title | Flexible Mining of Prefix Sequences from Time-Series Traces |
| Authors | Antonio Anastasio Bruto da Costa, Goran Frehse, Pallab Dasgupta |
| Abstract | Mining temporal assertions from time-series data using information theory to filter real properties from incidental ones is a practically significant challenge. The problem is complex for continuous or hybrid systems because the degrees of influence on a consequent from a timed-sequence of predicates (called its prefix sequence), varies continuously over dense time intervals. We propose a parameterized method that uses interval arithmetic for flexibly learning prefix sequences having influence on a defined consequent over various time scales and predicates over system variables. |
| Tasks | Time Series |
| Published | 2019-05-29 |
| URL | https://arxiv.org/abs/1905.12262v1 |
| https://arxiv.org/pdf/1905.12262v1.pdf | |
| PWC | https://paperswithcode.com/paper/flexible-mining-of-prefix-sequences-from-time |
| Repo | |
| Framework | |
Multi-modal segmentation with missing MR sequences using pre-trained fusion networks
| Title | Multi-modal segmentation with missing MR sequences using pre-trained fusion networks |
| Authors | Karin van Garderen, Marion Smits, Stefan Klein |
| Abstract | Missing data is a common problem in machine learning and in retrospective imaging research it is often encountered in the form of missing imaging modalities. We propose to take into account missing modalities in the design and training of neural networks, to ensure that they are capable of providing the best possible prediction even when multiple images are not available. The proposed network combines three modifications to the standard 3D UNet architecture: a training scheme with dropout of modalities, a multi-pathway architecture with fusion layer in the final stage, and the separate pre-training of these pathways. These modifications are evaluated incrementally in terms of performance on full and missing data, using the BraTS multi-modal segmentation challenge. The final model shows significant improvement with respect to the state of the art on missing data and requires less memory during training. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11464v1 |
| https://arxiv.org/pdf/1909.11464v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-modal-segmentation-with-missing-mr |
| Repo | |
| Framework | |
Affine Self Convolution
| Title | Affine Self Convolution |
| Authors | Nichita Diaconu, Daniel E Worrall |
| Abstract | Attention mechanisms, and most prominently self-attention, are a powerful building block for processing not only text but also images. These provide a parameter efficient method for aggregating inputs. We focus on self-attention in vision models, and we combine it with convolution, which as far as we know, are the first to do. What emerges is a convolution with data dependent filters. We call this an Affine Self Convolution. While this is applied differently at each spatial location, we show that it is translation equivariant. We also modify the Squeeze and Excitation variant of attention, extending both variants of attention to the roto-translation group. We evaluate these new models on CIFAR10 and CIFAR100 and show an improvement in the number of parameters, while reaching comparable or higher accuracy at test time against self-trained baselines. |
| Tasks | |
| Published | 2019-11-18 |
| URL | https://arxiv.org/abs/1911.07704v1 |
| https://arxiv.org/pdf/1911.07704v1.pdf | |
| PWC | https://paperswithcode.com/paper/affine-self-convolution-1 |
| Repo | |
| Framework | |
Analyzing the Structure of Attention in a Transformer Language Model
| Title | Analyzing the Structure of Attention in a Transformer Language Model |
| Authors | Jesse Vig, Yonatan Belinkov |
| Abstract | The Transformer is a fully attention-based alternative to recurrent networks that has achieved state-of-the-art results across a range of NLP tasks. In this paper, we analyze the structure of attention in a Transformer language model, the GPT-2 small pretrained model. We visualize attention for individual instances and analyze the interaction between attention and syntax over a large corpus. We find that attention targets different parts of speech at different layer depths within the model, and that attention aligns with dependency relations most strongly in the middle layers. We also find that the deepest layers of the model capture the most distant relationships. Finally, we extract exemplar sentences that reveal highly specific patterns targeted by particular attention heads. |
| Tasks | Language Modelling |
| Published | 2019-06-07 |
| URL | https://arxiv.org/abs/1906.04284v2 |
| https://arxiv.org/pdf/1906.04284v2.pdf | |
| PWC | https://paperswithcode.com/paper/analyzing-the-structure-of-attention-in-a |
| Repo | |
| Framework | |
Outcome-Driven Clustering of Acute Coronary Syndrome Patients using Multi-Task Neural Network with Attention
| Title | Outcome-Driven Clustering of Acute Coronary Syndrome Patients using Multi-Task Neural Network with Attention |
| Authors | Eryu Xia, Xin Du, Jing Mei, Wen Sun, Suijun Tong, Zhiqing Kang, Jian Sheng, Jian Li, Changsheng Ma, Jianzeng Dong, Shaochun Li |
| Abstract | Cluster analysis aims at separating patients into phenotypically heterogenous groups and defining therapeutically homogeneous patient subclasses. It is an important approach in data-driven disease classification and subtyping. Acute coronary syndrome (ACS) is a syndrome due to sudden decrease of coronary artery blood flow, where disease classification would help to inform therapeutic strategies and provide prognostic insights. Here we conducted outcome-driven cluster analysis of ACS patients, which jointly considers treatment and patient outcome as indicators for patient state. Multi-task neural network with attention was used as a modeling framework, including learning of the patient state, cluster analysis, and feature importance profiling. Seven patient clusters were discovered. The clusters have different characteristics, as well as different risk profiles to the outcome of in-hospital major adverse cardiac events. The results demonstrate cluster analysis using outcome-driven multi-task neural network as promising for patient classification and subtyping. |
| Tasks | Feature Importance |
| Published | 2019-03-01 |
| URL | http://arxiv.org/abs/1903.00197v2 |
| http://arxiv.org/pdf/1903.00197v2.pdf | |
| PWC | https://paperswithcode.com/paper/outcome-driven-clustering-of-acute-coronary |
| Repo | |
| Framework | |
Petrophysical Property Estimation from Seismic Data Using Recurrent Neural Networks
| Title | Petrophysical Property Estimation from Seismic Data Using Recurrent Neural Networks |
| Authors | Motaz Alfarraj, Ghassan AlRegib |
| Abstract | Reservoir characterization involves the estimation petrophysical properties from well-log data and seismic data. Estimating such properties is a challenging task due to the non-linearity and heterogeneity of the subsurface. Various attempts have been made to estimate petrophysical properties using machine learning techniques such as feed-forward neural networks and support vector regression (SVR). Recent advances in machine learning have shown promising results for recurrent neural networks (RNN) in modeling complex sequential data such as videos and speech signals. In this work, we propose an algorithm for property estimation from seismic data using recurrent neural networks. An applications of the proposed workflow to estimate density and p-wave impedance using seismic data shows promising results compared to feed-forward neural networks. |
| Tasks | |
| Published | 2019-01-24 |
| URL | http://arxiv.org/abs/1901.08623v2 |
| http://arxiv.org/pdf/1901.08623v2.pdf | |
| PWC | https://paperswithcode.com/paper/petrophysical-property-estimation-from |
| Repo | |
| Framework | |
Privacy Preserving Threat Hunting in Smart Home Environments
| Title | Privacy Preserving Threat Hunting in Smart Home Environments |
| Authors | Ahmed M. Elmisery, Mirela Sertovic |
| Abstract | The recent proliferation of smart home environments offers new and transformative circumstances for various domains with a commitment to enhancing the quality of life and experience. Most of these environments combine different gadgets offered by multiple stakeholders in a dynamic and decentralized manner, which in turn presents new challenges from the perspective of digital investigation. In addition, a plentiful amount of data records got generated because of the day to day interactions between these gadgets and homeowners, which poses difficulty in managing and analyzing such data. The analysts should endorse new digital investigation approaches to tackle the current limitations in traditional approaches when used in these environments. The digital evidence in such environments can be found inside the records of logfiles that store the historical events occurred inside the smart home. Threat hunting can leverage the collective nature of these gadgets to gain deeper insights into the best way for responding to new threats, which in turn can be valuable in reducing the impact of breaches. Nevertheless, this approach depends mainly on the readiness of smart homeowners to share their own personal usage logs that have been extracted from their smart home environments. However, they might disincline to employ such service due to the sensitive nature of the information logged by their personal gateways. In this paper, we presented an approach to enable smart homeowners to share their usage logs in a privacy preserving manner. A distributed threat hunting approach has been developed to permit the composition of diverse threat classes without revealing the logged records to other involved parties. Furthermore, a scenario was proposed to depict a proactive threat Intelligence sharing for the detection of potential threats in smart home environments with some experimental results. |
| Tasks | |
| Published | 2019-11-06 |
| URL | https://arxiv.org/abs/1911.02174v2 |
| https://arxiv.org/pdf/1911.02174v2.pdf | |
| PWC | https://paperswithcode.com/paper/privacy-preserving-threat-hunting-in-smart |
| Repo | |
| Framework | |
Partial Or Complete, That’s The Question
| Title | Partial Or Complete, That’s The Question |
| Authors | Qiang Ning, Hangfeng He, Chuchu Fan, Dan Roth |
| Abstract | For many structured learning tasks, the data annotation process is complex and costly. Existing annotation schemes usually aim at acquiring completely annotated structures, under the common perception that partial structures are of low quality and could hurt the learning process. This paper questions this common perception, motivated by the fact that structures consist of interdependent sets of variables. Thus, given a fixed budget, partly annotating each structure may provide the same level of supervision, while allowing for more structures to be annotated. We provide an information theoretic formulation for this perspective and use it, in the context of three diverse structured learning tasks, to show that learning from partial structures can sometimes outperform learning from complete ones. Our findings may provide important insights into structured data annotation schemes and could support progress in learning protocols for structured tasks. |
| Tasks | |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.04937v1 |
| https://arxiv.org/pdf/1906.04937v1.pdf | |
| PWC | https://paperswithcode.com/paper/partial-or-complete-thats-the-question-1 |
| Repo | |
| Framework | |
An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves
| Title | An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves |
| Authors | Christian Pieringer, Karim Pichara, Márcio Catelán, Pavlos Protopapas |
| Abstract | Within the last years, the classification of variable stars with Machine Learning has become a mainstream area of research. Recently, visualization of time series is attracting more attention in data science as a tool to visually help scientists to recognize significant patterns in complex dynamics. Within the Machine Learning literature, dictionary-based methods have been widely used to encode relevant parts of image data. These methods intrinsically assign a degree of importance to patches in pictures, according to their contribution in the image reconstruction. Inspired by dictionary-based techniques, we present an approach that naturally provides the visualization of salient parts in astronomical light curves, making the analogy between image patches and relevant pieces in time series. Our approach encodes the most meaningful patterns such that we can approximately reconstruct light curves by just using the encoded information. We test our method in light curves from the OGLE-III and StarLight databases. Our results show that the proposed model delivers an automatic and intuitive visualization of relevant light curve parts, such as local peaks and drops in magnitude. |
| Tasks | Classification Of Variable Stars, Image Reconstruction, Time Series |
| Published | 2019-03-08 |
| URL | http://arxiv.org/abs/1903.03254v1 |
| http://arxiv.org/pdf/1903.03254v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-algorithm-for-the-visualization-of |
| Repo | |
| Framework | |
Robust Deep Ordinal Regression Under Label Noise
| Title | Robust Deep Ordinal Regression Under Label Noise |
| Authors | Bhanu Garg, Naresh Manwani |
| Abstract | The real-world data is often susceptible to label noise, which might constrict the effectiveness of the existing state of the art algorithms for ordinal regression. Existing works on ordinal regression do not take label noise into account. We propose a theoretically grounded approach for class conditional label noise in ordinal regression problems. We present a deep learning implementation of two commonly used loss functions for ordinal regression that is both - 1) robust to label noise, and 2) rank consistent for a good ranking rule. We verify these properties of the algorithm empirically and show robustness to label noise on real data and rank consistency. To the best of our knowledge, this is the first approach for robust ordinal regression models. |
| Tasks | |
| Published | 2019-12-07 |
| URL | https://arxiv.org/abs/1912.03488v2 |
| https://arxiv.org/pdf/1912.03488v2.pdf | |
| PWC | https://paperswithcode.com/paper/robust-deep-ordinal-regression-under-label |
| Repo | |
| Framework | |
RL-Based User Association and Resource Allocation for Multi-UAV enabled MEC
| Title | RL-Based User Association and Resource Allocation for Multi-UAV enabled MEC |
| Authors | Liang Wang, Peiqiu Huang, Kezhi Wang, Guopeng Zhang, Lei Zhang, Nauman Aslam, Kun Yang |
| Abstract | In this paper, multi-unmanned aerial vehicle (UAV) enabled mobile edge computing (MEC), i.e., UAVE is studied, where several UAVs are deployed as flying MEC platform to provide computing resource to ground user equipments (UEs). Compared to the traditional fixed location MEC, UAV enabled MEC (i.e., UAVE) is particular useful in case of temporary events, emergency situations and on-demand services, due to its high flexibility, low cost and easy deployment features. However, operation of UAVE faces several challenges, two of which are how to achieve both 1) the association between multiple UEs and UAVs and 2) the resource allocation from UAVs to UEs, while minimizing the energy consumption for all the UEs. To address this, we formulate the above problem into a mixed integer nonlinear programming (MINLP), which is difficult to be solved in general, especially in the large-scale scenario. We then propose a Reinforcement Learning (RL)-based user Association and resource Allocation (RLAA) algorithm to tackle this problem efficiently and effectively. Numerical results show that the proposed RLAA can achieve the optimal performance with comparison to the exhaustive search in small scale, and have considerable performance gain over other typical algorithms in large-scale cases. |
| Tasks | |
| Published | 2019-04-08 |
| URL | http://arxiv.org/abs/1904.07961v1 |
| http://arxiv.org/pdf/1904.07961v1.pdf | |
| PWC | https://paperswithcode.com/paper/190407961 |
| Repo | |
| Framework | |
On the Adversarial Robustness of Multivariate Robust Estimation
| Title | On the Adversarial Robustness of Multivariate Robust Estimation |
| Authors | Erhan Bayraktar, Lifeng Lai |
| Abstract | In this paper, we investigate the adversarial robustness of multivariate $M$-Estimators. In the considered model, after observing the whole dataset, an adversary can modify all data points with the goal of maximizing inference errors. We use adversarial influence function (AIF) to measure the asymptotic rate at which the adversary can change the inference result. We first characterize the adversary’s optimal modification strategy and its corresponding AIF. From the defender’s perspective, we would like to design an estimator that has a small AIF. For the case of joint location and scale estimation problem, we characterize the optimal $M$-estimator that has the smallest AIF. We further identify a tradeoff between robustness against adversarial modifications and robustness against outliers, and derive the optimal $M$-estimator that achieves the best tradeoff. |
| Tasks | |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1903.11220v1 |
| http://arxiv.org/pdf/1903.11220v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-adversarial-robustness-of-multivariate |
| Repo | |
| Framework | |
Cataloging Accreted Stars within Gaia DR2 using Deep Learning
| Title | Cataloging Accreted Stars within Gaia DR2 using Deep Learning |
| Authors | Bryan Ostdiek, Lina Necib, Timothy Cohen, Marat Freytsis, Mariangela Lisanti, Shea Garrison-Kimmel, Andrew Wetzel, Robyn E. Sanderson, Philip F. Hopkins |
| Abstract | The goal of this paper is to develop a machine learning based approach that utilizes phase space alone to separate the Gaia DR2 stars into two categories: those accreted onto the Milky Way from in situ stars that were born within the Galaxy. Traditional selection methods that have been used to identify accreted stars typically rely on full 3D velocity and/or metallicity information, which significantly reduces the number of classifiable stars. The approach advocated here is applicable to a much larger fraction of Gaia DR2. A method known as transfer learning is shown to be effective through extensive testing on a set of mock Gaia catalogs that are based on the FIRE cosmological zoom-in hydrodynamic simulations of Milky Way-mass galaxies. The machine is first trained on simulated data using only 5D kinematics as inputs, and is then further trained on a cross-matched Gaia/RAVE data set, which improves sensitivity to properties of the real Milky Way. The result is a catalog that identifies ~650,000 accreted stars within Gaia DR2. This catalog can yield empirical insights into the merger history of the Milky Way, and could be used to infer properties of the dark matter distribution. |
| Tasks | Transfer Learning |
| Published | 2019-07-15 |
| URL | https://arxiv.org/abs/1907.06652v1 |
| https://arxiv.org/pdf/1907.06652v1.pdf | |
| PWC | https://paperswithcode.com/paper/cataloging-accreted-stars-within-gaia-dr2 |
| Repo | |
| Framework | |
Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training
| Title | Compression of Acoustic Event Detection Models with Low-rank Matrix Factorization and Quantization Training |
| Authors | Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Spyros Matsoukas, Chao Wang |
| Abstract | In this paper, we present a compression approach based on the combination of low-rank matrix factorization and quantization training, to reduce complexity for neural network based acoustic event detection (AED) models. Our experimental results show this combined compression approach is very effective. For a three-layer long short-term memory (LSTM) based AED model, the original model size can be reduced to 1% with negligible loss of accuracy. Our approach enables the feasibility of deploying AED for resource-constraint applications. |
| Tasks | Quantization |
| Published | 2019-05-02 |
| URL | https://arxiv.org/abs/1905.00855v1 |
| https://arxiv.org/pdf/1905.00855v1.pdf | |
| PWC | https://paperswithcode.com/paper/compression-of-acoustic-event-detection |
| Repo | |
| Framework | |
Learning by stochastic serializations
| Title | Learning by stochastic serializations |
| Authors | Pablo Strasser, Stephane Armand, Stephane Marchand-Maillet, Alexandros Kalousis |
| Abstract | Complex structures are typical in machine learning. Tailoring learning algorithms for every structure requires an effort that may be saved by defining a generic learning procedure adaptive to any complex structure. In this paper, we propose to map any complex structure onto a generic form, called serialization, over which we can apply any sequence-based density estimator. We then show how to transfer the learned density back onto the space of original structures. To expose the learning procedure to the structural particularities of the original structures, we take care that the serializations reflect accurately the structures’ properties. Enumerating all serializations is infeasible. We propose an effective way to sample representative serializations from the complete set of serializations which preserves the statistics of the complete set. Our method is competitive or better than state of the art learning algorithms that have been specifically designed for given structures. In addition, since the serialization involves sampling from a combinatorial process it provides considerable protection from overfitting, which we clearly demonstrate on a number of experiments. |
| Tasks | |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1905.11245v1 |
| https://arxiv.org/pdf/1905.11245v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-by-stochastic-serializations |
| Repo | |
| Framework | |
