January 31, 2020

3321 words 16 mins read

Paper Group ANR 6

A Deep-learning Approach for Prognosis of Age-Related Macular Degeneration Disease using SD-OCT Imaging Biomarkers. Numerically Recovering the Critical Points of a Deep Linear Autoencoder. A Comparative Survey of Recent Natural Language Interfaces for Databases. ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Powe …


Title	A Deep-learning Approach for Prognosis of Age-Related Macular Degeneration Disease using SD-OCT Imaging Biomarkers
Authors	Imon Banerjee, Luis de Sisternes, Joelle Hallak, Theodore Leng, Aaron Osborne, Mary Durbin, Daniel Rubin
Abstract	We propose a hybrid sequential deep learning model to predict the risk of AMD progression in non-exudative AMD eyes at multiple timepoints, starting from short-term progression (3-months) up to long-term progression (21-months). Proposed model combines radiomics and deep learning to handle challenges related to imperfect ratio of OCT scan dimension and training cohort size. We considered a retrospective clinical trial dataset that includes 671 fellow eyes with 13,954 dry AMD observations for training and validating the machine learning models on a 10-fold cross validation setting. The proposed RNN model achieved high accuracy (0.96 AUCROC) for the prediction of both short term and long-term AMD progression, and outperformed the traditional random forest model trained. High accuracy achieved by the RNN establishes the ability to identify AMD patients at risk of progressing to advanced AMD at an early stage which could have a high clinical impact as it allows for optimal clinical follow-up, with more frequent screening and potential earlier treatment for those patients at high risk.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10700v1
PDF	http://arxiv.org/pdf/1902.10700v1.pdf
PWC	https://paperswithcode.com/paper/a-deep-learning-approach-for-prognosis-of-age
Repo
Framework

Numerically Recovering the Critical Points of a Deep Linear Autoencoder


Title	Numerically Recovering the Critical Points of a Deep Linear Autoencoder
Authors	Charles G. Frye, Neha S. Wadia, Michael R. DeWeese, Kristofer E. Bouchard
Abstract	Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been explored to gain insight into outstanding questions in optimization, generalization, and network architecture design. However, the degree to which recently-proposed methods for numerically recovering critical points actually do so has not been thoroughly evaluated. In this paper, we examine this issue in a case for which the ground truth is known: the deep linear autoencoder. We investigate two sub-problems associated with numerical critical point identification: first, because of large parameter counts, it is infeasible to find all of the critical points for contemporary neural networks, necessitating sampling approaches whose characteristics are poorly understood; second, the numerical tolerance for accurately identifying a critical point is unknown, and conservative tolerances are difficult to satisfy. We first identify connections between recently-proposed methods and well-understood methods in other fields, including chemical physics, economics, and algebraic geometry. We find that several methods work well at recovering certain information about loss surfaces, but fail to take an unbiased sample of critical points. Furthermore, numerical tolerance must be very strict to ensure that numerically-identified critical points have similar properties to true analytical critical points. We also identify a recently-published Newton method for optimization that outperforms previous methods as a critical point-finding algorithm. We expect our results will guide future attempts to numerically study critical points in large nonlinear neural networks.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10603v1
PDF	http://arxiv.org/pdf/1901.10603v1.pdf
PWC	https://paperswithcode.com/paper/numerically-recovering-the-critical-points-of
Repo
Framework

A Comparative Survey of Recent Natural Language Interfaces for Databases


Title	A Comparative Survey of Recent Natural Language Interfaces for Databases
Authors	Katrin Affolter, Kurt Stockinger, Abraham Bernstein
Abstract	Over the last few years natural language interfaces (NLI) for databases have gained significant traction both in academia and industry. These systems use very different approaches as described in recent survey papers. However, these systems have not been systematically compared against a set of benchmark questions in order to rigorously evaluate their functionalities and expressive power. In this paper, we give an overview over 24 recently developed NLIs for databases. Each of the systems is evaluated using a curated list of ten sample questions to show their strengths and weaknesses. We categorize the NLIs into four groups based on the methodology they are using: keyword-, pattern-, parsing-, and grammar-based NLI. Overall, we learned that keyword-based systems are enough to answer simple questions. To solve more complex questions involving subqueries, the system needs to apply some sort of parsing to identify structural dependencies. Grammar-based systems are overall the most powerful ones, but are highly dependent on their manually designed rules. In addition to providing a systematic analysis of the major systems, we derive lessons learned that are vital for designing NLIs that can answer a wide range of user questions.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.08990v1
PDF	https://arxiv.org/pdf/1906.08990v1.pdf
PWC	https://paperswithcode.com/paper/a-comparative-survey-of-recent-natural
Repo
Framework

ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units using Chebyshev Approximations


Title	ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units using Chebyshev Approximations
Authors	Shanshan Tang, Bo Li, Haijun Yu
Abstract	In a recent paper [B. Li, S. Tang and H. Yu, arXiv:1903.05858], it was shown that deep neural networks built with rectified power units (RePU) can give better approximation for sufficient smooth functions than those with rectified linear units, by converting polynomial approximation given in power series into deep neural networks with optimal complexity and no approximation error. However, in practice, power series are not easy to compute. In this paper, we propose a new and more stable way to construct deep RePU neural networks based on Chebyshev polynomial approximations. By using a hierarchical structure of Chebyshev polynomial approximation in frequency domain, we build efficient and stable deep neural network constructions. In theory, ChebNets and the deep RePU nets based on Power series have the same upper error bounds for general function approximations. But numerically, ChebNets are much more stable. Numerical results show that the constructed ChebNets can be further trained and obtain much better results than those obtained by training deep RePU nets constructed basing on power series.
Tasks
Published	2019-11-07
URL	https://arxiv.org/abs/1911.05467v2
PDF	https://arxiv.org/pdf/1911.05467v2.pdf
PWC	https://paperswithcode.com/paper/chebnet-efficient-and-stable-constructions-of
Repo
Framework

Causal Modeling for Fairness in Dynamical Systems


Title	Causal Modeling for Fairness in Dynamical Systems
Authors	Elliot Creager, David Madras, Toniann Pitassi, Richard Zemel
Abstract	In this work, we present causal directed acyclic graphs (DAGs) as a unifying framework for the recent literature on fairness in dynamical systems. We advocate for the use of causal DAGs as a tool in both designing equitable policies and estimating their impacts. By visualizing models of dynamic unfairness graphically, we expose implicit causal assumptions which can then be more easily interpreted and scrutinized by domain experts. We demonstrate that this method of reinterpretation can be used to critique the robustness of an existing model/policy, or uncover new policy evaluation questions. Causal models also enable a rich set of options for evaluating a new candidate policy without incurring the risk of implementing the policy in the real world. We close the paper with causal analyses of several models from the recent literature, and provide an in-depth case study to demonstrate the utility of causal DAGs for modeling fairness in dynamical systems.
Tasks
Published	2019-09-18
URL	https://arxiv.org/abs/1909.09141v1
PDF	https://arxiv.org/pdf/1909.09141v1.pdf
PWC	https://paperswithcode.com/paper/causal-modeling-for-fairness-in-dynamical
Repo
Framework

Transfer Learning from an Auxiliary Discriminative Task for Unsupervised Anomaly Detection


Title	Transfer Learning from an Auxiliary Discriminative Task for Unsupervised Anomaly Detection
Authors	Urwa Muaz, Stanislav Sobolevsky
Abstract	Unsupervised anomaly detection from high dimensional data like mobility networks is a challenging task. Study of different approaches of feature engineering from such high dimensional data have been a focus of research in this field. This study aims to investigate the transferability of features learned by network classification to unsupervised anomaly detection. We propose use of an auxiliary classification task to extract features from unlabelled data by supervised learning, which can be used for unsupervised anomaly detection. We validate this approach by designing experiments to detect anomalies in mobility network data from New York and Taipei, and compare the results to traditional unsupervised feature learning approaches of PCA and autoencoders. We find that our feature learning approach yields best anomaly detection performance for both datasets, outperforming other studied approaches. This establishes the utility of this approach to feature engineering, which can be applied to other problems of similar nature.
Tasks	Anomaly Detection, Feature Engineering, Transfer Learning, Unsupervised Anomaly Detection
Published	2019-12-05
URL	https://arxiv.org/abs/1912.02864v1
PDF	https://arxiv.org/pdf/1912.02864v1.pdf
PWC	https://paperswithcode.com/paper/transfer-learning-from-an-auxiliary
Repo
Framework

Siamese Attentional Keypoint Network for High Performance Visual Tracking


Title	Siamese Attentional Keypoint Network for High Performance Visual Tracking
Authors	Peng Gao, Ruyue Yuan, Fei Wang, Liyi Xiao, Hamido Fujita, Yan Zhang
Abstract	In this paper, we investigate the impacts of three main aspects of visual tracking, i.e., the backbone network, the attentional mechanism, and the detection component, and propose a Siamese Attentional Keypoint Network, dubbed SATIN, for efficient tracking and accurate localization. Firstly, a new Siamese lightweight hourglass network is specially designed for visual tracking. It takes advantage of the benefits of the repeated bottom-up and top-down inference to capture more global and local contextual information at multiple scales. Secondly, a novel cross-attentional module is utilized to leverage both channel-wise and spatial intermediate attentional information, which can enhance both discriminative and localization capabilities of feature maps. Thirdly, a keypoints detection approach is invented to trace any target object by detecting the top-left corner point, the centroid point, and the bottom-right corner point of its bounding box. Therefore, our SATIN tracker not only has a strong capability to learn more effective object representations, but also is computational and memory storage efficiency, either during the training or testing stages. To the best of our knowledge, we are the first to propose this approach. Without bells and whistles, experimental results demonstrate that our approach achieves state-of-the-art performance on several recent benchmark datasets, at a speed far exceeding 27 frames per second.
Tasks	Visual Tracking
Published	2019-04-23
URL	https://arxiv.org/abs/1904.10128v2
PDF	https://arxiv.org/pdf/1904.10128v2.pdf
PWC	https://paperswithcode.com/paper/siamese-attentional-keypoint-network-for-high
Repo
Framework

BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services


Title	BottleNet: A Deep Learning Architecture for Intelligent Mobile Cloud Computing Services
Authors	Amir Erfan Eshratifar, Amirhossein Esmaili, Massoud Pedram
Abstract	Recent studies have shown the latency and energy consumption of deep neural networks can be significantly improved by splitting the network between the mobile device and cloud. This paper introduces a new deep learning architecture, called BottleNet, for reducing the feature size needed to be sent to the cloud. Furthermore, we propose a training method for compensating for the potential accuracy loss due to the lossy compression of features before transmitting them to the cloud. BottleNet achieves on average 30x improvement in end-to-end latency and 40x improvement in mobile energy consumption compared to the cloud-only approach with negligible accuracy loss.
Tasks
Published	2019-02-04
URL	http://arxiv.org/abs/1902.01000v1
PDF	http://arxiv.org/pdf/1902.01000v1.pdf
PWC	https://paperswithcode.com/paper/bottlenet-a-deep-learning-architecture-for
Repo
Framework

Predicting Next-Season Designs on High Fashion Runway


Title	Predicting Next-Season Designs on High Fashion Runway
Authors	Yusan Lin, Hao Yang
Abstract	Fashion is a large and fast-changing industry. Foreseeing the upcoming fashion trends is beneficial for fashion designers, consumers, and retailers. However, fashion trends are often perceived as unpredictable due to the enormous amount of factors involved into designers’ subjectivity. In this paper, we propose a fashion trend prediction framework and design neural network models to leverage structured fashion runway show data, learn the fashion collection embedding, and further train RNN/LSTM models to capture the designers’ style evolution. Our proposed framework consists of (1) a runway embedding learning model that uses fashion runway images to learn every season’s collection embedding, and (2) a next-season fashion design prediction model that leverage the concept of designer style and trend to predict next-season design given designers. Through experiments on a collected dataset across 32 years of fashion shows, our framework can achieve the best performance of 78.42% AUC on average and 95% for an individual designer when predicting the next season’s design.
Tasks
Published	2019-07-16
URL	https://arxiv.org/abs/1907.07161v1
PDF	https://arxiv.org/pdf/1907.07161v1.pdf
PWC	https://paperswithcode.com/paper/predicting-next-season-designs-on-high
Repo
Framework

An Attribute Oriented Induction based Methodology for Data Driven Predictive Maintenance


Title	An Attribute Oriented Induction based Methodology for Data Driven Predictive Maintenance
Authors	Javier Fernandez-Anakabe, Ekhi Zugasti Uriguen, Urko Zurutuza Ortega
Abstract	Attribute Oriented Induction (AOI) is a data mining algorithm used for extracting knowledge of relational data, taking into account expert knowledge. It is a clustering algorithm that works by transforming the values of the attributes and converting an instance into others that are more generic or ambiguous. In this way, it seeks similarities between elements to generate data groupings. AOI was initially conceived as an algorithm for knowledge discovery in databases, but over the years it has been applied to other areas such as spatial patterns, intrusion detection or strategy making. In this paper, AOI has been extended to the field of Predictive Maintenance. The objective is to demonstrate that combining expert knowledge and data collected from the machine can provide good results in the Predictive Maintenance of industrial assets. To this end we adapted the algorithm and used an LSTM approach to perform both the Anomaly Detection (AD) and the Remaining Useful Life (RUL). The results obtained confirm the validity of the proposal, as the methodology was able to detect anomalies, and calculate the RUL until breakage with considerable degree of accuracy.
Tasks	Anomaly Detection, Intrusion Detection
Published	2019-12-02
URL	https://arxiv.org/abs/1912.00662v1
PDF	https://arxiv.org/pdf/1912.00662v1.pdf
PWC	https://paperswithcode.com/paper/an-attribute-oriented-induction-based
Repo
Framework

Learning Wi-Fi Connection Loss Predictions for Seamless Vertical Handovers Using Multipath TCP


Title	Learning Wi-Fi Connection Loss Predictions for Seamless Vertical Handovers Using Multipath TCP
Authors	Jonas Höchst, Artur Sterz, Alexander Frömmgen, Denny Stohr, Ralf Steinmetz, Bernd Freisleben
Abstract	We present a novel data-driven approach to perform smooth Wi-Fi/cellular handovers on smartphones. Our approach relies on data provided by multiple smartphone sensors (e.g., Wi-Fi RSSI, acceleration, compass, step counter, air pressure) to predict Wi-Fi connection loss and uses Multipath TCP to dynamically switch between different connectivity modes. We train a random forest classifier and an artificial neural network on real-world sensor data collected by five smartphone users over a period of three months. The trained models are executed on smartphones to reliably predict Wi-Fi connection loss 15 seconds ahead of time, with a precision of up to 0.97 and a recall of up to 0.98. Furthermore, we present results for four DASH video streaming experiments that run on a Nexus 5 smartphone using available Wi-Fi/cellular networks. The neural network predictions for Wi-Fi connection loss are used to establish MPTCP subflows on the cellular link. The experiments show that our approach provides seamless wireless connectivity, improves quality of experience of DASH video streaming, and requires less cellular data compared to handover approaches without Wi-Fi connection loss predictions.
Tasks
Published	2019-07-24
URL	https://arxiv.org/abs/1907.10493v1
PDF	https://arxiv.org/pdf/1907.10493v1.pdf
PWC	https://paperswithcode.com/paper/learning-wi-fi-connection-loss-predictions
Repo
Framework

Neural Hybrid Recommender: Recommendation needs collaboration


Title	Neural Hybrid Recommender: Recommendation needs collaboration
Authors	Ezgi Yıldırım, Payam Azad, Şule Gündüz Öğüdücü
Abstract	In recent years, deep learning has gained an indisputable success in computer vision, speech recognition, and natural language processing. After its rising success on these challenging areas, it has been studied on recommender systems as well, but mostly to include content features into traditional methods. In this paper, we introduce a generalized neural network-based recommender framework that is easily extendable by additional networks. This framework named NHR, short for Neural Hybrid Recommender allows us to include more elaborate information from the same and different data sources. We have worked on item prediction problems, but the framework can be used for rating prediction problems as well with a single change on the loss function. To evaluate the effect of such a framework, we have tested our approach on benchmark and not yet experimented datasets. The results in these real-world datasets show the superior performance of our approach in comparison with the state-of-the-art methods.
Tasks	Recommendation Systems, Speech Recognition
Published	2019-09-29
URL	https://arxiv.org/abs/1909.13330v1
PDF	https://arxiv.org/pdf/1909.13330v1.pdf
PWC	https://paperswithcode.com/paper/neural-hybrid-recommender-recommendation
Repo
Framework

FNHSM_HRS: Hybrid recommender system using fuzzy clustering and heuristic similarity measure


Title	FNHSM_HRS: Hybrid recommender system using fuzzy clustering and heuristic similarity measure
Authors	Mostafa Khalaji, Chitra Dadkhah
Abstract	Nowadays, Recommender Systems have become a comprehensive system for helping and guiding users in a huge amount of data on the Internet. Collaborative Filtering offers to active users based on the rating of a set of users. One of the simplest and most comprehensible and successful models is to find users with a taste in recommender systems. In this model, with increasing number of users and items, the system is faced to scalability problem. On the other hand, improving system performance when there is little information available from ratings, that is important. In this paper, a hybrid recommender system called FNHSM_HRS which is based on the new heuristic similarity measure (NHSM) along with a fuzzy clustering is presented. Using the fuzzy clustering method in the proposed system improves the scalability problem and increases the accuracy of system recommendations. The proposed system is based on the collaborative filtering model and is partnered with the heuristic similarity measure to improve the system’s performance and accuracy. The evaluation of the proposed system based results on the MovieLens dataset carried out the results using MAE, Recall, Precision and Accuracy measures Indicating improvement in system performance and increasing the accuracy of recommendation to collaborative filtering methods which use other measures to find similarities.
Tasks	Recommendation Systems
Published	2019-09-26
URL	https://arxiv.org/abs/1909.13765v1
PDF	https://arxiv.org/pdf/1909.13765v1.pdf
PWC	https://paperswithcode.com/paper/fnhsm_hrs-hybrid-recommender-system-using
Repo
Framework

Let’s Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation


Title	Let’s Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation
Authors	Tommaso Cavallari, Luca Bertinetto, Jishnu Mukhoti, Philip Torr, Stuart Golodetz
Abstract	Many applications require a camera to be relocalised online, without expensive offline training on the target scene. Whilst both keyframe and sparse keypoint matching methods can be used online, the former often fail away from the training trajectory, and the latter can struggle in textureless regions. By contrast, scene coordinate regression (SCoRe) methods generalise to novel poses and can leverage dense correspondences to improve robustness, and recent work has shown how to adapt SCoRe forests between scenes, allowing their state-of-the-art performance to be leveraged online. However, because they use features hand-crafted for indoor use, they do not generalise well to harder outdoor scenes. Whilst replacing the forest with a neural network and learning suitable features for outdoor use is possible, the techniques used to adapt forests between scenes are unfortunately harder to transfer to a network context. In this paper, we address this by proposing a novel way of leveraging a network trained on one scene to predict points in another scene. Our approach replaces the appearance clustering performed by the branching structure of a regression forest with a two-step process that first uses the network to predict points in the original scene, and then uses these predicted points to look up clusters of points from the new scene. We show experimentally that our online approach achieves state-of-the-art performance on both the 7-Scenes and Cambridge Landmarks datasets, whilst running in under 300ms, making it highly effective in live scenarios.
Tasks	Camera Relocalization
Published	2019-06-20
URL	https://arxiv.org/abs/1906.08744v1
PDF	https://arxiv.org/pdf/1906.08744v1.pdf
PWC	https://paperswithcode.com/paper/lets-take-this-online-adapting-scene
Repo
Framework

Maximum Mean Discrepancy Gradient Flow


Title	Maximum Mean Discrepancy Gradient Flow
Authors	Michael Arbel, Anna Korba, Adil Salim, Arthur Gretton
Abstract	We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties. The MMD is an integral probability metric defined for a reproducing kernel Hilbert space (RKHS), and serves as a metric on probability measures for a sufficiently rich RKHS. We obtain conditions for convergence of the gradient flow towards a global optimum, that can be related to particle transport when optimizing neural networks. We also propose a way to regularize this MMD flow, based on an injection of noise in the gradient. This algorithmic fix comes with theoretical and empirical evidence. The practical implementation of the flow is straightforward, since both the MMD and its gradient have simple closed-form expressions, which can be easily estimated with samples.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04370v2
PDF	https://arxiv.org/pdf/1906.04370v2.pdf
PWC	https://paperswithcode.com/paper/maximum-mean-discrepancy-gradient-flow
Repo
Framework