October 17, 2019

3232 words 16 mins read

Paper Group ANR 820

Building robust prediction models for defective sensor data using Artificial Neural Networks. News-based forecasts of macroeconomic indicators: A semantic path model for interpretable predictions. Evaluation of Neural Networks for Image Recognition Applications: Designing a 0-1 MILP Model of a CNN to create adversarials. RELF: Robust Regression Ext …

Building robust prediction models for defective sensor data using Artificial Neural Networks


Title	Building robust prediction models for defective sensor data using Artificial Neural Networks
Authors	Arvind Kumar Shekar, Cláudio Rebelo de Sá, Hugo Ferreira, Carlos Soares
Abstract	Predicting the health of components in complex dynamic systems such as an automobile poses numerous challenges. The primary aim of such predictive systems is to use the high-dimensional data acquired from different sensors and predict the state-of-health of a particular component, e.g., brake pad. The classical approach involves selecting a smaller set of relevant sensor signals using feature selection and using them to train a machine learning algorithm. However, this fails to address two prominent problems: (1) sensors are susceptible to failure when exposed to extreme conditions over a long periods of time; (2) sensors are electrical devices that can be affected by noise or electrical interference. Using the failed and noisy sensor signals as inputs largely reduce the prediction accuracy. To tackle this problem, it is advantageous to use the information from all sensor signals, so that the failure of one sensor can be compensated by another. In this work, we propose an Artificial Neural Network (ANN) based framework to exploit the information from a large number of signals. Secondly, our framework introduces a data augmentation approach to perform accurate predictions in spite of noisy signals. The plausibility of our framework is validated on real life industrial application from Robert Bosch GmbH.
Tasks	Data Augmentation, Feature Selection
Published	2018-04-16
URL	http://arxiv.org/abs/1804.05544v1
PDF	http://arxiv.org/pdf/1804.05544v1.pdf
PWC	https://paperswithcode.com/paper/building-robust-prediction-models-for
Repo
Framework

News-based forecasts of macroeconomic indicators: A semantic path model for interpretable predictions


Title	News-based forecasts of macroeconomic indicators: A semantic path model for interpretable predictions
Authors	Stefan Feuerriegel, Julius Gordon
Abstract	The macroeconomic climate influences operations with regard to, e.g., raw material prices, financing, supply chain utilization and demand quotas. In order to adapt to the economic environment, decision-makers across the public and private sectors require accurate forecasts of the economic outlook. Existing predictive frameworks base their forecasts primarily on time series analysis, as well as the judgments of experts. As a consequence, current approaches are often biased and prone to error. In order to reduce forecast errors, this paper presents an innovative methodology that extends lag variables with unstructured data in the form of financial news: (1) we apply a variety of models from machine learning to word counts as a high-dimensional input. However, this approach suffers from low interpretability and overfitting, motivating the following remedies. (2) We follow the intuition that the economic climate is driven by general sentiments and suggest a projection of words onto latent semantic structures as a means of feature engineering. (3) We propose a semantic path model, together with estimation technique based on regularization, in order to yield full interpretability of the forecasts. We demonstrate the predictive performance of our approach by utilizing 80,813 ad hoc announcements in order to make long-term forecasts of up to 24 months ahead regarding key macroeconomic indicators. Back-testing reveals a considerable reduction in forecast errors.
Tasks	Feature Engineering, Time Series, Time Series Analysis
Published	2018-01-22
URL	http://arxiv.org/abs/1801.07047v2
PDF	http://arxiv.org/pdf/1801.07047v2.pdf
PWC	https://paperswithcode.com/paper/news-based-forecasts-of-macroeconomic
Repo
Framework

Evaluation of Neural Networks for Image Recognition Applications: Designing a 0-1 MILP Model of a CNN to create adversarials


Title	Evaluation of Neural Networks for Image Recognition Applications: Designing a 0-1 MILP Model of a CNN to create adversarials
Authors	Lucas Schelkes
Abstract	Image Recognition is a central task in computer vision with applications ranging across search, robotics, self-driving cars and many others. There are three purposes of this document: 1. We follow up on (Fischetti & Jo, December, 2017) and show how standard convolutional neural network can be optimized to a more sophisticated capsule architecture. 2. We introduce a MILP model based on CNN to create adversarials. 3. We compare and evaluate each network for image recognition tasks.
Tasks	Self-Driving Cars
Published	2018-09-01
URL	http://arxiv.org/abs/1809.00216v1
PDF	http://arxiv.org/pdf/1809.00216v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-neural-networks-for-image
Repo
Framework

RELF: Robust Regression Extended with Ensemble Loss Function


Title	RELF: Robust Regression Extended with Ensemble Loss Function
Authors	Hamideh Hajiabadi, Reza Monsefi, Hadi Sadoghi Yazdi
Abstract	Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regressor. We then propose a half-quadratic learning algorithm in order to find the parameter of the regressor and the optimal weights associated with each loss function. Moreover, we show that our proposed loss function is robust in noisy environments. For a particular class of loss functions, we show that our proposed ensemble loss function is Bayes consistent and robust. Experimental evaluations on several datasets demonstrate that our proposed ensemble loss function significantly improves the performance of a simple regressor in comparison with state-of-the-art methods.
Tasks	Meta-Learning
Published	2018-10-25
URL	http://arxiv.org/abs/1810.11071v1
PDF	http://arxiv.org/pdf/1810.11071v1.pdf
PWC	https://paperswithcode.com/paper/relf-robust-regression-extended-with-ensemble
Repo
Framework

Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit


Title	Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit
Authors	Yang Cao, Zheng Wen, Branislav Kveton, Yao Xie
Abstract	Multi-armed bandit (MAB) is a class of online learning problems where a learning agent aims to maximize its expected cumulative reward while repeatedly selecting to pull arms with unknown reward distributions. We consider a scenario where the reward distributions may change in a piecewise-stationary fashion at unknown time steps. We show that by incorporating a simple change-detection component with classic UCB algorithms to detect and adapt to changes, our so-called M-UCB algorithm can achieve nearly optimal regret bound on the order of $O(\sqrt{MKT\log T})$, where $T$ is the number of time steps, $K$ is the number of arms, and $M$ is the number of stationary segments. Comparison with the best available lower bound shows that our M-UCB is nearly optimal in $T$ up to a logarithmic factor. We also compare M-UCB with the state-of-the-art algorithms in numerical experiments using a public Yahoo! dataset to demonstrate its superior performance.
Tasks
Published	2018-02-11
URL	http://arxiv.org/abs/1802.03692v4
PDF	http://arxiv.org/pdf/1802.03692v4.pdf
PWC	https://paperswithcode.com/paper/nearly-optimal-adaptive-procedure-with-change
Repo
Framework

2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy


Title	2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy
Authors	Chuhan Min, Aosen Wang, Yiran Chen, Wenyao Xu, Xin Chen
Abstract	Deep Convolutional Neural Networks~(CNNs) offer remarkable performance of classifications and regressions in many high-dimensional problems and have been widely utilized in real-word cognitive applications. However, high computational cost of CNNs greatly hinder their deployment in resource-constrained applications, real-time systems and edge computing platforms. To overcome this challenge, we propose a novel filter-pruning framework, two-phase filter pruning based on conditional entropy, namely \textit{2PFPCE}, to compress the CNN models and reduce the inference time with marginal performance degradation. In our proposed method, we formulate filter pruning process as an optimization problem and propose a novel filter selection criteria measured by conditional entropy. Based on the assumption that the representation of neurons shall be evenly distributed, we also develop a maximum-entropy filter freeze technique that can reduce over fitting. Two filter pruning strategies – global and layer-wise strategies, are compared. Our experiment result shows that combining these two strategies can achieve a higher neural network compression ratio than applying only one of them under the same accuracy drop threshold. Two-phase pruning, that is, combining both global and layer-wise strategies, achieves 10 X FLOPs reduction and 46% inference time reduction on VGG-16, with 2% accuracy drop.
Tasks	Neural Network Compression
Published	2018-09-06
URL	http://arxiv.org/abs/1809.02220v1
PDF	http://arxiv.org/pdf/1809.02220v1.pdf
PWC	https://paperswithcode.com/paper/2pfpce-two-phase-filter-pruning-based-on
Repo
Framework

Object Localization and Size Estimation from RGB-D Images


Title	Object Localization and Size Estimation from RGB-D Images
Authors	ShreeRanjani SrirangamSridharan, Oytun Ulutan, Shehzad Noor Taus Priyo, Swati Rallapalli, Mudhakar Srivatsa
Abstract	Depth sensing cameras (e.g., Kinect sensor, Tango phone) can acquire color and depth images that are registered to a common viewpoint. This opens the possibility of developing algorithms that exploit the advantages of both sensing modalities. Traditionally, cues from color images have been used for object localization (e.g., YOLO). However, the addition of a depth image can be further used to segment images that might otherwise have identical color information. Further, the depth image can be used for object size (height/width) estimation (in real-world measurements units, such as meters) as opposed to image based segmentation that would only support drawing bounding boxes around objects of interest. In this paper, we first collect color camera information along with depth information using a custom Android application on Tango Phab2 phone. Second, we perform timing and spatial alignment between the two data sources. Finally, we evaluate several ways of measuring the height of the object of interest within the captured images under a variety of settings.
Tasks	Object Localization
Published	2018-08-02
URL	http://arxiv.org/abs/1808.00641v1
PDF	http://arxiv.org/pdf/1808.00641v1.pdf
PWC	https://paperswithcode.com/paper/object-localization-and-size-estimation-from
Repo
Framework

Optimizing the Wisdom of the Crowd: Inference, Learning, and Teaching


Title	Optimizing the Wisdom of the Crowd: Inference, Learning, and Teaching
Authors	Yao Zhou, Jingrui He
Abstract	The unprecedented demand for large amount of data has catalyzed the trend of combining human insights with machine learning techniques, which facilitate the use of crowdsourcing to enlist label information both effectively and efficiently. The classic work on crowdsourcing mainly focuses on the label inference problem under the categorization setting. However, inferring the true label requires sophisticated aggregation models that usually can only perform well under certain assumptions. Meanwhile, no matter how complicated the aggregation model is, the true model that generated the crowd labels remains unknown. Therefore, the label inference problem can never infer the ground truth perfectly. Based on the fact that the crowdsourcing labels are abundant and utilizing aggregation will lose such kind of rich annotation information (e.g., which worker provided which labels), we believe that it is critical to take the diverse labeling abilities of the crowdsourcing workers as well as their correlations into consideration. To address the above challenge, we propose to tackle three research problems, namely inference, learning, and teaching.
Tasks
Published	2018-06-23
URL	http://arxiv.org/abs/1806.09018v1
PDF	http://arxiv.org/pdf/1806.09018v1.pdf
PWC	https://paperswithcode.com/paper/optimizing-the-wisdom-of-the-crowd-inference
Repo
Framework

Explorations in Homeomorphic Variational Auto-Encoding


Title	Explorations in Homeomorphic Variational Auto-Encoding
Authors	Luca Falorsi, Pim de Haan, Tim R. Davidson, Nicola De Cao, Maurice Weiler, Patrick Forré, Taco S. Cohen
Abstract	The manifold hypothesis states that many kinds of high-dimensional data are concentrated near a low-dimensional manifold. If the topology of this data manifold is non-trivial, a continuous encoder network cannot embed it in a one-to-one manner without creating holes of low density in the latent space. This is at odds with the Gaussian prior assumption typically made in Variational Auto-Encoders (VAEs), because the density of a Gaussian concentrates near a blob-like manifold. In this paper we investigate the use of manifold-valued latent variables. Specifically, we focus on the important case of continuously differentiable symmetry groups (Lie groups), such as the group of 3D rotations $\operatorname{SO}(3)$. We show how a VAE with $\operatorname{SO}(3)$-valued latent variables can be constructed, by extending the reparameterization trick to compact connected Lie groups. Our experiments show that choosing manifold-valued latent variables that match the topology of the latent data manifold, is crucial to preserve the topological structure and learn a well-behaved latent space.
Tasks
Published	2018-07-12
URL	http://arxiv.org/abs/1807.04689v1
PDF	http://arxiv.org/pdf/1807.04689v1.pdf
PWC	https://paperswithcode.com/paper/explorations-in-homeomorphic-variational-auto
Repo
Framework

Learning Quadratic Games on Networks


Title	Learning Quadratic Games on Networks
Authors	Yan Leng, Xiaowen Dong, Alex Pentland
Abstract	Individuals, or organizations, cooperate with or compete against one another in a wide range of practical situations. In the economics literature, such strategic interactions are often modeled as games played on networks, where an individual’s payoff depends not only on her action but also that of her neighbors. The current literature has largely focused on analyzing the characteristics of network games in the scenario where the structure of the network, which is represented by a graph, is known beforehand. It is often the case, however, that the actions of the players are readily observable while the underlying interaction network remains hidden. In this paper, we propose two novel frameworks for learning, from the observations on individual actions, network games with linear-quadratic payoffs, and in particular the structure of the interaction network. Our frameworks are based on the Nash equilibrium of such games and involve solving a joint optimization problem for the graph structure and the individual marginal benefits. We test the proposed frameworks in synthetic settings and further study several factors that affect their learning performance. Moreover, with experiments on three real world examples, we show that our methods can effectively and more accurately learn the games than the baselines. The proposed approach is among the first of its kind for learning quadratic games, and have both theoretical and practical implications for understanding strategic interactions in a network environment.
Tasks
Published	2018-11-21
URL	http://arxiv.org/abs/1811.08790v2
PDF	http://arxiv.org/pdf/1811.08790v2.pdf
PWC	https://paperswithcode.com/paper/learning-quadratic-games-on-networks
Repo
Framework

Development and Evaluation of a Personalized Computer-aided Question Generation for English Learners to Improve Proficiency and Correct Mistakes


Title	Development and Evaluation of a Personalized Computer-aided Question Generation for English Learners to Improve Proficiency and Correct Mistakes
Authors	Yi-Ting Huang, Meng Chang Chen, Yeali S. Sun
Abstract	In the last several years, the field of computer assisted language learning has increasingly focused on computer aided question generation. However, this approach often provides test takers with an exhaustive amount of questions that are not designed for any specific testing purpose. In this work, we present a personalized computer aided question generation that generates multiple choice questions at various difficulty levels and types, including vocabulary, grammar and reading comprehension. In order to improve the weaknesses of test takers, it selects questions depending on an estimated proficiency level and unclear concepts behind incorrect responses. This results show that the students with the personalized automatic quiz generation corrected their mistakes more frequently than ones only with computer aided question generation. Moreover, students demonstrated the most progress between the pretest and post test and correctly answered more difficult questions. Finally, we investigated the personalizing strategy and found that a student could make a significant progress if the proposed system offered the vocabulary questions at the same level of his or her proficiency level, and if the grammar and reading comprehension questions were at a level lower than his or her proficiency level.
Tasks	Question Generation, Reading Comprehension
Published	2018-08-29
URL	http://arxiv.org/abs/1808.09732v1
PDF	http://arxiv.org/pdf/1808.09732v1.pdf
PWC	https://paperswithcode.com/paper/development-and-evaluation-of-a-personalized
Repo
Framework

Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models


Title	Hospital Readmission Prediction - Applying Hierarchical Sparsity Norms for Interpretable Models
Authors	Jialiang Jiang, Sharon Hewner, Varun Chandola
Abstract	Hospital readmissions have become one of the key measures of healthcare quality. Preventable readmissions have been identified as one of the primary targets for reducing costs and improving healthcare delivery. However, most data driven studies for understanding readmissions have produced black box classification and predictive models with moderate performance, which precludes them from being used effectively within the decision support systems in the hospitals. In this paper we present an application of structured sparsity-inducing norms for predicting readmission risk for patients based on their disease history and demographics. Most existing studies have focused on hospital utilization, test results, etc., to assign a readmission label to each episode of hospitalization. However, we focus on assigning a readmission risk label to a patient based on their disease history. Our emphasis is on interpreting the models to improve the understanding of the readmission problem. To achieve this, we exploit the domain induced hierarchical structure available for the disease codes which are the features for the classification algorithm. We use a tree based sparsity-inducing regularization strategy that explicitly uses the domain hierarchy. The resulting model not only outperforms standard regularization procedures but is also highly sparse and interpretable. We analyze the model and identify several significant factors that have an effect on readmission risk. Some of these factors conform to existing beliefs, e.g., impact of surgical complications and infections during hospital stay. Other factors, such as the impact of mental disorder and substance abuse on readmission, provide empirical evidence for several pre-existing but unverified hypotheses. The analysis also reveals previously undiscovered connections such as the influence of socioeconomic factors like lack of housing and malnutrition.
Tasks	Readmission Prediction
Published	2018-04-03
URL	http://arxiv.org/abs/1804.01188v1
PDF	http://arxiv.org/pdf/1804.01188v1.pdf
PWC	https://paperswithcode.com/paper/hospital-readmission-prediction-applying
Repo
Framework

Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation


Title	Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation
Authors	Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos, Arya D. McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, Philipp Koehn
Abstract	To better understand the effectiveness of continued training, we analyze the major components of a neural machine translation system (the encoder, decoder, and each embedding space) and consider each component’s contribution to, and capacity for, domain adaptation. We find that freezing any single component during continued training has minimal impact on performance, and that performance is surprisingly good when a single component is adapted while holding the rest of the model fixed. We also find that continued training does not move the model very far from the out-of-domain model, compared to a sensitivity analysis metric, suggesting that the out-of-domain model can provide a good generic initialization for the new domain.
Tasks	Domain Adaptation, Machine Translation
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05218v4
PDF	http://arxiv.org/pdf/1809.05218v4.pdf
PWC	https://paperswithcode.com/paper/freezing-subnetworks-to-analyze-domain
Repo
Framework

Why the Firefly Algorithm Works?


Title	Why the Firefly Algorithm Works?
Authors	Xin-She Yang, Xingshi He
Abstract	Firefly algorithm is a nature-inspired optimization algorithm and there have been significant developments since its appearance about ten years ago. This chapter summarizes the latest developments about the firefly algorithm and its variants as well as their diverse applications. Future research directions are also highlighted.
Tasks
Published	2018-04-22
URL	http://arxiv.org/abs/1806.01632v1
PDF	http://arxiv.org/pdf/1806.01632v1.pdf
PWC	https://paperswithcode.com/paper/why-the-firefly-algorithm-works
Repo
Framework

Monotone Learning with Rectified Wire Networks


Title	Monotone Learning with Rectified Wire Networks
Authors	Veit Elser, Dan Schmidt, Jonathan Yedidia
Abstract	We introduce a new neural network model, together with a tractable and monotone online learning algorithm. Our model describes feed-forward networks for classification, with one output node for each class. The only nonlinear operation is rectification using a ReLU function with a bias. However, there is a rectifier on every edge rather than at the nodes of the network. There are also weights, but these are positive, static, and associated with the nodes. Our “rectified wire” networks are able to represent arbitrary Boolean functions. Only the bias parameters, on the edges of the network, are learned. Another departure in our approach, from standard neural networks, is that the loss function is replaced by a constraint. This constraint is simply that the value of the output node associated with the correct class should be zero. Our model has the property that the exact norm-minimizing parameter update, required to correctly classify a training item, is the solution to a quadratic program that can be computed with a few passes through the network. We demonstrate a training algorithm using this update, called sequential deactivation (SDA), on MNIST and some synthetic datasets. Upon adopting a natural choice for the nodal weights, SDA has no hyperparameters other than those describing the network structure. Our experiments explore behavior with respect to network size and depth in a family of sparse expander networks.
Tasks
Published	2018-05-10
URL	http://arxiv.org/abs/1805.03963v4
PDF	http://arxiv.org/pdf/1805.03963v4.pdf
PWC	https://paperswithcode.com/paper/monotone-learning-with-rectified-wire
Repo
Framework