January 30, 2020

3138 words 15 mins read

Paper Group ANR 479

Efficient Discovery of Expressive Multi-label Rules using Relaxed Pruning. Generating and Exploiting Probabilistic Monocular Depth Estimates. The Non-IID Data Quagmire of Decentralized Machine Learning. Gradient-based Optimization for Bayesian Preference Elicitation. A Survey of the Recent Architectures of Deep Convolutional Neural Networks. Electr …

Efficient Discovery of Expressive Multi-label Rules using Relaxed Pruning


Title	Efficient Discovery of Expressive Multi-label Rules using Relaxed Pruning
Authors	Yannik Klein, Michael Rapp, Eneldo Loza Mencía
Abstract	Being able to model correlations between labels is considered crucial in multi-label classification. Rule-based models enable to expose such dependencies, e.g., implications, subsumptions, or exclusions, in an interpretable and human-comprehensible manner. Albeit the number of possible label combinations increases exponentially with the number of available labels, it has been shown that rules with multiple labels in their heads, which are a natural form to model local label dependencies, can be induced efficiently by exploiting certain properties of rule evaluation measures and pruning the label search space accordingly. However, experiments have revealed that multi-label heads are unlikely to be learned by existing methods due to their restrictiveness. To overcome this limitation, we propose a plug-in approach that relaxes the search space pruning used by existing methods in order to introduce a bias towards larger multi-label heads resulting in more expressive rules. We further demonstrate the effectiveness of our approach empirically and show that it does not come with drawbacks in terms of training time or predictive performance.
Tasks	Multi-Label Classification
Published	2019-08-19
URL	https://arxiv.org/abs/1908.06874v1
PDF	https://arxiv.org/pdf/1908.06874v1.pdf
PWC	https://paperswithcode.com/paper/efficient-discovery-of-expressive-multi-label
Repo
Framework

Generating and Exploiting Probabilistic Monocular Depth Estimates


Title	Generating and Exploiting Probabilistic Monocular Depth Estimates
Authors	Zhihao Xia, Patrick Sullivan, Ayan Chakrabarti
Abstract	Beyond depth estimation from a single image, the monocular cue is useful in a broader range of depth inference applications and settings—such as when one can leverage other available depth cues for improved accuracy. Currently, different applications, with different inference tasks and combinations of depth cues, are solved via different specialized networks—trained separately for each application. Instead, we propose a versatile task-agnostic monocular model that outputs a probability distribution over scene depth given an input color image, as a sample approximation of outputs from a patch-wise conditional VAE. We show that this distributional output can be used to enable a variety of inference tasks in different settings, without needing to retrain for each application. Across a diverse set of applications (depth completion, user guided estimation, etc.), our common model yields results with high accuracy—comparable to or surpassing that of state-of-the-art methods dependent on application-specific networks.
Tasks	Depth Completion, Depth Estimation, Monocular Depth Estimation
Published	2019-06-13
URL	https://arxiv.org/abs/1906.05739v2
PDF	https://arxiv.org/pdf/1906.05739v2.pdf
PWC	https://paperswithcode.com/paper/generating-and-exploiting-probabilistic
Repo
Framework

The Non-IID Data Quagmire of Decentralized Machine Learning


Title	The Non-IID Data Quagmire of Decentralized Machine Learning
Authors	Kevin Hsieh, Amar Phanishayee, Onur Mutlu, Phillip B. Gibbons
Abstract	Many large-scale machine learning (ML) applications need to train ML models over decentralized datasets that are generated at different devices and locations. These decentralized datasets pose a fundamental challenge to ML because they are typically generated in very different contexts, which leads to significant differences in data distribution across devices/locations (i.e., they are not independent and identically distributed (IID)). In this work, we take a step toward better understanding this challenge, by presenting the first detailed experimental study of the impact of such non-IID data on the decentralized training of deep neural networks (DNNs). Our study shows that: (i) the problem of non-IID data partitions is fundamental and pervasive, as it exists in all ML applications, DNN models, training datasets, and decentralized learning algorithms in our study; (ii) this problem is particularly difficult for DNN models with batch normalization layers; and (iii) the degree of deviation from IID (the skewness) is a key determinant of the difficulty level of the problem. With these findings in mind, we present SkewScout, a system-level approach that adapts the communication frequency of decentralized learning algorithms to the (skew-induced) accuracy loss between data partitions. We also show that group normalization can recover much of the skew-induced accuracy loss of batch normalization.
Tasks
Published	2019-10-01
URL	https://arxiv.org/abs/1910.00189v1
PDF	https://arxiv.org/pdf/1910.00189v1.pdf
PWC	https://paperswithcode.com/paper/the-non-iid-data-quagmire-of-decentralized
Repo
Framework

Gradient-based Optimization for Bayesian Preference Elicitation


Title	Gradient-based Optimization for Bayesian Preference Elicitation
Authors	Ivan Vendrov, Tyler Lu, Qingqing Huang, Craig Boutilier
Abstract	Effective techniques for eliciting user preferences have taken on added importance as recommender systems (RSs) become increasingly interactive and conversational. A common and conceptually appealing Bayesian criterion for selecting queries is expected value of information (EVOI). Unfortunately, it is computationally prohibitive to construct queries with maximum EVOI in RSs with large item spaces. We tackle this issue by introducing a continuous formulation of EVOI as a differentiable network that can be optimized using gradient methods available in modern machine learning (ML) computational frameworks (e.g., TensorFlow, PyTorch). We exploit this to develop a novel, scalable Monte Carlo method for EVOI optimization, which is more scalable for large item spaces than methods requiring explicit enumeration of items. While we emphasize the use of this approach for pairwise (or k-wise) comparisons of items, we also demonstrate how our method can be adapted to queries involving subsets of item attributes or “partial items,” which are often more cognitively manageable for users. Experiments show that our gradient-based EVOI technique achieves state-of-the-art performance across several domains while scaling to large item spaces.
Tasks	Recommendation Systems
Published	2019-11-20
URL	https://arxiv.org/abs/1911.09153v1
PDF	https://arxiv.org/pdf/1911.09153v1.pdf
PWC	https://paperswithcode.com/paper/gradient-based-optimization-for-bayesian
Repo
Framework

A Survey of the Recent Architectures of Deep Convolutional Neural Networks


Title	A Survey of the Recent Architectures of Deep Convolutional Neural Networks
Authors	Asifullah Khan, Anabia Sohail, Umme Zahoora, Aqsa Saeed Qureshi
Abstract	Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability of deep CNN is primarily due to the use of multiple feature extraction stages that can automatically learn representations from the data. The availability of a large amount of data and improvement in the hardware technology has accelerated the research in CNNs, and recently interesting deep CNN architectures have been reported. Several inspiring ideas to bring advancements in CNNs have been explored, such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the significant improvement in the representational capacity of the deep CNN is achieved through architectural innovations. Notably, the ideas of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing have gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and, consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges, and applications of CNN are also provided.
Tasks	Image Classification, Object Detection, Speech Recognition
Published	2019-01-17
URL	https://arxiv.org/abs/1901.06032v6
PDF	https://arxiv.org/pdf/1901.06032v6.pdf
PWC	https://paperswithcode.com/paper/a-survey-of-the-recent-architectures-of-deep
Repo
Framework

Electric Load and Power Forecasting Using Ensemble Gaussian Process Regression


Title	Electric Load and Power Forecasting Using Ensemble Gaussian Process Regression
Authors	Tong Ma, Renke Huang, David Barajas-Solano, Ramakrishna Tipireddy, Alexandre M. Tartakovsky
Abstract	We propose a new forecasting method for predicting load demand and generation scheduling. Accurate week-long forecasting of load demand and optimal power generation is critical for efficient operation of power grid systems. In this work, we use a synthetic data set describing a power grid with 700 buses and 134 generators over a 365-days period with data synthetically generated at an hourly rate. The proposed approach for week-long forecasting is based on the Gaussian process regression (GPR) method, with prior covariance matrices of the quantities of interest (QoI) computed from ensembles formed by up to twenty preceding weeks of QoI observations. Then, we use these covariances within the GPR framework to forecast the QoIs for the following week. We demonstrate that the the proposed ensemble GPR (EGPR) method is capable of accurately forecasting weekly total load demand and power generation profiles. The EGPR method is shown to outperform traditional forecasting methods including the standard GPR and autoregressive integrated moving average (ARIMA) methods.
Tasks
Published	2019-10-09
URL	https://arxiv.org/abs/1910.03783v1
PDF	https://arxiv.org/pdf/1910.03783v1.pdf
PWC	https://paperswithcode.com/paper/electric-load-and-power-forecasting-using
Repo
Framework

AQUALOC: An Underwater Dataset for Visual-Inertial-Pressure Localization


Title	AQUALOC: An Underwater Dataset for Visual-Inertial-Pressure Localization
Authors	Maxime Ferrera, Vincent Creuze, Julien Moras, Pauline Trouvé-Peloux
Abstract	We present a new dataset, dedicated to the development of simultaneous localization and mapping methods for underwater vehicles navigating close to the seabed. The data sequences composing this dataset are recorded in three different environments: a harbor at a depth of a few meters, a first archaeological site at a depth of 270 meters and a second site at a depth of 380 meters. The data acquisition is performed using Remotely Operated Vehicles equipped with a monocular monochromatic camera, a low-cost inertial measurement unit, a pressure sensor and a computing unit, all embedded in a single enclosure. The sensors’ measurements are recorded synchronously on the computing unit and seventeen sequences have been created from all the acquired data. These sequences are made available in the form of ROS bags and as raw data. For each sequence, a trajectory has also been computed offline using a Structure-from-Motion library in order to allow the comparison with real-time localization methods. With the release of this dataset, we wish to provide data difficult to acquire and to encourage the development of vision-based localization methods dedicated to the underwater environment. The dataset can be downloaded from: http://www.lirmm.fr/aqualoc/
Tasks	Simultaneous Localization and Mapping
Published	2019-10-31
URL	https://arxiv.org/abs/1910.14532v1
PDF	https://arxiv.org/pdf/1910.14532v1.pdf
PWC	https://paperswithcode.com/paper/aqualoc-an-underwater-dataset-for-visual
Repo
Framework

On the Trade-off Between Consistency and Coverage in Multi-label Rule Learning Heuristics


Title	On the Trade-off Between Consistency and Coverage in Multi-label Rule Learning Heuristics
Authors	Michael Rapp, Eneldo Loza Mencía, Johannes Fürnkranz
Abstract	Recently, several authors have advocated the use of rule learning algorithms to model multi-label data, as rules are interpretable and can be comprehended, analyzed, or qualitatively evaluated by domain experts. Many rule learning algorithms employ a heuristic-guided search for rules that model regularities contained in the training data and it is commonly accepted that the choice of the heuristic has a significant impact on the predictive performance of the learner. Whereas the properties of rule learning heuristics have been studied in the realm of single-label classification, there is no such work taking into account the particularities of multi-label classification. This is surprising, as the quality of multi-label predictions is usually assessed in terms of a variety of different, potentially competing, performance measures that cannot all be optimized by a single learner at the same time. In this work, we show empirically that it is crucial to trade off the consistency and coverage of rules differently, depending on which multi-label measure should be optimized by a model. Based on these findings, we emphasize the need for configurable learners that can flexibly use different heuristics. As our experiments reveal, the choice of the heuristic is not straight-forward, because a search for rules that optimize a measure locally does usually not result in a model that maximizes that measure globally.
Tasks	Multi-Label Classification
Published	2019-08-08
URL	https://arxiv.org/abs/1908.03032v1
PDF	https://arxiv.org/pdf/1908.03032v1.pdf
PWC	https://paperswithcode.com/paper/on-the-trade-off-between-consistency-and
Repo
Framework

A Human Action Descriptor Based on Motion Coordination


Title	A Human Action Descriptor Based on Motion Coordination
Authors	Pietro Falco, Matteo Saveriano, Eka Gibran Hasany, Nicholas H. Kirk, Dongheui Lee
Abstract	In this paper, we present a descriptor for human whole-body actions based on motion coordination. We exploit the principle, well known in neuromechanics, that humans move their joints in a coordinated fashion. Our coordination-based descriptor (CODE) is computed by two main steps. The first step is to identify the most informative joints which characterize the motion. The second step enriches the descriptor considering minimum and maximum joint velocities and the correlations between the most informative joints. In order to compute the distances between action descriptors, we propose a novel correlation-based similarity measure. The performance of CODE is tested on two public datasets, namely HDM05 and Berkeley MHAD, and compared with state-of-the-art approaches, showing recognition results.
Tasks
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08928v1
PDF	https://arxiv.org/pdf/1911.08928v1.pdf
PWC	https://paperswithcode.com/paper/a-human-action-descriptor-based-on-motion
Repo
Framework

Semi-supervised Logistic Learning Based on Exponential Tilt Mixture Models


Title	Semi-supervised Logistic Learning Based on Exponential Tilt Mixture Models
Authors	Xinwei Zhang, Zhiqiang Tan
Abstract	Consider semi-supervised learning for classification, where both labeled and unlabeled data are available for training. The goal is to exploit both datasets to achieve higher prediction accuracy than just using labeled data alone. We develop a semi-supervised logistic learning method based on exponential tilt mixture models, by extending a statistical equivalence between logistic regression and exponential tilt modeling. We study maximum nonparametric likelihood estimation and derive novel objective functions which are shown to be Fisher consistent. We also propose regularized estimation and construct simple and highly interpretable EM algorithms. Finally, we present numerical results which demonstrate the advantage of the proposed methods compared with existing methods.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.07882v1
PDF	https://arxiv.org/pdf/1906.07882v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-logistic-learning-based-on
Repo
Framework

Machine Learning for removing EEG artifacts: Setting the benchmark


Title	Machine Learning for removing EEG artifacts: Setting the benchmark
Authors	Subhrajit Roy
Abstract	Electroencephalograms (EEG) are often contaminated by artifacts which make interpreting them more challenging for clinicians. Hence, automated artifact recognition systems have the potential to aid the clinical workflow. In this abstract, we share the first results on applying various machine learning algorithms to the recently released world’s largest open-source artifact recognition dataset. We envision that these results will serve as a benchmark for researchers who might work with this dataset in future.
Tasks	EEG
Published	2019-03-19
URL	http://arxiv.org/abs/1903.07825v1
PDF	http://arxiv.org/pdf/1903.07825v1.pdf
PWC	https://paperswithcode.com/paper/machine-learning-for-removing-eeg-artifacts
Repo
Framework

Emotion Recognition with Machine Learning Using EEG Signals


Title	Emotion Recognition with Machine Learning Using EEG Signals
Authors	Omid Bazgir, Zeynab Mohammadi, Seyed Amir Hassan Habibi
Abstract	In this research, an emotion recognition system is developed based on valence/arousal model using electroencephalography (EEG) signals. EEG signals are decomposed into the gamma, beta, alpha and theta frequency bands using discrete wavelet transform (DWT), and spectral features are extracted from each frequency band. Principle component analysis (PCA) is applied to the extracted features by preserving the same dimensionality, as a transform, to make the features mutually uncorrelated. Support vector machine (SVM), K-nearest neighbor (KNN) and artificial neural network (ANN) are used to classify emotional states. The cross-validated SVM with radial basis function (RBF) kernel using extracted features of 10 EEG channels, performs with 91.3% accuracy for arousal and 91.1% accuracy for valence, both in the beta frequency band. Our approach shows better performance compared to existing algorithms applied to the “DEAP” dataset.
Tasks	EEG, Emotion Recognition
Published	2019-03-18
URL	https://arxiv.org/abs/1903.07272v2
PDF	https://arxiv.org/pdf/1903.07272v2.pdf
PWC	https://paperswithcode.com/paper/emotion-recognition-with-machine-learning
Repo
Framework

Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks


Title	Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks
Authors	Guangyong Chen, Pengfei Chen, Yujun Shi, Chang-Yu Hsieh, Benben Liao, Shengyu Zhang
Abstract	In this work, we propose a novel technique to boost training efficiency of a neural network. Our work is based on an excellent idea that whitening the inputs of neural networks can achieve a fast convergence speed. Given the well-known fact that independent components must be whitened, we introduce a novel Independent-Component (IC) layer before each weight layer, whose inputs would be made more independent. However, determining independent components is a computationally intensive task. To overcome this challenge, we propose to implement an IC layer by combining two popular techniques, Batch Normalization and Dropout, in a new manner that we can rigorously prove that Dropout can quadratically reduce the mutual information and linearly reduce the correlation between any pair of neurons with respect to the dropout layer parameter $p$. As demonstrated experimentally, the IC layer consistently outperforms the baseline approaches with more stable training process, faster convergence speed and better convergence limit on CIFAR10/100 and ILSVRC2012 datasets. The implementation of our IC layer makes us rethink the common practices in the design of neural networks. For example, we should not place Batch Normalization before ReLU since the non-negative responses of ReLU will make the weight layer updated in a suboptimal way, and we can achieve better performance by combining Batch Normalization and Dropout together as an IC layer.
Tasks
Published	2019-05-15
URL	https://arxiv.org/abs/1905.05928v1
PDF	https://arxiv.org/pdf/1905.05928v1.pdf
PWC	https://paperswithcode.com/paper/rethinking-the-usage-of-batch-normalization
Repo
Framework

Multi-view shape estimation of transparent containers


Title	Multi-view shape estimation of transparent containers
Authors	Alessio Xompero, Ricardo Sanchez-Matilla, Apostolos Modas, Pascal Frossard, Andrea Cavallaro
Abstract	The 3D localisation of an object and the estimation of its properties, such as shape and dimensions, are challenging under varying degrees of transparency and lighting conditions. In this paper, we propose a method for jointly localising container-like objects and estimating their dimensions using two wide-baseline, calibrated RGB cameras. Under the assumption of circular symmetry along the vertical axis, we estimate the dimensions of an object with a generative 3D sampling model of sparse circumferences, iterative shape fitting and image re-projection to verify the sampling hypotheses in each camera using semantic segmentation masks. We evaluate the proposed method on a novel dataset of objects with different degrees of transparency and captured under different backgrounds and illumination conditions. Our method, which is based on RGB images only, outperforms in terms of localisation success and dimension estimation accuracy a deep-learning based approach that uses depth maps.
Tasks	Semantic Segmentation
Published	2019-11-27
URL	https://arxiv.org/abs/1911.12354v2
PDF	https://arxiv.org/pdf/1911.12354v2.pdf
PWC	https://paperswithcode.com/paper/multi-view-shape-estimation-of-transparent
Repo
Framework

Privacy Preserving QoE Modeling using Collaborative Learning


Title	Privacy Preserving QoE Modeling using Collaborative Learning
Authors	Selim Ickin, Konstantinos Vandikas, Markus Fiedler
Abstract	Machine Learning based Quality of Experience (QoE) models potentially suffer from over-fitting due to limitations including low data volume, and limited participant profiles. This prevents models from becoming generic. Consequently, these trained models may under-perform when tested outside the experimented population. One reason for the limited datasets, which we refer in this paper as small QoE data lakes, is due to the fact that often these datasets potentially contain user sensitive information and are only collected throughout expensive user studies with special user consent. Thus, sharing of datasets amongst researchers is often not allowed. In recent years, privacy preserving machine learning models have become important and so have techniques that enable model training without sharing datasets but instead relying on secure communication protocols. Following this trend, in this paper, we present Round-Robin based Collaborative Machine Learning model training, where the model is trained in a sequential manner amongst the collaborated partner nodes. We benchmark this work using our customized Federated Learning mechanism as well as conventional Centralized and Isolated Learning methods.
Tasks
Published	2019-06-21
URL	https://arxiv.org/abs/1906.09248v2
PDF	https://arxiv.org/pdf/1906.09248v2.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-qoe-modeling-using
Repo
Framework