April 2, 2020

3307 words 16 mins read

Paper Group ANR 223

Reservoir computing model of two-dimensional turbulent convection. Contextual Constrained Learning for Dose-Finding Clinical Trials. A Theory of Usable Information Under Computational Constraints. Fairness in ad auctions through inverse proportionality. Practical Annotation Strategies for Question Answering Datasets. Differentially Private k-Means …

Reservoir computing model of two-dimensional turbulent convection


Title	Reservoir computing model of two-dimensional turbulent convection
Authors	Sandeep Pandey, Jörg Schumacher
Abstract	Reservoir computing is applied to model the large-scale evolution and the resulting low-order turbulence statistics of a two-dimensional turbulent Rayleigh-B'{e}nard convection flow at a Rayleigh number ${\rm Ra}=10^7$ and a Prandtl number ${\rm Pr}=7$ in an extended domain with an aspect ratio of 6. Our data-driven approach which is based on a long-term direct numerical simulation of the convection flow comprises a two-step procedure. (1) Reduction of the original simulation data by a Proper Orthogonal Decomposition (POD) snapshot analysis and subsequent truncation to the first 150 POD modes which are associated with the largest total energy amplitudes. (2) Setup and optimization of a reservoir computing model to describe the dynamical evolution of these 150 degrees of freedom and thus the large-scale evolution of the convection flow. The quality of the prediction of the reservoir computing model is comprehensively tested. At the core of the model is the reservoir, a very large sparse random network charcterized by the spectral radius of the corresponding adjacency matrix and a few further hyperparameters which are varied to investigate the quality of the prediction. Our work demonstrates that the reservoir computing model is capable to model the large-scale structure and low-order statistics of turbulent convection which can open new avenues for modeling mesoscale convection processes in larger circulation models.
Tasks
Published	2020-01-28
URL	https://arxiv.org/abs/2001.10280v1
PDF	https://arxiv.org/pdf/2001.10280v1.pdf
PWC	https://paperswithcode.com/paper/reservoir-computing-model-of-two-dimensional
Repo
Framework

Contextual Constrained Learning for Dose-Finding Clinical Trials


Title	Contextual Constrained Learning for Dose-Finding Clinical Trials
Authors	Hyun-Suk Lee, Cong Shen, James Jordon, Mihaela van der Schaar
Abstract	Clinical trials in the medical domain are constrained by budgets. The number of patients that can be recruited is therefore limited. When a patient population is heterogeneous, this creates difficulties in learning subgroup specific responses to a particular drug and especially for a variety of dosages. In addition, patient recruitment can be difficult by the fact that clinical trials do not aim to provide a benefit to any given patient in the trial. In this paper, we propose C3T-Budget, a contextual constrained clinical trial algorithm for dose-finding under both budget and safety constraints. The algorithm aims to maximize drug efficacy within the clinical trial while also learning about the drug being tested. C3T-Budget recruits patients with consideration of the remaining budget, the remaining time, and the characteristics of each group, such as the population distribution, estimated expected efficacy, and estimation credibility. In addition, the algorithm aims to avoid unsafe dosages. These characteristics are further illustrated in a simulated clinical trial study, which corroborates the theoretical analysis and demonstrates an efficient budget usage as well as a balanced learning-treatment trade-off.
Tasks
Published	2020-01-08
URL	https://arxiv.org/abs/2001.02463v2
PDF	https://arxiv.org/pdf/2001.02463v2.pdf
PWC	https://paperswithcode.com/paper/contextual-constrained-learning-for-dose
Repo
Framework

A Theory of Usable Information Under Computational Constraints


Title	A Theory of Usable Information Under Computational Constraints
Authors	Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, Stefano Ermon
Abstract	We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon’s information theory that takes into account the modeling power and computational constraints of the observer. The resulting \emph{predictive $\mathcal{V}$-information} encompasses mutual information and other notions of informativeness such as the coefficient of determination. Unlike Shannon’s mutual information and in violation of the data processing inequality, $\mathcal{V}$-information can be created through computation. This is consistent with deep neural networks extracting hierarchies of progressively more informative features in representation learning. Additionally, we show that by incorporating computational constraints, $\mathcal{V}$-information can be reliably estimated from data even in high dimensions with PAC-style guarantees. Empirically, we demonstrate predictive $\mathcal{V}$-information is more effective than mutual information for structure learning and fair representation learning.
Tasks	Representation Learning
Published	2020-02-25
URL	https://arxiv.org/abs/2002.10689v1
PDF	https://arxiv.org/pdf/2002.10689v1.pdf
PWC	https://paperswithcode.com/paper/a-theory-of-usable-information-under-1
Repo
Framework

Fairness in ad auctions through inverse proportionality


Title	Fairness in ad auctions through inverse proportionality
Authors	Shuchi Chawla, Meena Jagadeesan
Abstract	We study the tradeoff between social welfare maximization and fairness in the context of ad auctions. We study an ad auction setting where users arrive in an online fashion, $k$ advertisers submit bids for each user, and the auction assigns a distribution over ads to the user. Following the works of Dwork and Ilvento (2019) and Chawla et al. (2020), our goal is to design a truthful auction that satisfies multiple-task fairness in its outcomes: informally speaking, users that are similar to each other should obtain similar allocations of ads. We develop a new class of allocation algorithms that we call inverse-proportional allocation. These allocation algorithms are truthful, online, and do not explicitly need to know the fairness constraint over the users. In terms of fairness, they guarantee fair outcomes as long as every advertiser’s bids are non-discriminatory across users. In terms of social welfare, inverse-proportional allocation achieves a constant factor approximation in social welfare against the optimal (unfair) allocation, independent of the number of advertisers in the system. In this respect, these allocation algorithms greatly surpass the guarantees achieved in previous work; in fact, they achieve the optimal tradeoffs between fairness and social welfare in some contexts. We also extend our results to broader notions of fairness that we call subset fairness.
Tasks
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13966v1
PDF	https://arxiv.org/pdf/2003.13966v1.pdf
PWC	https://paperswithcode.com/paper/fairness-in-ad-auctions-through-inverse
Repo
Framework

Practical Annotation Strategies for Question Answering Datasets


Title	Practical Annotation Strategies for Question Answering Datasets
Authors	Bernhard Kratzwald, Xiang Yue, Huan Sun, Stefan Feuerriegel
Abstract	Annotating datasets for question answering (QA) tasks is very costly, as it requires intensive manual labor and often domain-specific knowledge. Yet strategies for annotating QA datasets in a cost-effective manner are scarce. To provide a remedy for practitioners, our objective is to develop heuristic rules for annotating a subset of questions, so that the annotation cost is reduced while maintaining both in- and out-of-domain performance. For this, we conduct a large-scale analysis in order to derive practical recommendations. First, we demonstrate experimentally that more training samples contribute often only to a higher in-domain test-set performance, but do not help the model in generalizing to unseen datasets. Second, we develop a model-guided annotation strategy: it makes a recommendation with regard to which subset of samples should be annotated. Its effectiveness is demonstrated in a case study based on domain customization of QA to a clinical setting. Here, remarkably, annotating a stratified subset with only 1.2% of the original training set achieves 97.7% of the performance as if the complete dataset was annotated. Hence, the labeling effort can be reduced immensely. Altogether, our work fulfills a demand in practice when labeling budgets are limited and where thus recommendations are needed for annotating QA datasets more cost-effectively.
Tasks	Question Answering
Published	2020-03-06
URL	https://arxiv.org/abs/2003.03235v1
PDF	https://arxiv.org/pdf/2003.03235v1.pdf
PWC	https://paperswithcode.com/paper/practical-annotation-strategies-for-question
Repo
Framework

Differentially Private k-Means Clustering with Guaranteed Convergence


Title	Differentially Private k-Means Clustering with Guaranteed Convergence
Authors	Zhigang Lu, Hong Shen
Abstract	Iterative clustering algorithms help us to learn the insights behind the data. Unfortunately, this may allow adversaries to infer the privacy of individuals with some background knowledge. In the worst case, the adversaries know the centroids of an arbitrary iteration and the information of n-1 out of n items. To protect individual privacy against such an inference attack, preserving differential privacy (DP) for the iterative clustering algorithms has been extensively studied in the interactive settings. However, existing interactive differentially private clustering algorithms suffer from a non-convergence problem, i.e., these algorithms may not terminate without a predefined number of iterations. This problem severely impacts the clustering quality and the efficiency of a differentially private algorithm. To resolve this problem, in this paper, we propose a novel differentially private clustering framework in the interactive settings which controls the orientation of the movement of the centroids over the iterations to ensure the convergence by injecting DP noise in a selected area. We prove that, in the expected case, algorithm under our framework converges in at most twice the iterations of Lloyd’s algorithm. We perform experimental evaluations on real-world datasets to show that our algorithm outperforms the state-of-the-art of the interactive differentially private clustering algorithms with guaranteed convergence and better clustering quality to meet the same DP requirement.
Tasks	Inference Attack
Published	2020-02-03
URL	https://arxiv.org/abs/2002.01043v1
PDF	https://arxiv.org/pdf/2002.01043v1.pdf
PWC	https://paperswithcode.com/paper/differentially-private-k-means-clustering
Repo
Framework

Concept-aware Geographic Information Retrieval


Title	Concept-aware Geographic Information Retrieval
Authors	Noemi Mauro, Liliana Ardissono, Adriano Savoca
Abstract	Textual queries are largely employed in information retrieval to let users specify search goals in a natural way. However, differences in user and system terminologies can challenge the identification of the user’s information needs, and thus the generation of relevant results. We argue that the explicit management of ontological knowledge, and of the meaning of concepts (by integrating linguistic and encyclopedic knowledge in the system ontology), can improve the analysis of search queries, because it enables a flexible identification of the topics the user is searching for, regardless of the adopted vocabulary. This paper proposes an information retrieval support model based on semantic concept identification. Starting from the recognition of the ontology concepts that the search query refers to, this model exploits the qualifiers specified in the query to select information items on the basis of possibly fine-grained features. Moreover, it supports query expansion and reformulation by suggesting the exploration of semantically similar concepts, as well as of concepts related to those referred in the query through thematic relations. A test on a data-set collected using the OnToMap Participatory GIS has shown that this approach provides accurate results.
Tasks	Information Retrieval
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13481v1
PDF	https://arxiv.org/pdf/2003.13481v1.pdf
PWC	https://paperswithcode.com/paper/concept-aware-geographic-information
Repo
Framework

Comparison Between Traditional Machine Learning Models And Neural Network Models For Vietnamese Hate Speech Detection


Title	Comparison Between Traditional Machine Learning Models And Neural Network Models For Vietnamese Hate Speech Detection
Authors	Son T. Luu, Hung P. Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
Abstract	Hate-speech detection on social network language has become one of the main researching fields recently due to the spreading of social networks like Facebook and Twitter. In Vietnam, the threat of offensive and harassment cause bad impacts for online user. The VLSP - Shared task about Hate Speech Detection on social networks showed many proposed approaches for detecting whatever comment is clean or not. However, this problem still needs further researching. Consequently, we compare traditional machine learning and deep learning on a large dataset about the user’s comments on social network in Vietnamese and find out what is the advantage and disadvantage of each model by comparing their accuracy on F1-score, then we pick two models in which has highest accuracy in traditional machine learning models and deep neural models respectively. Next, we compare these two models capable of predicting the right label by referencing their confusion matrices and considering the advantages and disadvantages of each model. Finally, from the comparison result, we propose our ensemble method that concentrates the abilities of traditional methods and deep learning methods.
Tasks	Hate Speech Detection
Published	2020-01-31
URL	https://arxiv.org/abs/2002.00759v1
PDF	https://arxiv.org/pdf/2002.00759v1.pdf
PWC	https://paperswithcode.com/paper/comparison-between-traditional-machine
Repo
Framework

Are Accelerometers for Activity Recognition a Dead-end?


Title	Are Accelerometers for Activity Recognition a Dead-end?
Authors	Catherine Tong, Shyam A. Tailor, Nicholas D. Lane
Abstract	Accelerometer-based (and by extension other inertial sensors) research for Human Activity Recognition (HAR) is a dead-end. This sensor does not offer enough information for us to progress in the core domain of HAR - to recognize everyday activities from sensor data. Despite continued and prolonged efforts in improving feature engineering and machine learning models, the activities that we can recognize reliably have only expanded slightly and many of the same flaws of early models are still present today. Instead of relying on acceleration data, we should instead consider modalities with much richer information - a logical choice are images. With the rapid advance in image sensing hardware and modelling techniques, we believe that a widespread adoption of image sensors will open many opportunities for accurate and robust inference across a wide spectrum of human activities. In this paper, we make the case for imagers in place of accelerometers as the default sensor for human activity recognition. Our review of past works has led to the observation that progress in HAR had stalled, caused by our reliance on accelerometers. We further argue for the suitability of images for activity recognition by illustrating their richness of information and the marked progress in computer vision. Through a feasibility analysis, we find that deploying imagers and CNNs on device poses no substantial burden on modern mobile hardware. Overall, our work highlights the need to move away from accelerometers and calls for further exploration of using imagers for activity recognition.
Tasks	Activity Recognition, Feature Engineering, Human Activity Recognition
Published	2020-01-22
URL	https://arxiv.org/abs/2001.08111v2
PDF	https://arxiv.org/pdf/2001.08111v2.pdf
PWC	https://paperswithcode.com/paper/are-accelerometers-for-activity-recognition-a
Repo
Framework

A neural network model of perception and reasoning


Title	A neural network model of perception and reasoning
Authors	Paul J. Blazek, Milo M. Lin
Abstract	How perception and reasoning arise from neuronal network activity is poorly understood. This is reflected in the fundamental limitations of connectionist artificial intelligence, typified by deep neural networks trained via gradient-based optimization. Despite success on many tasks, such networks remain unexplainable black boxes incapable of symbolic reasoning and concept generalization. Here we show that a simple set of biologically consistent organizing principles confer these capabilities to neuronal networks. To demonstrate, we implement these principles in a novel machine learning algorithm, based on concept construction instead of optimization, to design deep neural networks that reason with explainable neuron activity. On a range of tasks including NP-hard problems, their reasoning capabilities grant additional cognitive functions, like deliberating through self-analysis, tolerating adversarial attacks, and learning transferable rules from simple examples to solve problems of unencountered complexity. The networks also naturally display properties of biological nervous systems inherently absent in current deep neural networks, including sparsity, modularity, and both distributed and localized firing patterns. Because they do not sacrifice performance, compactness, or training time on standard learning tasks, these networks provide a new black-box-free approach to artificial intelligence. They likewise serve as a quantitative framework to understand the emergence of cognition from neuronal networks.
Tasks
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11319v1
PDF	https://arxiv.org/pdf/2002.11319v1.pdf
PWC	https://paperswithcode.com/paper/a-neural-network-model-of-perception-and
Repo
Framework

Motion Classification using Kinematically Sifted ACGAN-Synthesized Radar Micro-Doppler Signatures


Title	Motion Classification using Kinematically Sifted ACGAN-Synthesized Radar Micro-Doppler Signatures
Authors	Baris Erol, Sevgi Zubeyde Gurbuz, Moeness G. Amin
Abstract	Deep neural networks (DNNs) have recently received vast attention in applications requiring classification of radar returns, including radar-based human activity recognition for security, smart homes, assisted living, and biomedicine. However,acquiring a sufficiently large training dataset remains a daunting task due to the high human costs and resources required for radar data collection. In this paper, an extended approach to adversarial learning is proposed for generation of synthetic radar micro-Doppler signatures that are well-adapted to different environments. The synthetic data is evaluated using visual interpretation, analysis of kinematic consistency, data diversity, dimensions of the latent space, and saliency maps. A principle-component analysis (PCA) based kinematic-sifting algorithm is introduced to ensure that synthetic signatures are consistent with physically possible human motions. The synthetic dataset is used to train a 19-layer deep convolutional neural network (DCNN) to classify micro-Doppler signatures acquired from an environment different from that of the dataset supplied to the adversarial network. An overall accuracy 93% is achieved on a dataset that contains multiple aspect angles (0 deg., 30 deg., and 45 deg. as well as 60 deg.), with 9% improvement as a result of kinematic sifting.
Tasks	Activity Recognition, Human Activity Recognition
Published	2020-01-19
URL	https://arxiv.org/abs/2001.08582v1
PDF	https://arxiv.org/pdf/2001.08582v1.pdf
PWC	https://paperswithcode.com/paper/motion-classification-using-kinematically
Repo
Framework

Improve Unsupervised Domain Adaptation with Mixup Training


Title	Improve Unsupervised Domain Adaptation with Mixup Training
Authors	Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, Liu Ren
Abstract	Unsupervised domain adaptation studies the problem of utilizing a relevant source domain with abundant labels to build predictive modeling for an unannotated target domain. Recent work observe that the popular adversarial approach of learning domain-invariant features is insufficient to achieve desirable target domain performance and thus introduce additional training constraints, e.g. cluster assumption. However, these approaches impose the constraints on source and target domains individually, ignoring the important interplay between them. In this work, we propose to enforce training constraints across domains using mixup formulation to directly address the generalization performance for target data. In order to tackle potentially huge domain discrepancy, we further propose a feature-level consistency regularizer to facilitate the inter-domain constraint. When adding intra-domain mixup and domain adversarial learning, our general framework significantly improves state-of-the-art performance on several important tasks from both image classification and human activity recognition.
Tasks	Activity Recognition, Domain Adaptation, Human Activity Recognition, Image Classification, Unsupervised Domain Adaptation
Published	2020-01-03
URL	https://arxiv.org/abs/2001.00677v1
PDF	https://arxiv.org/pdf/2001.00677v1.pdf
PWC	https://paperswithcode.com/paper/improve-unsupervised-domain-adaptation-with
Repo
Framework

TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks


Title	TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks
Authors	Xinyu Liu, Xiaoguang Di
Abstract	Lightweight or mobile neural networks used for real-time computer vision tasks contain fewer parameters than normal networks, which lead to a constrained performance. In this work, we proposed a novel activation function named Tanh Exponential Activation Function (TanhExp) which can improve the performance for these networks on image classification task significantly. The definition of TanhExp is f(x) = xtanh(e^x). We demonstrate the simplicity, efficiency, and robustness of TanhExp on various datasets and network models and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour also remains stable even with noise added and dataset altered. We show that without increasing the size of the network, the capacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters added.
Tasks	Image Classification
Published	2020-03-22
URL	https://arxiv.org/abs/2003.09855v1
PDF	https://arxiv.org/pdf/2003.09855v1.pdf
PWC	https://paperswithcode.com/paper/tanhexp-a-smooth-activation-function-with
Repo
Framework

Some Insights into Lifelong Reinforcement Learning Systems


Title	Some Insights into Lifelong Reinforcement Learning Systems
Authors	Changjian Li
Abstract	A lifelong reinforcement learning system is a learning system that has the ability to learn through trail-and-error interaction with the environment over its lifetime. In this paper, I give some arguments to show that the traditional reinforcement learning paradigm fails to model this type of learning system. Some insights into lifelong reinforcement learning are provided, along with a simplistic prototype lifelong reinforcement learning system.
Tasks
Published	2020-01-27
URL	https://arxiv.org/abs/2001.09608v1
PDF	https://arxiv.org/pdf/2001.09608v1.pdf
PWC	https://paperswithcode.com/paper/some-insights-into-lifelong-reinforcement
Repo
Framework

Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation


Title	Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation
Authors	Jongbin Ryu, Jiun Bae, Jongwoo Lim
Abstract	In this paper, we introduce a collaborative training algorithm of balanced random forests with convolutional neural networks for domain adaptation tasks. In real scenarios, most domain adaptation algorithms face the challenges from noisy, insufficient training data and open set categorization. In such cases, conventional methods suffer from overfitting and fail to successfully transfer the knowledge of the source to the target domain. To address these issues, the following two techniques are proposed. First, we introduce the optimized decision tree construction method with convolutional neural networks, in which the data at each node are split into equal sizes while maximizing the information gain. It generates balanced decision trees on deep features because of the even-split constraint, which contributes to enhanced discrimination power and reduced overfitting problem. Second, to tackle the domain misalignment problem, we propose the domain alignment loss which penalizes uneven splits of the source and target domain data. By collaboratively optimizing the information gain of the labeled source data as well as the entropy of unlabeled target data distributions, the proposed CoBRF algorithm achieves significantly better performance than the state-of-the-art methods.
Tasks	Domain Adaptation
Published	2020-02-10
URL	https://arxiv.org/abs/2002.03642v1
PDF	https://arxiv.org/pdf/2002.03642v1.pdf
PWC	https://paperswithcode.com/paper/collaborative-training-of-balanced-random-1
Repo
Framework