January 31, 2020

3023 words 15 mins read

Paper Group ANR 170

On Data-Selective Learning. Hotel Recommendation System. Temporally-Biased Sampling Schemes for Online Model Management. Input-Output Equivalence of Unitary and Contractive RNNs. Supervised learning improves disease outbreak detection. Adaptive Sequence Submodularity. HEMELB Acceleration and Visualization for Cerebral Aneurysms. Motorway Traffic Fl …

On Data-Selective Learning


Title	On Data-Selective Learning
Authors	Hamed Yazdanpanah
Abstract	Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancellation, signal prediction, and stock market. Therefore, reducing the energy consumption of the adaptive filtering algorithms has great importance, particularly in green technologies and in devices using battery. In this thesis, data-selective adaptive filters, in particular the set-membership (SM) adaptive filters, are the tools to reach the goal. There are well known SM adaptive filters in literature. This work introduces new algorithms based on the classical ones in order to improve their performances and reduce the number of required arithmetic operations at the same time. Therefore, firstly, we analyze the robustness of the classical SM adaptive filtering algorithms. Secondly, we extend the SM technique to trinion and quaternion systems. Thirdly, by combining SM filtering and partial-updating, we introduce a new improved set-membership affine projection algorithm with constrained step size to improve its stability behavior. Fourthly, we propose some new least-mean-square (LMS) based and recursive least-squares based adaptive filtering algorithms with low computational complexity for sparse systems. Finally, we derive some feature LMS algorithms to exploit the hidden sparsity in the parameters.
Tasks
Published	2019-09-06
URL	https://arxiv.org/abs/1909.03891v1
PDF	https://arxiv.org/pdf/1909.03891v1.pdf
PWC	https://paperswithcode.com/paper/on-data-selective-learning
Repo
Framework

Hotel Recommendation System


Title	Hotel Recommendation System
Authors	Aditi A. Mavalankar, Ajitesh Gupta, Chetan Gandotra, Rishabh Misra
Abstract	One of the first things to do while planning a trip is to book a good place to stay. Booking a hotel online can be an overwhelming task with thousands of hotels to choose from, for every destination. Motivated by the importance of these situations, we decided to work on the task of recommending hotels to users. We used Expedia’s hotel recommendation dataset, which has a variety of features that helped us achieve a deep understanding of the process that makes a user choose certain hotels over others. The aim of this hotel recommendation task is to predict and recommend five hotel clusters to a user that he/she is more likely to book given hundred distinct clusters.
Tasks
Published	2019-08-20
URL	https://arxiv.org/abs/1908.07498v2
PDF	https://arxiv.org/pdf/1908.07498v2.pdf
PWC	https://paperswithcode.com/paper/hotel-recommendation-system
Repo
Framework

Temporally-Biased Sampling Schemes for Online Model Management


Title	Temporally-Biased Sampling Schemes for Online Model Management
Authors	Brian Hentschel, Peter J. Haas, Yuanyuan Tian
Abstract	To maintain the accuracy of supervised learning models in the presence of evolving data streams, we provide temporally-biased sampling schemes that weight recent data most heavily, with inclusion probabilities for a given data item decaying over time according to a specified “decay function”. We then periodically retrain the models on the current sample. This approach speeds up the training process relative to training on all of the data. Moreover, time-biasing lets the models adapt to recent changes in the data while—unlike in a sliding-window approach—still keeping some old data to ensure robustness in the face of temporary fluctuations and periodicities in the data values. In addition, the sampling-based approach allows existing analytic algorithms for static data to be applied to dynamic streaming data essentially without change. We provide and analyze both a simple sampling scheme (T-TBS) that probabilistically maintains a target sample size and a novel reservoir-based scheme (R-TBS) that is the first to provide both control over the decay rate and a guaranteed upper bound on the sample size. If the decay function is exponential, then control over the decay rate is complete, and R-TBS maximizes both expected sample size and sample-size stability. For general decay functions, the actual item inclusion probabilities can be made arbitrarily close to the nominal probabilities, and we provide a scheme that allows a trade-off between sample footprint and sample-size stability. The R-TBS and T-TBS schemes are of independent interest, extending the known set of unequal-probability sampling schemes. We discuss distributed implementation strategies; experiments in Spark illuminate the performance and scalability of the algorithms, and show that our approach can increase machine learning robustness in the face of evolving data.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.05677v1
PDF	https://arxiv.org/pdf/1906.05677v1.pdf
PWC	https://paperswithcode.com/paper/temporally-biased-sampling-schemes-for-online
Repo
Framework

Input-Output Equivalence of Unitary and Contractive RNNs


Title	Input-Output Equivalence of Unitary and Contractive RNNs
Authors	M. Emami, M. Sahraee-Ardakan, S. Rangan, A. K. Fletcher
Abstract	Unitary recurrent neural networks (URNNs) have been proposed as a method to overcome the vanishing and exploding gradient problem in modeling data with long-term dependencies. A basic question is how restrictive is the unitary constraint on the possible input-output mappings of such a network? This work shows that for any contractive RNN with ReLU activations, there is a URNN with at most twice the number of hidden states and the identical input-output mapping. Hence, with ReLU activations, URNNs are as expressive as general RNNs. In contrast, for certain smooth activations, it is shown that the input-output mapping of an RNN cannot be matched with a URNN, even with an arbitrary number of states. The theoretical results are supported by experiments on modeling of slowly-varying dynamical systems.
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13672v1
PDF	https://arxiv.org/pdf/1910.13672v1.pdf
PWC	https://paperswithcode.com/paper/input-output-equivalence-of-unitary-and
Repo
Framework

Supervised learning improves disease outbreak detection


Title	Supervised learning improves disease outbreak detection
Authors	Benedikt Zacher, Irina Czogiel
Abstract	The early detection of infectious disease outbreaks is a crucial task to protect population health. To this end, public health surveillance systems have been established to systematically collect and analyse infectious disease data. A variety of statistical tools are available, which detect potential outbreaks as abberations from an expected endemic level using these data. Here, we develop the first supervised learning approach based on hidden Markov models for disease outbreak detection, which leverages data that is routinely collected within a public health surveillance system. We evaluate our model using real Salmonella and Campylobacter data, as well as simulations. In comparison to a state-of-the-art approach, which is applied in multiple European countries including Germany, our proposed model reduces the false positive rate by up to 50% while retaining the same sensitivity. We see our supervised learning approach as a significant step to further develop machine learning applications for disease outbreak detection, which will be instrumental to improve public health surveillance systems.
Tasks
Published	2019-02-06
URL	http://arxiv.org/abs/1902.10061v1
PDF	http://arxiv.org/pdf/1902.10061v1.pdf
PWC	https://paperswithcode.com/paper/supervised-learning-improves-disease-outbreak
Repo
Framework

Adaptive Sequence Submodularity


Title	Adaptive Sequence Submodularity
Authors	Marko Mitrovic, Ehsan Kazemi, Moran Feldman, Andreas Krause, Amin Karbasi
Abstract	In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user’s feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of future outcomes. Without further structure, finding an optimal sequence is notoriously challenging, if not completely intractable. In this paper, we view the problem of adaptive and sequential decision making through the lens of submodularity and propose an adaptive greedy policy with strong theoretical guarantees. Additionally, to demonstrate the practical utility of our results, we run experiments on Amazon product recommendation and Wikipedia link prediction tasks.
Tasks	Decision Making, Link Prediction, Product Recommendation
Published	2019-02-15
URL	https://arxiv.org/abs/1902.05981v2
PDF	https://arxiv.org/pdf/1902.05981v2.pdf
PWC	https://paperswithcode.com/paper/adaptive-sequence-submodularity
Repo
Framework

HEMELB Acceleration and Visualization for Cerebral Aneurysms


Title	HEMELB Acceleration and Visualization for Cerebral Aneurysms
Authors	Sahar Soheilian Esfahani, Xiaojun Zhai, Minsi Chen, Abbes Amira, Faycal Bensaali, Julien AbiNahed, Sarada Dakua, Georges Younes, Robin A. Richardson, Peter V. Coveney
Abstract	A weakness in the wall of a cerebral artery causing a dilation or ballooning of the blood vessel is known as a cerebral aneurysm. Optimal treatment requires fast and accurate diagnosis of the aneurysm. HemeLB is a fluid dynamics solver for complex geometries developed to provide neurosurgeons with information related to the flow of blood in and around aneurysms. On a cost efficient platform, HemeLB could be employed in hospitals to provide surgeons with the simulation results in real-time. In this work, we developed an improved version of HemeLB for GPU implementation and result visualization. A visualization platform for smooth interaction with end users is also presented. Finally, a comprehensive evaluation of this implementation is reported. The results demonstrate that the proposed implementation achieves a maximum performance of 15,168,964 site updates per second, and is capable of speeding up HemeLB for deployment in hospitals and clinical investigations.
Tasks
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11925v1
PDF	https://arxiv.org/pdf/1906.11925v1.pdf
PWC	https://paperswithcode.com/paper/hemelb-acceleration-and-visualization-for
Repo
Framework

Motorway Traffic Flow Prediction using Advanced Deep Learning


Title	Motorway Traffic Flow Prediction using Advanced Deep Learning
Authors	Adriana-Simona Mihaita, Haowen Li, Zongyang He, Marian-Andrei Rizoiu
Abstract	Congestion prediction represents a major priority for traffic management centres around the world to ensure timely incident response handling. The increasing amounts of generated traffic data have been used to train machine learning predictors for traffic, however this is a challenging task due to inter-dependencies of traffic flow both in time and space. Recently, deep learning techniques have shown significant prediction improvements over traditional models, however open questions remain around their applicability, accuracy and parameter tuning. This paper proposes an advanced deep learning framework for simultaneously predicting the traffic flow on a large number of monitoring stations along a highly circulated motorway in Sydney, Australia, including exit and entry loop count stations, and over varying training and prediction time horizons. The spatial and temporal features extracted from the 36.34 million data points are used in various deep learning architectures that exploit their spatial structure (convolutional neuronal networks), their temporal dynamics (recurrent neuronal networks), or both through a hybrid spatio-temporal modelling (CNN-LSTM). We show that our deep learning models consistently outperform traditional methods, and we conduct a comparative analysis of the optimal time horizon of historical data required to predict traffic flow at different time points in the future.
Tasks
Published	2019-07-15
URL	https://arxiv.org/abs/1907.06356v3
PDF	https://arxiv.org/pdf/1907.06356v3.pdf
PWC	https://paperswithcode.com/paper/motorway-traffic-flow-prediction-using
Repo
Framework

Model Fusion via Optimal Transport


Title	Model Fusion via Optimal Transport
Authors	Sidak Pal Singh, Martin Jaggi
Abstract	Combining different models is a widely used paradigm in machine learning applications. While the most common approach is to form an ensemble of models and average their individual predictions, this approach is often rendered infeasible by given resource constraints in terms of memory and computation, which grow linearly with the number of models. We present a layer-wise model fusion algorithm for neural networks that utilizes optimal transport to (soft-) align neurons across the models before averaging their associated parameters. We discuss two main strategies for fusing neural networks in this “one-shot” manner, without requiring any retraining. Next, we illustrate how this significantly outperforms vanilla averaging on convolutional networks (like VGG11), residual networks (like ResNet18), and multi-layer perceptrons, on CIFAR10 and MNIST. Finally, we show applications to transfer tasks (where our fused model even surpasses the performance of both the original models) as well as for compressing models. Code will be made available under the following link https://github.com/modelfusion.
Tasks
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05653v3
PDF	https://arxiv.org/pdf/1910.05653v3.pdf
PWC	https://paperswithcode.com/paper/model-fusion-via-optimal-transport
Repo
Framework

Brain-mediated Transfer Learning of Convolutional Neural Networks


Title	Brain-mediated Transfer Learning of Convolutional Neural Networks
Authors	Satoshi Nishida, Yusuke Nakano, Antoine Blanc, Naoya Maeda, Masataka Kado, Shinji Nishimoto
Abstract	The human brain can effectively learn a new task from a small number of samples, which indicate that the brain can transfer its prior knowledge to solve tasks in different domains. This function is analogous to transfer learning (TL) in the field of machine learning. TL uses a well-trained feature space in a specific task domain to improve performance in new tasks with insufficient training data. TL with rich feature representations, such as features of convolutional neural networks (CNNs), shows high generalization ability across different task domains. However, such TL is still insufficient in making machine learning attain generalization ability comparable to that of the human brain. To examine if the internal representation of the brain could be used to achieve more efficient TL, we introduce a method for TL mediated by human brains. Our method transforms feature representations of audiovisual inputs in CNNs into those in activation patterns of individual brains via their association learned ahead using measured brain responses. Then, to estimate labels reflecting human cognition and behavior induced by the audiovisual inputs, the transformed representations are used for TL. We demonstrate that our brain-mediated TL (BTL) shows higher performance in the label estimation than the standard TL. In addition, we illustrate that the estimations mediated by different brains vary from brain to brain, and the variability reflects the individual variability in perception. Thus, our BTL provides a framework to improve the generalization ability of machine-learning feature representations and enable machine learning to estimate human-like cognition and behavior, including individual variability.
Tasks	Transfer Learning
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10037v3
PDF	https://arxiv.org/pdf/1905.10037v3.pdf
PWC	https://paperswithcode.com/paper/brain-mediated-transfer-learning-of
Repo
Framework

Learning Causal State Representations of Partially Observable Environments


Title	Learning Causal State Representations of Partially Observable Environments
Authors	Amy Zhang, Zachary C. Lipton, Luis Pineda, Kamyar Azizzadenesheli, Anima Anandkumar, Laurent Itti, Joelle Pineau, Tommaso Furlanello
Abstract	Intelligent agents can cope with sensory-rich environments by learning task-agnostic state abstractions. In this paper, we propose mechanisms to approximate causal states, which optimally compress the joint history of actions and observations in partially-observable Markov decision processes. Our proposed algorithm extracts causal state representations from RNNs that are trained to predict subsequent observations given the history. We demonstrate that these learned task-agnostic state abstractions can be used to efficiently learn policies for reinforcement learning problems with rich observation spaces. We evaluate agents using multiple partially observable navigation tasks with both discrete (GridWorld) and continuous (VizDoom, ALE) observation processes that cannot be solved by traditional memory-limited methods. Our experiments demonstrate systematic improvement of the DQN and tabular models using approximate causal state representations with respect to recurrent-DQN baselines trained with raw inputs.
Tasks
Published	2019-06-25
URL	https://arxiv.org/abs/1906.10437v1
PDF	https://arxiv.org/pdf/1906.10437v1.pdf
PWC	https://paperswithcode.com/paper/learning-causal-state-representations-of
Repo
Framework

Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform


Title	Privacy Accounting and Quality Control in the Sage Differentially Private ML Platform
Authors	Mathias Lecuyer, Riley Spahn, Kiran Vodrahalli, Roxana Geambasu, Daniel Hsu
Abstract	Companies increasingly expose machine learning (ML) models trained over sensitive user data to untrusted domains, such as end-user devices and wide-access model stores. We present Sage, a differentially private (DP) ML platform that bounds the cumulative leakage of training data through models. Sage builds upon the rich literature on DP ML algorithms and contributes pragmatic solutions to two of the most pressing systems challenges of global DP: running out of privacy budget and the privacy-utility tradeoff. To address the former, we develop block composition, a new privacy loss accounting method that leverages the growing database regime of ML workloads to keep training models endlessly on a sensitive data stream while enforcing a global DP guarantee for the stream. To address the latter, we develop privacy-adaptive training, a process that trains a model on growing amounts of data and/or with increasing privacy parameters until, with high probability, the model meets developer-configured quality criteria. They illustrate how a systems focus on characteristics of ML workloads enables pragmatic solutions that are not apparent when one focuses on individual algorithms, as most DP ML literature does.
Tasks
Published	2019-09-04
URL	https://arxiv.org/abs/1909.01502v2
PDF	https://arxiv.org/pdf/1909.01502v2.pdf
PWC	https://paperswithcode.com/paper/privacy-accounting-and-quality-control-in-the
Repo
Framework

Integrating Artificial and Human Intelligence for Efficient Translation


Title	Integrating Artificial and Human Intelligence for Efficient Translation
Authors	Nico Herbig, Santanu Pal, Josef van Genabith, Antonio Krüger
Abstract	Current advances in machine translation increase the need for translators to switch from traditional translation to post-editing of machine-translated text, a process that saves time and improves quality. Human and artificial intelligence need to be integrated in an efficient way to leverage the advantages of both for the translation task. This paper outlines approaches at this boundary of AI and HCI and discusses open research questions to further advance the field.
Tasks	Machine Translation
Published	2019-03-07
URL	http://arxiv.org/abs/1903.02978v1
PDF	http://arxiv.org/pdf/1903.02978v1.pdf
PWC	https://paperswithcode.com/paper/integrating-artificial-and-human-intelligence
Repo
Framework

Declarative Learning-Based Programming as an Interface to AI Systems


Title	Declarative Learning-Based Programming as an Interface to AI Systems
Authors	Parisa Kordjamshidi, Dan Roth, Kristian Kersting
Abstract	Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry. In most cases, machine learning models are the key component of these solutions, but a solution involves multiple such models, along with significant levels of reasoning with the models’ output and input. Current technologies do not make such techniques easy to use for application experts who are not fluent in machine learning nor for machine learning experts who aim at testing ideas and models on real-world data in the context of the overall AI system. We review key efforts made by various AI communities to provide languages for high-level abstractions over learning and reasoning techniques needed for designing complex AI systems. We classify the existing frameworks based on the type of techniques and the data and knowledge representations they use, provide a comparative study of the way they address the challenges of programming real-world applications, and highlight some shortcomings and future directions.
Tasks
Published	2019-06-18
URL	https://arxiv.org/abs/1906.07809v1
PDF	https://arxiv.org/pdf/1906.07809v1.pdf
PWC	https://paperswithcode.com/paper/declarative-learning-based-programming-as-an
Repo
Framework

Douglas-Quaid – Open Source Image Matching Library


Title	Douglas-Quaid – Open Source Image Matching Library
Authors	Vincent Falconieri
Abstract	Security analysts need to classify, search and correlate numerous images. Automatic classification tools improve the efficiency of such tasks. However, no open-source and turnkey library was found able to reach this goal. The present paper introduces an Open-Source modular library for the specific cases of visual correlation and Image Matching named Douglas-Quaid. The design of the library, chosen tradeoffs, encountered challenges, envisioned solutions as well as quality and speed results are presented in this paper. We also explore researches directions and future potential developments of the library. Our claim is that even partial automation of screenshots classification would reduce the burden on security teams and that Douglas-Quaid is a step forward in this direction.
Tasks
Published	2019-08-12
URL	https://arxiv.org/abs/1908.04014v1
PDF	https://arxiv.org/pdf/1908.04014v1.pdf
PWC	https://paperswithcode.com/paper/douglas-quaid-open-source-image-matching
Repo
Framework