January 27, 2020

3397 words 16 mins read

Paper Group ANR 1340

Modelling of Sickle Cell Anemia Patients Response to Hydroxyurea using Artificial Neural Networks. Attract or Distract: Exploit the Margin of Open Set. FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control. Analysis of an Automated Machine Learning Approach in Brain Pred …

Modelling of Sickle Cell Anemia Patients Response to Hydroxyurea using Artificial Neural Networks


Title	Modelling of Sickle Cell Anemia Patients Response to Hydroxyurea using Artificial Neural Networks
Authors	Brendan E. Odigwe, Jesuloluwa S. Eyitayo, Celestine I. Odigwe, Homayoun Valafar
Abstract	Hydroxyurea (HU) has been shown to be effective in alleviating the symptoms of Sickle Cell Anemia disease. While Hydroxyurea reduces the complications associated with Sickle Cell Anemia in some patients, others do not benefit from this drug and experience deleterious effects since it is also a chemotherapeutic agent. Therefore, to whom, should the administration of HU be considered as a viable option, is the main question asked by the responsible physician. We address this question by developing modeling techniques that can predict a patient’s response to HU and therefore spare the non-responsive patients from the unnecessary effects of HU on the values of 22 parameters that can be obtained from blood samples in 122 patients. Using this data, we developed Deep Artificial Neural Network models that can predict with 92.6% accuracy, the final HbF value of a subject after undergoing HU therapy. Our current studies are focussing on forecasting a patient’s HbF response, 30 days ahead of time.
Tasks
Published	2019-11-25
URL	https://arxiv.org/abs/1911.10978v1
PDF	https://arxiv.org/pdf/1911.10978v1.pdf
PWC	https://paperswithcode.com/paper/modelling-of-sickle-cell-anemia-patients
Repo
Framework

Attract or Distract: Exploit the Margin of Open Set


Title	Attract or Distract: Exploit the Margin of Open Set
Authors	Qianyu Feng, Guoliang Kang, Hehe Fan, Yi Yang
Abstract	Open set domain adaptation aims to diminish the domain shift across domains, with partially shared classes. There exist unknown target samples out of the knowledge of source domain. Compared to the close set setting, how to separate the unknown (unshared) class from the known (shared) ones plays a key role. Whereas, previous methods did not emphasize the semantic structure of the open set data, which may introduce bias into the domain alignment and confuse the classifier around the decision boundary. In this paper, we exploit the semantic structure of open set data from two aspects: 1) Semantic Categorical Alignment, which aims to achieve good separability of target known classes by categorically aligning the centroid of target with the source. 2)Semantic Contrastive Mapping, which aims to push the unknown class away from the decision boundary. Empirically, we demonstrate that our method performs favourably against the state-of-the-art methods on representative benchmarks, e.g. Digit datasets and Office-31 datasets.
Tasks	Domain Adaptation
Published	2019-08-06
URL	https://arxiv.org/abs/1908.01925v2
PDF	https://arxiv.org/pdf/1908.01925v2.pdf
PWC	https://paperswithcode.com/paper/attract-or-distract-exploit-the-margin-of
Repo
Framework

FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control


Title	FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control
Authors	Longxiang Shi, Shijian Li, Longbing Cao, Long Yang, Gang Zheng, Gang Pan
Abstract	In recent years significant progress has been made in dealing with challenging problems using reinforcement learning.Despite its great success, reinforcement learning still faces challenge in continuous control tasks. Conventional methods always compute the derivatives of the optimal goal with a costly computation resources, and are inefficient, unstable and lack of robust-ness when dealing with such tasks. Alternatively, derivative-based methods treat the optimization process as a blackbox and show robustness and stability in learning continuous control tasks, but not data efficient in learning. The combination of both methods so as to get the best of the both has raised attention. However, most of the existing combination works adopt complex neural networks (NNs) as the policy for control. The double-edged sword of deep NNs can yield better performance, but also makes it difficult for parameter tuning and computation. To this end, in this paper we presents a novel method called FiDi-RL, which incorporates deep RL with Finite-Difference (FiDi) policy search.FiDi-RL combines Deep Deterministic Policy Gradients (DDPG)with Augment Random Search (ARS) and aims at improving the data efficiency of ARS. The empirical results show that FiDi-RL can improves the performance and stability of ARS, and provide competitive results against some existing deep reinforcement learning methods
Tasks	Continuous Control
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00526v3
PDF	https://arxiv.org/pdf/1907.00526v3.pdf
PWC	https://paperswithcode.com/paper/fidi-rl-incorporating-deep-reinforcement
Repo
Framework

Analysis of an Automated Machine Learning Approach in Brain Predictive Modelling: A data-driven approach to Predict Brain Age from Cortical Anatomical Measures


Title	Analysis of an Automated Machine Learning Approach in Brain Predictive Modelling: A data-driven approach to Predict Brain Age from Cortical Anatomical Measures
Authors	Jessica Dafflon, Walter H. L Pinaya, Federico Turkheimer, James H. Cole, Robert Leech, Mathew A. Harris, Simon R. Cox, Heather C. Whalley, Andrew M. McIntosh, Peter J. Hellyer
Abstract	The use of machine learning (ML) algorithms has significantly increased in neuroscience. However, from the vast extent of possible ML algorithms, which one is the optimal model to predict the target variable? What are the hyperparameters for such a model? Given the plethora of possible answers to these questions, in the last years, automated machine learning (autoML) has been gaining attention. Here, we apply an autoML library called TPOT which uses a tree-based representation of machine learning pipelines and conducts a genetic-programming based approach to find the model and its hyperparameters that more closely predicts the subject’s true age. To explore autoML and evaluate its efficacy within neuroimaging datasets, we chose a problem that has been the focus of previous extensive study: brain age prediction. Without any prior knowledge, TPOT was able to scan through the model space and create pipelines that outperformed the state-of-the-art accuracy for Freesurfer-based models using only thickness and volume information for anatomical structure. In particular, we compared the performance of TPOT (mean accuracy error (MAE): $4.612 \pm .124$ years) and a Relevance Vector Regression (MAE $5.474 \pm .140$ years). TPOT also suggested interesting combinations of models that do not match the current most used models for brain prediction but generalise well to unseen data. AutoML showed promising results as a data-driven approach to find optimal models for neuroimaging applications.
Tasks	AutoML
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03349v1
PDF	https://arxiv.org/pdf/1910.03349v1.pdf
PWC	https://paperswithcode.com/paper/analysis-of-an-automated-machine-learning
Repo
Framework

A deep Convolutional Neural Network for topology optimization with strong generalization ability


Title	A deep Convolutional Neural Network for topology optimization with strong generalization ability
Authors	Yiquan Zhang, Bo Peng, Xiaoyi Zhou, Cheng Xiang, Dalei Wang
Abstract	This paper proposes a deep Convolutional Neural Network(CNN) with strong generalization ability for structural topology optimization. The architecture of the neural network is made up of encoding and decoding parts, which provide down- and up-sampling operations. In addition, a popular technique, namely U-Net, was adopted to improve the performance of the proposed neural network. The input of the neural network is a well-designed tensor with each channel includes different information for the problem, and the output is the layout of the optimal structure. To train the neural network, a large dataset is generated by a conventional topology optimization approach, i.e. SIMP. The performance of the proposed method was evaluated by comparing its efficiency and accuracy with SIMP on a series of typical optimization problems. Results show that a significant reduction in computation cost was achieved with little sacrifice on the optimality of design solutions. Furthermore, the proposed method can intelligently solve problems under boundary conditions not being included in the training dataset.
Tasks
Published	2019-01-23
URL	https://arxiv.org/abs/1901.07761v3
PDF	https://arxiv.org/pdf/1901.07761v3.pdf
PWC	https://paperswithcode.com/paper/a-deep-convolutional-neural-network-for-1
Repo
Framework


Title	Learning Joint Embedding for Cross-Modal Retrieval
Authors	Donghuo Zeng
Abstract	A cross-modal retrieval process is to use a query in one modality to obtain relevant data in another modality. The challenging issue of cross-modal retrieval lies in bridging the heterogeneous gap for similarity computation, which has been broadly discussed in image-text, audio-text, and video-text cross-modal multimedia data mining and retrieval. However, the gap in temporal structures of different data modalities is not well addressed due to the lack of alignment relationship between temporal cross-modal structures. Our research focuses on learning the correlation between different modalities for the task of cross-modal retrieval. We have proposed an architecture: Supervised-Deep Canonical Correlation Analysis (S-DCCA), for cross-modal retrieval. In this forum paper, we will talk about how to exploit triplet neural networks (TNN) to enhance the correlation learning for cross-modal retrieval. The experimental result shows the proposed TNN-based supervised correlation learning architecture can get the best result when the data representation extracted by supervised learning.
Tasks	Cross-Modal Retrieval
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07673v1
PDF	https://arxiv.org/pdf/1908.07673v1.pdf
PWC	https://paperswithcode.com/paper/190807673
Repo
Framework


Title	Deep Triplet Neural Networks with Cluster-CCA for Audio-Visual Cross-modal Retrieval
Authors	Donghuo Zeng, Yi Yu, Keizo Oyama
Abstract	Cross-modal retrieval aims to retrieve data in one modality by a query in another modality, which has been avery interesting research issue in the filed of multimedia, information retrieval, and computer vision, anddatabase. Most existing works focus on cross-modal retrieval between text-image, text-video, and lyrics-audio.little research addresses cross-modal retrieval between audio and video due to limited audio-video paireddataset and semantic information. The main challenge of audio-visual cross-modal retrieval task focuses on learning joint embeddings from a shared subspace for computing the similarity across different modalities, were generating new representations is to maximize the correlation between audio and visual modalities space. In this work, we propose a novel deep triplet neural network with cluster-based canonical correlationanalysis (TNN-C-CCA), which is an end-to-end supervised learning architecture with audio branch and videobranch. we not only consider the matching pairs in the common space but also compute the mismatching pairs when maximizing the correlation. In particular, two significant contributions are made in this work: i) abetter representation by constructing deep triplet neural network with triplet loss for optimal projections canbe generated to maximize correlation in the shared subspace. ii) positive examples and negative examplesare used in the learning stage to improve the capability of embedding learning between audio and video. Our experiment is run over 5-fold cross-validation, where average performance is applied to demonstratethe performance of audio-video cross-modal retrieval. The experimental results achieved on two different audio-visual datasets show the proposed learning architecture with two branches outperforms the state-of-art cross-modal retrieval methods.
Tasks	Cross-Modal Retrieval, Information Retrieval
Published	2019-08-10
URL	https://arxiv.org/abs/1908.03737v1
PDF	https://arxiv.org/pdf/1908.03737v1.pdf
PWC	https://paperswithcode.com/paper/deep-triplet-neural-networks-with-cluster-cca
Repo
Framework

Multisensory Learning Framework for Robot Drumming


Title	Multisensory Learning Framework for Robot Drumming
Authors	A. Barsky, C. Zito, H. Mori, T. Ogata, J. L. Wyatt
Abstract	The hype about sensorimotor learning is currently reaching high fever, thanks to the latest advancement in deep learning. In this paper, we present an open-source framework for collecting large-scale, time-synchronised synthetic data from highly disparate sensory modalities, such as audio, video, and proprioception, for learning robot manipulation tasks. We demonstrate the learning of non-linear sensorimotor mappings for a humanoid drumming robot that generates novel motion sequences from desired audio data using cross-modal correspondences. We evaluate our system through the quality of its cross-modal retrieval, for generating suitable motion sequences to match desired unseen audio or video sequences.
Tasks	Cross-Modal Retrieval
Published	2019-07-23
URL	https://arxiv.org/abs/1907.09775v1
PDF	https://arxiv.org/pdf/1907.09775v1.pdf
PWC	https://paperswithcode.com/paper/multisensory-learning-framework-for-robot
Repo
Framework

Quantifying Transparency of Machine Learning Systems through Analysis of Contributions


Title	Quantifying Transparency of Machine Learning Systems through Analysis of Contributions
Authors	Iain Barclay, Alun Preece, Ian Taylor, Dinesh Verma
Abstract	Increased adoption and deployment of machine learning (ML) models into business, healthcare and other organisational processes, will result in a growing disconnect between the engineers and researchers who developed the models and the model’s users and other stakeholders, such as regulators or auditors. This disconnect is inevitable, as models begin to be used over a number of years or are shared among third parties through user communities or via commercial marketplaces, and it will become increasingly difficult for users to maintain ongoing insight into the suitability of the parties who created the model, or the data that was used to train it. This could become problematic, particularly where regulations change and once-acceptable standards become outdated, or where data sources are discredited, perhaps judged to be biased or corrupted, either deliberately or unwittingly. In this paper we present a method for arriving at a quantifiable metric capable of ranking the transparency of the process pipelines used to generate ML models and other data assets, such that users, auditors and other stakeholders can gain confidence that they will be able to validate and trust the data sources and human contributors in the systems that they rely on for their business operations. The methodology for calculating the transparency metric, and the type of criteria that could be used to make judgements on the visibility of contributions to systems are explained and illustrated through an example scenario.
Tasks
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03483v1
PDF	https://arxiv.org/pdf/1907.03483v1.pdf
PWC	https://paperswithcode.com/paper/quantifying-transparency-of-machine-learning
Repo
Framework

Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval


Title	Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval
Authors	Stefan Balke, Matthias Dorfer, Luis Carvalho, Andreas Arzt, Gerhard Widmer
Abstract	Connecting large libraries of digitized audio recordings to their corresponding sheet music images has long been a motivation for researchers to develop new cross-modal retrieval systems. In recent years, retrieval systems based on embedding space learning with deep neural networks got a step closer to fulfilling this vision. However, global and local tempo deviations in the music recordings still require careful tuning of the amount of temporal context given to the system. In this paper, we address this problem by introducing an additional soft-attention mechanism on the audio input. Quantitative and qualitative results on synthesized piano data indicate that this attention increases the robustness of the retrieval system by focusing on different parts of the input representation based on the tempo of the audio. Encouraged by these results, we argue for the potential of attention models as a very general tool for many MIR tasks.
Tasks	Cross-Modal Retrieval
Published	2019-06-26
URL	https://arxiv.org/abs/1906.10996v1
PDF	https://arxiv.org/pdf/1906.10996v1.pdf
PWC	https://paperswithcode.com/paper/learning-soft-attention-models-for-tempo
Repo
Framework

Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation


Title	Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation
Authors	Simone Bonechi, Paolo Andreini, Monica Bianchini, Franco Scarselli
Abstract	Providing pixel-level supervisions for scene text segmentation is inherently difficult and costly, so that only few small datasets are available for this task. To face the scarcity of training data, previous approaches based on Convolutional Neural Networks (CNNs) rely on the use of a synthetic dataset for pre-training. However, synthetic data cannot reproduce the complexity and variability of natural images. In this work, we propose to use a weakly supervised learning approach to reduce the domain-shift between synthetic and real data. Leveraging the bounding-box supervision of the COCO-Text and the MLT datasets, we generate weak pixel-level supervisions of real images. In particular, the COCO-Text-Segmentation (COCO_TS) and the MLT-Segmentation (MLT_S) datasets are created and released. These two datasets are used to train a CNN, the Segmentation Multiscale Attention Network (SMANet), which is specifically designed to face some peculiarities of the scene text segmentation task. The SMANet is trained end-to-end on the proposed datasets, and the experiments show that COCO_TS and MLT_S are a valid alternative to synthetic images, allowing to use only a fraction of the training samples and improving significantly the performances.
Tasks
Published	2019-11-19
URL	https://arxiv.org/abs/1911.09026v1
PDF	https://arxiv.org/pdf/1911.09026v1.pdf
PWC	https://paperswithcode.com/paper/weak-supervision-for-generating-pixel-level
Repo
Framework

Soft Constraints for Inference with Declarative Knowledge


Title	Soft Constraints for Inference with Declarative Knowledge
Authors	Zenna Tavares, Javier Burroni, Edgar Minaysan, Armando Solar Lezama, Rajesh Ranganath
Abstract	We develop a likelihood free inference procedure for conditioning a probabilistic model on a predicate. A predicate is a Boolean valued function which expresses a yes/no question about a domain. Our contribution, which we call predicate exchange, constructs a softened predicate which takes value in the unit interval [0, 1] as opposed to a simply true or false. Intuitively, 1 corresponds to true, and a high value (such as 0.999) corresponds to “nearly true” as determined by a distance metric. We define Boolean algebra for soft predicates, such that they can be negated, conjoined and disjoined arbitrarily. A softened predicate can serve as a tractable proxy to a likelihood function for approximate posterior inference. However, to target exact inference, we temper the relaxation by a temperature parameter, and add a accept/reject phase use to replica exchange Markov Chain Mont Carlo, which exchanges states between a sequence of models conditioned on predicates at varying temperatures. We describe a lightweight implementation of predicate exchange that it provides a language independent layer that can be implemented on top of existingn modeling formalisms.
Tasks
Published	2019-01-16
URL	http://arxiv.org/abs/1901.05437v1
PDF	http://arxiv.org/pdf/1901.05437v1.pdf
PWC	https://paperswithcode.com/paper/soft-constraints-for-inference-with
Repo
Framework

Learning to Super Resolve Intensity Images from Events


Title	Learning to Super Resolve Intensity Images from Events
Authors	S. Mohammad Mostafavi I., Jonghyun Choi, Kuk-Jin Yoon
Abstract	An event camera detects per-pixel intensity difference and produces asynchronous event stream with low latency, high dynamic sensing range, and low power consumption. As a trade-off, the event camera has low spatial resolution. We propose an end-to-end network to reconstruct high resolution, high dynamic range (HDR) images from the event streams. The reconstructed images using the proposed method is in better quality than the combination of state-of-the-art intensity image reconstruction algorithms and the state-of-the-art super resolution schemes. We further evaluate our algorithm on multiple real-world sequences showing the ability to generate high quality images in the zero-shot cross dataset transfer setting.
Tasks	Image Reconstruction, Super-Resolution
Published	2019-12-03
URL	https://arxiv.org/abs/1912.01196v1
PDF	https://arxiv.org/pdf/1912.01196v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-super-resolve-intensity-images
Repo
Framework

Thieves on Sesame Street! Model Extraction of BERT-based APIs


Title	Thieves on Sesame Street! Model Extraction of BERT-based APIs
Authors	Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, Mohit Iyyer
Abstract	We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model. Assuming that both the adversary and victim model fine-tune a large pretrained language model such as BERT (Devlin et al. 2019), we show that the adversary does not need any real training data to successfully mount the attack. In fact, the attacker need not even use grammatical or semantically meaningful queries: we show that random sequences of words coupled with task-specific heuristics form effective queries for model extraction on a diverse set of NLP tasks, including natural language inference and question answering. Our work thus highlights an exploit only made feasible by the shift towards transfer learning methods within the NLP community: for a query budget of a few hundred dollars, an attacker can extract a model that performs only slightly worse than the victim model. Finally, we study two defense strategies against model extraction—membership classification and API watermarking—which while successful against naive adversaries, are ineffective against more sophisticated ones.
Tasks	Language Modelling, Natural Language Inference, Question Answering, Transfer Learning
Published	2019-10-27
URL	https://arxiv.org/abs/1910.12366v2
PDF	https://arxiv.org/pdf/1910.12366v2.pdf
PWC	https://paperswithcode.com/paper/thieves-on-sesame-street-model-extraction-of
Repo
Framework

Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics


Title	Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
Authors	Niru Maheswaranathan, Alex Williams, Matthew D. Golub, Surya Ganguli, David Sussillo
Abstract	Recurrent neural networks (RNNs) are a widely used tool for modeling sequential data, yet they are often treated as inscrutable black boxes. Given a trained recurrent network, we would like to reverse engineer it–to obtain a quantitative, interpretable description of how it solves a particular task. Even for simple tasks, a detailed understanding of how recurrent networks work, or a prescription for how to develop such an understanding, remains elusive. In this work, we use tools from dynamical systems analysis to reverse engineer recurrent networks trained to perform sentiment classification, a foundational natural language processing task. Given a trained network, we find fixed points of the recurrent dynamics and linearize the nonlinear system around these fixed points. Despite their theoretical capacity to implement complex, high-dimensional computations, we find that trained networks converge to highly interpretable, low-dimensional representations. In particular, the topological structure of the fixed points and corresponding linearized dynamics reveal an approximate line attractor within the RNN, which we can use to quantitatively understand how the RNN solves the sentiment analysis task. Finally, we find this mechanism present across RNN architectures (including LSTMs, GRUs, and vanilla RNNs) trained on multiple datasets, suggesting that our findings are not unique to a particular architecture or dataset. Overall, these results demonstrate that surprisingly universal and human interpretable computations can arise across a range of recurrent networks.
Tasks	Sentiment Analysis
Published	2019-06-25
URL	https://arxiv.org/abs/1906.10720v2
PDF	https://arxiv.org/pdf/1906.10720v2.pdf
PWC	https://paperswithcode.com/paper/reverse-engineering-recurrent-networks-for
Repo
Framework