January 28, 2020

3100 words 15 mins read

Paper Group ANR 829

Autonomous discovery of battery electrolytes with robotic experimentation and machine-learning. Equivariant Hamiltonian Flows. Proceedings of FACTS-IR 2019. Semi-Supervised Learning of Bearing Anomaly Detection via Deep Variational Autoencoders. ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard. Gated Recurrent Neural Network …

Autonomous discovery of battery electrolytes with robotic experimentation and machine-learning


Title	Autonomous discovery of battery electrolytes with robotic experimentation and machine-learning
Authors	Adarsh Dave, Jared Mitchell, Kirthevasan Kandasamy, Sven Burke, Biswajit Paria, Barnabas Poczos, Jay Whitacre, Venkatasubramanian Viswanathan
Abstract	Innovations in batteries take years to formulate and commercialize, requiring extensive experimentation during the design and optimization phases. We approached the design and selection of a battery electrolyte through a black-box optimization algorithm directly integrated into a robotic test-stand. We report here the discovery of a novel battery electrolyte by this experiment completely guided by the machine-learning software without human intervention. Motivated by the recent trend toward super-concentrated aqueous electrolytes for high-performance batteries, we utilize Dragonfly - a Bayesian machine-learning software package - to search mixtures of commonly used lithium and sodium salts for super-concentrated aqueous electrolytes with wide electrochemical stability windows. Dragonfly autonomously managed the robotic test-stand, recommending electrolyte designs to test and receiving experimental feedback in real time. In 40 hours of continuous experimentation over a four-dimensional design space with millions of potential candidates, Dragonfly discovered a novel, mixed-anion aqueous sodium electrolyte with a wider electrochemical stability window than state-of-the-art sodium electrolyte. A human-guided design process may have missed this optimal electrolyte. This result demonstrates the possibility of integrating robotics with machine-learning to rapidly and autonomously discover novel battery materials.
Tasks
Published	2019-10-22
URL	https://arxiv.org/abs/2001.09938v1
PDF	https://arxiv.org/pdf/2001.09938v1.pdf
PWC	https://paperswithcode.com/paper/autonomous-discovery-of-battery-electrolytes
Repo
Framework

Equivariant Hamiltonian Flows


Title	Equivariant Hamiltonian Flows
Authors	Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins, Peter Toth
Abstract	This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformations while providing an equivariant representation of the data. We provide proof of principle demonstrations of how such flows can be learnt, as well as how the addition of symmetry invariance constraints can improve data efficiency and generalisation. Finally, we make connections to disentangled representation learning and show how this work relates to a recently proposed definition.
Tasks	Representation Learning
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13739v1
PDF	https://arxiv.org/pdf/1909.13739v1.pdf
PWC	https://paperswithcode.com/paper/equivariant-hamiltonian-flows
Repo
Framework

Proceedings of FACTS-IR 2019


Title	Proceedings of FACTS-IR 2019
Authors	Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D. Ekstrand
Abstract	The proceedings list for the program of FACTS-IR 2019, the Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval held at SIGIR 2019.
Tasks	Information Retrieval
Published	2019-07-12
URL	https://arxiv.org/abs/1907.05755v1
PDF	https://arxiv.org/pdf/1907.05755v1.pdf
PWC	https://paperswithcode.com/paper/proceedings-of-facts-ir-2019
Repo
Framework

Semi-Supervised Learning of Bearing Anomaly Detection via Deep Variational Autoencoders


Title	Semi-Supervised Learning of Bearing Anomaly Detection via Deep Variational Autoencoders
Authors	Shen Zhang, Fei Ye, Bingnan Wang, Thomas G. Habetler
Abstract	Most of the data-driven approaches applied to bearing fault diagnosis up to date are established in the supervised learning paradigm, which usually requires a large set of labeled data collected a priori. In practical applications, however, obtaining accurate labels based on real-time bearing conditions can be far more challenging than simply collecting a huge amount of unlabeled data using various sensors. In this paper, we thus propose a semi-supervised learning approach for bearing anomaly detection using variational autoencoder (VAE) based deep generative models, which allows for effective utilization of dataset when only a small subset of data have labels. Finally, a series of experiments is performed using both the Case Western Reserve University (CWRU) bearing dataset and the University of Cincinnati’s Center for Intelligent Maintenance Systems (IMS) dataset. The experimental results demonstrate that the proposed semi-supervised learning scheme greatly outperforms two mainstream semi-supervised learning approaches and a baseline supervised convolutional neural network approach, with the overall accuracy improvement ranging between 3% to 30% using different proportions of labeled samples.
Tasks	Anomaly Detection
Published	2019-12-02
URL	https://arxiv.org/abs/1912.01096v2
PDF	https://arxiv.org/pdf/1912.01096v2.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-learning-of-bearing-anomaly
Repo
Framework

ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard


Title	ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard
Authors	Xi Liu, Rui Zhang, Yongsheng Zhou, Qianyi Jiang, Qi Song, Nan Li, Kai Zhou, Lei Wang, Dong Wang, Minghui Liao, Mingkun Yang, Xiang Bai, Baoguang Shi, Dimosthenis Karatzas, Shijian Lu, C. V. Jawahar
Abstract	Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinesecharacters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is http://rrc.cvc.uab.es/?ch=12.
Tasks
Published	2019-12-20
URL	https://arxiv.org/abs/1912.09641v1
PDF	https://arxiv.org/pdf/1912.09641v1.pdf
PWC	https://paperswithcode.com/paper/icdar-2019-robust-reading-challenge-on
Repo
Framework

Gated Recurrent Neural Network Approach for Multilabel Emotion Detection in Microblogs


Title	Gated Recurrent Neural Network Approach for Multilabel Emotion Detection in Microblogs
Authors	Prabod Rathnayaka, Supun Abeysinghe, Chamod Samarajeewa, Isura Manchanayake, Malaka J. Walpola, Rashmika Nawaratne, Tharindu Bandaragoda, Damminda Alahakoon
Abstract	People express their opinions and emotions freely in social media posts and online reviews that contain valuable feedback for multiple stakeholders such as businesses and political campaigns. Manually extracting opinions and emotions from large volumes of such posts is an impossible task. Therefore, automated processing of these posts to extract opinions and emotions is an important research problem. However, human emotion detection is a challenging task due to the complexity and nuanced nature. To overcome these barriers, researchers have extensively used techniques such as deep learning, distant supervision, and transfer learning. In this paper, we propose a novel Pyramid Attention Network (PAN) based model for emotion detection in microblogs. The main advantage of our approach is that PAN has the capability to evaluate sentences in different perspectives to capture multiple emotions existing in a single text. The proposed model was evaluated on a recently released dataset and the results achieved the state-of-the-art accuracy of 58.9%.
Tasks	Transfer Learning
Published	2019-07-17
URL	https://arxiv.org/abs/1907.07653v1
PDF	https://arxiv.org/pdf/1907.07653v1.pdf
PWC	https://paperswithcode.com/paper/gated-recurrent-neural-network-approach-for
Repo
Framework

Categorical Feature Compression via Submodular Optimization


Title	Categorical Feature Compression via Submodular Optimization
Authors	MohammadHossein Bateni, Lin Chen, Hossein Esfandiari, Thomas Fu, Vahab S. Mirrokni, Afshin Rostamizadeh
Abstract	In the era of big data, learning from categorical features with very large vocabularies (e.g., 28 million for the Criteo click prediction dataset) has become a practical challenge for machine learning researchers and practitioners. We design a highly-scalable vocabulary compression algorithm that seeks to maximize the mutual information between the compressed categorical feature and the target binary labels and we furthermore show that its solution is guaranteed to be within a $1-1/e \approx 63%$ factor of the global optimal solution. To achieve this, we introduce a novel re-parametrization of the mutual information objective, which we prove is submodular, and design a data structure to query the submodular function in amortized $O(\log n )$ time (where $n$ is the input vocabulary size). Our complete algorithm is shown to operate in $O(n \log n )$ time. Additionally, we design a distributed implementation in which the query data structure is decomposed across $O(k)$ machines such that each machine only requires $O(\frac n k)$ space, while still preserving the approximation guarantee and using only logarithmic rounds of computation. We also provide analysis of simple alternative heuristic compression methods to demonstrate they cannot achieve any approximation guarantee. Using the large-scale Criteo learning task, we demonstrate better performance in retaining mutual information and also verify competitive learning performance compared to other baseline methods.
Tasks
Published	2019-04-30
URL	http://arxiv.org/abs/1904.13389v1
PDF	http://arxiv.org/pdf/1904.13389v1.pdf
PWC	https://paperswithcode.com/paper/categorical-feature-compression-via
Repo
Framework

DSReg: Using Distant Supervision as a Regularizer


Title	DSReg: Using Distant Supervision as a Regularizer
Authors	Yuxian Meng, Muyu Li, Xiaoya Li, Wei Wu, Jiwei Li
Abstract	In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples. We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. The original task is converted to a multi-task learning problem, in which distant supervision is used to retrieve hard-negative examples. The obtained hard-negative examples are then used as a regularizer. The original target objective of distinguishing positive examples from negative examples is jointly optimized with the auxiliary task objective of distinguishing softened positive (i.e., hard-negative examples plus positive examples) from easy-negative examples. In the neural context, this can be done by outputting the same representation from the last neural layer to different $softmax$ functions. Using this strategy, we can improve the performance of baseline models in a range of different NLP tasks, including text classification, sequence labeling and reading comprehension.
Tasks	Multi-Task Learning, Reading Comprehension, Text Classification
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11658v3
PDF	https://arxiv.org/pdf/1905.11658v3.pdf
PWC	https://paperswithcode.com/paper/dsreg-using-distant-supervision-as-a
Repo
Framework

Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering


Title	Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering
Authors	Siheng Chen, Chaojing Duan, Yaoqing Yang, Duanshun Li, Chen Feng, Dong Tian
Abstract	We propose a deep autoencoder with graph topology inference and filtering to achieve compact representations of unorganized 3D point clouds in an unsupervised manner. Many previous works discretize 3D points to voxels and then use lattice-based methods to process and learn 3D spatial information; however, this leads to inevitable discretization errors. In this work, we handle raw 3D points without such compromise. The proposed networks follow the autoencoder framework with a focus on designing the decoder. The encoder adopts similar architectures as in PointNet. The decoder involves three novel modules. The folding module folds a canonical 2D lattice to the underlying surface of a 3D point cloud, achieving coarse reconstruction; the graph-topology-inference module learns a graph topology to represent pairwise relationships between 3D points, pushing the latent code to preserve both coordinates and pairwise relationships of points in 3D point clouds; and the graph-filtering module couples the above two modules, refining the coarse reconstruction through a learnt graph topology to obtain the final reconstruction. The proposed decoder leverages a learnable graph topology to push the codeword to preserve representative features and further improve the unsupervised-learning performance. We further provide theoretical analyses of the proposed architecture. In the experiments, we validate the proposed networks in three tasks, including 3D point cloud reconstruction, visualization, and transfer classification. The experimental results show that (1) the proposed networks outperform the state-of-the-art methods in various tasks; (2) a graph topology can be inferred as auxiliary information without specific supervision on graph topology inference; and (3) graph filtering refines the reconstruction, leading to better performances.
Tasks
Published	2019-05-11
URL	https://arxiv.org/abs/1905.04571v2
PDF	https://arxiv.org/pdf/1905.04571v2.pdf
PWC	https://paperswithcode.com/paper/deep-unsupervised-learning-of-3d-point-clouds
Repo
Framework

On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraints


Title	On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraints
Authors	Damien Teney, Ehsan Abbasnejad, Anton van den Hengel
Abstract	The knowledge that humans hold about a problem often extends far beyond a set of training data and output labels. While the success of deep learning mostly relies on supervised training, important properties cannot be inferred efficiently from end-to-end annotations alone, for example causal relations or domain-specific invariances. We present a general technique to supplement supervised training with prior knowledge expressed as relations between training instances. We illustrate the method on the task of visual question answering to exploit various auxiliary annotations, including relations of equivalence and of logical entailment between questions. Existing methods to use these annotations, including auxiliary losses and data augmentation, cannot guarantee the strict inclusion of these relations into the model since they require a careful balancing against the end-to-end objective. Our method uses these relations to shape the embedding space of the model, and treats them as strict constraints on its learned representations. In the context of VQA, this approach brings significant improvements in accuracy and robustness, in particular over the common practice of incorporating the constraints as a soft regularizer. We also show that incorporating this type of prior knowledge with our method brings consistent improvements, independently from the amount of supervised data used. It demonstrates the value of an additional training signal that is otherwise difficult to extract from end-to-end annotations alone.
Tasks	Data Augmentation, Question Answering, Visual Question Answering
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13471v2
PDF	https://arxiv.org/pdf/1909.13471v2.pdf
PWC	https://paperswithcode.com/paper/on-incorporating-semantic-prior-knowlegde-in-1
Repo
Framework

Towards Understanding of Medical Randomized Controlled Trials by Conclusion Generation


Title	Towards Understanding of Medical Randomized Controlled Trials by Conclusion Generation
Authors	Alexander Te-Wei Shieh, Yung-Sung Chuang, Shang-Yu Su, Yun-Nung Chen
Abstract	Randomized controlled trials (RCTs) represent the paramount evidence of clinical medicine. Using machines to interpret the massive amount of RCTs has the potential of aiding clinical decision-making. We propose a RCT conclusion generation task from the PubMed 200k RCT sentence classification dataset to examine the effectiveness of sequence-to-sequence models on understanding RCTs. We first build a pointer-generator baseline model for conclusion generation. Then we fine-tune the state-of-the-art GPT-2 language model, which is pre-trained with general domain data, for this new medical domain task. Both automatic and human evaluation show that our GPT-2 fine-tuned models achieve improved quality and correctness in the generated conclusions compared to the baseline pointer-generator model. Further inspection points out the limitations of this current approach and future directions to explore.
Tasks	Decision Making, Language Modelling, Sentence Classification
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01462v1
PDF	https://arxiv.org/pdf/1910.01462v1.pdf
PWC	https://paperswithcode.com/paper/towards-understanding-of-medical-randomized
Repo
Framework

Learning Surrogate Losses


Title	Learning Surrogate Losses
Authors	Josif Grabocka, Randolf Scholz, Lars Schmidt-Thieme
Abstract	The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc.) seamlessly. Our strategy learns smooth relaxation versions of the true losses by approximating them through a surrogate neural network. The proposed loss networks are set-wise models which are invariant to the order of mini-batch instances. Ultimately, the surrogate losses are learned jointly with the prediction model via bilevel optimization. Empirical results on multiple datasets with diverse real-life loss functions compared with state-of-the-art baselines demonstrate the efficiency of learning surrogate losses.
Tasks	bilevel optimization
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10108v1
PDF	https://arxiv.org/pdf/1905.10108v1.pdf
PWC	https://paperswithcode.com/paper/learning-surrogate-losses
Repo
Framework

Capturing human categorization of natural images at scale by combining deep networks and cognitive models


Title	Capturing human categorization of natural images at scale by combining deep networks and cognitive models
Authors	Ruairidh M. Battleday, Joshua C. Peterson, Thomas L. Griffiths
Abstract	Human categorization is one of the most important and successful targets of cognitive modeling in psychology, yet decades of development and assessment of competing models have been contingent on small sets of simple, artificial experimental stimuli. Here we extend this modeling paradigm to the domain of natural images, revealing the crucial role that stimulus representation plays in categorization and its implications for conclusions about how people form categories. Applying psychological models of categorization to natural images required two significant advances. First, we conducted the first large-scale experimental study of human categorization, involving over 500,000 human categorization judgments of 10,000 natural images from ten non-overlapping object categories. Second, we addressed the traditional bottleneck of representing high-dimensional images in cognitive models by exploring the best of current supervised and unsupervised deep and shallow machine learning methods. We find that selecting sufficiently expressive, data-driven representations is crucial to capturing human categorization, and using these representations allows simple models that represent categories with abstract prototypes to outperform the more complex memory-based exemplar accounts of categorization that have dominated in studies using less naturalistic stimuli.
Tasks
Published	2019-04-26
URL	http://arxiv.org/abs/1904.12690v1
PDF	http://arxiv.org/pdf/1904.12690v1.pdf
PWC	https://paperswithcode.com/paper/capturing-human-categorization-of-natural
Repo
Framework

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference


Title	White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
Authors	Alexandre Sablayrolles, Matthijs Douze, Yann Ollivier, Cordelia Schmid, Hervé Jégou
Abstract	Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depend on the loss function, and thus black-box attacks are as good as white-box attacks. As the optimal strategy is not tractable, we provide approximations of it leading to several inference methods, and show that existing membership inference methods are coarser approximations of this optimal strategy. Our membership attacks outperform the state of the art in various settings, ranging from a simple logistic regression to more complex architectures and datasets, such as ResNet-101 and Imagenet.
Tasks
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11229v1
PDF	https://arxiv.org/pdf/1908.11229v1.pdf
PWC	https://paperswithcode.com/paper/white-box-vs-black-box-bayes-optimal
Repo
Framework

Bone Age Estimation by Deep Learning in X-Ray Medical Images


Title	Bone Age Estimation by Deep Learning in X-Ray Medical Images
Authors	Behnam Kiani Kalejahi, Saeed Meshgini, Sabalan Daneshvar
Abstract	Patient skeletal age estimation using a skeletal bone age assessment method is a time consuming and very boring process. Today, in order to overcome these deficiencies, computerized techniques are used to replace hand-held techniques in the medical industry, to the extent that this results in the better evaluation. The purpose of this research is to minimize the problems of the division of existing systems with deep learning algorithms and the high accuracy of diagnosis. The evaluation of skeletal bone age is the most clinical application for the study of endocrinology, genetic disorders and growth in young people. This assessment is usually performed using the radiologic analysis of the left wrist using the GP(Greulich-Pyle) technique or the TW(Tanner-Whitehouse) technique. Both techniques have many disadvantages, including a lack of human deductions from observations as well as being time-consuming.
Tasks	Age Estimation
Published	2019-12-15
URL	https://arxiv.org/abs/1912.06650v1
PDF	https://arxiv.org/pdf/1912.06650v1.pdf
PWC	https://paperswithcode.com/paper/bone-age-estimation-by-deep-learning-in-x-ray
Repo
Framework