January 31, 2020

3182 words 15 mins read

Paper Group ANR 40

Simple but effective techniques to reduce biases. Learning the Right Expansion-ordering Heuristics for Satisfiability Testing in OWL Reasoners. DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD. 3DBGrowth: volumetric vertebrae segmentation and reconstruction in magnetic resonance imaging. Multilingual Bottleneck Featur …

Simple but effective techniques to reduce biases


Title	Simple but effective techniques to reduce biases
Authors	Rabeeh Karimi Mahabadi, James Henderson
Abstract	There have been several studies recently showing that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models which fail to generalize to out-of-domain datasets, and are likely to perform poorly in real-world scenarios. We propose several learning strategies to train neural models which are more robust to such biases and transfer better to out-of-domain datasets. We introduce an additional lightweight bias-only model which learns dataset biases and uses its prediction to adjust the loss of the base model to reduce the biases. In other words, our methods down-weight the importance of the biased examples, and focus training on hard examples, i.e. examples that cannot be correctly classified by only relying on biases. Our approaches are model agnostic and simple to implement. We experiment on large-scale natural language inference and fact verification datasets and their out-of-domain datasets and show that our debiased models significantly improve the robustness in all settings, including gaining 9.76 points on the FEVER symmetric evaluation dataset, 5.45 on the HANS dataset and 4.78 points on the SNLI hard set. These datasets are specifically designed to assess the robustness of models in the out-of-domain setting where typical biases in the training data do not exist in the evaluation set.
Tasks	Natural Language Inference
Published	2019-09-13
URL	https://arxiv.org/abs/1909.06321v2
PDF	https://arxiv.org/pdf/1909.06321v2.pdf
PWC	https://paperswithcode.com/paper/simple-but-effective-techniques-to-reduce
Repo
Framework

Learning the Right Expansion-ordering Heuristics for Satisfiability Testing in OWL Reasoners


Title	Learning the Right Expansion-ordering Heuristics for Satisfiability Testing in OWL Reasoners
Authors	Razieh Mehri, Volker Haarslev, Hamidreza Chinaei
Abstract	Web Ontology Language (OWL) reasoners are used to infer new logical relations from ontologies. While inferring new facts, these reasoners can be further optimized, e.g., by properly ordering disjuncts in disjunction expressions of ontologies for satisfiability testing of concepts. Different expansion-ordering heuristics have been developed for this purpose. The built-in heuristics in these reasoners determine the order for branches in search trees while each heuristic choice causes different effects for various ontologies depending on the ontologies’ syntactic structure and probably other features as well. A learning-based approach that takes into account the features aims to select an appropriate expansion-ordering heuristic for each ontology. The proper choice is expected to accelerate the reasoning process for the reasoners. In this paper, the effect of our methodology is investigated on a well-known reasoner that is JFact. Our experiments show the average speedup by a factor of one to two orders of magnitude for satisfiability testing after applying learning methodology for selecting the right expansion-ordering heuristics.
Tasks
Published	2019-04-20
URL	http://arxiv.org/abs/1904.09443v1
PDF	http://arxiv.org/pdf/1904.09443v1.pdf
PWC	https://paperswithcode.com/paper/learning-the-right-expansion-ordering
Repo
Framework

DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD


Title	DTN: A Learning Rate Scheme with Convergence Rate of $\mathcal{O}(1/t)$ for SGD
Authors	Lam M. Nguyen, Phuong Ha Nguyen, Dzung T. Phan, Jayant R. Kalagnanam, Marten van Dijk
Abstract	This paper has some inconsistent results, i.e., we made some failed claims because we did some mistakes for using the test criterion for a series. Precisely, our claims on the convergence rate of $\mathcal{O}(1/t)$ of SGD presented in Theorem 1, Corollary 1, Theorem 2 and Corollary 2 are wrongly derived because they are based on Lemma 5. In Lemma 5, we do not correctly use the test criterion for a series. Hence, the result of Lemma 5 is not valid. We would like to thank the community for pointing out this mistake!
Tasks
Published	2019-01-22
URL	http://arxiv.org/abs/1901.07634v3
PDF	http://arxiv.org/pdf/1901.07634v3.pdf
PWC	https://paperswithcode.com/paper/dtn-a-learning-rate-scheme-with-convergence
Repo
Framework

3DBGrowth: volumetric vertebrae segmentation and reconstruction in magnetic resonance imaging


Title	3DBGrowth: volumetric vertebrae segmentation and reconstruction in magnetic resonance imaging
Authors	Jonathan S. Ramos, Mirela T. Cazzolato, Bruno S. Faiçal, Marcello H. Nogueira-Barbosa, Caetano Traina Jr., Agma J. M. Traina
Abstract	Segmentation of medical images is critical for making several processes of analysis and classification more reliable. With the growing number of people presenting back pain and related problems, the semi-automatic segmentation and 3D reconstruction of vertebral bodies became even more important to support decision making. A 3D reconstruction allows a fast and objective analysis of each vertebrae condition, which may play a major role in surgical planning and evaluation of suitable treatments. In this paper, we propose 3DBGrowth, which develops a 3D reconstruction over the efficient Balanced Growth method for 2D images. We also take advantage of the slope coefficient from the annotation time to reduce the total number of annotated slices, reducing the time spent on manual annotation. We show experimental results on a representative dataset with 17 MRI exams demonstrating that our approach significantly outperforms the competitors and, on average, only 37% of the total slices with vertebral body content must be annotated without losing performance/accuracy. Compared to the state-of-the-art methods, we have achieved a Dice Score gain of over 5% with comparable processing time. Moreover, 3DBGrowth works well with imprecise seed points, which reduces the time spent on manual annotation by the specialist.
Tasks	3D Reconstruction, Decision Making
Published	2019-06-25
URL	https://arxiv.org/abs/1906.10288v2
PDF	https://arxiv.org/pdf/1906.10288v2.pdf
PWC	https://paperswithcode.com/paper/3dbgrowth-volumetric-vertebrae-segmentation
Repo
Framework

Multilingual Bottleneck Features for Query by Example Spoken Term Detection


Title	Multilingual Bottleneck Features for Query by Example Spoken Term Detection
Authors	Dhananjay Ram, Lesly Miculicich, Hervé Bourlard
Abstract	State of the art solutions to query by example spoken term detection (QbE-STD) usually rely on bottleneck feature representation of the query and audio document to perform dynamic time warping (DTW) based template matching. Here, we present a study on QbE-STD performance using several monolingual as well as multilingual bottleneck features extracted from feed forward networks. Then, we propose to employ residual networks (ResNet) to estimate the bottleneck features and show significant improvements over the corresponding feed forward network based features. The neural networks are trained on GlobalPhone corpus and QbE-STD experiments are performed on a very challenging QUESST 2014 database.
Tasks
Published	2019-06-30
URL	https://arxiv.org/abs/1907.00443v1
PDF	https://arxiv.org/pdf/1907.00443v1.pdf
PWC	https://paperswithcode.com/paper/multilingual-bottleneck-features-for-query-by
Repo
Framework

Better Paracoherent Answer Sets with Less Resources


Title	Better Paracoherent Answer Sets with Less Resources
Authors	Giovanni Amendola, Carmine Dodaro, Francesco Ricca
Abstract	Answer Set Programming (ASP) is a well-established formalism for logic programming. Problem solving in ASP requires to write an ASP program whose answers sets correspond to solutions. Albeit the non-existence of answer sets for some ASP programs can be considered as a modeling feature, it turns out to be a weakness in many other cases, and especially for query answering. Paracoherent answer set semantics extend the classical semantics of ASP to draw meaningful conclusions also from incoherent programs, with the result of increasing the range of applications of ASP. State of the art implementations of paracoherent ASP adopt the semi-equilibrium semantics, but cannot be lifted straightforwardly to compute efficiently the (better) split semi-equilibrium semantics that discards undesirable semi-equilibrium models. In this paper an efficient evaluation technique for computing a split semi-equilibrium model is presented. An experiment on hard benchmarks shows that better paracoherent answer sets can be computed consuming less computational resources than existing methods. Under consideration for acceptance in TPLP.
Tasks
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09560v1
PDF	https://arxiv.org/pdf/1907.09560v1.pdf
PWC	https://paperswithcode.com/paper/better-paracoherent-answer-sets-with-less
Repo
Framework

Biophysical models of cis-regulation as interpretable neural networks


Title	Biophysical models of cis-regulation as interpretable neural networks
Authors	Ammar Tareen, Justin B. Kinney
Abstract	The adoption of deep learning techniques in genomics has been hindered by the difficulty of mechanistically interpreting the models that these techniques produce. In recent years, a variety of post-hoc attribution methods have been proposed for addressing this neural network interpretability problem in the context of gene regulation. Here we describe a complementary way of approaching this problem. Our strategy is based on the observation that two large classes of biophysical models of cis-regulatory mechanisms can be expressed as deep neural networks in which nodes and weights have explicit physiochemical interpretations. We also demonstrate how such biophysical networks can be rapidly inferred, using modern deep learning frameworks, from the data produced by certain types of massively parallel reporter assays (MPRAs). These results suggest a scalable strategy for using MPRAs to systematically characterize the biophysical basis of gene regulation in a wide range of biological contexts. They also highlight gene regulation as a promising venue for the development of scientifically interpretable approaches to deep learning.
Tasks
Published	2019-12-30
URL	https://arxiv.org/abs/2001.03560v2
PDF	https://arxiv.org/pdf/2001.03560v2.pdf
PWC	https://paperswithcode.com/paper/biophysical-models-of-cis-regulation-as
Repo
Framework

Improved Techniques for Training Adaptive Deep Networks


Title	Improved Techniques for Training Adaptive Deep Networks
Authors	Hao Li, Hong Zhang, Xiaojuan Qi, Ruigang Yang, Gao Huang
Abstract	Adaptive inference is a promising technique to improve the computational efficiency of deep models at test time. In contrast to static models which use the same computation graph for all instances, adaptive networks can dynamically adjust their structure conditioned on each input. While existing research on adaptive inference mainly focuses on designing more advanced architectures, this paper investigates how to train such networks more effectively. Specifically, we consider a typical adaptive deep network with multiple intermediate classifiers. We present three techniques to improve its training efficacy from two aspects: 1) a Gradient Equilibrium algorithm to resolve the conflict of learning of different classifiers; 2) an Inline Subnetwork Collaboration approach and a One-for-all Knowledge Distillation algorithm to enhance the collaboration among classifiers. On multiple datasets (CIFAR-10, CIFAR-100 and ImageNet), we show that the proposed approach consistently leads to further improved efficiency on top of state-of-the-art adaptive deep networks.
Tasks
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06294v1
PDF	https://arxiv.org/pdf/1908.06294v1.pdf
PWC	https://paperswithcode.com/paper/improved-techniques-for-training-adaptive
Repo
Framework

Tomographic reconstruction to detect evolving structures


Title	Tomographic reconstruction to detect evolving structures
Authors	Preeti Gopal, Sharat Chandran, Imants Svalbe, Ajit Rajwade
Abstract	The need for tomographic reconstruction from sparse measurements arises when the measurement process is potentially harmful, needs to be rapid, or is uneconomical. In such cases, information from previous longitudinal scans of the same object helps to reconstruct the current object while requiring significantly fewer updating measurements. Our work is based on longitudinal data acquisition scenarios where we wish to study new changes that evolve within an object over time, such as in repeated scanning for disease monitoring, or in tomography-guided surgical procedures. While this is easily feasible when measurements are acquired from a large number of projection views, it is challenging when the number of views is limited. If the goal is to track the changes while simultaneously reducing sub-sampling artefacts, we propose (1) acquiring measurements from a small number of views and using a global unweighted prior-based reconstruction. If the goal is to observe details of new changes, we propose (2) acquiring measurements from a moderate number of views and using a more involved reconstruction routine. We show that in the latter case, a weighted technique is necessary in order to prevent the prior from adversely affecting the reconstruction of new structures that are absent in any of the earlier scans. The reconstruction of new regions is safeguarded from the bias of the prior by computing regional weights that moderate the local influence of the priors. We are thus able to effectively reconstruct both the old and the new structures in the test. In addition to testing on simulated data, we have validated the efficacy of our method on real tomographic data. The results demonstrate the use of both unweighted and weighted priors in different scenarios.
Tasks
Published	2019-09-11
URL	https://arxiv.org/abs/1909.05686v1
PDF	https://arxiv.org/pdf/1909.05686v1.pdf
PWC	https://paperswithcode.com/paper/tomographic-reconstruction-to-detect-evolving
Repo
Framework

Character n-gram Embeddings to Improve RNN Language Models


Title	Character n-gram Embeddings to Improve RNN Language Models
Authors	Sho Takase, Jun Suzuki, Masaaki Nagata
Abstract	This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information. We focus on character n-grams based on research in the field of word embedding construction (Wieting et al. 2016). Our proposed method constructs word embeddings from character n-gram embeddings and combines them with ordinary word embeddings. We demonstrate that the proposed method achieves the best perplexities on the language modeling datasets: Penn Treebank, WikiText-2, and WikiText-103. Moreover, we conduct experiments on application tasks: machine translation and headline generation. The experimental results indicate that our proposed method also positively affects these tasks.
Tasks	Language Modelling, Machine Translation, Word Embeddings
Published	2019-06-13
URL	https://arxiv.org/abs/1906.05506v1
PDF	https://arxiv.org/pdf/1906.05506v1.pdf
PWC	https://paperswithcode.com/paper/character-n-gram-embeddings-to-improve-rnn
Repo
Framework

The Price of Local Fairness in Multistage Selection


Title	The Price of Local Fairness in Multistage Selection
Authors	Vitalii Emelianov, George Arvanitakis, Nicolas Gast, Krishna Gummadi, Patrick Loiseau
Abstract	The rise of algorithmic decision making led to active researches on how to define and guarantee fairness, mostly focusing on one-shot decision making. In several important applications such as hiring, however, decisions are made in multiple stage with additional information at each stage. In such cases, fairness issues remain poorly understood. In this paper we study fairness in $k$-stage selection problems where additional features are observed at every stage. We first introduce two fairness notions, local (per stage) and global (final stage) fairness, that extend the classical fairness notions to the $k$-stage setting. We propose a simple model based on a probabilistic formulation and show that the locally and globally fair selections that maximize precision can be computed via a linear program. We then define the price of local fairness to measure the loss of precision induced by local constraints; and investigate theoretically and empirically this quantity. In particular, our experiments show that the price of local fairness is generally smaller when the sensitive attribute is observed at the first stage; but globally fair selections are more locally fair when the sensitive attribute is observed at the second stage—hence in both cases it is often possible to have a selection that has a small price of local fairness and is close to locally fair.
Tasks	Decision Making
Published	2019-06-15
URL	https://arxiv.org/abs/1906.06613v1
PDF	https://arxiv.org/pdf/1906.06613v1.pdf
PWC	https://paperswithcode.com/paper/the-price-of-local-fairness-in-multistage
Repo
Framework

Stochastic AUC Maximization with Deep Neural Networks


Title	Stochastic AUC Maximization with Deep Neural Networks
Authors	Mingrui Liu, Zhuoning Yuan, Yiming Ying, Tianbao Yang
Abstract	Stochastic AUC maximization has garnered an increasing interest due to better fit to imbalanced data classification. However, existing works are limited to stochastic AUC maximization with a linear predictive model, which restricts its predictive power when dealing with extremely complex data. In this paper, we consider stochastic AUC maximization problem with a deep neural network as the predictive model. Building on the saddle point reformulation of a surrogated loss of AUC, the problem can be cast into a {\it non-convex concave} min-max problem. The main contribution made in this paper is to make stochastic AUC maximization more practical for deep neural networks and big data with theoretical insights as well. In particular, we propose to explore Polyak-\L{}ojasiewicz (PL) condition that has been proved and observed in deep learning, which enables us to develop new stochastic algorithms with even faster convergence rate and more practical step size scheme. An AdaGrad-style algorithm is also analyzed under the PL condition with adaptive convergence rate. Our experimental results demonstrate the effectiveness of the proposed algorithms.
Tasks
Published	2019-08-28
URL	https://arxiv.org/abs/1908.10831v3
PDF	https://arxiv.org/pdf/1908.10831v3.pdf
PWC	https://paperswithcode.com/paper/stochastic-auc-maximization-with-deep-neural
Repo
Framework

Fairness Sample Complexity and the Case for Human Intervention


Title	Fairness Sample Complexity and the Case for Human Intervention
Authors	Ananth Balashankar, Alyssa Lees
Abstract	With the aim of building machine learning systems that incorporate standards of fairness and accountability, we explore explicit subgroup sample complexity bounds. The work is motivated by the observation that classifier predictions for real world datasets often demonstrate drastically different metrics, such as accuracy, when subdivided by specific sensitive variable subgroups. The reasons for these discrepancies are varied and not limited to the influence of mitigating variables, institutional bias, underlying population distributions as well as sampling bias. Among the numerous definitions of fairness that exist, we argue that at a minimum, principled ML practices should ensure that classification predictions are able to mirror the underlying sub-population distributions. However, as the number of sensitive variables increase, populations meeting at the intersectionality of these variables may simply not exist or may not be large enough to provide accurate samples for classification. In these increasingly likely scenarios, we make the case for human intervention and applying situational and individual definitions of fairness. In this paper we present lower bounds of subgroup sample complexity for metric-fair learning based on the theory of Probably Approximately Metric Fair Learning. We demonstrate that for a classifier to approach a definition of fairness in terms of specific sensitive variables, adequate subgroup population samples need to exist and the model dimensionality has to be aligned with subgroup population distributions. In cases where this is not feasible, we propose an approach using individual fairness definitions for achieving alignment. We look at two commonly explored UCI datasets under this lens and suggest human interventions for data collection for specific subgroups to achieve approximate individual fairness for linear hypotheses.
Tasks
Published	2019-10-24
URL	https://arxiv.org/abs/1910.11452v1
PDF	https://arxiv.org/pdf/1910.11452v1.pdf
PWC	https://paperswithcode.com/paper/fairness-sample-complexity-and-the-case-for
Repo
Framework

Visualizing and Describing Fine-grained Categories as Textures


Title	Visualizing and Describing Fine-grained Categories as Textures
Authors	Tsung-Yu Lin, Mikayla Timm, Chenyun Wu, Subhransu Maji
Abstract	We analyze how categories from recent FGVC challenges can be described by their textural content. The motivation is that subtle differences between species of birds or butterflies can often be described in terms of the texture associated with them and that several top-performing networks are inspired by texture-based representations. These representations are characterized by orderless pooling of second-order filter activations such as in bilinear CNNs and the winner of the iNaturalist 2018 challenge. Concretely, for each category we (i) visualize the “maximal images” by obtaining inputs x that maximize the probability of the particular class according to a texture-based deep network, and (ii) automatically describe the maximal images using a set of texture attributes. The models for texture captioning were trained on our ongoing efforts on collecting a dataset of describable textures building on the DTD dataset. These visualizations indicate what aspects of the texture is most discriminative for each category while the descriptions provide a language-based explanation of the same.
Tasks
Published	2019-07-02
URL	https://arxiv.org/abs/1907.05288v1
PDF	https://arxiv.org/pdf/1907.05288v1.pdf
PWC	https://paperswithcode.com/paper/visualizing-and-describing-fine-grained
Repo
Framework


Title	Examining the Role of Clickbait Headlines to Engage Readers with Reliable Health-related Information
Authors	Sima Bhowmik, Md Main Uddin Rony, Md Mahfuzul Haque, Kristen Alley Swain, Naeemul Hassan
Abstract	Clickbait headlines are frequently used to attract readers to read articles. Although this headline type has turned out to be a technique to engage readers with misleading items, it is still unknown whether the technique can be used to attract readers to reliable pieces. This study takes the opportunity to test its efficacy to engage readers with reliable health articles. A set of online surveys would be conducted to test readers’ engagement with and perception about clickbait headlines with reliable articles. After that, we would design an automation system to generate clickabit headlines to maximize user engagement.
Tasks
Published	2019-11-25
URL	https://arxiv.org/abs/1911.11214v1
PDF	https://arxiv.org/pdf/1911.11214v1.pdf
PWC	https://paperswithcode.com/paper/examining-the-role-of-clickbait-headlines-to
Repo
Framework