Paper Group ANR 424
Distributed, partially collapsed MCMC for Bayesian Nonparametrics. Variational Transformers for Diverse Response Generation. SynCGAN: Using learnable class specific priors to generate synthetic data for improving classifier performance on cytological images. Quantum subspace alignment for domain adaptation. Machine Learning the Phenomenology of COV …
Distributed, partially collapsed MCMC for Bayesian Nonparametrics
| Title | Distributed, partially collapsed MCMC for Bayesian Nonparametrics |
| Authors | Avinava Dubey, Michael Minyi Zhang, Eric P. Xing, Sinead A. Williamson |
| Abstract | Bayesian nonparametric (BNP) models provide elegant methods for discovering underlying latent features within a data set, but inference in such models can be slow. We exploit the fact that completely random measures, which commonly used models like the Dirichlet process and the beta-Bernoulli process can be expressed as, are decomposable into independent sub-measures. We use this decomposition to partition the latent measure into a finite measure containing only instantiated components, and an infinite measure containing all other components. We then select different inference algorithms for the two components: uncollapsed samplers mix well on the finite measure, while collapsed samplers mix well on the infinite, sparsely occupied tail. The resulting hybrid algorithm can be applied to a wide class of models, and can be easily distributed to allow scalable inference without sacrificing asymptotic convergence guarantees. |
| Tasks | |
| Published | 2020-01-15 |
| URL | https://arxiv.org/abs/2001.05591v3 |
| https://arxiv.org/pdf/2001.05591v3.pdf | |
| PWC | https://paperswithcode.com/paper/distributed-partially-collapsed-mcmc-for |
| Repo | |
| Framework | |
Variational Transformers for Diverse Response Generation
| Title | Variational Transformers for Diverse Response Generation |
| Authors | Zhaojiang Lin, Genta Indra Winata, Peng Xu, Zihan Liu, Pascale Fung |
| Abstract | Despite the great promise of Transformers in many sequence modeling tasks (e.g., machine translation), their deterministic nature hinders them from generalizing to high entropy tasks such as dialogue response generation. Previous work proposes to capture the variability of dialogue responses with a recurrent neural network (RNN)-based conditional variational autoencoder (CVAE). However, the autoregressive computation of the RNN limits the training efficiency. Therefore, we propose the Variational Transformer (VT), a variational self-attentive feed-forward sequence model. The VT combines the parallelizability and global receptive field of the Transformer with the variational nature of the CVAE by incorporating stochastic latent variables into Transformers. We explore two types of the VT: 1) modeling the discourse-level diversity with a global latent variable; and 2) augmenting the Transformer decoder with a sequence of fine-grained latent variables. Then, the proposed models are evaluated on three conversational datasets with both automatic metric and human evaluation. The experimental results show that our models improve standard Transformers and other baselines in terms of diversity, semantic relevance, and human judgment. |
| Tasks | Machine Translation |
| Published | 2020-03-28 |
| URL | https://arxiv.org/abs/2003.12738v1 |
| https://arxiv.org/pdf/2003.12738v1.pdf | |
| PWC | https://paperswithcode.com/paper/variational-transformers-for-diverse-response |
| Repo | |
| Framework | |
SynCGAN: Using learnable class specific priors to generate synthetic data for improving classifier performance on cytological images
| Title | SynCGAN: Using learnable class specific priors to generate synthetic data for improving classifier performance on cytological images |
| Authors | Soumyajyoti Dey, Soham Das, Swarnendu Ghosh, Shyamali Mitra, Sukanta Chakrabarty, Nibaran Das |
| Abstract | One of the most challenging aspects of medical image analysis is the lack of a high quantity of annotated data. This makes it difficult for deep learning algorithms to perform well due to a lack of variations in the input space. While generative adversarial networks have shown promise in the field of synthetic data generation, but without a carefully designed prior the generation procedure can not be performed well. In the proposed approach we have demonstrated the use of automatically generated segmentation masks as learnable class-specific priors to guide a conditional GAN for the generation of patho-realistic samples for cytology image. We have observed that augmentation of data using the proposed pipeline called “SynCGAN” improves the performance of state of the art classifiers such as ResNet-152, DenseNet-161, Inception-V3 significantly. |
| Tasks | Synthetic Data Generation |
| Published | 2020-03-12 |
| URL | https://arxiv.org/abs/2003.05712v1 |
| https://arxiv.org/pdf/2003.05712v1.pdf | |
| PWC | https://paperswithcode.com/paper/syncgan-using-learnable-class-specific-priors |
| Repo | |
| Framework | |
Quantum subspace alignment for domain adaptation
| Title | Quantum subspace alignment for domain adaptation |
| Authors | Xi He, Xiaoting Wang |
| Abstract | Domain adaptation (DA) is used for adaptively obtaining labels of an unprocessed data set with given a related, but different labelled data set. Subspace alignment (SA), a representative DA algorithm, attempts to find a linear transformation to align the two different data sets. The classifier trained on the aligned labelled data set can be transferred to the unlabelled data set to classify the target labels. In this paper, a quantum version of the SA algorithm is proposed to implement the domain adaptation procedure on a quantum computer. Compared with the classical SA algorithm, the quantum algorithm presented in our work achieves at least quadratic speedup in the number of given samples and the data dimension. In addition, the kernel method is applied to the quantum SA algorithm to capture the nonlinear characteristics of the data sets. |
| Tasks | Domain Adaptation |
| Published | 2020-01-08 |
| URL | https://arxiv.org/abs/2001.02472v1 |
| https://arxiv.org/pdf/2001.02472v1.pdf | |
| PWC | https://paperswithcode.com/paper/quantum-subspace-alignment-for-domain |
| Repo | |
| Framework | |
Machine Learning the Phenomenology of COVID-19 From Early Infection Dynamics
| Title | Machine Learning the Phenomenology of COVID-19 From Early Infection Dynamics |
| Authors | Malik Magdon-Ismail |
| Abstract | We present a data-driven machine learning analysis of COVID-19 from its \emph{early} infection dynamics, with the goal of extracting actionable public health insights. We focus on the transmission dynamics in the USA starting from the first confirmed infection on January 21 2020. We find that COVID-19 has a strong infectious force if left unchecked, with a doubling time of under 3 days. However it is not particularly virulent. Our methods may be of general interest. |
| Tasks | |
| Published | 2020-03-17 |
| URL | https://arxiv.org/abs/2003.07602v2 |
| https://arxiv.org/pdf/2003.07602v2.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-the-phenomenology-of-covid |
| Repo | |
| Framework | |
Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning
| Title | Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning |
| Authors | Esther Rolf, Max Simchowitz, Sarah Dean, Lydia T. Liu, Daniel Björkegren, Moritz Hardt, Joshua Blumenstock |
| Abstract | While real-world decisions involve many competing objectives, algorithmic decisions are often evaluated with a single objective function. In this paper, we study algorithmic policies which explicitly trade off between a private objective (such as profit) and a public objective (such as social welfare). We analyze a natural class of policies which trace an empirical Pareto frontier based on learned scores, and focus on how such decisions can be made in noisy or data-limited regimes. Our theoretical results characterize the optimal strategies in this class, bound the Pareto errors due to inaccuracies in the scores, and show an equivalence between optimal strategies and a rich class of fairness-constrained profit-maximizing policies. We then present empirical results in two different contexts — online content recommendation and sustainable abalone fisheries — to underscore the applicability of our approach to a wide range of practical decisions. Taken together, these results shed light on inherent trade-offs in using machine learning for decisions that impact social welfare. |
| Tasks | |
| Published | 2020-03-15 |
| URL | https://arxiv.org/abs/2003.06740v1 |
| https://arxiv.org/pdf/2003.06740v1.pdf | |
| PWC | https://paperswithcode.com/paper/balancing-competing-objectives-with-noisy |
| Repo | |
| Framework | |
Improving STDP-based Visual Feature Learning with Whitening
| Title | Improving STDP-based Visual Feature Learning with Whitening |
| Authors | Pierre Falez, Pierre Tirilly, Ioan Marius Bilasco |
| Abstract | In recent years, spiking neural networks (SNNs) emerge as an alternative to deep neural networks (DNNs). SNNs present a higher computational efficiency using low-power neuromorphic hardware and require less labeled data for training using local and unsupervised learning rules such as spike timing-dependent plasticity (STDP). SNN have proven their effectiveness in image classification on simple datasets such as MNIST. However, to process natural images, a pre-processing step is required. Difference-of-Gaussians (DoG) filtering is typically used together with on-center/off-center coding, but it results in a loss of information that is detrimental to the classification performance. In this paper, we propose to use whitening as a pre-processing step before learning features with STDP. Experiments on CIFAR-10 show that whitening allows STDP to learn visual features that are closer to the ones learned with standard neural networks, with a significantly increased classification performance as compared to DoG filtering. We also propose an approximation of whitening as convolution kernels that is computationally cheaper to learn and more suited to be implemented on neuromorphic hardware. Experiments on CIFAR-10 show that it performs similarly to regular whitening. Cross-dataset experiments on CIFAR-10 and STL-10 also show that it is fairly stable across datasets, making it possible to learn a single whitening transformation to process different datasets. |
| Tasks | Image Classification |
| Published | 2020-02-24 |
| URL | https://arxiv.org/abs/2002.10177v1 |
| https://arxiv.org/pdf/2002.10177v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-stdp-based-visual-feature-learning |
| Repo | |
| Framework | |
Fictitious Play Outperforms Counterfactual Regret Minimization
| Title | Fictitious Play Outperforms Counterfactual Regret Minimization |
| Authors | Sam Ganzfried |
| Abstract | We compare the performance of two popular iterative algorithms, fictitious play and counterfactual regret minimization, in approximating Nash equilibrium in multiplayer games. Despite recent success of counterfactual regret minimization in multiplayer poker and conjectures of its superiority, we show that fictitious play leads to improved Nash equilibrium approximation with statistical significance over a variety of game sizes. |
| Tasks | |
| Published | 2020-01-30 |
| URL | https://arxiv.org/abs/2001.11165v2 |
| https://arxiv.org/pdf/2001.11165v2.pdf | |
| PWC | https://paperswithcode.com/paper/fictitious-play-outperforms-counterfactual |
| Repo | |
| Framework | |
Automated Detection of Cribriform Growth Patterns in Prostate Histology Images
| Title | Automated Detection of Cribriform Growth Patterns in Prostate Histology Images |
| Authors | Pierre Ambrosini, Eva Hollemans, Charlotte F. Kweldam, Geert J. L. H. van Leenders, Sjoerd Stallinga, Frans Vos |
| Abstract | Cribriform growth patterns in prostate carcinoma are associated with poor prognosis. We aimed to introduce a deep learning method to detect such patterns automatically. To do so, convolutional neural network was trained to detect cribriform growth patterns on 128 prostate needle biopsies. Ensemble learning taking into account other tumor growth patterns during training was used to cope with heterogeneous and limited tumor tissue occurrences. ROC and FROC analyses were applied to assess network performance regarding detection of biopsies harboring cribriform growth pattern. The ROC analysis yielded an area under the curve up to 0.82. FROC analysis demonstrated a sensitivity of 0.9 for regions larger than 0.0150 mm2 with on average 6.8 false positives. To benchmark method performance for intra-observer annotation variability, false positive and negative detections were re-evaluated by the pathologists. Pathologists considered 9% of the false positive regions as cribriform, and 11% as possibly cribriform; 44% of the false negative regions were not annotated as cribriform. As a final experiment, the network was also applied on a dataset of 60 biopsy regions annotated by 23 pathologists. With the cut-off reaching highest sensitivity, all images annotated as cribriform by at least 7/23 of the pathologists, were all detected as cribriform by the network. In conclusion, the proposed deep learning method has high sensitivity for detecting cribriform growth patterns at the expense of a limited number of false positives. It can detect cribriform regions that are labelled as such by at least a minority of pathologists. Therefore, it could assist clinical decision making by suggesting suspicious regions. |
| Tasks | Decision Making |
| Published | 2020-03-23 |
| URL | https://arxiv.org/abs/2003.10543v1 |
| https://arxiv.org/pdf/2003.10543v1.pdf | |
| PWC | https://paperswithcode.com/paper/automated-detection-of-cribriform-growth |
| Repo | |
| Framework | |
Automated Relational Meta-learning
| Title | Automated Relational Meta-learning |
| Authors | Huaxiu Yao, Xian Wu, Zhiqiang Tao, Yaliang Li, Bolin Ding, Ruirui Li, Zhenhui Li |
| Abstract | In order to efficiently learn with small amount of data on new tasks, meta-learning transfers knowledge learned from previous tasks to the new ones. However, a critical challenge in meta-learning is the task heterogeneity which cannot be well handled by traditional globally shared meta-learning methods. In addition, current task-specific meta-learning methods may either suffer from hand-crafted structure design or lack the capability to capture complex relations between tasks. In this paper, motivated by the way of knowledge organization in knowledge bases, we propose an automated relational meta-learning (ARML) framework that automatically extracts the cross-task relations and constructs the meta-knowledge graph. When a new task arrives, it can quickly find the most relevant structure and tailor the learned structure knowledge to the meta-learner. As a result, the proposed framework not only addresses the challenge of task heterogeneity by a learned meta-knowledge graph, but also increases the model interpretability. We conduct extensive experiments on 2D toy regression and few-shot image classification and the results demonstrate the superiority of ARML over state-of-the-art baselines. |
| Tasks | Few-Shot Image Classification, Image Classification, Meta-Learning |
| Published | 2020-01-03 |
| URL | https://arxiv.org/abs/2001.00745v1 |
| https://arxiv.org/pdf/2001.00745v1.pdf | |
| PWC | https://paperswithcode.com/paper/automated-relational-meta-learning-1 |
| Repo | |
| Framework | |
PoET-BiN: Power Efficient Tiny Binary Neurons
| Title | PoET-BiN: Power Efficient Tiny Binary Neurons |
| Authors | Sivakumar Chidambaram, J. M. Pierre Langlois, Jean Pierre David |
| Abstract | The success of neural networks in image classification has inspired various hardware implementations on embedded platforms such as Field Programmable Gate Arrays, embedded processors and Graphical Processing Units. These embedded platforms are constrained in terms of power, which is mainly consumed by the Multiply Accumulate operations and the memory accesses for weight fetching. Quantization and pruning have been proposed to address this issue. Though effective, these techniques do not take into account the underlying architecture of the embedded hardware. In this work, we propose PoET-BiN, a Look-Up Table based power efficient implementation on resource constrained embedded devices. A modified Decision Tree approach forms the backbone of the proposed implementation in the binary domain. A LUT access consumes far less power than the equivalent Multiply Accumulate operation it replaces, and the modified Decision Tree algorithm eliminates the need for memory accesses. We applied the PoET-BiN architecture to implement the classification layers of networks trained on MNIST, SVHN and CIFAR-10 datasets, with near state-of-the art results. The energy reduction for the classifier portion reaches up to six orders of magnitude compared to a floating point implementations and up to three orders of magnitude when compared to recent binary quantized neural networks. |
| Tasks | Image Classification, Quantization |
| Published | 2020-02-23 |
| URL | https://arxiv.org/abs/2002.09794v1 |
| https://arxiv.org/pdf/2002.09794v1.pdf | |
| PWC | https://paperswithcode.com/paper/poet-bin-power-efficient-tiny-binary-neurons |
| Repo | |
| Framework | |
Captioning Images with Novel Objects via Online Vocabulary Expansion
| Title | Captioning Images with Novel Objects via Online Vocabulary Expansion |
| Authors | Mikihiro Tanaka, Tatsuya Harada |
| Abstract | In this study, we introduce a low cost method for generating descriptions from images containing novel objects. Generally, constructing a model, which can explain images with novel objects, is costly because of the following: (1) collecting a large amount of data for each category, and (2) retraining the entire system. If humans see a small number of novel objects, they are able to estimate their properties by associating their appearance with known objects. Accordingly, we propose a method that can explain images with novel objects without retraining using the word embeddings of the objects estimated from only a small number of image features of the objects. The method can be integrated with general image-captioning models. The experimental results show the effectiveness of our approach. |
| Tasks | Image Captioning, Word Embeddings |
| Published | 2020-03-06 |
| URL | https://arxiv.org/abs/2003.03305v1 |
| https://arxiv.org/pdf/2003.03305v1.pdf | |
| PWC | https://paperswithcode.com/paper/captioning-images-with-novel-objects-via |
| Repo | |
| Framework | |
SOS: Selective Objective Switch for Rapid Immunofluorescence Whole Slide Image Classification
| Title | SOS: Selective Objective Switch for Rapid Immunofluorescence Whole Slide Image Classification |
| Authors | Sam Maksoud, Kun Zhao, Peter Hobson, Anthony Jennings, Brian Lovell |
| Abstract | The difficulty of processing gigapixel whole slide images (WSIs) in clinical microscopy has been a long-standing barrier to implementing computer aided diagnostic systems. Since modern computing resources are unable to perform computations at this extremely large scale, current state of the art methods utilize patch-based processing to preserve the resolution of WSIs. However, these methods are often resource intensive and make significant compromises on processing time. In this paper, we demonstrate that conventional patch-based processing is redundant for certain WSI classification tasks where high resolution is only required in a minority of cases. This reflects what is observed in clinical practice; where a pathologist may screen slides using a low power objective and only switch to a high power in cases where they are uncertain about their findings. To eliminate these redundancies, we propose a method for the selective use of high resolution processing based on the confidence of predictions on downscaled WSIs — we call this the Selective Objective Switch (SOS). Our method is validated on a novel dataset of 684 Liver-Kidney-Stomach immunofluorescence WSIs routinely used in the investigation of autoimmune liver disease. By limiting high resolution processing to cases which cannot be classified confidently at low resolution, we maintain the accuracy of patch-level analysis whilst reducing the inference time by a factor of 7.74. |
| Tasks | Image Classification |
| Published | 2020-03-11 |
| URL | https://arxiv.org/abs/2003.05080v1 |
| https://arxiv.org/pdf/2003.05080v1.pdf | |
| PWC | https://paperswithcode.com/paper/sos-selective-objective-switch-for-rapid |
| Repo | |
| Framework | |
Predicting Subjective Features from Questions on QA Websites using BERT
| Title | Predicting Subjective Features from Questions on QA Websites using BERT |
| Authors | Issa Annamoradnejad, Mohammadamin Fazli, Jafar Habibi |
| Abstract | Community Question-Answering websites, such as StackOverflow and Quora, expect users to follow specific guidelines in order to maintain content quality. These systems mainly rely on community reports for assessing contents, which has serious problems such as the slow handling of violations, the loss of normal and experienced users’ time, the low quality of some reports, and discouraging feedback to new users. Therefore, with the overall goal of providing solutions for automating moderation actions in Q&A websites, we aim to provide a model to predict 20 quality or subjective aspects of questions in QA websites. To this end, we used data gathered by the CrowdSource team at Google Research in 2019 and fine-tuned pre-trained BERT model on our problem. Based on evaluation by Mean-Squared-Error (MSE), model achieved the value of 0.046 after 2 epochs of training, which did not improve substantially in the next ones. Results confirm that by simple fine-tuning, we can achieve accurate models in little time and on less amount of data. |
| Tasks | Community Question Answering, Question Answering |
| Published | 2020-02-24 |
| URL | https://arxiv.org/abs/2002.10107v2 |
| https://arxiv.org/pdf/2002.10107v2.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-subjective-features-from-questions |
| Repo | |
| Framework | |
Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages
| Title | Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages |
| Authors | Machel Reid, Edison Marrese-Taylor, Yutaka Matsuo |
| Abstract | The contrast between the need for large amounts of data for current Natural Language Processing (NLP) techniques, and the lack thereof, is accentuated in the case of African languages, most of which are considered low-resource. To help circumvent this issue, we explore techniques exploiting the qualities of morphologically rich languages (MRLs), while leveraging pretrained word vectors in well-resourced languages. In our exploration, we show that a meta-embedding approach combining both pretrained and morphologically-informed word embeddings performs best in the downstream task of Xhosa-English translation. |
| Tasks | Word Embeddings |
| Published | 2020-03-09 |
| URL | https://arxiv.org/abs/2003.04419v2 |
| https://arxiv.org/pdf/2003.04419v2.pdf | |
| PWC | https://paperswithcode.com/paper/combining-pretrained-high-resource-embeddings |
| Repo | |
| Framework | |
