April 1, 2020

3155 words 15 mins read

Paper Group ANR 463

Deep Learning for Hindi Text Classification: A Comparison. Endoscopy disease detection challenge 2020. Beyond No-Regret: Competitive Control via Online Optimization with Memory. Resolving the Scope of Speculation and Negation using Transformer-Based Architectures. Collaborative Learning of Semi-Supervised Clustering and Classification for Labeling …

Deep Learning for Hindi Text Classification: A Comparison


Title	Deep Learning for Hindi Text Classification: A Comparison
Authors	Ramchandra Joshi, Purvi Goel, Raviraj Joshi
Abstract	Natural Language Processing (NLP) and especially natural language text analysis have seen great advances in recent times. Usage of deep learning in text processing has revolutionized the techniques for text processing and achieved remarkable results. Different deep learning architectures like CNN, LSTM, and very recent Transformer have been used to achieve state of the art results variety on NLP tasks. In this work, we survey a host of deep learning architectures for text classification tasks. The work is specifically concerned with the classification of Hindi text. The research in the classification of morphologically rich and low resource Hindi language written in Devanagari script has been limited due to the absence of large labeled corpus. In this work, we used translated versions of English data-sets to evaluate models based on CNN, LSTM and Attention. Multilingual pre-trained sentence embeddings based on BERT and LASER are also compared to evaluate their effectiveness for the Hindi language. The paper also serves as a tutorial for popular text classification techniques.
Tasks	Sentence Embeddings, Text Classification
Published	2020-01-19
URL	https://arxiv.org/abs/2001.10340v1
PDF	https://arxiv.org/pdf/2001.10340v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-hindi-text-classification-a
Repo
Framework

Endoscopy disease detection challenge 2020


Title	Endoscopy disease detection challenge 2020
Authors	Sharib Ali, Noha Ghatwary, Barbara Braden, Dominique Lamarque, Adam Bailey, Stefano Realdon, Renato Cannizzaro, Jens Rittscher, Christian Daul, James East
Abstract	Whilst many technologies are built around endoscopy, there is a need to have a comprehensive dataset collected from multiple centers to address the generalization issues with most deep learning frameworks. What could be more important than disease detection and localization? Through our extensive network of clinical and computational experts, we have collected, curated and annotated gastrointestinal endoscopy video frames. We have released this dataset and have launched disease detection and segmentation challenge EDD2020 https://edd2020.grand-challenge.org to address the limitations and explore new directions. EDD2020 is a crowd sourcing initiative to test the feasibility of recent deep learning methods and to promote research for building robust technologies. In this paper, we provide an overview of the EDD2020 dataset, challenge tasks, evaluation strategies and a short summary of results on test data. A detailed paper will be drafted after the challenge workshop with more detailed analysis of the results.
Tasks
Published	2020-03-07
URL	https://arxiv.org/abs/2003.03376v1
PDF	https://arxiv.org/pdf/2003.03376v1.pdf
PWC	https://paperswithcode.com/paper/endoscopy-disease-detection-challenge-2020
Repo
Framework

Beyond No-Regret: Competitive Control via Online Optimization with Memory


Title	Beyond No-Regret: Competitive Control via Online Optimization with Memory
Authors	Guanya Shi, Yiheng Lin, Soon-Jo Chung, Yisong Yue, Adam Wierman
Abstract	This paper studies online control with adversarial disturbances using tools from online optimization with memory. Most work that bridges learning and control theory focuses on designing policies that are no-regret with respect to the best static linear controller in hindsight. However, the optimal offline controller can have orders-of-magnitude lower cost than the best linear controller. We instead focus on achieving constant competitive ratio compared to the offline optimal controller, which need not be linear or static. We provide a novel reduction from online control of a class of controllable systems to online convex optimization with memory. We then design a new algorithm for online convex optimization with memory, Optimistic Regularized Online Balanced Descent, that has a constant, dimension-free competitive ratio. This result, in turn, leads to a new constant-competitive approach for online control.
Tasks
Published	2020-02-13
URL	https://arxiv.org/abs/2002.05318v1
PDF	https://arxiv.org/pdf/2002.05318v1.pdf
PWC	https://paperswithcode.com/paper/beyond-no-regret-competitive-control-via
Repo
Framework

Resolving the Scope of Speculation and Negation using Transformer-Based Architectures


Title	Resolving the Scope of Speculation and Negation using Transformer-Based Architectures
Authors	Benita Kathleen Britto, Aditya Khandelwal
Abstract	Speculation is a naturally occurring phenomena in textual data, forming an integral component of many systems, especially in the biomedical information retrieval domain. Previous work addressing cue detection and scope resolution (the two subtasks of speculation detection) have ranged from rule-based systems to deep learning-based approaches. In this paper, we apply three popular transformer-based architectures, BERT, XLNet and RoBERTa to this task, on two publicly available datasets, BioScope Corpus and SFU Review Corpus, reporting substantial improvements over previously reported results (by at least 0.29 F1 points on cue detection and 4.27 F1 points on scope resolution). We also experiment with joint training of the model on multiple datasets, which outperforms the single dataset training approach by a good margin. We observe that XLNet consistently outperforms BERT and RoBERTa, contrary to results on other benchmark datasets. To confirm this observation, we apply XLNet and RoBERTa to negation detection and scope resolution, reporting state-of-the-art results on negation scope resolution for the BioScope Corpus (increase of 3.16 F1 points on the BioScope Full Papers, 0.06 F1 points on the BioScope Abstracts) and the SFU Review Corpus (increase of 0.3 F1 points).
Tasks	Information Retrieval, Negation Detection, Negation Scope Resolution, Speculation Detection, Speculation Scope Resolution
Published	2020-01-09
URL	https://arxiv.org/abs/2001.02885v1
PDF	https://arxiv.org/pdf/2001.02885v1.pdf
PWC	https://paperswithcode.com/paper/resolving-the-scope-of-speculation-and
Repo
Framework

Collaborative Learning of Semi-Supervised Clustering and Classification for Labeling Uncurated Data


Title	Collaborative Learning of Semi-Supervised Clustering and Classification for Labeling Uncurated Data
Authors	Sara Mousavi, Dylan Lee, Tatianna Griffin, Dawnie Steadman, Audris Mockus
Abstract	Domain-specific image collections present potential value in various areas of science and business but are often not curated nor have any way to readily extract relevant content. To employ contemporary supervised image analysis methods on such image data, they must first be cleaned and organized, and then manually labeled for the nomenclature employed in the specific domain, which is a time consuming and expensive endeavor. To address this issue, we designed and implemented the Plud system. Plud provides an iterative semi-supervised workflow to minimize the effort spent by an expert and handles realistic large collections of images. We believe it can support labeling datasets regardless of their size and type. Plud is an iterative sequence of unsupervised clustering, human assistance, and supervised classification. With each iteration 1) the labeled dataset grows, 2) the generality of the classification method and its accuracy increases, and 3) manual effort is reduced. We evaluated the effectiveness of our system, by applying it on over a million images documenting human decomposition. In our experiment comparing manual labeling with labeling conducted with the support of Plud, we found that it reduces the time needed to label data and produces highly accurate models for this new domain.
Tasks
Published	2020-03-09
URL	https://arxiv.org/abs/2003.04261v1
PDF	https://arxiv.org/pdf/2003.04261v1.pdf
PWC	https://paperswithcode.com/paper/collaborative-learning-of-semi-supervised-1
Repo
Framework

HP2IFS: Head Pose estimation exploiting Partitioned Iterated Function Systems


Title	HP2IFS: Head Pose estimation exploiting Partitioned Iterated Function Systems
Authors	Carmen Bisogni, Michele Nappi, Chiara Pero, Stefano Ricciardi
Abstract	Estimating the actual head orientation from 2D images, with regard to its three degrees of freedom, is a well known problem that is highly significant for a large number of applications involving head pose knowledge. Consequently, this topic has been tackled by a plethora of methods and algorithms the most part of which exploits neural networks. Machine learning methods, indeed, achieve accurate head rotation values yet require an adequate training stage and, to that aim, a relevant number of positive and negative examples. In this paper we take a different approach to this topic by using fractal coding theory and particularly Partitioned Iterated Function Systems to extract the fractal code from the input head image and to compare this representation to the fractal code of a reference model through Hamming distance. According to experiments conducted on both the BIWI and the AFLW2000 databases, the proposed PIFS based head pose estimation method provides accurate yaw/pitch/roll angular values, with a performance approaching that of state of the art of machine-learning based algorithms and exceeding most of non-training based approaches.
Tasks	Head Pose Estimation, Pose Estimation
Published	2020-03-25
URL	https://arxiv.org/abs/2003.11536v1
PDF	https://arxiv.org/pdf/2003.11536v1.pdf
PWC	https://paperswithcode.com/paper/hp2ifs-head-pose-estimation-exploiting
Repo
Framework

Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization


Title	Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization
Authors	Saehyung Lee, Hyungyu Lee, Sungroh Yoon
Abstract	Adversarial examples cause neural networks to produce incorrect outputs with high confidence. Although adversarial training is one of the most effective forms of defense against adversarial examples, unfortunately, a large gap exists between test accuracy and training accuracy in adversarial training. In this paper, we identify Adversarial Feature Overfitting (AFO), which may cause poor adversarially robust generalization, and we show that adversarial training can overshoot the optimal point in terms of robust generalization, leading to AFO in our simple Gaussian model. Considering these theoretical results, we present soft labeling as a solution to the AFO problem. Furthermore, we propose Adversarial Vertex mixup (AVmixup), a soft-labeled data augmentation approach for improving adversarially robust generalization. We complement our theoretical analysis with experiments on CIFAR10, CIFAR100, SVHN, and Tiny ImageNet, and show that AVmixup significantly improves the robust generalization performance and that it reduces the trade-off between standard accuracy and adversarial robustness.
Tasks	Data Augmentation
Published	2020-03-05
URL	https://arxiv.org/abs/2003.02484v2
PDF	https://arxiv.org/pdf/2003.02484v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-vertex-mixup-toward-better
Repo
Framework

Efficient algorithms for multivariate shape-constrained convex regression problems


Title	Efficient algorithms for multivariate shape-constrained convex regression problems
Authors	Meixia Lin, Defeng Sun, Kim-Chuan Toh
Abstract	Shape-constrained convex regression problem deals with fitting a convex function to the observed data, where additional constraints are imposed, such as component-wise monotonicity and uniform Lipschitz continuity. This paper provides a comprehensive mechanism for computing the least squares estimator of a multivariate shape-constrained convex regression function in $\mathbb{R}^d$. We prove that the least squares estimator is computable via solving a constrained convex quadratic programming (QP) problem with $(n+1)d$ variables and at least $n(n-1)$ linear inequality constraints, where $n$ is the number of data points. For solving the generally very large-scale convex QP, we design two efficient algorithms, one is the symmetric Gauss-Seidel based alternating direction method of multipliers ({\tt sGS-ADMM}), and the other is the proximal augmented Lagrangian method ({\tt pALM}) with the subproblems solved by the semismooth Newton method ({\tt SSN}). Comprehensive numerical experiments, including those in the pricing of basket options and estimation of production functions in economics, demonstrate that both of our proposed algorithms outperform the state-of-the-art algorithm. The {\tt pALM} is more efficient than the {\tt sGS-ADMM} but the latter has the advantage of being simpler to implement.
Tasks
Published	2020-02-26
URL	https://arxiv.org/abs/2002.11410v1
PDF	https://arxiv.org/pdf/2002.11410v1.pdf
PWC	https://paperswithcode.com/paper/efficient-algorithms-for-multivariate-shape
Repo
Framework

BitTensor: An Intermodel Intelligence Measure


Title	BitTensor: An Intermodel Intelligence Measure
Authors	Jacob Steeves, Ala Shaabana, Matthew McAteer
Abstract	A purely inter-model version of a machine intelligence benchmark would allow us to measure intelligence directly as information without projecting that information onto labeled datasets. We propose a framework in which other learners measure the informational significance of their peers across a network and use a digital ledger to negotiate the scores. However, the main benefits of measuring intelligence with other learners are lost if the underlying scores are dishonest. As a solution, we show how competition for connectivity in the network can be used to force honest bidding. We first prove that selecting inter-model scores using gradient descent is a regret-free strategy: one which generates the best subjective outcome regardless of the behavior of others. We then empirically show that when nodes apply this strategy, the network converges to a ranking that correlates with the one found in a fully coordinated and centralized setting. The result is a fair mechanism for training an internet-wide, decentralized and incentivized machine learning system, one which produces a continually hardening and expanding benchmark at the generalized intersection of the participants.
Tasks
Published	2020-03-09
URL	https://arxiv.org/abs/2003.03917v1
PDF	https://arxiv.org/pdf/2003.03917v1.pdf
PWC	https://paperswithcode.com/paper/bittensor-an-intermodel-intelligence-measure
Repo
Framework

Space-Time-Aware Multi-Resolution Video Enhancement


Title	Space-Time-Aware Multi-Resolution Video Enhancement
Authors	Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
Abstract	We consider the problem of space-time super-resolution (ST-SR): increasing spatial resolution of video frames and simultaneously interpolating frames to increase the frame rate. Modern approaches handle these axes one at a time. In contrast, our proposed model called STARnet super-resolves jointly in space and time. This allows us to leverage mutually informative relationships between time and space: higher resolution can provide more detailed information about motion, and higher frame-rate can provide better pixel alignment. The components of our model that generate latent low- and high-resolution representations during ST-SR can be used to finetune a specialized mechanism for just spatial or just temporal super-resolution. Experimental results demonstrate that STARnet improves the performances of space-time, spatial, and temporal video super-resolution by substantial margins on publicly available datasets.
Tasks	Super-Resolution, Video Super-Resolution
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13170v1
PDF	https://arxiv.org/pdf/2003.13170v1.pdf
PWC	https://paperswithcode.com/paper/space-time-aware-multi-resolution-video
Repo
Framework

End-To-End Trainable Video Super-Resolution Based on a New Mechanism for Implicit Motion Estimation and Compensation


Title	End-To-End Trainable Video Super-Resolution Based on a New Mechanism for Implicit Motion Estimation and Compensation
Authors	Xiaohong Liu, Lingshi Kong, Yang Zhou, Jiying Zhao, Jun Chen
Abstract	Video super-resolution aims at generating a high-resolution video from its low-resolution counterpart. With the rapid rise of deep learning, many recently proposed video super-resolution methods use convolutional neural networks in conjunction with explicit motion compensation to capitalize on statistical dependencies within and across low-resolution frames. Two common issues of such methods are noteworthy. Firstly, the quality of the final reconstructed HR video is often very sensitive to the accuracy of motion estimation. Secondly, the warp grid needed for motion compensation, which is specified by the two flow maps delineating pixel displacements in horizontal and vertical directions, tends to introduce additional errors and jeopardize the temporal consistency across video frames. To address these issues, we propose a novel dynamic local filter network to perform implicit motion estimation and compensation by employing, via locally connected layers, sample-specific and position-specific dynamic local filters that are tailored to the target pixels. We also propose a global refinement network based on ResBlock and autoencoder structures to exploit non-local correlations and enhance the spatial consistency of super-resolved frames. The experimental results demonstrate that the proposed method outperforms the state-of-the-art, and validate its strength in terms of local transformation handling, temporal consistency as well as edge sharpness.
Tasks	Motion Compensation, Motion Estimation, Super-Resolution, Video Super-Resolution
Published	2020-01-05
URL	https://arxiv.org/abs/2001.01162v1
PDF	https://arxiv.org/pdf/2001.01162v1.pdf
PWC	https://paperswithcode.com/paper/end-to-end-trainable-video-super-resolution
Repo
Framework

Evaluating Logical Generalization in Graph Neural Networks


Title	Evaluating Logical Generalization in Graph Neural Networks
Authors	Koustuv Sinha, Shagun Sodhani, Joelle Pineau, William L. Hamilton
Abstract	Recent research has highlighted the role of relational inductive biases in building learning agents that can generalize and reason in a compositional manner. However, while relational learning algorithms such as graph neural networks (GNNs) show promise, we do not understand how effectively these approaches can adapt to new tasks. In this work, we study the task of logical generalization using GNNs by designing a benchmark suite grounded in first-order logic. Our benchmark suite, GraphLog, requires that learning algorithms perform rule induction in different synthetic logics, represented as knowledge graphs. GraphLog consists of relation prediction tasks on 57 distinct logical domains. We use GraphLog to evaluate GNNs in three different setups: single-task supervised learning, multi-task pretraining, and continual learning. Unlike previous benchmarks, our approach allows us to precisely control the logical relationship between the different tasks. We find that the ability for models to generalize and adapt is strongly determined by the diversity of the logical rules they encounter during training, and our results highlight new challenges for the design of GNN models. We publicly release the dataset and code used to generate and interact with the dataset at https://www.cs.mcgill.ca/~ksinha4/graphlog.
Tasks	Continual Learning, Knowledge Graphs, Relational Reasoning
Published	2020-03-14
URL	https://arxiv.org/abs/2003.06560v1
PDF	https://arxiv.org/pdf/2003.06560v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-logical-generalization-in-graph
Repo
Framework

Voice Separation with an Unknown Number of Multiple Speakers


Title	Voice Separation with an Unknown Number of Multiple Speakers
Authors	Eliya Nachmani, Yossi Adi, Lior Wolf
Abstract	We present a new method for separating a mixed audio sequence, in which multiple voices speak simultaneously. The new method employs gated neural networks that are trained to separate the voices at multiple processing steps, while maintaining the speaker in each output channel fixed. A different model is trained for every number of possible speakers, and a the model with the largest number of speakers is employed to select the actual number of speakers in a given sample. Our method greatly outperforms the current state of the art, which, as we show, is not competitive for more than two speakers.
Tasks
Published	2020-02-29
URL	https://arxiv.org/abs/2003.01531v1
PDF	https://arxiv.org/pdf/2003.01531v1.pdf
PWC	https://paperswithcode.com/paper/voice-separation-with-an-unknown-number-of
Repo
Framework

On Implicit Regularization in $β$-VAEs


Title	On Implicit Regularization in $β$-VAEs
Authors	Abhishek Kumar, Ben Poole
Abstract	While the impact of variational inference (VI) on posterior inference in a fixed generative model is well-characterized, its role in regularizing a learned generative model when used in variational autoencoders (VAEs) is poorly understood. We study the regularizing effects of variational distributions on learning in generative models from two perspectives. First, we analyze the role that the choice of variational family plays in imparting uniqueness to the learned model by restricting the set of optimal generative models. Second, we study the regularization effect of the variational family on the local geometry of the decoding model. This analysis uncovers the regularizer implicit in the $\beta$-VAE objective, and leads to an approximation consisting of a deterministic autoencoding objective plus analytic regularizers that depend on the Hessian or Jacobian of the decoding model, unifying VAEs with recent heuristics proposed for training regularized autoencoders. We empirically verify these findings, observing that the proposed deterministic objective exhibits similar behavior to the $\beta$-VAE in terms of objective value and sample quality.
Tasks
Published	2020-01-31
URL	https://arxiv.org/abs/2002.00041v2
PDF	https://arxiv.org/pdf/2002.00041v2.pdf
PWC	https://paperswithcode.com/paper/on-implicit-regularization-in-vaes
Repo
Framework

Feature selection in machine learning: Rényi min-entropy vs Shannon entropy


Title	Feature selection in machine learning: Rényi min-entropy vs Shannon entropy
Authors	Catuscia Palamidessi, Marco Romanelli
Abstract	Feature selection, in the context of machine learning, is the process of separating the highly predictive feature from those that might be irrelevant or redundant. Information theory has been recognized as a useful concept for this task, as the prediction power stems from the correlation, i.e., the mutual information, between features and labels. Many algorithms for feature selection in the literature have adopted the Shannon-entropy-based mutual information. In this paper, we explore the possibility of using R'enyi min-entropy instead. In particular, we propose an algorithm based on a notion of conditional R'enyi min-entropy that has been recently adopted in the field of security and privacy, and which is strictly related to the Bayes error. We prove that in general the two approaches are incomparable, in the sense that we show that we can construct datasets on which the R'enyi-based algorithm performs better than the corresponding Shannon-based one, and datasets on which the situation is reversed. In practice, however, when considering datasets of real data, it seems that the R'enyi-based algorithm tends to outperform the other one. We have effectuate several experiments on the BASEHOCK, SEMEION, and GISETTE datasets, and in all of them we have indeed observed that the R'enyi-based algorithm gives better results.
Tasks	Feature Selection
Published	2020-01-27
URL	https://arxiv.org/abs/2001.09654v1
PDF	https://arxiv.org/pdf/2001.09654v1.pdf
PWC	https://paperswithcode.com/paper/feature-selection-in-machine-learning-renyi
Repo
Framework