Paper Group ANR 14
Tractable Querying and Learning in Hybrid Domains via Sum-Product Networks. Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding. Self Organizing Classifiers: First Steps in Structured Evolutionary Machine Learning. MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sent …
Tractable Querying and Learning in Hybrid Domains via Sum-Product Networks
| Title | Tractable Querying and Learning in Hybrid Domains via Sum-Product Networks |
| Authors | Andreas Bueff, Stefanie Speichert, Vaishak Belle |
| Abstract | Probabilistic representations, such as Bayesian and Markov networks, are fundamental to much of statistical machine learning. Thus, learning probabilistic representations directly from data is a deep challenge, the main computational bottleneck being inference that is intractable. Tractable learning is a powerful new paradigm that attempts to learn distributions that support efficient probabilistic querying. By leveraging local structure, representations such as sum-product networks (SPNs) can capture high tree-width models with many hidden layers, essentially a deep architecture, while still admitting a range of probabilistic queries to be computable in time polynomial in the network size. The leaf nodes in SPNs, from which more intricate mixtures are formed, are tractable univariate distributions, and so the literature has focused on Bernoulli and Gaussian random variables. This is clearly a restriction for handling mixed discrete-continuous data, especially if the continuous features are generated from non-parametric and non-Gaussian distribution families. In this work, we present a framework that systematically integrates SPN structure learning with weighted model integration, a recently introduced computational abstraction for performing inference in hybrid domains, by means of piecewise polynomial approximations of density functions of arbitrary shape. Our framework is instantiated by exploiting the notion of propositional abstractions, thus minimally interfering with the SPN structure learning module, and supports a powerful query interface for conditioning on interval constraints. Our empirical results show that our approach is effective, and allows a study of the trade off between the granularity of the learned model and its predictive power. |
| Tasks | |
| Published | 2018-07-14 |
| URL | http://arxiv.org/abs/1807.05464v3 |
| http://arxiv.org/pdf/1807.05464v3.pdf | |
| PWC | https://paperswithcode.com/paper/tractable-querying-and-learning-in-hybrid |
| Repo | |
| Framework | |
Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding
| Title | Optimal Transport Classifier: Defending Against Adversarial Attacks by Regularized Deep Embedding |
| Authors | Yao Li, Martin Renqiang Min, Wenchao Yu, Cho-Jui Hsieh, Thomas C. M. Lee, Erik Kruus |
| Abstract | Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples. Inspired by the observation that the intrinsic dimension of image data is much smaller than its pixel space dimension and the vulnerability of neural networks grows with the input dimension, we propose to embed high-dimensional input images into a low-dimensional space to perform classification. However, arbitrarily projecting the input images to a low-dimensional space without regularization will not improve the robustness of deep neural networks. Leveraging optimal transport theory, we propose a new framework, Optimal Transport Classifier (OT-Classifier), and derive an objective that minimizes the discrepancy between the distribution of the true label and the distribution of the OT-Classifier output. Experimental results on several benchmark datasets show that, our proposed framework achieves state-of-the-art performance against strong adversarial attack methods. |
| Tasks | Adversarial Attack |
| Published | 2018-11-19 |
| URL | http://arxiv.org/abs/1811.07950v2 |
| http://arxiv.org/pdf/1811.07950v2.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-transport-classifier-defending |
| Repo | |
| Framework | |
Self Organizing Classifiers: First Steps in Structured Evolutionary Machine Learning
| Title | Self Organizing Classifiers: First Steps in Structured Evolutionary Machine Learning |
| Authors | Danilo Vasconcellos Vargas, Hirotaka Takano, Junichi Murata |
| Abstract | Learning classifier systems (LCSs) are evolutionary machine learning algorithms, flexible enough to be applied to reinforcement, supervised and unsupervised learning problems with good performance. Recently, self organizing classifiers were proposed which are similar to LCSs but have the advantage that in its structured population no balance between niching and fitness pressure is necessary. However, more tests and analysis are required to verify its benefits. Here, a variation of the first algorithm is proposed which uses a parameterless self organizing map (SOM). This algorithm is applied in challenging problems such as big, noisy as well as dynamically changing continuous input-action mazes (growing and compressing mazes are included) with good performance. Moreover, a genetic operator is proposed which utilizes the topological information of the SOM’s population structure, improving the results. Thus, the first steps in structured evolutionary machine learning are shown, nonetheless, the problems faced are more difficult than the state-of-art continuous input-action multi-step ones. |
| Tasks | |
| Published | 2018-11-20 |
| URL | http://arxiv.org/abs/1811.08225v1 |
| http://arxiv.org/pdf/1811.08225v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-organizing-classifiers-first-steps-in |
| Repo | |
| Framework | |
MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classification
| Title | MultiBooked: A Corpus of Basque and Catalan Hotel Reviews Annotated for Aspect-level Sentiment Classification |
| Authors | Jeremy Barnes, Patrik Lambert, Toni Badia |
| Abstract | While sentiment analysis has become an established field in the NLP community, research into languages other than English has been hindered by the lack of resources. Although much research in multi-lingual and cross-lingual sentiment analysis has focused on unsupervised or semi-supervised approaches, these still require a large number of resources and do not reach the performance of supervised approaches. With this in mind, we introduce two datasets for supervised aspect-level sentiment analysis in Basque and Catalan, both of which are under-resourced languages. We provide high-quality annotations and benchmarks with the hope that they will be useful to the growing community of researchers working on these languages. |
| Tasks | Sentiment Analysis |
| Published | 2018-03-22 |
| URL | http://arxiv.org/abs/1803.08614v1 |
| http://arxiv.org/pdf/1803.08614v1.pdf | |
| PWC | https://paperswithcode.com/paper/multibooked-a-corpus-of-basque-and-catalan |
| Repo | |
| Framework | |
Modified SMOTE Using Mutual Information and Different Sorts of Entropies
| Title | Modified SMOTE Using Mutual Information and Different Sorts of Entropies |
| Authors | Sima Sharifirad, Azra Nazari, Mehdi Ghatee |
| Abstract | SMOTE is one of the oversampling techniques for balancing the datasets and it is considered as a pre-processing step in learning algorithms. In this paper, four new enhanced SMOTE are proposed that include an improved version of KNN in which the attribute weights are defined by mutual information firstly and then they are replaced by maximum entropy, Renyi entropy and Tsallis entropy. These four pre-processing methods are combined with 1NN and J48 classifiers and their performance are compared with the previous methods on 11 imbalanced datasets from KEEL repository. The results show that these pre-processing methods improves the accuracy compared with the previous stablished works. In addition, as a case study, the first pre-processing method is applied on transportation data of Tehran-Bazargan Highway in Iran with IR equal to 36. |
| Tasks | |
| Published | 2018-03-29 |
| URL | http://arxiv.org/abs/1803.11002v1 |
| http://arxiv.org/pdf/1803.11002v1.pdf | |
| PWC | https://paperswithcode.com/paper/modified-smote-using-mutual-information-and |
| Repo | |
| Framework | |
How Much Does Tokenization Affect Neural Machine Translation?
| Title | How Much Does Tokenization Affect Neural Machine Translation? |
| Authors | Miguel Domingo, Mercedes Garcıa-Martınez, Alexandre Helle, Francisco Casacuberta, Manuel Herranz |
| Abstract | Tokenization or segmentation is a wide concept that covers simple processes such as separating punctuation from words, or more sophisticated processes such as applying morphological knowledge. Neural Machine Translation (NMT) requires a limited-size vocabulary for computational cost and enough examples to estimate word embeddings. Separating punctuation and splitting tokens into words or subwords has proven to be helpful to reduce vocabulary and increase the number of examples of each word, improving the translation quality. Tokenization is more challenging when dealing with languages with no separator between words. In order to assess the impact of the tokenization in the quality of the final translation on NMT, we experimented on five tokenizers over ten language pairs. We reached the conclusion that the tokenization significantly affects the final translation quality and that the best tokenizer differs for different language pairs. |
| Tasks | Machine Translation, Tokenization, Word Embeddings |
| Published | 2018-12-20 |
| URL | https://arxiv.org/abs/1812.08621v4 |
| https://arxiv.org/pdf/1812.08621v4.pdf | |
| PWC | https://paperswithcode.com/paper/how-much-does-tokenization-affect-neural |
| Repo | |
| Framework | |
Predicting Language Recovery after Stroke with Convolutional Networks on Stitched MRI
| Title | Predicting Language Recovery after Stroke with Convolutional Networks on Stitched MRI |
| Authors | Yusuf H. Roohani, Noor Sajid, Pranava Madhyastha, Cathy J. Price, Thomas M. H. Hope |
| Abstract | One third of stroke survivors have language difficulties. Emerging evidence suggests that their likelihood of recovery depends mainly on the damage to language centers. Thus previous research for predicting language recovery post-stroke has focused on identifying damaged regions of the brain. In this paper, we introduce a novel method where we only make use of stitched 2-dimensional cross-sections of raw MRI scans in a deep convolutional neural network setup to predict language recovery post-stroke. Our results show: a) the proposed model that only uses MRI scans has comparable performance to models that are dependent on lesion specific information; b) the features learned by our model are complementary to the lesion specific information and the combination of both appear to outperform previously reported results in similar settings. We further analyse the CNN model for understanding regions in brain that are responsible for arriving at these predictions using gradient based saliency maps. Our findings are in line with previous lesion studies. |
| Tasks | |
| Published | 2018-11-26 |
| URL | http://arxiv.org/abs/1811.10520v1 |
| http://arxiv.org/pdf/1811.10520v1.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-language-recovery-after-stroke |
| Repo | |
| Framework | |
Learning High-level Representations from Demonstrations
| Title | Learning High-level Representations from Demonstrations |
| Authors | Garrett Andersen, Peter Vrancx, Haitham Bou-Ammar |
| Abstract | Hierarchical learning (HL) is key to solving complex sequential decision problems with long horizons and sparse rewards. It allows learning agents to break-up large problems into smaller, more manageable subtasks. A common approach to HL, is to provide the agent with a number of high-level skills that solve small parts of the overall problem. A major open question, however, is how to identify a suitable set of reusable skills. We propose a principled approach that uses human demonstrations to infer a set of subgoals based on changes in the demonstration dynamics. Using these subgoals, we decompose the learning problem into an abstract high-level representation and a set of low-level subtasks. The abstract description captures the overall problem structure, while subtasks capture desired skills. We demonstrate that we can jointly optimize over both levels of learning. We show that the resulting method significantly outperforms previous baselines on two challenging problems: the Atari 2600 game Montezuma’s Revenge, and a simulated robotics problem moving the ant robot through a maze. |
| Tasks | Montezuma’s Revenge |
| Published | 2018-02-19 |
| URL | http://arxiv.org/abs/1802.06604v3 |
| http://arxiv.org/pdf/1802.06604v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-high-level-representations-from |
| Repo | |
| Framework | |
Focusing on What is Relevant: Time-Series Learning and Understanding using Attention
| Title | Focusing on What is Relevant: Time-Series Learning and Understanding using Attention |
| Authors | Phongtharin Vinayavekhin, Subhajit Chaudhury, Asim Munawar, Don Joven Agravante, Giovanni De Magistris, Daiki Kimura, Ryuki Tachibana |
| Abstract | This paper is a contribution towards interpretability of the deep learning models in different applications of time-series. We propose a temporal attention layer that is capable of selecting the relevant information to perform various tasks, including data completion, key-frame detection and classification. The method uses the whole input sequence to calculate an attention value for each time step. This results in more focused attention values and more plausible visualisation than previous methods. We apply the proposed method to three different tasks. Experimental results show that the proposed network produces comparable results to a state of the art. In addition, the network provides better interpretability of the decision, that is, it generates more significant attention weight to related frames compared to similar techniques attempted in the past. |
| Tasks | Time Series |
| Published | 2018-06-22 |
| URL | http://arxiv.org/abs/1806.08523v1 |
| http://arxiv.org/pdf/1806.08523v1.pdf | |
| PWC | https://paperswithcode.com/paper/focusing-on-what-is-relevant-time-series |
| Repo | |
| Framework | |
Balanced News Using Constrained Bandit-based Personalization
| Title | Balanced News Using Constrained Bandit-based Personalization |
| Authors | Sayash Kapoor, Vijay Keswani, Nisheeth K. Vishnoi, L. Elisa Celis |
| Abstract | We present a prototype for a news search engine that presents balanced viewpoints across liberal and conservative articles with the goal of de-polarizing content and allowing users to escape their filter bubble. The balancing is done according to flexible user-defined constraints, and leverages recent advances in constrained bandit optimization. We showcase our balanced news feed by displaying it side-by-side with the news feed produced by a traditional (polarized) feed. |
| Tasks | |
| Published | 2018-06-24 |
| URL | http://arxiv.org/abs/1806.09202v1 |
| http://arxiv.org/pdf/1806.09202v1.pdf | |
| PWC | https://paperswithcode.com/paper/balanced-news-using-constrained-bandit-based |
| Repo | |
| Framework | |
Advice from the Oracle: Really Intelligent Information Retrieval
| Title | Advice from the Oracle: Really Intelligent Information Retrieval |
| Authors | Michael J. Kurtz |
| Abstract | What is “intelligent” information retrieval? Essentially this is asking what is intelligence, in this article I will attempt to show some of the aspects of human intelligence, as related to information retrieval. I will do this by the device of a semi-imaginary Oracle. Every Observatory has an oracle, someone who is a distinguished scientist, has great administrative responsibilities, acts as mentor to a number of less senior people, and as trusted advisor to even the most accomplished scientists, and knows essentially everyone in the field. In an appendix I will present a brief summary of the Statistical Factor Space method for text indexing and retrieval, and indicate how it will be used in the Astrophysics Data System Abstract Service. 2018 Keywords: Personal Digital Assistant; Supervised Topic Models |
| Tasks | Information Retrieval, Topic Models |
| Published | 2018-01-02 |
| URL | http://arxiv.org/abs/1801.00815v1 |
| http://arxiv.org/pdf/1801.00815v1.pdf | |
| PWC | https://paperswithcode.com/paper/advice-from-the-oracle-really-intelligent |
| Repo | |
| Framework | |
Clumped Nuclei Segmentation with Adjacent Point Match and Local Shape based Intensity Analysis for Overlapped Nuclei in Fluorescence In-Situ Hybridization Images
| Title | Clumped Nuclei Segmentation with Adjacent Point Match and Local Shape based Intensity Analysis for Overlapped Nuclei in Fluorescence In-Situ Hybridization Images |
| Authors | Xiaoyuan Guo, Hanyi Yu, Blair Rossetti, George Teodoro, Daniel Brat, Jun Kong |
| Abstract | Highly clumped nuclei clusters captured in fluorescence in situ hybridization microscopy images are common histology entities under investigations in a wide spectrum of tissue-related biomedical investigations. Due to their large scale in presence, computer based image analysis is used to facilitate such analysis with improved analysis efficiency and reproducibility. To ensure the quality of downstream biomedical analyses, it is essential to segment clustered nuclei with high quality. However, this presents a technical challenge commonly encountered in a large number of biomedical research, as nuclei are often overlapped due to a high cell density. In this paper, we propose an segmentation algorithm that identifies point pair connection candidates and evaluates adjacent point connections with a formulated ellipse fitting quality indicator. After connection relationships are determined, we recover the resulting dividing paths by following points with specific eigenvalues from Hessian in a constrained searching space. We validate our algorithm with 560 image patches from two classes of tumor regions of seven brain tumor patients. Both qualitative and quantitative experimental results suggest that our algorithm is promising for dividing overlapped nuclei in fluorescence in situ hybridization microscopy images widely used in various biomedical research. |
| Tasks | |
| Published | 2018-08-14 |
| URL | http://arxiv.org/abs/1808.04795v1 |
| http://arxiv.org/pdf/1808.04795v1.pdf | |
| PWC | https://paperswithcode.com/paper/clumped-nuclei-segmentation-with-adjacent |
| Repo | |
| Framework | |
Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points
| Title | Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points |
| Authors | Krishnakumar Balasubramanian, Saeed Ghadimi |
| Abstract | In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting and saddle-point avoiding. To handle constrained optimization, we first propose generalizations of the conditional gradient algorithm achieving rates similar to the standard stochastic gradient algorithm using only zeroth-order information. To facilitate zeroth-order optimization in high-dimensions, we explore the advantages of structural sparsity assumptions. Specifically, (i) we highlight an implicit regularization phenomenon where the standard stochastic gradient algorithm with zeroth-order information adapts to the sparsity of the problem at hand by just varying the step-size and (ii) propose a truncated stochastic gradient algorithm with zeroth-order information, whose rate of convergence depends only poly-logarithmically on the dimensionality. We next focus on avoiding saddle-points in non-convex setting. Towards that, we interpret the Gaussian smoothing technique for estimating gradient based on zeroth-order information as an instantiation of first-order Stein’s identity. Based on this, we provide a novel linear-(in dimension) time estimator of the Hessian matrix of a function using only zeroth-order information, which is based on second-order Stein’s identity. We then provide an algorithm for avoiding saddle-points, which is based on a zeroth-order cubic regularization Newton’s method and discuss its convergence rates. |
| Tasks | Stochastic Optimization |
| Published | 2018-09-17 |
| URL | http://arxiv.org/abs/1809.06474v2 |
| http://arxiv.org/pdf/1809.06474v2.pdf | |
| PWC | https://paperswithcode.com/paper/zeroth-order-nonconvex-stochastic |
| Repo | |
| Framework | |
The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD
| Title | The risk of sub-optimal use of Open Source NLP Software: UKB is inadvertently state-of-the-art in knowledge-based WSD |
| Authors | Eneko Agirre, Oier López de Lacalle, Aitor Soroa |
| Abstract | UKB is an open source collection of programs for performing, among other tasks, knowledge-based Word Sense Disambiguation (WSD). Since it was released in 2009 it has been often used out-of-the-box in sub-optimal settings. We show that nine years later it is the state-of-the-art on knowledge-based WSD. This case shows the pitfalls of releasing open source NLP software without optimal default settings and precise instructions for reproducibility. |
| Tasks | Word Sense Disambiguation |
| Published | 2018-05-11 |
| URL | http://arxiv.org/abs/1805.04277v1 |
| http://arxiv.org/pdf/1805.04277v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-risk-of-sub-optimal-use-of-open-source |
| Repo | |
| Framework | |
A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation
| Title | A Network Structure to Explicitly Reduce Confusion Errors in Semantic Segmentation |
| Authors | Qichuan Geng, Xinyu Huang, Zhong Zhou, Ruigang Yang |
| Abstract | Confusing classes that are ubiquitous in real world often degrade performance for many vision related applications like object detection, classification, and segmentation. The confusion errors are not only caused by similar visual patterns but also amplified by various factors during the training of our designed models, such as reduced feature resolution in the encoding process or imbalanced data distributions. A large amount of deep learning based network structures has been proposed in recent years to deal with these individual factors and improve network performance. However, to our knowledge, no existing work in semantic image segmentation is designed to tackle confusion errors explicitly. In this paper, we present a novel and general network structure that reduces confusion errors in more direct manner and apply the network for semantic segmentation. There are two major contributions in our network structure: 1) We ensemble subnets with heterogeneous output spaces based on the discriminative confusing groups. The training for each subnet can distinguish confusing classes within the group without affecting unrelated classes outside the group. 2) We propose an improved cross-entropy loss function that maximizes the probability assigned to the correct class and penalizes the probabilities assigned to the confusing classes at the same time. Our network structure is a general structure and can be easily adapted to any other networks to further reduce confusion errors. Without any changes in the feature encoder and post-processing steps, our experiments demonstrate consistent and significant improvements on different baseline models on Cityscapes and PASCAL VOC datasets (e.g., 3.05% over ResNet-101 and 1.30% over ResNet-38). |
| Tasks | Object Detection, Semantic Segmentation |
| Published | 2018-08-01 |
| URL | http://arxiv.org/abs/1808.00313v1 |
| http://arxiv.org/pdf/1808.00313v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-network-structure-to-explicitly-reduce |
| Repo | |
| Framework | |
