October 17, 2019

2761 words 13 mins read

Paper Group ANR 961

Gradient descent in higher codimension. Automated Segmentation of Cervical Nuclei in Pap Smear Images using Deformable Multi-path Ensemble Model. Sigsoftmax: Reanalysis of the Softmax Bottleneck. An Image Processing based Object Counting Approach for Machine Vision Application. Exploration on Grounded Word Embedding: Matching Words and Images with …

Gradient descent in higher codimension


Title	Gradient descent in higher codimension
Authors	Y. Cooper
Abstract	We consider the behavior of gradient flow and of discrete and noisy gradient descent. It is commonly noted that the addition of noise to the process of discrete gradient descent can affect the trajectory of gradient descent. In previous work, we observed such effects. There, we considered the case where the minima had codimension 1. In this note, we do some computer experiments and observe the behavior of noisy gradient descent in the more complex setting of minima of higher codimension.
Tasks
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05527v2
PDF	http://arxiv.org/pdf/1809.05527v2.pdf
PWC	https://paperswithcode.com/paper/secondary-gradient-descent-in-higher
Repo
Framework

Automated Segmentation of Cervical Nuclei in Pap Smear Images using Deformable Multi-path Ensemble Model


Title	Automated Segmentation of Cervical Nuclei in Pap Smear Images using Deformable Multi-path Ensemble Model
Authors	Jie Zhao, Quanzheng Li, Xiang Li, Hongfeng Li, Li Zhang
Abstract	Pap smear testing has been widely used for detecting cervical cancers based on the morphology properties of cell nuclei in microscopic image. An accurate nuclei segmentation could thus improve the success rate of cervical cancer screening. In this work, a method of automated cervical nuclei segmentation using Deformable Multipath Ensemble Model (D-MEM) is proposed. The approach adopts a U-shaped convolutional network as a backbone network, in which dense blocks are used to transfer feature information more effectively. To increase the flexibility of the model, we then use deformable convolution to deal with different nuclei irregular shapes and sizes. To reduce the predictive bias, we further construct multiple networks with different settings, which form an ensemble model. The proposed segmentation framework has achieved state-of-the-art accuracy on Herlev dataset with Zijdenbos similarity index (ZSI) of 0.933, and has the potential to be extended for solving other medical image segmentation tasks.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2018-12-03
URL	https://arxiv.org/abs/1812.00527v2
PDF	https://arxiv.org/pdf/1812.00527v2.pdf
PWC	https://paperswithcode.com/paper/automated-segmentation-of-cervical-nuclei-in
Repo
Framework

Sigsoftmax: Reanalysis of the Softmax Bottleneck


Title	Sigsoftmax: Reanalysis of the Softmax Bottleneck
Authors	Sekitoshi Kanai, Yasuhiro Fujiwara, Yuki Yamanaka, Shuichi Adachi
Abstract	Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning. However, a recent study revealed that softmax can be a bottleneck of representational capacity of neural networks in language modeling (the softmax bottleneck). In this paper, we propose an output activation function for breaking the softmax bottleneck without additional parameters. We re-analyze the softmax bottleneck from the perspective of the output set of log-softmax and identify the cause of the softmax bottleneck. On the basis of this analysis, we propose sigsoftmax, which is composed of a multiplication of an exponential function and sigmoid function. Sigsoftmax can break the softmax bottleneck. The experiments on language modeling demonstrate that sigsoftmax and mixture of sigsoftmax outperform softmax and mixture of softmax, respectively.
Tasks	Language Modelling
Published	2018-05-28
URL	http://arxiv.org/abs/1805.10829v1
PDF	http://arxiv.org/pdf/1805.10829v1.pdf
PWC	https://paperswithcode.com/paper/sigsoftmax-reanalysis-of-the-softmax
Repo
Framework

An Image Processing based Object Counting Approach for Machine Vision Application


Title	An Image Processing based Object Counting Approach for Machine Vision Application
Authors	Mehmet Baygin, Mehmet Karakose, Alisan Sarimaden, Erhan Akin
Abstract	Machine vision applications are low cost and high precision measurement systems which are frequently used in production lines. With these systems that provide contactless control and measurement, production facilities are able to reach high production numbers without errors. Machine vision operations such as product counting, error control, dimension measurement can be performed through a camera. In this paper, a machine vision application is proposed, which can perform object-independent product counting. The proposed approach is based on Otsu thresholding and Hough transformation and performs automatic counting independently of product type and color. Basically one camera is used in the system. Through this camera, an image of the products passing through a conveyor is taken and various image processing algorithms are applied to these images. In this approach using images obtained from a real experimental setup, a real-time machine vision application was installed. As a result of the experimental studies performed, it has been determined that the proposed approach gives fast, accurate and reliable results.
Tasks	Object Counting
Published	2018-02-16
URL	http://arxiv.org/abs/1802.05911v1
PDF	http://arxiv.org/pdf/1802.05911v1.pdf
PWC	https://paperswithcode.com/paper/an-image-processing-based-object-counting
Repo
Framework

Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model


Title	Exploration on Grounded Word Embedding: Matching Words and Images with Image-Enhanced Skip-Gram Model
Authors	Ruixuan Luo
Abstract	Word embedding is designed to represent the semantic meaning of a word with low dimensional vectors. The state-of-the-art methods of learning word embeddings (word2vec and GloVe) only use the word co-occurrence information. The learned embeddings are real number vectors, which are obscure to human. In this paper, we propose an Image-Enhanced Skip-Gram Model to learn grounded word embeddings by representing the word vectors in the same hyper-plane with image vectors. Experiments show that the image vectors and word embeddings learned by our model are highly correlated, which indicates that our model is able to provide a vivid image-based explanation to the word embeddings.
Tasks	Learning Word Embeddings, Word Embeddings
Published	2018-09-08
URL	http://arxiv.org/abs/1809.02765v1
PDF	http://arxiv.org/pdf/1809.02765v1.pdf
PWC	https://paperswithcode.com/paper/exploration-on-grounded-word-embedding
Repo
Framework

Multilingual Cross-domain Perspectives on Online Hate Speech


Title	Multilingual Cross-domain Perspectives on Online Hate Speech
Authors	Tom De Smedt, Sylvia Jaki, Eduan Kotzé, Leïla Saoud, Maja Gwóźdź, Guy De Pauw, Walter Daelemans
Abstract	In this report, we present a study of eight corpora of online hate speech, by demonstrating the NLP techniques that we used to collect and analyze the jihadist, extremist, racist, and sexist content. Analysis of the multilingual corpora shows that the different contexts share certain characteristics in their hateful rhetoric. To expose the main features, we have focused on text classification, text profiling, keyword and collocation extraction, along with manual annotation and qualitative study.
Tasks	Text Classification
Published	2018-09-11
URL	http://arxiv.org/abs/1809.03944v1
PDF	http://arxiv.org/pdf/1809.03944v1.pdf
PWC	https://paperswithcode.com/paper/multilingual-cross-domain-perspectives-on
Repo
Framework

Extracting Epistatic Interactions in Type 2 Diabetes Genome-Wide Data Using Stacked Autoencoder


Title	Extracting Epistatic Interactions in Type 2 Diabetes Genome-Wide Data Using Stacked Autoencoder
Authors	Basma Abdulaimma, Paul Fergus, Carl Chalmers
Abstract	2 Diabetes is a leading worldwide public health concern, and its increasing prevalence has significant health and economic importance in all nations. The condition is a multifactorial disorder with a complex aetiology. The genetic determinants remain largely elusive, with only a handful of identified candidate genes. Genome wide association studies (GWAS) promised to significantly enhance our understanding of genetic based determinants of common complex diseases. To date, 83 single nucleotide polymorphisms (SNPs) for type 2 diabetes have been identified using GWAS. Standard statistical tests for single and multi-locus analysis such as logistic regression, have demonstrated little effect in understanding the genetic architecture of complex human diseases. Logistic regression is modelled to capture linear interactions but neglects the non-linear epistatic interactions present within genetic data. There is an urgent need to detect epistatic interactions in complex diseases as this may explain the remaining missing heritability in such diseases. In this paper, we present a novel framework based on deep learning algorithms that deal with non-linear epistatic interactions that exist in genome wide association data. Logistic association analysis under an additive genetic model, adjusted for genomic control inflation factor, is conducted to remove statistically improbable SNPs to minimize computational overheads.
Tasks
Published	2018-08-28
URL	http://arxiv.org/abs/1808.09517v1
PDF	http://arxiv.org/pdf/1808.09517v1.pdf
PWC	https://paperswithcode.com/paper/extracting-epistatic-interactions-in-type-2
Repo
Framework

Graph filtering for data reduction and reconstruction


Title	Graph filtering for data reduction and reconstruction
Authors	Ioannis D. Schizas
Abstract	A novel approach is put forth that utilizes data similarity, quantified on a graph, to improve upon the reconstruction performance of principal component analysis. The tasks of data dimensionality reduction and reconstruction are formulated as graph filtering operations, that enable the exploitation of data node connectivity in a graph via the adjacency matrix. The unknown reducing and reconstruction filters are determined by optimizing a mean-square error cost that entails the data, as well as their graph adjacency matrix. Working in the graph spectral domain enables the derivation of simple gradient descent recursions used to update the matrix filter taps. Numerical tests in real image datasets demonstrate the better reconstruction performance of the novel method over standard principal component analysis.
Tasks	Dimensionality Reduction
Published	2018-09-25
URL	http://arxiv.org/abs/1809.09266v1
PDF	http://arxiv.org/pdf/1809.09266v1.pdf
PWC	https://paperswithcode.com/paper/graph-filtering-for-data-reduction-and
Repo
Framework

Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism


Title	Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism
Authors	Sha Yuan, Yu Zhang, Jie Tang, Huawei Shen, Xingxing Wei
Abstract	An ability to predict the popularity dynamics of individual items within a complex evolving system has important implications in a wide range of domains. Here we propose a deep learning attention mechanism to model the process through which individual items gain their popularity. We analyze the interpretability of the model with the four key phenomena confirmed independently in the previous studies of long-term popularity dynamics quantification, including the intrinsic quality, the aging effect, the recency effect and the Matthew effect. We analyze the effectiveness of introducing attention model in popularity dynamics prediction. Extensive experiments on a real-large citation data set demonstrate that the designed deep learning attention mechanism possesses remarkable power at predicting the long-term popularity dynamics. It consistently outperforms the existing methods, and achieves a significant performance improvement.
Tasks
Published	2018-11-06
URL	http://arxiv.org/abs/1811.02117v1
PDF	http://arxiv.org/pdf/1811.02117v1.pdf
PWC	https://paperswithcode.com/paper/modeling-and-predicting-popularity-dynamics
Repo
Framework

Learning to Optimize with Hidden Constraints


Title	Learning to Optimize with Hidden Constraints
Authors	Aaron Babier, Timothy C. Y. Chan, Adam Diamant, Rafid Mahmood
Abstract	We consider a data-driven framework for learning to generate decisions to instances of continuous optimization problems where the feasible set varies with an instance-specific auxiliary input. We use a data set of inputs and feasible solutions, as well as an oracle of feasibility, to iteratively train two machine learning models. The first model is a binary classifier for feasibility, which then serves as a barrier function to train the second model via an interior point method. We develop theory and optimality guarantees for interior point methods when given a barrier that relaxes the feasible set, and extend these results to obtain probabilistic out-of-sample guarantees for our learning framework. Finally, we implement our method on a radiation therapy treatment planning problem to predict personalized treatments for head-and-neck cancer patients.
Tasks
Published	2018-05-23
URL	https://arxiv.org/abs/1805.09293v2
PDF	https://arxiv.org/pdf/1805.09293v2.pdf
PWC	https://paperswithcode.com/paper/interior-point-methods-with-adversarial
Repo
Framework

Distributed learning of deep neural network over multiple agents


Title	Distributed learning of deep neural network over multiple agents
Authors	Otkrist Gupta, Ramesh Raskar
Abstract	In domains such as health care and finance, shortage of labeled data and computational resources is a critical issue while developing machine learning algorithms. To address the issue of labeled data scarcity in training and deployment of neural network-based systems, we propose a new technique to train deep neural networks over several data sources. Our method allows for deep neural networks to be trained using data from multiple entities in a distributed fashion. We evaluate our algorithm on existing datasets and show that it obtains performance which is similar to a regular neural network trained on a single machine. We further extend it to incorporate semi-supervised learning when training with few labeled samples, and analyze any security concerns that may arise. Our algorithm paves the way for distributed training of deep neural networks in data sensitive applications when raw data may not be shared directly.
Tasks
Published	2018-10-14
URL	http://arxiv.org/abs/1810.06060v1
PDF	http://arxiv.org/pdf/1810.06060v1.pdf
PWC	https://paperswithcode.com/paper/distributed-learning-of-deep-neural-network
Repo
Framework

Improving Solvability for Procedurally Generated Challenges in Physical Solitaire Games Through Entangled Components


Title	Improving Solvability for Procedurally Generated Challenges in Physical Solitaire Games Through Entangled Components
Authors	Mark Goadrich, James Droscha
Abstract	Challenges for physical solitaire puzzle games are typically designed in advance by humans and limited in number. Alternatively, some games incorporate rules for stochastic setup, where the human solver randomly sets up the game board before solving the challenge. These setup rules greatly increase the number of possible challenges, but can often generate unsolvable or uninteresting challenges. To better understand the compromises involved in minimizing undesirable challenges, we examine three games where component design choices can influence the stochastic nature of the resulting challenge generation algorithms. We evaluate the effect of these components and algorithms on challenge solvability and challenge engagement. We find that algorithms which control randomness through entangling components based on sub-elements of the puzzle mechanics can generate interesting challenges with a high probability of being solvable.
Tasks
Published	2018-10-03
URL	https://arxiv.org/abs/1810.01926v3
PDF	https://arxiv.org/pdf/1810.01926v3.pdf
PWC	https://paperswithcode.com/paper/improving-solvability-for-procedurally
Repo
Framework

ExFuse: Enhancing Feature Fusion for Semantic Segmentation


Title	ExFuse: Enhancing Feature Fusion for Semantic Segmentation
Authors	Zhenli Zhang, Xiangyu Zhang, Chao Peng, Dazhi Cheng, Jian Sun
Abstract	Modern semantic segmentation frameworks usually combine low-level and high-level features from pre-trained backbone convolutional models to boost performance. In this paper, we first point out that a simple fusion of low-level and high-level features could be less effective because of the gap in semantic levels and spatial resolution. We find that introducing semantic information into low-level features and high-resolution details into high-level features is more effective for the later fusion. Based on this observation, we propose a new framework, named ExFuse, to bridge the gap between low-level and high-level features thus significantly improve the segmentation quality by 4.0% in total. Furthermore, we evaluate our approach on the challenging PASCAL VOC 2012 segmentation benchmark and achieve 87.9% mean IoU, which outperforms the previous state-of-the-art results.
Tasks	Semantic Segmentation
Published	2018-04-11
URL	http://arxiv.org/abs/1804.03821v1
PDF	http://arxiv.org/pdf/1804.03821v1.pdf
PWC	https://paperswithcode.com/paper/exfuse-enhancing-feature-fusion-for-semantic
Repo
Framework

Learning compositionally through attentive guidance


Title	Learning compositionally through attentive guidance
Authors	Dieuwke Hupkes, Anand Singh, Kris Korrel, German Kruszewski, Elia Bruni
Abstract	While neural network models have been successfully applied to domains that require substantial generalisation skills, recent studies have implied that they struggle when solving the task they are trained on requires inferring its underlying compositional structure. In this paper, we introduce Attentive Guidance, a mechanism to direct a sequence to sequence model equipped with attention to find more compositional solutions. We test it on two tasks, devised precisely to assess the compositional capabilities of neural models, and we show that vanilla sequence to sequence models with attention overfit the training distribution, while the guided versions come up with compositional solutions that fit the training and testing distributions almost equally well. Moreover, the learned solutions generalise even in cases where the training and testing distributions strongly diverge. In this way, we demonstrate that sequence to sequence models are capable of finding compositional solutions without requiring extra components. These results helps to disentangle the causes for the lack of systematic compositionality in neural networks, which can in turn fuel future work.
Tasks
Published	2018-05-20
URL	https://arxiv.org/abs/1805.09657v4
PDF	https://arxiv.org/pdf/1805.09657v4.pdf
PWC	https://paperswithcode.com/paper/learning-compositionally-through-attentive
Repo
Framework

The Effect of Learning Strategy versus Inherent Architecture Properties on the Ability of Convolutional Neural Networks to Develop Transformation Invariance


Title	The Effect of Learning Strategy versus Inherent Architecture Properties on the Ability of Convolutional Neural Networks to Develop Transformation Invariance
Authors	Megha Srivastava, Kalanit Grill-Spector
Abstract	As object recognition becomes an increasingly common ML task, and recent research demonstrating CNNs vulnerability to attacks and small image perturbations necessitate fully understanding the foundations of object recognition. We focus on understanding the mechanisms behind how neural networks generalize to spatial transformations of complex objects. While humans excel at discriminating between objects shown at new positions, orientations, and scales, past results demonstrate that this may be limited to familiar objects - humans demonstrate low tolerance of spatial-variances for purposefully constructed novel objects. Because training artificial neural networks from scratch is similar to showing novel objects to humans, we seek to understand the factors influencing the tolerance of CNNs to spatial transformations. We conduct a thorough empirical examination of seven Convolutional Neural Network (CNN) architectures. By training on a controlled face image dataset, we measure model accuracy across different degrees of 5 transformations: position, size, rotation, Gaussian blur, and resolution transformation due to resampling. We also examine how learning strategy affects generalizability by examining how different amounts of pre-training have on model robustness. Overall, we find that the most significant contributor to transformation invariance is pre-training on a large, diverse image dataset. Moreover, while AlexNet tends to be the least robust network, VGG and ResNet architectures demonstrate higher robustness for different transformations. Along with kernel visualizations and qualitative analyses, we examine differences between learning strategy and inherent architectural properties in contributing to invariance of transformations, providing valuable information towards understanding how to achieve greater robustness to transformations in CNNs.
Tasks	Object Recognition
Published	2018-10-31
URL	http://arxiv.org/abs/1810.13128v1
PDF	http://arxiv.org/pdf/1810.13128v1.pdf
PWC	https://paperswithcode.com/paper/the-effect-of-learning-strategy-versus
Repo
Framework