Paper Group ANR 1624
Is perturbation an effective restart strategy?. Surprises in High-Dimensional Ridgeless Least Squares Interpolation. Handwritten Text Segmentation via End-to-End Learning of Convolutional Neural Network. Discriminative structural graph classification. Estimation of 2D Velocity Model using Acoustic Signals and Convolutional Neural Networks. Realism …
Is perturbation an effective restart strategy?
| Title | Is perturbation an effective restart strategy? |
| Authors | Aldeida Aleti, Mark Wallace, Markus Wagner |
| Abstract | Premature convergence can be detrimental to the performance of search methods, which is why many search algorithms include restart strategies to deal with it. While it is common to perturb the incumbent solution with diversification steps of various sizes with the hope that the search method will find a new basin of attraction leading to a better local optimum, it is usually not clear how big the perturbation step should be. We introduce a new property of fitness landscapes termed “Neighbours with Similar Fitness” and we demonstrate that the effectiveness of a restart strategy depends on this property. |
| Tasks | |
| Published | 2019-12-05 |
| URL | https://arxiv.org/abs/1912.02535v1 |
| https://arxiv.org/pdf/1912.02535v1.pdf | |
| PWC | https://paperswithcode.com/paper/is-perturbation-an-effective-restart-strategy |
| Repo | |
| Framework | |
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
| Title | Surprises in High-Dimensional Ridgeless Least Squares Interpolation |
| Authors | Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J. Tibshirani |
| Abstract | Interpolators—estimators that achieve zero training error—have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum $\ell_2$ norm (“ridgeless”) interpolation in high-dimensional least squares regression. We consider two different models for the feature distribution: a linear model, where the feature vectors $x_i \in \mathbb{R}^p$ are obtained by applying a linear transform to a vector of i.i.d. entries, $x_i = \Sigma^{1/2} z_i$ (with $z_i \in \mathbb{R}^p$); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, $x_i = \varphi(W z_i)$ (with $z_i \in \mathbb{R}^d$, $W \in \mathbb{R}^{p \times d}$ a matrix of i.i.d. entries, and $\varphi$ an activation function acting componentwise on $W z_i$). We recover—in a precise quantitative way—several phenomena that have been observed in large-scale neural networks and kernel machines, including the “double descent” behavior of the prediction risk, and the potential benefits of overparametrization. |
| Tasks | |
| Published | 2019-03-19 |
| URL | https://arxiv.org/abs/1903.08560v4 |
| https://arxiv.org/pdf/1903.08560v4.pdf | |
| PWC | https://paperswithcode.com/paper/surprises-in-high-dimensional-ridgeless-least |
| Repo | |
| Framework | |
Handwritten Text Segmentation via End-to-End Learning of Convolutional Neural Network
| Title | Handwritten Text Segmentation via End-to-End Learning of Convolutional Neural Network |
| Authors | Junho Jo, Hyung Il Koo, Jae Woong Soh, Nam Ik Cho |
| Abstract | We present a new handwritten text segmentation method by training a convolutional neural network (CNN) in an end-to-end manner. Many conventional methods addressed this problem by extracting connected components and then classifying them. However, this two-step approach has limitations when handwritten components and machine-printed parts are overlapping. Unlike conventional methods, we develop an end-to-end deep CNN for this problem, which does not need any preprocessing steps. Since there is no publicly available dataset for this goal and pixel-wise annotations are time-consuming and costly, we also propose a data synthesis algorithm that generates realistic training samples. For training our network, we develop a cross-entropy based loss function that addresses the imbalance problems. Experimental results on synthetic and real images show the effectiveness of the proposed method. Specifically, the proposed network has been trained solely on synthetic images, nevertheless the removal of handwritten text in real documents improves OCR performance from 71.13% to 92.50%, showing the generalization performance of our network and synthesized images. |
| Tasks | Optical Character Recognition |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05229v1 |
| https://arxiv.org/pdf/1906.05229v1.pdf | |
| PWC | https://paperswithcode.com/paper/handwritten-text-segmentation-via-end-to-end |
| Repo | |
| Framework | |
Discriminative structural graph classification
| Title | Discriminative structural graph classification |
| Authors | Younjoo Seo, Andreas Loukas, Nathanaël Perraudin |
| Abstract | This paper focuses on the discrimination capacity of aggregation functions: these are the permutation invariant functions used by graph neural networks to combine the features of nodes. Realizing that the most powerful aggregation functions suffer from a dimensionality curse, we consider a restricted setting. In particular, we show that the standard sum and a novel histogram-based function have the capacity to discriminate between any fixed number of inputs chosen by an adversary. Based on our insights, we design a graph neural network aiming, not to maximize discrimination capacity, but to learn discriminative graph representations that generalize well. Our empirical evaluation provides evidence that our choices can yield benefits to the problem of structural graph classification. |
| Tasks | Graph Classification |
| Published | 2019-05-31 |
| URL | https://arxiv.org/abs/1905.13422v2 |
| https://arxiv.org/pdf/1905.13422v2.pdf | |
| PWC | https://paperswithcode.com/paper/discriminative-structural-graph |
| Repo | |
| Framework | |
Estimation of 2D Velocity Model using Acoustic Signals and Convolutional Neural Networks
| Title | Estimation of 2D Velocity Model using Acoustic Signals and Convolutional Neural Networks |
| Authors | Marco Apolinario, Samuel Huaman Bustamante, Giorgio Morales, Joel Telles, Daniel Diaz |
| Abstract | The parameters estimation of a system using indirect measurements over the same system is a problem that occurs in many fields of engineering, known as the inverse problem. It also happens in the field of underwater acoustic, especially in mediums that are not transparent enough. In those cases, shape identification of objects using only acoustic signals is a challenge because it is carried out with information of echoes that are produced by objects with different densities from that of the medium. In general, these echoes are difficult to understand since their information is usually noisy and redundant. In this paper, we propose a model of convolutional neural network with an Encoder-Decoder configuration to estimate both localization and shape of objects, which produce reflected signals. This model allows us to obtain a 2D velocity model. The model was trained with data generated by the finite-difference method, and it achieved a value of 98.58% in the intersection over union metric 75.88% in precision and 64.69% in sensibility. |
| Tasks | |
| Published | 2019-06-10 |
| URL | https://arxiv.org/abs/1906.04310v1 |
| https://arxiv.org/pdf/1906.04310v1.pdf | |
| PWC | https://paperswithcode.com/paper/estimation-of-2d-velocity-model-using |
| Repo | |
| Framework | |
Realism Index: Interpolation in Generative Models With Arbitrary Prior
| Title | Realism Index: Interpolation in Generative Models With Arbitrary Prior |
| Authors | Łukasz Struski, Jacek Tabor, Igor Podolak, Aleksandra Nowak, Krzysztof Maziarz |
| Abstract | In order to perform plausible interpolations in the latent space of a generative model, we need a measure that credibly reflects if a point in an interpolation is close to the data manifold being modelled, i.e. if it is convincing. In this paper, we introduce a realism index of a point, which can be constructed from an arbitrary prior density, or based on FID score approach in case a prior is not available. We propose a numerically efficient algorithm that directly maximises the realism index of an interpolation which, as we theoretically prove, leads to a search of a geodesic with respect to the corresponding Riemann structure. We show that we obtain better interpolations then the classical linear ones, in particular when either the prior density is not convex shaped, or when the soap bubble effect appears. |
| Tasks | |
| Published | 2019-04-06 |
| URL | https://arxiv.org/abs/1904.03445v2 |
| https://arxiv.org/pdf/1904.03445v2.pdf | |
| PWC | https://paperswithcode.com/paper/interpolation-in-generative-models |
| Repo | |
| Framework | |
Crowd Management in Open Spaces
| Title | Crowd Management in Open Spaces |
| Authors | Tauseef Ali, Ahmed B. Altamimi |
| Abstract | Crowd analysis and management is a challenging problem to ensure public safety and security. For this purpose, many techniques have been proposed to cope with various problems. However, the generalization capabilities of these techniques is limited due to ignoring the fact that the density of crowd changes from low to extreme high depending on the scene under observation. We propose robust feature based approach to deal with the problem of crowd management for people safety and security. We have evaluated our method using a benchmark dataset and have presented details analysis. |
| Tasks | |
| Published | 2019-04-18 |
| URL | http://arxiv.org/abs/1904.12625v1 |
| http://arxiv.org/pdf/1904.12625v1.pdf | |
| PWC | https://paperswithcode.com/paper/190412625 |
| Repo | |
| Framework | |
Adaptive Initialization Method for K-means Algorithm
| Title | Adaptive Initialization Method for K-means Algorithm |
| Authors | Jie Yang, Yu-Kai Wang, Xin Yao, Chin-Teng Lin |
| Abstract | The K-means algorithm is a widely used clustering algorithm that offers simplicity and efficiency. However, the traditional K-means algorithm uses the random method to determine the initial cluster centers, which make clustering results prone to local optima and then result in worse clustering performance. Many initialization methods have been proposed, but none of them can dynamically adapt to datasets with various characteristics. In our previous research, an initialization method for K-means based on hybrid distance was proposed, and this algorithm can adapt to datasets with different characteristics. However, it has the following drawbacks: (a) When calculating density, the threshold cannot be uniquely determined, resulting in unstable results. (b) Heavily depending on adjusting the parameter, the parameter must be adjusted five times to obtain better clustering results. (c) The time complexity of the algorithm is quadratic, which is difficult to apply to large datasets. In the current paper, we proposed an adaptive initialization method for the K-means algorithm (AIMK) to improve our previous work. AIMK can not only adapt to datasets with various characteristics but also obtain better clustering results within two interactions. In addition, we then leverage random sampling in AIMK, which is named as AIMK-RS, to reduce the time complexity. AIMK-RS is easily applied to large and high-dimensional datasets. We compared AIMK and AIMK-RS with 10 different algorithms on 16 normal and six extra-large datasets. The experimental results show that AIMK and AIMK-RS outperform the current initialization methods and several well-known clustering algorithms. Furthermore, AIMK-RS can significantly reduce the complexity of applying it to extra-large datasets with high dimensions. The time complexity of AIMK-RS is O(n). |
| Tasks | |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.12104v1 |
| https://arxiv.org/pdf/1911.12104v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-initialization-method-for-k-means |
| Repo | |
| Framework | |
Meta-Transfer Networks for Zero-Shot Learning
| Title | Meta-Transfer Networks for Zero-Shot Learning |
| Authors | Yunlong Yu, Zhongfei Zhang, Jungong Han |
| Abstract | Zero-Shot Learning (ZSL) aims at recognizing unseen categories using some class semantics of the categories. The existing studies mostly leverage the seen categories to learn a visual-semantic interaction model to infer the unseen categories. However, the disjointness between the seen and unseen categories cannot ensure that the models trained on the seen categories generalize well to the unseen categories. In this work, we propose an episode-based approach to accumulate experiences on addressing disjointness issue by mimicking extensive classification scenarios where training classes and test classes are disjoint. In each episode, a visual-semantic interaction model is first trained on a subset of seen categories as a learner that provides an initial prediction for the rest disjoint seen categories and then a meta-learner fine-tunes the learner by minimizing the differences between the prediction and the ground-truth labels in a pre-defined space. By training extensive episodes on the seen categories, the model is trained to be an expert in predicting the mimetic unseen categories, which will generalize well to the real unseen categories. Extensive experiments on four datasets under both the traditional ZSL and generalized ZSL tasks show that our framework outperforms the state-of-the-art approaches by large margins. |
| Tasks | Zero-Shot Learning |
| Published | 2019-09-08 |
| URL | https://arxiv.org/abs/1909.03360v1 |
| https://arxiv.org/pdf/1909.03360v1.pdf | |
| PWC | https://paperswithcode.com/paper/meta-transfer-networks-for-zero-shot-learning |
| Repo | |
| Framework | |
Listen to the Image
| Title | Listen to the Image |
| Authors | Di Hu, Dong Wang, Xuelong Li, Feiping Nie, Qi Wang |
| Abstract | Visual-to-auditory sensory substitution devices can assist the blind in sensing the visual environment by translating the visual information into a sound pattern. To improve the translation quality, the task performances of the blind are usually employed to evaluate different encoding schemes. In contrast to the toilsome human-based assessment, we argue that machine model can be also developed for evaluation, and more efficient. To this end, we firstly propose two distinct cross-modal perception model w.r.t. the late-blind and congenitally-blind cases, which aim to generate concrete visual contents based on the translated sound. To validate the functionality of proposed models, two novel optimization strategies w.r.t. the primary encoding scheme are presented. Further, we conduct sets of human-based experiments to evaluate and compare them with the conducted machine-based assessments in the cross-modal generation task. Their highly consistent results w.r.t. different encoding schemes indicate that using machine model to accelerate optimization evaluation and reduce experimental cost is feasible to some extent, which could dramatically promote the upgrading of encoding scheme then help the blind to improve their visual perception ability. |
| Tasks | |
| Published | 2019-04-19 |
| URL | http://arxiv.org/abs/1904.09115v1 |
| http://arxiv.org/pdf/1904.09115v1.pdf | |
| PWC | https://paperswithcode.com/paper/listen-to-the-image |
| Repo | |
| Framework | |
Active Deep Decoding of Linear Codes
| Title | Active Deep Decoding of Linear Codes |
| Authors | Ishay Be’ery, Nir Raviv, Tomer Raviv, Yair Be’ery |
| Abstract | High quality data is essential in deep learning to train a robust model. While in other fields data is sparse and costly to collect, in error decoding it is free to query and label thus allowing potential data exploitation. Utilizing this fact and inspired by active learning, two novel methods are introduced to improve Weighted Belief Propagation (WBP) decoding. These methods incorporate machine-learning concepts with error decoding measures. For BCH(63,36), (63,45) and (127,64) codes, with cycle-reduced parity-check matrices, improvement of up to 0.4dB at the waterfall region, and of up to 1.5dB at the errorfloor region in FER, over the original WBP, is demonstrated by smartly sampling the data, without increasing inference (decoding) complexity. The proposed methods constitutes an example guidelines for model enhancement by incorporation of domain knowledge from error-correcting field into a deep learning model. These guidelines can be adapted to any other deep learning based communication block. |
| Tasks | Active Learning |
| Published | 2019-06-06 |
| URL | https://arxiv.org/abs/1906.02778v2 |
| https://arxiv.org/pdf/1906.02778v2.pdf | |
| PWC | https://paperswithcode.com/paper/active-deep-decoding-of-linear-codes |
| Repo | |
| Framework | |
Bayesian Active Learning With Abstention Feedbacks
| Title | Bayesian Active Learning With Abstention Feedbacks |
| Authors | Cuong V. Nguyen, Lam Si Tung Ho, Huan Xu, Vu Dinh, Binh Nguyen |
| Abstract | We study pool-based active learning with abstention feedbacks where a labeler can abstain from labeling a queried example with some unknown abstention rate. Using the Bayesian approach, we develop two new greedy algorithms that learn both the classification problem and the unknown abstention rate at the same time. These are achieved by incorporating the estimated average abstention rate into the greedy criteria. We prove that both algorithms have near-optimality guarantees: they respectively achieve a ${(1-\frac{1}{e})}$ constant factor approximation of the optimal expected or worst-case value of a useful utility function. Our experiments show the algorithms perform well in various practical scenarios. |
| Tasks | Active Learning |
| Published | 2019-06-04 |
| URL | https://arxiv.org/abs/1906.02179v1 |
| https://arxiv.org/pdf/1906.02179v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-active-learning-with-abstention |
| Repo | |
| Framework | |
Label-Removed Generative Adversarial Networks Incorporating with K-Means
| Title | Label-Removed Generative Adversarial Networks Incorporating with K-Means |
| Authors | Ce Wang, Zhangling Chen, Kun Shang |
| Abstract | Generative Adversarial Networks (GANs) have achieved great success in generating realistic images. Most of these are conditional models, although acquisition of class labels is expensive and time-consuming in practice. To reduce the dependence on labeled data, we propose an un-conditional generative adversarial model, called K-Means-GAN (KM-GAN), which incorporates the idea of updating centers in K-Means into GANs. Specifically, we redesign the framework of GANs by applying K-Means on the features extracted from the discriminator. With obtained labels from K-Means, we propose new objective functions from the perspective of deep metric learning (DML). Distinct from previous works, the discriminator is treated as a feature extractor rather than a classifier in KM-GAN, meanwhile utilization of K-Means makes features of the discriminator more representative. Experiments are conducted on various datasets, such as MNIST, Fashion-10, CIFAR-10 and CelebA, and show that the quality of samples generated by KM-GAN is comparable to some conditional generative adversarial models. |
| Tasks | Metric Learning |
| Published | 2019-02-19 |
| URL | http://arxiv.org/abs/1902.06938v1 |
| http://arxiv.org/pdf/1902.06938v1.pdf | |
| PWC | https://paperswithcode.com/paper/label-removed-generative-adversarial-networks |
| Repo | |
| Framework | |
From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
| Title | From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group) |
| Authors | Zied Bouraoui, Antoine Cornuéjols, Thierry Denœux, Sébastien Destercke, Didier Dubois, Romain Guillaume, João Marques-Silva, Jérôme Mengin, Henri Prade, Steven Schockaert, Mathieu Serrurier, Christel Vrain |
| Abstract | This paper proposes a tentative and original survey of meeting points between Knowledge Representation and Reasoning (KRR) and Machine Learning (ML), two areas which have been developing quite separately in the last three decades. Some common concerns are identified and discussed such as the types of used representation, the roles of knowledge and data, the lack or the excess of information, or the need for explanations and causal understanding. Then some methodologies combining reasoning and learning are reviewed (such as inductive logic programming, neuro-symbolic reasoning, formal concept analysis, rule-based representations and ML, uncertainty in ML, or case-based reasoning and analogical reasoning), before discussing examples of synergies between KRR and ML (including topics such as belief functions on regression, EM algorithm versus revision, the semantic description of vector representations, the combination of deep learning with high level inference, knowledge graph completion, declarative frameworks for data mining, or preferences and recommendation). This paper is the first step of a work in progress aiming at a better mutual understanding of research in KRR and ML, and how they could cooperate. |
| Tasks | Knowledge Graph Completion |
| Published | 2019-12-13 |
| URL | https://arxiv.org/abs/1912.06612v1 |
| https://arxiv.org/pdf/1912.06612v1.pdf | |
| PWC | https://paperswithcode.com/paper/from-shallow-to-deep-interactions-between |
| Repo | |
| Framework | |
Interpretable Dynamics Models for Data-Efficient Reinforcement Learning
| Title | Interpretable Dynamics Models for Data-Efficient Reinforcement Learning |
| Authors | Markus Kaiser, Clemens Otte, Thomas Runkler, Carl Henrik Ek |
| Abstract | In this paper, we present a Bayesian view on model-based reinforcement learning. We use expert knowledge to impose structure on the transition model and present an efficient learning scheme based on variational inference. This scheme is applied to a heteroskedastic and bimodal benchmark problem on which we compare our results to NFQ and show how our approach yields human-interpretable insight about the underlying dynamics while also increasing data-efficiency. |
| Tasks | |
| Published | 2019-07-10 |
| URL | https://arxiv.org/abs/1907.04902v1 |
| https://arxiv.org/pdf/1907.04902v1.pdf | |
| PWC | https://paperswithcode.com/paper/interpretable-dynamics-models-for-data |
| Repo | |
| Framework | |
