Paper Group ANR 92
A Note on Kernel Methods for Multiscale Systems with Critical Transitions. A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers. Fast Efficient Object Detection Using Selective Attention. Instance Selection Improves Geometric Mean Accuracy: A Study on Imbalanced Data Classification. Unsupervised and Efficient V …
A Note on Kernel Methods for Multiscale Systems with Critical Transitions
| Title | A Note on Kernel Methods for Multiscale Systems with Critical Transitions |
| Authors | Boumediene Hamzi, Christian Kuehn, Sameh Mohamed |
| Abstract | We study the maximum mean discrepancy (MMD) in the context of critical transitions modelled by fast-slow stochastic dynamical systems. We establish a new link between the dynamical theory of critical transitions with the statistical aspects of the MMD. In particular, we show that a formal approximation of the MMD near fast subsystem bifurcation points can be computed to leading-order. In particular, this leading order approximation shows that the MMD depends intricately on the fast-slow systems parameters and one can only expect to extract warning signs under rather stringent conditions. However, the MMD turns out to be an excellent binary classifier to detect the change point induced by the critical transition. We cross-validate our results by numerical simulations for a van der Pol-type model. |
| Tasks | |
| Published | 2018-04-25 |
| URL | http://arxiv.org/abs/1804.09415v1 |
| http://arxiv.org/pdf/1804.09415v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-note-on-kernel-methods-for-multiscale |
| Repo | |
| Framework | |
A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers
| Title | A Droplet Approach Based on Raptor Codes for Distributed Computing With Straggling Servers |
| Authors | Albin Severinson, Alexandre Graell i Amat, Eirik Rosnes, Francisco Lazaro, Gianluigi Liva |
| Abstract | We propose a coded distributed computing scheme based on Raptor codes to address the straggler problem. In particular, we consider a scheme where each server computes intermediate values, referred to as droplets, that are either stored locally or sent over the network. Once enough droplets are collected, the computation can be completed. Compared to previous schemes in the literature, our proposed scheme achieves lower computational delay when the decoding time is taken into account. |
| Tasks | |
| Published | 2018-10-08 |
| URL | http://arxiv.org/abs/1810.03488v1 |
| http://arxiv.org/pdf/1810.03488v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-droplet-approach-based-on-raptor-codes-for |
| Repo | |
| Framework | |
Fast Efficient Object Detection Using Selective Attention
| Title | Fast Efficient Object Detection Using Selective Attention |
| Authors | Shivanthan Yohanandan, Andy Song, Adrian G. Dyer, Angela Faragasso, Subhrajit Roy, Dacheng Tao |
| Abstract | Retraction due to significant oversight |
| Tasks | Object Detection |
| Published | 2018-11-19 |
| URL | https://arxiv.org/abs/1811.07502v3 |
| https://arxiv.org/pdf/1811.07502v3.pdf | |
| PWC | https://paperswithcode.com/paper/fast-efficient-object-detection-using |
| Repo | |
| Framework | |
Instance Selection Improves Geometric Mean Accuracy: A Study on Imbalanced Data Classification
| Title | Instance Selection Improves Geometric Mean Accuracy: A Study on Imbalanced Data Classification |
| Authors | Ludmila I. Kuncheva, Álvar Arnaiz-González, José-Francisco Díez-Pastor, Iain A. D. Gunn |
| Abstract | A natural way of handling imbalanced data is to attempt to equalise the class frequencies and train the classifier of choice on balanced data. For two-class imbalanced problems, the classification success is typically measured by the geometric mean (GM) of the true positive and true negative rates. Here we prove that GM can be improved upon by instance selection, and give the theoretical conditions for such an improvement. We demonstrate that GM is non-monotonic with respect to the number of retained instances, which discourages systematic instance selection. We also show that balancing the distribution frequencies is inferior to a direct maximisation of GM. To verify our theoretical findings, we carried out an experimental study of 12 instance selection methods for imbalanced data, using 66 standard benchmark data sets. The results reveal possible room for new instance selection methods for imbalanced data. |
| Tasks | |
| Published | 2018-04-19 |
| URL | http://arxiv.org/abs/1804.07155v1 |
| http://arxiv.org/pdf/1804.07155v1.pdf | |
| PWC | https://paperswithcode.com/paper/instance-selection-improves-geometric-mean |
| Repo | |
| Framework | |
Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR
| Title | Unsupervised and Efficient Vocabulary Expansion for Recurrent Neural Network Language Models in ASR |
| Authors | Yerbolat Khassanov, Eng Siong Chng |
| Abstract | In automatic speech recognition (ASR) systems, recurrent neural network language models (RNNLM) are used to rescore a word lattice or N-best hypotheses list. Due to the expensive training, the RNNLM’s vocabulary set accommodates only small shortlist of most frequent words. This leads to suboptimal performance if an input speech contains many out-of-shortlist (OOS) words. An effective solution is to increase the shortlist size and retrain the entire network which is highly inefficient. Therefore, we propose an efficient method to expand the shortlist set of a pretrained RNNLM without incurring expensive retraining and using additional training data. Our method exploits the structure of RNNLM which can be decoupled into three parts: input projection layer, middle layers, and output projection layer. Specifically, our method expands the word embedding matrices in projection layers and keeps the middle layers unchanged. In this approach, the functionality of the pretrained RNNLM will be correctly maintained as long as OOS words are properly modeled in two embedding spaces. We propose to model the OOS words by borrowing linguistic knowledge from appropriate in-shortlist words. Additionally, we propose to generate the list of OOS words to expand vocabulary in unsupervised manner by automatically extracting them from ASR output. |
| Tasks | Speech Recognition |
| Published | 2018-06-27 |
| URL | http://arxiv.org/abs/1806.10306v1 |
| http://arxiv.org/pdf/1806.10306v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-and-efficient-vocabulary |
| Repo | |
| Framework | |
Black-Box Reductions for Parameter-free Online Learning in Banach Spaces
| Title | Black-Box Reductions for Parameter-free Online Learning in Banach Spaces |
| Authors | Ashok Cutkosky, Francesco Orabona |
| Abstract | We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce optimization in a Banach space to one-dimensional optimization, and we reduce optimization over a constrained domain to unconstrained optimization. All of our reductions run as fast as online gradient descent. We use our new techniques to improve upon the previously best regret bounds for parameter-free learning, and do so for arbitrary norms. |
| Tasks | |
| Published | 2018-02-17 |
| URL | http://arxiv.org/abs/1802.06293v2 |
| http://arxiv.org/pdf/1802.06293v2.pdf | |
| PWC | https://paperswithcode.com/paper/black-box-reductions-for-parameter-free |
| Repo | |
| Framework | |
Feature Selection for Gender Classification in TUIK Life Satisfaction Survey
| Title | Feature Selection for Gender Classification in TUIK Life Satisfaction Survey |
| Authors | Adil Çoban, Ilhan Tarımer |
| Abstract | As known, attribute selection is a method that is used before the classification of data mining. In this study, a new data set has been created by using attributes expressing overall satisfaction in Turkey Statistical Institute (TSI) Life Satisfaction Survey dataset. Attributes are sorted by Ranking search method using attribute selection algorithms in a data mining application. These selected attributes were subjected to a classification test with Naive Bayes and Random Forest from machine learning algorithms. The feature selection algorithms are compared according to the number of attributes selected and the classification accuracy rates achievable with them. In this study, which is aimed at reducing the dataset volume, the best classification result comes up with 3 attributes selected by the Chi2 algorithm. The best classification rate was 73% with the Random Forest classification algorithm. |
| Tasks | Feature Selection |
| Published | 2018-07-12 |
| URL | http://arxiv.org/abs/1807.04800v1 |
| http://arxiv.org/pdf/1807.04800v1.pdf | |
| PWC | https://paperswithcode.com/paper/feature-selection-for-gender-classification |
| Repo | |
| Framework | |
Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences
| Title | Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences |
| Authors | Yi Tay, Luu Anh Tuan, Siu Cheung Hui |
| Abstract | Learning a matching function between two text sequences is a long standing problem in NLP research. This task enables many potential applications such as question answering and paraphrase identification. This paper proposes Co-Stack Residual Affinity Networks (CSRAN), a new and universal neural architecture for this problem. CSRAN is a deep architecture, involving stacked (multi-layered) recurrent encoders. Stacked/Deep architectures are traditionally difficult to train, due to the inherent weaknesses such as difficulty with feature propagation and vanishing gradients. CSRAN incorporates two novel components to take advantage of the stacked architecture. Firstly, it introduces a new bidirectional alignment mechanism that learns affinity weights by fusing sequence pairs across stacked hierarchies. Secondly, it leverages a multi-level attention refinement component between stacked recurrent layers. The key intuition is that, by leveraging information across all network hierarchies, we can not only improve gradient flow but also improve overall performance. We conduct extensive experiments on six well-studied text sequence matching datasets, achieving state-of-the-art performance on all. |
| Tasks | Paraphrase Identification, Question Answering |
| Published | 2018-10-06 |
| URL | http://arxiv.org/abs/1810.02938v1 |
| http://arxiv.org/pdf/1810.02938v1.pdf | |
| PWC | https://paperswithcode.com/paper/co-stack-residual-affinity-networks-with |
| Repo | |
| Framework | |
Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions
| Title | Convolutional Spatial Attention Model for Reading Comprehension with Multiple-Choice Questions |
| Authors | Zhipeng Chen, Yiming Cui, Wentao Ma, Shijin Wang, Guoping Hu |
| Abstract | Machine Reading Comprehension (MRC) with multiple-choice questions requires the machine to read given passage and select the correct answer among several candidates. In this paper, we propose a novel approach called Convolutional Spatial Attention (CSA) model which can better handle the MRC with multiple-choice questions. The proposed model could fully extract the mutual information among the passage, question, and the candidates, to form the enriched representations. Furthermore, to merge various attention results, we propose to use convolutional operation to dynamically summarize the attention values within the different size of regions. Experimental results show that the proposed model could give substantial improvements over various state-of-the-art systems on both RACE and SemEval-2018 Task11 datasets. |
| Tasks | Machine Reading Comprehension, Reading Comprehension |
| Published | 2018-11-21 |
| URL | http://arxiv.org/abs/1811.08610v1 |
| http://arxiv.org/pdf/1811.08610v1.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-spatial-attention-model-for |
| Repo | |
| Framework | |
Improved Sentence Modeling using Suffix Bidirectional LSTM
| Title | Improved Sentence Modeling using Suffix Bidirectional LSTM |
| Authors | Siddhartha Brahma |
| Abstract | Recurrent neural networks have become ubiquitous in computing representations of sequential data, especially textual data in natural language processing. In particular, Bidirectional LSTMs are at the heart of several neural models achieving state-of-the-art performance in a wide variety of tasks in NLP. However, BiLSTMs are known to suffer from sequential bias - the contextual representation of a token is heavily influenced by tokens close to it in a sentence. We propose a general and effective improvement to the BiLSTM model which encodes each suffix and prefix of a sequence of tokens in both forward and reverse directions. We call our model Suffix Bidirectional LSTM or SuBiLSTM. This introduces an alternate bias that favors long range dependencies. We apply SuBiLSTMs to several tasks that require sentence modeling. We demonstrate that using SuBiLSTM instead of a BiLSTM in existing models leads to improvements in performance in learning general sentence representations, text classification, textual entailment and paraphrase detection. Using SuBiLSTM we achieve new state-of-the-art results for fine-grained sentiment classification and question classification. |
| Tasks | Natural Language Inference, Sentiment Analysis, Text Classification |
| Published | 2018-05-18 |
| URL | http://arxiv.org/abs/1805.07340v2 |
| http://arxiv.org/pdf/1805.07340v2.pdf | |
| PWC | https://paperswithcode.com/paper/improved-sentence-modeling-using-suffix |
| Repo | |
| Framework | |
3D Surface Reconstruction by Pointillism
| Title | 3D Surface Reconstruction by Pointillism |
| Authors | Olivia Wiles, Andrew Zisserman |
| Abstract | The objective of this work is to infer the 3D shape of an object from a single image. We use sculptures as our training and test bed, as these have great variety in shape and appearance. To achieve this we build on the success of multiple view geometry (MVG) which is able to accurately provide correspondences between images of 3D objects under varying viewpoint and illumination conditions, and make the following contributions: first, we introduce a new loss function that can harness image-to-image correspondences to provide a supervisory signal to train a deep network to infer a depth map. The network is trained end-to-end by differentiating through the camera. Second, we develop a processing pipeline to automatically generate a large scale multi-view set of correspondences for training the network. Finally, we demonstrate that we can indeed obtain a depth map of a novel object from a single image for a variety of sculptures with varying shape/texture, and that the network generalises at test time to new domains (e.g. synthetic images). |
| Tasks | |
| Published | 2018-09-06 |
| URL | http://arxiv.org/abs/1809.02002v2 |
| http://arxiv.org/pdf/1809.02002v2.pdf | |
| PWC | https://paperswithcode.com/paper/3d-surface-reconstruction-by-pointillism |
| Repo | |
| Framework | |
Landmark Weighting for 3DMM Shape Fitting
| Title | Landmark Weighting for 3DMM Shape Fitting |
| Authors | Yu Yanga, Xiao-Jun Wu, Josef Kittler |
| Abstract | Human face is a 3D object with shape and surface texture. 3D Morphable Model (3DMM) is a powerful tool for reconstructing the 3D face from a single 2D face image. In the shape fitting process, 3DMM estimates the correspondence between 2D and 3D landmarks. Most traditional 3DMM fitting methods fail to reconstruct an accurate model because face shape fitting is a difficult non-linear optimization problem. In this paper we show that landmark weighting is instrumental to improve the accuracy of shape reconstruction and propose a novel 3D Morphable Model Fitting method. Different from previous works that treat all landmarks equally, we take into consideration the estimated errors for each pair of 2D and 3D corresponding landmarks. The landmark points are weighted in the optimization cost function based on these errors. Obviously, these landmarks have different semantics because they locate on different facial components. In the context of the solution of fitting is approximated, there are deviations in landmarks matching. However, these landmarks with different semantics have different effects on reconstructing 3D faces. Thus, it is necessary to consider each landmark individually. To our knowledge, we are the first to analyze each feature point for 3D face reconstruction by 3DMM. The weight is adaptive with the estimation residuals of landmarks. Experimental results show that the proposed method significantly reduces the reconstruction error and improves the authenticity of the 3D model expression. |
| Tasks | 3D Face Reconstruction, Face Reconstruction |
| Published | 2018-08-16 |
| URL | http://arxiv.org/abs/1808.05399v1 |
| http://arxiv.org/pdf/1808.05399v1.pdf | |
| PWC | https://paperswithcode.com/paper/landmark-weighting-for-3dmm-shape-fitting |
| Repo | |
| Framework | |
Can Machines Design? An Artificial General Intelligence Approach
| Title | Can Machines Design? An Artificial General Intelligence Approach |
| Authors | Andreas Makoto Hein, Hélène Condat |
| Abstract | Can machines design? Can they come up with creative solutions to problems and build tools and artifacts across a wide range of domains? Recent advances in the field of computational creativity and formal Artificial General Intelligence (AGI) provide frameworks for machines with the general ability to design. In this paper we propose to integrate a formal computational creativity framework into the G"odel machine framework. We call the resulting framework design G"odel machine. Such a machine could solve a variety of design problems by generating novel concepts. In addition, it could change the way these concepts are generated by modifying itself. The design G"odel machine is able to improve its initial design program, once it has proven that a modification would increase its return on the utility function. Finally, we sketch out a specific version of the design G"odel machine which specifically addresses the design of complex software and hardware systems. Future work aims at the development of a more formal version of the design G"odel machine and a proof of concept implementation. |
| Tasks | |
| Published | 2018-06-06 |
| URL | http://arxiv.org/abs/1806.02091v4 |
| http://arxiv.org/pdf/1806.02091v4.pdf | |
| PWC | https://paperswithcode.com/paper/can-machines-design-an-artificial-general |
| Repo | |
| Framework | |
EigenNetworks
| Title | EigenNetworks |
| Authors | Jonathan Mei, José M. F. Moura |
| Abstract | Many applications donot have the benefit of the laws of physics to derive succinct descriptive models for observed data. In alternative, interdependencies among $N$ time series ${ x_{nk}, k>0 }_{n=1}^{N}$ are nowadays often captured by a graph or network $G$ that in practice may be very large. The network itself may change over time as well (i.e., as $G_k$). Tracking brute force the changes of time varying networks presents major challenges, including the associated computational problems. Further, a large set of networks may not lend itself to useful analysis. This paper approximates the time varying networks $\left{G_k\right}$ as weighted linear combinations of eigennetworks. The eigennetworks are fixed building blocks that are estimated by first learning the time series of graphs $G_k$ from the data ${ x_{nk}, k>0 }_{n=1}^{N}$, followed by a Principal Network Analysis procedure. The weights of the eigennetwork representation are eigenfeatures and the time varying networks $\left{G_k\right}$ describe a trajectory in eigennetwork space. These eigentrajectories should be smooth since the networks $G_k$ vary at a much slower rate than the data $x_{nk}$, except when structural network shifts occur reflecting potentially an abrupt change in the underlying application and sources of the data. Algorithms for learning the time series of graphs $\left{G_k\right}$, deriving the eigennetworks, eigenfeatures and eigentrajectories, and detecting changepoints are presented. Experiments on simulated data and with two real time series data (a voting record of the US senate and genetic expression data for the \textit{Drosophila Melanogaster} as it goes through its life cycle) demonstrate the performance of the learning and provide interesting interpretations of the eigennetworks. |
| Tasks | Time Series |
| Published | 2018-06-05 |
| URL | http://arxiv.org/abs/1806.01455v2 |
| http://arxiv.org/pdf/1806.01455v2.pdf | |
| PWC | https://paperswithcode.com/paper/eigennetworks |
| Repo | |
| Framework | |
Exploiting Tournament Selection for Efficient Parallel Genetic Programming
| Title | Exploiting Tournament Selection for Efficient Parallel Genetic Programming |
| Authors | Darren M. Chitty |
| Abstract | Genetic Programming (GP) is a computationally intensive technique which is naturally parallel in nature. Consequently, many attempts have been made to improve its run-time from exploiting highly parallel hardware such as GPUs. However, a second methodology of improving the speed of GP is through efficiency techniques such as subtree caching. However achieving parallel performance and efficiency is a difficult task. This paper will demonstrate an efficiency saving for GP compatible with the harnessing of parallel CPU hardware by exploiting tournament selection. Significant efficiency savings are demonstrated whilst retaining the capability of a high performance parallel implementation of GP. Indeed, a 74% improvement in the speed of GP is achieved with a peak rate of 96 billion GPop/s for classification type problems. |
| Tasks | |
| Published | 2018-09-19 |
| URL | http://arxiv.org/abs/1809.07406v1 |
| http://arxiv.org/pdf/1809.07406v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploiting-tournament-selection-for-efficient |
| Repo | |
| Framework | |
