Paper Group ANR 760
Preprocessing Methods and Pipelines of Data Mining: An Overview. ArCo: the Italian Cultural Heritage Knowledge Graph. Deep Detector Health Management under Adversarial Campaigns. Hyperparameter Optimisation with Early Termination of Poor Performers. A Targeted Acceleration and Compression Framework for Low bit Neural Networks. Algorithms for certai …
Preprocessing Methods and Pipelines of Data Mining: An Overview
| Title | Preprocessing Methods and Pipelines of Data Mining: An Overview |
| Authors | Canchen Li |
| Abstract | Data mining is about obtaining new knowledge from existing datasets. However, the data in the existing datasets can be scattered, noisy, and even incomplete. Although lots of effort is spent on developing or fine-tuning data mining models to make them more robust to the noise of the input data, their qualities still strongly depend on the quality of it. The article starts with an overview of the data mining pipeline, where the procedures in a data mining task are briefly introduced. Then an overview of the data preprocessing techniques which are categorized as the data cleaning, data transformation and data preprocessing is given. Detailed preprocessing methods, as well as their influenced on the data mining models, are covered in this article. |
| Tasks | |
| Published | 2019-06-20 |
| URL | https://arxiv.org/abs/1906.08510v1 |
| https://arxiv.org/pdf/1906.08510v1.pdf | |
| PWC | https://paperswithcode.com/paper/preprocessing-methods-and-pipelines-of-data |
| Repo | |
| Framework | |
ArCo: the Italian Cultural Heritage Knowledge Graph
| Title | ArCo: the Italian Cultural Heritage Knowledge Graph |
| Authors | Valentina Anita Carriero, Aldo Gangemi, Maria Letizia Mancinelli, Ludovica Marinucci, Andrea Giovanni Nuzzolese, Valentina Presutti, Chiara Veninata |
| Abstract | ArCo is the Italian Cultural Heritage knowledge graph, consisting of a network of seven vocabularies and 169 million triples about 820 thousand cultural entities. It is distributed jointly with a SPARQL endpoint, a software for converting catalogue records to RDF, and a rich suite of documentation material (testing, evaluation, how-to, examples, etc.). ArCo is based on the official General Catalogue of the Italian Ministry of Cultural Heritage and Activities (MiBAC) - and its associated encoding regulations - which collects and validates the catalogue records of (ideally) all Italian Cultural Heritage properties (excluding libraries and archives), contributed by CH administrators from all over Italy. We present its structure, design methods and tools, its growing community, and delineate its importance, quality, and impact. |
| Tasks | |
| Published | 2019-05-07 |
| URL | https://arxiv.org/abs/1905.02840v1 |
| https://arxiv.org/pdf/1905.02840v1.pdf | |
| PWC | https://paperswithcode.com/paper/arco-the-italian-cultural-heritage-knowledge |
| Repo | |
| Framework | |
Deep Detector Health Management under Adversarial Campaigns
| Title | Deep Detector Health Management under Adversarial Campaigns |
| Authors | Javier Echauz, Keith Kenemer, Sarfaraz Hussein, Jay Dhaliwal, Saurabh Shintre, Slawomir Grzonkowski, Andrew Gardner |
| Abstract | Machine learning models are vulnerable to adversarial inputs that induce seemingly unjustifiable errors. As automated classifiers are increasingly used in industrial control systems and machinery, these adversarial errors could grow to be a serious problem. Despite numerous studies over the past few years, the field of adversarial ML is still considered alchemy, with no practical unbroken defenses demonstrated to date, leaving PHM practitioners with few meaningful ways of addressing the problem. We introduce turbidity detection as a practical superset of the adversarial input detection problem, coping with adversarial campaigns rather than statistically invisible one-offs. This perspective is coupled with ROC-theoretic design guidance that prescribes an inexpensive domain adaptation layer at the output of a deep learning model during an attack campaign. The result aims to approximate the Bayes optimal mitigation that ameliorates the detection model’s degraded health. A proactively reactive type of prognostics is achieved via Monte Carlo simulation of various adversarial campaign scenarios, by sampling from the model’s own turbidity distribution to quickly deploy the correct mitigation during a real-world campaign. |
| Tasks | Domain Adaptation |
| Published | 2019-11-19 |
| URL | https://arxiv.org/abs/1911.08090v1 |
| https://arxiv.org/pdf/1911.08090v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-detector-health-management-under |
| Repo | |
| Framework | |
Hyperparameter Optimisation with Early Termination of Poor Performers
| Title | Hyperparameter Optimisation with Early Termination of Poor Performers |
| Authors | Dobromir Marinov, Daniel Karapetyan |
| Abstract | It is typical for a machine learning system to have numerous hyperparameters that affect its learning rate and prediction quality. Finding a good combination of the hyperparameters is, however, a challenging job. This is mainly because evaluation of each combination is extremely expensive computationally; indeed, training a machine learning system on real data with just a single combination of hyperparameters usually takes hours or even days. In this paper, we address this challenge by trying to predict the performance of the machine learning system with a given combination of hyperparameters without completing the expensive learning process. Instead, we terminate the training process at an early stage, collect the model performance data and use it to predict which of the combinations of hyperparameters is most promising. Our preliminary experiments show that such a prediction improves the performance of the commonly used random search approach. |
| Tasks | |
| Published | 2019-07-19 |
| URL | https://arxiv.org/abs/1907.08651v2 |
| https://arxiv.org/pdf/1907.08651v2.pdf | |
| PWC | https://paperswithcode.com/paper/hyperparameter-optimisation-with-early |
| Repo | |
| Framework | |
A Targeted Acceleration and Compression Framework for Low bit Neural Networks
| Title | A Targeted Acceleration and Compression Framework for Low bit Neural Networks |
| Authors | Biao Qian, Yang Wang |
| Abstract | 1 bit deep neural networks (DNNs), of which both the activations and weights are binarized , are attracting more and more attention due to their high computational efficiency and low memory requirement . However, the drawback of large accuracy dropping also restrict s its application. In this paper, we propose a novel Targeted Acceleration and Compression (TAC) framework to improve the performance of 1 bit deep neural networks W e consider that the acceleration and compression effects of binarizing fully connected layer s are not sufficient to compensate for the accuracy loss caused by it In the proposed framework, t he convolutional and fully connected layer are separated and optimized i ndividually . F or the convolutional layer s , both the activations and weights are binarized. For the fully connected layer s, the binarization operation is re placed by network pruning and low bit quantization. The proposed framework is implemented on the CIFAR 10, CIFAR 100 and ImageNet ( ILSVRC 12 ) datasets , and experimental results show that the proposed TAC can significantly improve the accuracy of 1 bit deep neural networks and outperforms the state of the art by more than 6 percentage points . |
| Tasks | Network Pruning, Quantization |
| Published | 2019-07-09 |
| URL | https://arxiv.org/abs/1907.05271v1 |
| https://arxiv.org/pdf/1907.05271v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-targeted-acceleration-and-compression |
| Repo | |
| Framework | |
Algorithms for certain classes of Tamil Spelling correction
| Title | Algorithms for certain classes of Tamil Spelling correction |
| Authors | Muthiah Annamalai, T. Shrinivasan |
| Abstract | Tamil language has an agglutinative, diglossic, alpha-syllabary structure which provides a significant combinatorial explosion of morphological forms all of which are effectively used in Tamil prose, poetry from antiquity to the modern age in an unbroken chain of continuity. However, for the language understanding, spelling correction purposes some of these present challenges as out-of-dictionary words. In this paper the authors propose algorithmic techniques to handle specific problems of conjoined-words (out-of-dictionary) (transliteration)[thendRalkattRu] = [thendRal]+[kattRu] when parts are alone present in word-list in efficient way. Morphological structure of Tamil makes it necessary to depend on synthesis-analysis approach and dictionary lists will never be sufficient to truly capture the language. In this paper we have attempted to make a summary of various known algorithms for specific classes of Tamil spelling errors. We believe this collection of suggestions to improve future spelling checkers. We also note do not cover many important techniques like affix removal and other such techniques of key importance in rule-based spell checkers. |
| Tasks | Spelling Correction, Transliteration |
| Published | 2019-09-22 |
| URL | https://arxiv.org/abs/1909.10063v1 |
| https://arxiv.org/pdf/1909.10063v1.pdf | |
| PWC | https://paperswithcode.com/paper/190910063 |
| Repo | |
| Framework | |
Causal Induction from Visual Observations for Goal Directed Tasks
| Title | Causal Induction from Visual Observations for Goal Directed Tasks |
| Authors | Suraj Nair, Yuke Zhu, Silvio Savarese, Li Fei-Fei |
| Abstract | Causal reasoning has been an indispensable capability for humans and other intelligent animals to interact with the physical world. In this work, we propose to endow an artificial agent with the capability of causal reasoning for completing goal-directed tasks. We develop learning-based approaches to inducing causal knowledge in the form of directed acyclic graphs, which can be used to contextualize a learned goal-conditional policy to perform tasks in novel environments with latent causal structures. We leverage attention mechanisms in our causal induction model and goal-conditional policy, enabling us to incrementally generate the causal graph from the agent’s visual observations and to selectively use the induced graph for determining actions. Our experiments show that our method effectively generalizes towards completing new tasks in novel environments with previously unseen causal structures. |
| Tasks | |
| Published | 2019-10-03 |
| URL | https://arxiv.org/abs/1910.01751v1 |
| https://arxiv.org/pdf/1910.01751v1.pdf | |
| PWC | https://paperswithcode.com/paper/causal-induction-from-visual-observations-for |
| Repo | |
| Framework | |
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
| Title | WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection |
| Authors | Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang |
| Abstract | We study on weakly-supervised object detection (WSOD) which plays a vital role in relieving human involvement from object-level annotations. Predominant works integrate region proposal mechanisms with convolutional neural networks (CNN). Although CNN is proficient in extracting discriminative local features, grand challenges still exist to measure the likelihood of a bounding box containing a complete object (i.e., “objectness”). In this paper, we propose a novel WSOD framework with Objectness Distillation (i.e., WSOD^2) by designing a tailored training mechanism for weakly-supervised object detection. Multiple regression targets are specifically determined by jointly considering bottom-up (BU) and top-down (TD) objectness from low-level measurement and CNN confidences with an adaptive linear combination. As bounding box regression can facilitate a region proposal learning to approach its regression target with high objectness during training, deep objectness representation learned from bottom-up evidences can be gradually distilled into CNN by optimization. We explore different adaptive training curves for BU/TD objectness, and show that the proposed WSOD^2 can achieve state-of-the-art results. |
| Tasks | Object Detection, Weakly Supervised Object Detection |
| Published | 2019-09-11 |
| URL | https://arxiv.org/abs/1909.04972v1 |
| https://arxiv.org/pdf/1909.04972v1.pdf | |
| PWC | https://paperswithcode.com/paper/wsod2-learning-bottom-up-and-top-down-1 |
| Repo | |
| Framework | |
Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems
| Title | Bayesian Generative Models for Knowledge Transfer in MRI Semantic Segmentation Problems |
| Authors | Anna Kuzina, Evgenii Egorov, Evgeny Burnaev |
| Abstract | Automatic segmentation methods based on deep learning have recently demonstrated state-of-the-art performance, outperforming the ordinary methods. Nevertheless, these methods are inapplicable for small datasets, which are very common in medical problems. To this end, we propose a knowledge transfer method between diseases via the Generative Bayesian Prior network. Our approach is compared to a pre-train approach and random initialization and obtains the best results in terms of Dice Similarity Coefficient metric for the small subsets of the Brain Tumor Segmentation 2018 database (BRATS2018). |
| Tasks | Brain Tumor Segmentation, Semantic Segmentation, Transfer Learning |
| Published | 2019-08-15 |
| URL | https://arxiv.org/abs/1908.05480v1 |
| https://arxiv.org/pdf/1908.05480v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-generative-models-for-knowledge |
| Repo | |
| Framework | |
Automatic Spelling Correction with Transformer for CTC-based End-to-End Speech Recognition
| Title | Automatic Spelling Correction with Transformer for CTC-based End-to-End Speech Recognition |
| Authors | Shiliang Zhang, Ming Lei, Zhijie Yan |
| Abstract | Connectionist Temporal Classification (CTC) based end-to-end speech recognition system usually need to incorporate an external language model by using WFST-based decoding in order to achieve promising results. This is more essential to Mandarin speech recognition since it owns a special phenomenon, namely homophone, which causes a lot of substitution errors. The linguistic information introduced by language model will help to distinguish these substitution errors. In this work, we propose a transformer based spelling correction model to automatically correct errors especially the substitution errors made by CTC-based Mandarin speech recognition system. Specifically, we investigate using the recognition results generated by CTC-based systems as input and the ground-truth transcriptions as output to train a transformer with encoder-decoder architecture, which is much similar to machine translation. Results in a 20,000 hours Mandarin speech recognition task show that the proposed spelling correction model can achieve a CER of 3.41%, which results in 22.9% and 53.2% relative improvement compared to the baseline CTC-based systems decoded with and without language model respectively. |
| Tasks | End-To-End Speech Recognition, Language Modelling, Machine Translation, Speech Recognition, Spelling Correction |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1904.10045v1 |
| http://arxiv.org/pdf/1904.10045v1.pdf | |
| PWC | https://paperswithcode.com/paper/190410045 |
| Repo | |
| Framework | |
Adversarial recovery of agent rewards from latent spaces of the limit order book
| Title | Adversarial recovery of agent rewards from latent spaces of the limit order book |
| Authors | Jacobo Roa-Vicens, Yuanbo Wang, Virgile Mison, Yarin Gal, Ricardo Silva |
| Abstract | Inverse reinforcement learning has proved its ability to explain state-action trajectories of expert agents by recovering their underlying reward functions in increasingly challenging environments. Recent advances in adversarial learning have allowed extending inverse RL to applications with non-stationary environment dynamics unknown to the agents, arbitrary structures of reward functions and improved handling of the ambiguities inherent to the ill-posed nature of inverse RL. This is particularly relevant in real time applications on stochastic environments involving risk, like volatile financial markets. Moreover, recent work on simulation of complex environments enable learning algorithms to engage with real market data through simulations of its latent space representations, avoiding a costly exploration of the original environment. In this paper, we explore whether adversarial inverse RL algorithms can be adapted and trained within such latent space simulations from real market data, while maintaining their ability to recover agent rewards robust to variations in the underlying dynamics, and transfer them to new regimes of the original environment. |
| Tasks | |
| Published | 2019-12-09 |
| URL | https://arxiv.org/abs/1912.04242v1 |
| https://arxiv.org/pdf/1912.04242v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-recovery-of-agent-rewards-from |
| Repo | |
| Framework | |
ReNeg and Backseat Driver: Learning from Demonstration with Continuous Human Feedback
| Title | ReNeg and Backseat Driver: Learning from Demonstration with Continuous Human Feedback |
| Authors | Jacob Beck, Zoe Papakipos, Michael Littman |
| Abstract | In autonomous vehicle (AV) control, allowing mistakes can be quite dangerous and costly in the real world. For this reason we investigate methods of training an AV without allowing the agent to explore and instead having a human explorer collect the data. Supervised learning has been explored for AV control, but it encounters the issue of the covariate shift. That is, training data collected from an optimal demonstration consists only of the states induced by the optimal control policy, but at runtime, the trained agent may encounter a vastly different state distribution with little relevant training data. To mitigate this issue, we have our human explorer make sub-optimal decisions. In order to have our agent not replicate these sub-optimal decisions, supervised learning requires that we either erase these actions, or replace these action with the correct action. Erasing is wasteful and replacing is difficult, since it is not easy to know the correct action without driving. We propose an alternate framework that includes continuous scalar feedback for each action, marking which actions we should replicate, which we should avoid, and how sure we are. Our framework learns continuous control from sub-optimal demonstration and evaluative feedback collected before training. We find that a human demonstrator can explore sub-optimal states in a safe manner, while still getting enough gradation to benefit learning. The collection method for data and feedback we call “Backseat Driver.” We call the more general learning framework ReNeg, since it learns a regression from states to actions given negative as well as positive examples. We empirically validate several models in the ReNeg framework, testing on lane-following with limited data. We find that the best solution is a generalization of mean-squared error and outperforms supervised learning on the positive examples alone. |
| Tasks | Continuous Control |
| Published | 2019-01-16 |
| URL | http://arxiv.org/abs/1901.05101v1 |
| http://arxiv.org/pdf/1901.05101v1.pdf | |
| PWC | https://paperswithcode.com/paper/reneg-and-backseat-driver-learning-from |
| Repo | |
| Framework | |
Deep neural network for fringe pattern filtering and normalisation
| Title | Deep neural network for fringe pattern filtering and normalisation |
| Authors | Alan Reyes-Figueroa, Mariano Rivera |
| Abstract | We propose a new framework for processing Fringe Patterns (FP). Our novel approach builds upon the hypothesis that the denoising and normalisation of FPs can be learned by a deep neural network if enough pairs of corrupted and cleaned FPs are provided. Although similar proposals have been reported in the literature, we propose an improvement of a well-known deep neural network architecture, which produces high-quality results in terms of stability and repeatability. We test the performance of our method in various scenarios: FPs corrupted with different degrees of noise, and corrupted with different noise distributions. We compare our methodology versus other state-of-the-art methods. The experimental results (on both synthetic and real data) demonstrate the capabilities and potential of this new paradigm for processing interferograms. We expect our work would motivate more sophisticated developments in this direction. |
| Tasks | Denoising |
| Published | 2019-06-14 |
| URL | https://arxiv.org/abs/1906.06224v1 |
| https://arxiv.org/pdf/1906.06224v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-neural-network-for-fringe-pattern |
| Repo | |
| Framework | |
Optimal Randomness in Swarm-Based Search
| Title | Optimal Randomness in Swarm-Based Search |
| Authors | Jiamin Wei, YangQuan Chen, Yongguang Yu, Yuquan Chen |
| Abstract | L'{e}vy flights is a random walk where the step-lengths have a probability distribution that is heavy-tailed. It has been shown that L'{e}vy flights can maximize the efficiency of resource searching in uncertain environments, and also movements of many foragers and wandering animals have been shown to follow a L'{e}vy distribution. The reason mainly comes from that the L'{e}vy distribution, has an infinite second moment, and hence is more likely to generate an offspring that is farther away from its parent. However, the investigation into the efficiency of other different heavy-tailed probability distributions in swarm-based searches is still insufficient up to now. For swarm-based search algorithms, randomness plays a significant role in both exploration and exploitation, or diversification and intensification. Therefore, it’s necessary to discuss the optimal randomness in swarm-based search algorithms. In this study, CS is taken as a representative method of swarm-based optimization algorithms, and the results can be generalized to other swarm-based search algorithms. In this paper, four different types of commonly used heavy-tailed distributions, including Mittag-Leffler distribution, Pareto distribution, Cauchy distribution, and Weibull distribution, are considered to enhance the searching ability of CS. Then four novel CS algorithms are proposed and experiments are carried out on 20 benchmark functions to compare their searching performances. Finally, the proposed methods are used to system identification to demonstrate the effectiveness. |
| Tasks | |
| Published | 2019-05-07 |
| URL | https://arxiv.org/abs/1905.02776v2 |
| https://arxiv.org/pdf/1905.02776v2.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-randomness-in-swarm-based-search |
| Repo | |
| Framework | |
AVT: Unsupervised Learning of Transformation Equivariant Representations by Autoencoding Variational Transformations
| Title | AVT: Unsupervised Learning of Transformation Equivariant Representations by Autoencoding Variational Transformations |
| Authors | Guo-Jun Qi, Liheng Zhang, Chang Wen Chen, Qi Tian |
| Abstract | The learning of Transformation-Equivariant Representations (TERs), which is introduced by Hinton et al. \cite{hinton2011transforming}, has been considered as a principle to reveal visual structures under various transformations. It contains the celebrated Convolutional Neural Networks (CNNs) as a special case that only equivary to the translations. In contrast, we seek to train TERs for a generic class of transformations and train them in an {\em unsupervised} fashion. To this end, we present a novel principled method by Autoencoding Variational Transformations (AVT), compared with the conventional approach to autoencoding data. Formally, given transformed images, the AVT seeks to train the networks by maximizing the mutual information between the transformations and representations. This ensures the resultant TERs of individual images contain the {\em intrinsic} information about their visual structures that would equivary {\em extricably} under various transformations in a generalized {\em nonlinear} case. Technically, we show that the resultant optimization problem can be efficiently solved by maximizing a variational lower-bound of the mutual information. This variational approach introduces a transformation decoder to approximate the intractable posterior of transformations, resulting in an autoencoding architecture with a pair of the representation encoder and the transformation decoder. Experiments demonstrate the proposed AVT model sets a new record for the performances on unsupervised tasks, greatly closing the performance gap to the supervised models. |
| Tasks | |
| Published | 2019-03-23 |
| URL | https://arxiv.org/abs/1903.10863v3 |
| https://arxiv.org/pdf/1903.10863v3.pdf | |
| PWC | https://paperswithcode.com/paper/avt-unsupervised-learning-of-transformation |
| Repo | |
| Framework | |
