Paper Group ANR 570
Robust Implicit Backpropagation. Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes. On transfer learning using a MAC model variant. Data-driven model for the identification of the rock type at a drilling bit. An Iterative Spanning Forest Framework for Superpixel Segmentation. Graph Convolut …
Robust Implicit Backpropagation
| Title | Robust Implicit Backpropagation |
| Authors | Francois Fagan, Garud Iyengar |
| Abstract | Arguably the biggest challenge in applying neural networks is tuning the hyperparameters, in particular the learning rate. The sensitivity to the learning rate is due to the reliance on backpropagation to train the network. In this paper we present the first application of Implicit Stochastic Gradient Descent (ISGD) to train neural networks, a method known in convex optimization to be unconditionally stable and robust to the learning rate. Our key contribution is a novel layer-wise approximation of ISGD which makes its updates tractable for neural networks. Experiments show that our method is more robust to high learning rates and generally outperforms standard backpropagation on a variety of tasks. |
| Tasks | |
| Published | 2018-08-07 |
| URL | http://arxiv.org/abs/1808.02433v1 |
| http://arxiv.org/pdf/1808.02433v1.pdf | |
| PWC | https://paperswithcode.com/paper/robust-implicit-backpropagation |
| Repo | |
| Framework | |
Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes
| Title | Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes |
| Authors | Johannes Dornheim, Norbert Link |
| Abstract | In industrial applications of adaptive optimal control often multiple contrary objectives have to be considered. The weights (relative importance) of the objectives are often not known during the design of the control and can change with changing production conditions and requirements. In this work a novel model-free multiobjective reinforcement learning approach for adaptive optimal control of manufacturing processes is proposed. The approach enables sample-efficient learning in sequences of control configurations, given by particular objective weights. |
| Tasks | |
| Published | 2018-09-18 |
| URL | http://arxiv.org/abs/1809.06750v2 |
| http://arxiv.org/pdf/1809.06750v2.pdf | |
| PWC | https://paperswithcode.com/paper/multiobjective-reinforcement-learning-for |
| Repo | |
| Framework | |
On transfer learning using a MAC model variant
| Title | On transfer learning using a MAC model variant |
| Authors | Vincent Marois, T. S. Jayram, Vincent Albouy, Tomasz Kornuta, Younes Bouhadjar, Ahmet S. Ozcan |
| Abstract | We introduce a variant of the MAC model (Hudson and Manning, ICLR 2018) with a simplified set of equations that achieves comparable accuracy, while training faster. We evaluate both models on CLEVR and CoGenT, and show that, transfer learning with fine-tuning results in a 15 point increase in accuracy, matching the state of the art. Finally, in contrast, we demonstrate that improper fine-tuning can actually reduce a model’s accuracy as well. |
| Tasks | Transfer Learning |
| Published | 2018-11-15 |
| URL | http://arxiv.org/abs/1811.06529v2 |
| http://arxiv.org/pdf/1811.06529v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-transfer-learning-using-a-mac-model |
| Repo | |
| Framework | |
Data-driven model for the identification of the rock type at a drilling bit
| Title | Data-driven model for the identification of the rock type at a drilling bit |
| Authors | Nikita Klyuchnikov, Alexey Zaytsev, Arseniy Gruzdev, Georgiy Ovchinnikov, Ksenia Antipova, Leyla Ismailova, Ekaterina Muravleva, Evgeny Burnaev, Artyom Semenikhin, Alexey Cherepanov, Vitaliy Koryabkin, Igor Simon, Alexey Tsurgan, Fedor Krasnov, Dmitry Koroteev |
| Abstract | Directional oil well drilling requires high precision of the wellbore positioning inside the productive area. However, due to specifics of engineering design, sensors that explicitly determine the type of the drilled rock are located farther than 15m from the drilling bit. As a result, the target area runaways can be detected only after this distance, which in turn, leads to a loss in well productivity and the risk of the need for an expensive re-boring operation. We present a novel approach for identifying rock type at the drilling bit based on machine learning classification methods and data mining on sensors readings. We compare various machine-learning algorithms, examine extra features coming from mathematical modeling of drilling mechanics, and show that the real-time rock type classification error can be reduced from 13.5 % to 9 %. The approach is applicable for precise directional drilling in relatively thin target intervals of complex shapes and generalizes appropriately to new wells that are different from the ones used for training the machine learning model. |
| Tasks | |
| Published | 2018-06-08 |
| URL | http://arxiv.org/abs/1806.03218v3 |
| http://arxiv.org/pdf/1806.03218v3.pdf | |
| PWC | https://paperswithcode.com/paper/data-driven-model-for-the-identification-of |
| Repo | |
| Framework | |
An Iterative Spanning Forest Framework for Superpixel Segmentation
| Title | An Iterative Spanning Forest Framework for Superpixel Segmentation |
| Authors | John E. Vargas-Muñoz, Ananda S. Chowdhury, Eduardo B. Alexandre, Felipe L. Galvão, Paulo A. Vechiatto Miranda, Alexandre X. Falcão |
| Abstract | Superpixel segmentation has become an important research problem in image processing. In this paper, we propose an Iterative Spanning Forest (ISF) framework, based on sequences of Image Foresting Transforms, where one can choose i) a seed sampling strategy, ii) a connectivity function, iii) an adjacency relation, and iv) a seed pixel recomputation procedure to generate improved sets of connected superpixels (supervoxels in 3D) per iteration. The superpixels in ISF structurally correspond to spanning trees rooted at those seeds. We present five ISF methods to illustrate different choices of its components. These methods are compared with approaches from the state-of-the-art in effectiveness and efficiency. The experiments involve 2D and 3D datasets with distinct characteristics, and a high level application, named sky image segmentation. The theoretical properties of ISF are demonstrated in the supplementary material and the results show that some of its methods are competitive with or superior to the best baselines in effectiveness and efficiency. |
| Tasks | Semantic Segmentation |
| Published | 2018-01-30 |
| URL | http://arxiv.org/abs/1801.10041v1 |
| http://arxiv.org/pdf/1801.10041v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-iterative-spanning-forest-framework-for |
| Repo | |
| Framework | |
Graph Convolutional Neural Networks via Motif-based Attention
| Title | Graph Convolutional Neural Networks via Motif-based Attention |
| Authors | Hao Peng, Jianxin Li, Qiran Gong, Senzhang Wang, Yuanxing Ning, Philip S. Yu |
| Abstract | Many real-world problems can be represented as graph-based learning problems. In this paper, we propose a novel framework for learning spatial and attentional convolution neural networks on arbitrary graphs. Different from previous convolutional neural networks on graphs, we first design a motif-matching guided subgraph normalization method to capture neighborhood information. Then we implement subgraph-level self-attentional layers to learn different importances from different subgraphs to solve graph classification problems. Analogous to image-based attentional convolution networks that operate on locally connected and weighted regions of the input, we also extend graph normalization from one-dimensional node sequence to two-dimensional node grid by leveraging motif-matching, and design self-attentional layers without requiring any kinds of cost depending on prior knowledge of the graph structure. Our results on both bioinformatics and social network datasets show that we can significantly improve graph classification benchmarks over traditional graph kernel and existing deep models. |
| Tasks | Graph Classification |
| Published | 2018-11-11 |
| URL | http://arxiv.org/abs/1811.08270v2 |
| http://arxiv.org/pdf/1811.08270v2.pdf | |
| PWC | https://paperswithcode.com/paper/graph-convolutional-neural-networks-via-motif |
| Repo | |
| Framework | |
A Retrieve-and-Edit Framework for Predicting Structured Outputs
| Title | A Retrieve-and-Edit Framework for Predicting Structured Outputs |
| Authors | Tatsunori B. Hashimoto, Kelvin Guu, Yonatan Oren, Percy Liang |
| Abstract | For the task of generating complex outputs such as source code, editing existing outputs can be easier than generating complex outputs from scratch. With this motivation, we propose an approach that first retrieves a training example based on the input (e.g., natural language description) and then edits it to the desired output (e.g., code). Our contribution is a computationally efficient method for learning a retrieval model that embeds the input in a task-dependent way without relying on a hand-crafted metric or incurring the expense of jointly training the retriever with the editor. Our retrieve-and-edit framework can be applied on top of any base model. We show that on a new autocomplete task for GitHub Python code and the Hearthstone cards benchmark, retrieve-and-edit significantly boosts the performance of a vanilla sequence-to-sequence model on both tasks. |
| Tasks | |
| Published | 2018-12-04 |
| URL | http://arxiv.org/abs/1812.01194v1 |
| http://arxiv.org/pdf/1812.01194v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-retrieve-and-edit-framework-for-predicting |
| Repo | |
| Framework | |
Deep Representation for Patient Visits from Electronic Health Records
| Title | Deep Representation for Patient Visits from Electronic Health Records |
| Authors | Jean-Baptiste Escudié, Alaa Saade, Alice Coucke, Marc Lelarge |
| Abstract | We show how to learn low-dimensional representations (embeddings) of patient visits from the corresponding electronic health record (EHR) where International Classification of Diseases (ICD) diagnosis codes are removed. We expect that these embeddings will be useful for the construction of predictive statistical models anticipated to drive personalized medicine and improve healthcare quality. These embeddings are learned using a deep neural network trained to predict ICD diagnosis categories. We show that our embeddings capture relevant clinical informations and can be used directly as input to standard machine learning algorithms like multi-output classifiers for ICD code prediction. We also show that important medical informations correspond to particular directions in our embedding space. |
| Tasks | |
| Published | 2018-03-26 |
| URL | http://arxiv.org/abs/1803.09533v1 |
| http://arxiv.org/pdf/1803.09533v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-representation-for-patient-visits-from |
| Repo | |
| Framework | |
A New Finitely Controllable Class of Tuple Generating Dependencies: The Triangularly-Guarded Class
| Title | A New Finitely Controllable Class of Tuple Generating Dependencies: The Triangularly-Guarded Class |
| Authors | Vernon Asuncion, Yan Zhang |
| Abstract | In this paper we introduce a new class of tuple-generating dependencies (TGDs) called triangularly-guarded (TG) TGDs. We show that conjunctive query answering under this new class of TGDs is decidable since this new class of TGDs also satisfies the finite controllability (FC) property. We further show that this new class strictly contains some other decidable classes such as weak-acyclic, guarded, sticky and shy. In this sense, the class TG provides a unified representation of all these aforementioned classes of TGDs. |
| Tasks | |
| Published | 2018-05-21 |
| URL | http://arxiv.org/abs/1805.09157v1 |
| http://arxiv.org/pdf/1805.09157v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-new-finitely-controllable-class-of-tuple |
| Repo | |
| Framework | |
Time Series Deinterleaving of DNS Traffic
| Title | Time Series Deinterleaving of DNS Traffic |
| Authors | Amir Asiaee, Hardik Goel, Shalini Ghosh, Vinod Yegneswaran, Arindam Banerjee |
| Abstract | Stream deinterleaving is an important problem with various applications in the cybersecurity domain. In this paper, we consider the specific problem of deinterleaving DNS data streams using machine-learning techniques, with the objective of automating the extraction of malware domain sequences. We first develop a generative model for user request generation and DNS stream interleaving. Based on these we evaluate various inference strategies for deinterleaving including augmented HMMs and LSTMs on synthetic datasets. Our results demonstrate that state-of-the-art LSTMs outperform more traditional augmented HMMs in this application domain. |
| Tasks | Time Series |
| Published | 2018-07-16 |
| URL | http://arxiv.org/abs/1807.05650v1 |
| http://arxiv.org/pdf/1807.05650v1.pdf | |
| PWC | https://paperswithcode.com/paper/time-series-deinterleaving-of-dns-traffic |
| Repo | |
| Framework | |
Matrix Completion under Low-Rank Missing Mechanism
| Title | Matrix Completion under Low-Rank Missing Mechanism |
| Authors | Xiaojun Mao, Raymond K. W. Wong, Song Xi Chen |
| Abstract | Matrix completion is a modern missing data problem where both the missing structure and the underlying parameter are high dimensional. Although missing structure is a key component to any missing data problems, existing matrix completion methods often assume a simple uniform missing mechanism. In this work, we study matrix completion from corrupted data under a novel low-rank missing mechanism. The probability matrix of observation is estimated via a high dimensional low-rank matrix estimation procedure, and further used to complete the target matrix via inverse probabilities weighting. Due to both high dimensional and extreme (i.e., very small) nature of the true probability matrix, the effect of inverse probability weighting requires careful study. We derive optimal asymptotic convergence rates of the proposed estimators for both the observation probabilities and the target matrix. |
| Tasks | Matrix Completion |
| Published | 2018-12-19 |
| URL | https://arxiv.org/abs/1812.07813v2 |
| https://arxiv.org/pdf/1812.07813v2.pdf | |
| PWC | https://paperswithcode.com/paper/matrix-completion-under-low-rank-missing |
| Repo | |
| Framework | |
MITOS-RCNN: A Novel Approach to Mitotic Figure Detection in Breast Cancer Histopathology Images using Region Based Convolutional Neural Networks
| Title | MITOS-RCNN: A Novel Approach to Mitotic Figure Detection in Breast Cancer Histopathology Images using Region Based Convolutional Neural Networks |
| Authors | Siddhant Rao |
| Abstract | Studies estimate that there will be 266,120 new cases of invasive breast cancer and 40,920 breast cancer induced deaths in the year of 2018 alone. Despite the pervasiveness of this affliction, the current process to obtain an accurate breast cancer prognosis is tedious and time consuming, requiring a trained pathologist to manually examine histopathological images in order to identify the features that characterize various cancer severity levels. We propose MITOS-RCNN: a novel region based convolutional neural network (RCNN) geared for small object detection to accurately grade one of the three factors that characterize tumor belligerence described by the Nottingham Grading System: mitotic count. Other computational approaches to mitotic figure counting and detection do not demonstrate ample recall or precision to be clinically viable. Our models outperformed all previous participants in the ICPR 2012 challenge, the AMIDA 2013 challenge and the MITOS-ATYPIA-14 challenge along with recently published works. Our model achieved an F-measure score of 0.955, a 6.11% improvement in accuracy from the most accurate of the previously proposed models. |
| Tasks | Object Detection, Small Object Detection |
| Published | 2018-07-04 |
| URL | http://arxiv.org/abs/1807.01788v1 |
| http://arxiv.org/pdf/1807.01788v1.pdf | |
| PWC | https://paperswithcode.com/paper/mitos-rcnn-a-novel-approach-to-mitotic-figure |
| Repo | |
| Framework | |
BiographyNet: Extracting Relations Between People and Events
| Title | BiographyNet: Extracting Relations Between People and Events |
| Authors | Antske Fokkens, Serge ter Braake, Niels Ockeloen, Piek Vossen, Susan Legêne, Guus Schreiber, Victor de Boer |
| Abstract | This paper describes BiographyNet, a digital humanities project (2012-2016) that brings together researchers from history, computational linguistics and computer science. The project uses data from the Biography Portal of the Netherlands (BPN), which contains approximately 125,000 biographies from a variety of Dutch biographical dictionaries from the eighteenth century until now, describing around 76,000 individuals. BiographyNet’s aim is to strengthen the value of the portal and comparable biographical datasets for historical research, by improving the search options and the presentation of its outcome, with a historically justified NLP pipeline that works through a user evaluated demonstrator. The project’s main target group are professional historians. The project therefore worked with two key concepts: “provenance” -understood as a term allowing for both historical source criticism and for references to data-management and programming interventions in digitized sources; and “perspective” interpreted as inherent uncertainty concerning the interpretation of historical results. |
| Tasks | |
| Published | 2018-01-22 |
| URL | http://arxiv.org/abs/1801.07073v2 |
| http://arxiv.org/pdf/1801.07073v2.pdf | |
| PWC | https://paperswithcode.com/paper/biographynet-extracting-relations-between |
| Repo | |
| Framework | |
Adapted and Oversegmenting Graphs: Application to Geometric Deep Learning
| Title | Adapted and Oversegmenting Graphs: Application to Geometric Deep Learning |
| Authors | Alberto Gomez, Veronika A. Zimmer, Bishesh Khanal, Nicolas Toussaint, Julia A. Schnabel |
| Abstract | We propose a novel iterative method to adapt a a graph to d-dimensional image data. The method drives the nodes of the graph towards image features. The adaptation process naturally lends itself to a measure of feature saliency which can then be used to retain meaningful nodes and edges in the graph. From the adapted graph, we also propose the computation of a dual graph, which inherits the saliency measure from the adapted graph, and whose edges run along image features, hence producing an oversegmenting graph. The proposed method is computationally efficient and fully parallelisable. We propose two distance measures to find image saliency along graph edges, and evaluate the performance on synthetic images and on natural images from publicly available databases. In both cases, the most salient nodes of the graph achieve average boundary recall over 90%. We also apply our method to image classification on the MNIST hand-written digit dataset, using a recently proposed Deep Geometric Learning architecture, and achieving state-of-the-art classification accuracy, for a graph-based method, of 97.86%. |
| Tasks | Image Classification |
| Published | 2018-06-01 |
| URL | https://arxiv.org/abs/1806.00411v2 |
| https://arxiv.org/pdf/1806.00411v2.pdf | |
| PWC | https://paperswithcode.com/paper/oversegmenting-graphs |
| Repo | |
| Framework | |
Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning
| Title | Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning |
| Authors | Jongchan Park, Joon-Young Lee, Donggeun Yoo, In So Kweon |
| Abstract | Learning-based color enhancement approaches typically learn to map from input images to retouched images. Most of existing methods require expensive pairs of input-retouched images or produce results in a non-interpretable way. In this paper, we present a deep reinforcement learning (DRL) based method for color enhancement to explicitly model the step-wise nature of human retouching process. We cast a color enhancement process as a Markov Decision Process where actions are defined as global color adjustment operations. Then we train our agent to learn the optimal global enhancement sequence of the actions. In addition, we present a ‘distort-and-recover’ training scheme which only requires high-quality reference images for training instead of input and retouched image pairs. Given high-quality reference images, we distort the images’ color distribution and form distorted-reference image pairs for training. Through extensive experiments, we show that our method produces decent enhancement results and our DRL approach is more suitable for the ‘distort-and-recover’ training scheme than previous supervised approaches. Supplementary material and code are available at https://sites.google.com/view/distort-and-recover/ |
| Tasks | |
| Published | 2018-04-12 |
| URL | http://arxiv.org/abs/1804.04450v2 |
| http://arxiv.org/pdf/1804.04450v2.pdf | |
| PWC | https://paperswithcode.com/paper/distort-and-recover-color-enhancement-using |
| Repo | |
| Framework | |
