Paper Group ANR 761
One-Two-One Networks for Compression Artifacts Reduction in Remote Sensing. Verification of Recurrent Neural Networks Through Rule Extraction. An iterative method for classification of binary data. The 30-Year Cycle In The AI Debate. The Impact of Automatic Pre-annotation in Clinical Note Data Element Extraction - the CLEAN Tool. Local Explanation …
One-Two-One Networks for Compression Artifacts Reduction in Remote Sensing
| Title | One-Two-One Networks for Compression Artifacts Reduction in Remote Sensing |
| Authors | Baochang Zhang, Jiaxin Gu, Chen Chen, Jungong Han, Xiangbo Su, Xianbin Cao, Jianzhuang Liu |
| Abstract | Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing. Most recent deep learning based methods have demonstrated superior performance over the previous hand-crafted methods. In this paper, we propose an end-to-end one-two-one (OTO) network, to combine different deep models, i.e., summation and difference models, to solve the CAR problem. Particularly, the difference model motivated by the Laplacian pyramid is designed to obtain the high frequency information, while the summation model aggregates the low frequency information. We provide an in-depth investigation into our OTO architecture based on the Taylor expansion, which shows that these two kinds of information can be fused in a nonlinear scheme to gain more capacity of handling complicated image compression artifacts, especially the blocking effect in compression. Extensive experiments are conducted to demonstrate the superior performance of the OTO networks, as compared to the state-of-the-arts on remote sensing datasets and other benchmark datasets. |
| Tasks | Image Compression |
| Published | 2018-04-01 |
| URL | http://arxiv.org/abs/1804.00256v1 |
| http://arxiv.org/pdf/1804.00256v1.pdf | |
| PWC | https://paperswithcode.com/paper/one-two-one-networks-for-compression |
| Repo | |
| Framework | |
Verification of Recurrent Neural Networks Through Rule Extraction
| Title | Verification of Recurrent Neural Networks Through Rule Extraction |
| Authors | Qinglong Wang, Kaixuan Zhang, Xue Liu, C. Lee Giles |
| Abstract | The verification problem for neural networks is verifying whether a neural network will suffer from adversarial samples, or approximating the maximal allowed scale of adversarial perturbation that can be endured. While most prior work contributes to verifying feed-forward networks, little has been explored for verifying recurrent networks. This is due to the existence of a more rigorous constraint on the perturbation space for sequential data, and the lack of a proper metric for measuring the perturbation. In this work, we address these challenges by proposing a metric which measures the distance between strings, and use deterministic finite automata (DFA) to represent a rigorous oracle which examines if the generated adversarial samples violate certain constraints on a perturbation. More specifically, we empirically show that certain recurrent networks allow relatively stable DFA extraction. As such, DFAs extracted from these recurrent networks can serve as a surrogate oracle for when the ground truth DFA is unknown. We apply our verification mechanism to several widely used recurrent networks on a set of the Tomita grammars. The results demonstrate that only a few models remain robust against adversarial samples. In addition, we show that for grammars with different levels of complexity, there is also a difference in the difficulty of robust learning of these grammars. |
| Tasks | |
| Published | 2018-11-14 |
| URL | http://arxiv.org/abs/1811.06029v1 |
| http://arxiv.org/pdf/1811.06029v1.pdf | |
| PWC | https://paperswithcode.com/paper/verification-of-recurrent-neural-networks |
| Repo | |
| Framework | |
An iterative method for classification of binary data
| Title | An iterative method for classification of binary data |
| Authors | Denali Molitor, Deanna Needell |
| Abstract | In today’s data driven world, storing, processing, and gleaning insights from large-scale data are major challenges. Data compression is often required in order to store large amounts of high-dimensional data, and thus, efficient inference methods for analyzing compressed data are necessary. Building on a recently designed simple framework for classification using binary data, we demonstrate that one can improve classification accuracy of this approach through iterative applications whose output serves as input to the next application. As a side consequence, we show that the original framework can be used as a data preprocessing step to improve the performance of other methods, such as support vector machines. For several simple settings, we showcase the ability to obtain theoretical guarantees for the accuracy of the iterative classification method. The simplicity of the underlying classification framework makes it amenable to theoretical analysis and studying this approach will hopefully serve as a step toward developing theory for more sophisticated deep learning technologies. |
| Tasks | |
| Published | 2018-09-09 |
| URL | http://arxiv.org/abs/1809.03041v1 |
| http://arxiv.org/pdf/1809.03041v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-iterative-method-for-classification-of |
| Repo | |
| Framework | |
The 30-Year Cycle In The AI Debate
| Title | The 30-Year Cycle In The AI Debate |
| Authors | Jean-Marie Chauvet |
| Abstract | In the last couple of years, the rise of Artificial Intelligence and the successes of academic breakthroughs in the field have been inescapable. Vast sums of money have been thrown at AI start-ups. Many existing tech companies – including the giants like Google, Amazon, Facebook, and Microsoft – have opened new research labs. The rapid changes in these everyday work and entertainment tools have fueled a rising interest in the underlying technology itself; journalists write about AI tirelessly, and companies – of tech nature or not – brand themselves with AI, Machine Learning or Deep Learning whenever they get a chance. Confronting squarely this media coverage, several analysts are starting to voice concerns about over-interpretation of AI’s blazing successes and the sometimes poor public reporting on the topic. This paper reviews briefly the track-record in AI and Machine Learning and finds this pattern of early dramatic successes, followed by philosophical critique and unexpected difficulties, if not downright stagnation, returning almost to the clock in 30-year cycles since 1958. |
| Tasks | |
| Published | 2018-10-08 |
| URL | http://arxiv.org/abs/1810.04053v1 |
| http://arxiv.org/pdf/1810.04053v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-30-year-cycle-in-the-ai-debate |
| Repo | |
| Framework | |
The Impact of Automatic Pre-annotation in Clinical Note Data Element Extraction - the CLEAN Tool
| Title | The Impact of Automatic Pre-annotation in Clinical Note Data Element Extraction - the CLEAN Tool |
| Authors | Tsung-Ting Kuo, Jina Huh, Jihoon Kim, Robert El-Kareh, Siddharth Singh, Stephanie Feudjio Feupe, Vincent Kuri, Gordon Lin, Michele E. Day, Lucila Ohno-Machado, Chun-Nan Hsu |
| Abstract | Objective. Annotation is expensive but essential for clinical note review and clinical natural language processing (cNLP). However, the extent to which computer-generated pre-annotation is beneficial to human annotation is still an open question. Our study introduces CLEAN (CLinical note rEview and ANnotation), a pre-annotation-based cNLP annotation system to improve clinical note annotation of data elements, and comprehensively compares CLEAN with the widely-used annotation system Brat Rapid Annotation Tool (BRAT). Materials and Methods. CLEAN includes an ensemble pipeline (CLEAN-EP) with a newly developed annotation tool (CLEAN-AT). A domain expert and a novice user/annotator participated in a comparative usability test by tagging 87 data elements related to Congestive Heart Failure (CHF) and Kawasaki Disease (KD) cohorts in 84 public notes. Results. CLEAN achieved higher note-level F1-score (0.896) over BRAT (0.820), with significant difference in correctness (P-value < 0.001), and the mostly related factor being system/software (P-value < 0.001). No significant difference (P-value 0.188) in annotation time was observed between CLEAN (7.262 minutes/note) and BRAT (8.286 minutes/note). The difference was mostly associated with note length (P-value < 0.001) and system/software (P-value 0.013). The expert reported CLEAN to be useful/satisfactory, while the novice reported slight improvements. Discussion. CLEAN improves the correctness of annotation and increases usefulness/satisfaction with the same level of efficiency. Limitations include untested impact of pre-annotation correctness rate, small sample size, small user size, and restrictedly validated gold standard. Conclusion. CLEAN with pre-annotation can be beneficial for an expert to deal with complex annotation tasks involving numerous and diverse target data elements. |
| Tasks | |
| Published | 2018-08-11 |
| URL | http://arxiv.org/abs/1808.03806v1 |
| http://arxiv.org/pdf/1808.03806v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-impact-of-automatic-pre-annotation-in |
| Repo | |
| Framework | |
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
| Title | Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values |
| Authors | Julius Adebayo, Justin Gilmer, Ian Goodfellow, Been Kim |
| Abstract | Explaining the output of a complicated machine learning model like a deep neural network (DNN) is a central challenge in machine learning. Several proposed local explanation methods address this issue by identifying what dimensions of a single input are most responsible for a DNN’s output. The goal of this work is to assess the sensitivity of local explanations to DNN parameter values. Somewhat surprisingly, we find that DNNs with randomly-initialized weights produce explanations that are both visually and quantitatively similar to those produced by DNNs with learned weights. Our conjecture is that this phenomenon occurs because these explanations are dominated by the lower level features of a DNN, and that a DNN’s architecture provides a strong prior which significantly affects the representations learned at these lower layers. NOTE: This work is now subsumed by our recent manuscript, Sanity Checks for Saliency Maps (to appear NIPS 2018), where we expand on findings and address concerns raised in Sundararajan et. al. (2018). |
| Tasks | |
| Published | 2018-10-08 |
| URL | http://arxiv.org/abs/1810.03307v1 |
| http://arxiv.org/pdf/1810.03307v1.pdf | |
| PWC | https://paperswithcode.com/paper/local-explanation-methods-for-deep-neural |
| Repo | |
| Framework | |
Continuous Relaxation of MAP Inference: A Nonconvex Perspective
| Title | Continuous Relaxation of MAP Inference: A Nonconvex Perspective |
| Authors | D. Khuê Lê-Huu, Nikos Paragios |
| Abstract | In this paper, we study a nonconvex continuous relaxation of MAP inference in discrete Markov random fields (MRFs). We show that for arbitrary MRFs, this relaxation is tight, and a discrete stationary point of it can be easily reached by a simple block coordinate descent algorithm. In addition, we study the resolution of this relaxation using popular gradient methods, and further propose a more effective solution using a multilinear decomposition framework based on the alternating direction method of multipliers (ADMM). Experiments on many real-world problems demonstrate that the proposed ADMM significantly outperforms other nonconvex relaxation based methods, and compares favorably with state of the art MRF optimization algorithms in different settings. |
| Tasks | |
| Published | 2018-02-21 |
| URL | http://arxiv.org/abs/1802.07796v2 |
| http://arxiv.org/pdf/1802.07796v2.pdf | |
| PWC | https://paperswithcode.com/paper/continuous-relaxation-of-map-inference-a |
| Repo | |
| Framework | |
On Hyperparameter Search in Cluster Ensembles
| Title | On Hyperparameter Search in Cluster Ensembles |
| Authors | Luzie Helfmann, Johannes von Lindheim, Mattes Mollenhauer, Ralf Banisch |
| Abstract | Quality assessments of models in unsupervised learning and clustering verification in particular have been a long-standing problem in the machine learning research. The lack of robust and universally applicable cluster validity scores often makes the algorithm selection and hyperparameter evaluation a tough guess. In this paper, we show that cluster ensemble aggregation techniques such as consensus clustering may be used to evaluate clusterings and their hyperparameter configurations. We use normalized mutual information to compare individual objects of a clustering ensemble to the constructed consensus of the whole ensemble and show, that the resulting score can serve as an overall quality measure for clustering problems. This method is capable of highlighting the standout clustering and hyperparameter configuration in the ensemble even in the case of a distorted consensus. We apply this very general framework to various data sets and give possible directions for future research. |
| Tasks | |
| Published | 2018-03-29 |
| URL | http://arxiv.org/abs/1803.11008v1 |
| http://arxiv.org/pdf/1803.11008v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-hyperparameter-search-in-cluster-ensembles |
| Repo | |
| Framework | |
Dense Light Field Reconstruction From Sparse Sampling Using Residual Network
| Title | Dense Light Field Reconstruction From Sparse Sampling Using Residual Network |
| Authors | Mantang Guo, Hao Zhu, Guoqing Zhou, Qing Wang |
| Abstract | A light field records numerous light rays from a real-world scene. However, capturing a dense light field by existing devices is a time-consuming process. Besides, reconstructing a large amount of light rays equivalent to multiple light fields using sparse sampling arises a severe challenge for existing methods. In this paper, we present a learning based method to reconstruct multiple novel light fields between two mutually independent light fields. We indicate that light rays distributed in different light fields have the same consistent constraints under a certain condition. The most significant constraint is a depth related correlation between angular and spatial dimensions. Our method avoids working out the error-sensitive constraint by employing a deep neural network. We solve residual values of pixels on epipolar plane image (EPI) to reconstruct novel light fields. Our method is able to reconstruct 2 to 4 novel light fields between two mutually independent input light fields. We also compare our results with those yielded by a number of alternatives elsewhere in the literature, which shows our reconstructed light fields have better structure similarity and occlusion relationship. |
| Tasks | |
| Published | 2018-06-14 |
| URL | http://arxiv.org/abs/1806.05506v2 |
| http://arxiv.org/pdf/1806.05506v2.pdf | |
| PWC | https://paperswithcode.com/paper/dense-light-field-reconstruction-from-sparse |
| Repo | |
| Framework | |
Apuntes de Redes Neuronales Artificiales
| Title | Apuntes de Redes Neuronales Artificiales |
| Authors | J. C. Cuevas-Tello |
| Abstract | These handouts are designed for people who is just starting involved with the topic artificial neural networks. We show how it works a single artificial neuron (McCulloch & Pitt model), mathematically and graphically. We do explain the delta rule, a learning algorithm to find the neuron weights. We also present some examples in MATLAB/Octave. There are examples for classification task for lineal and non-lineal problems. At the end, we present an artificial neural network, a feed-forward neural network along its learning algorithm backpropagation. —– Estos apuntes est'an dise~nados para personas que por primera vez se introducen en el tema de las redes neuronales artificiales. Se muestra el funcionamiento b'asico de una neurona, matem'aticamente y gr'aficamente. Se explica la Regla Delta, algoritmo deaprendizaje para encontrar los pesos de una neurona. Tambi'en se muestran ejemplos en MATLAB/Octave. Hay ejemplos para problemas de clasificaci'on, para problemas lineales y no-lineales. En la parte final se muestra la arquitectura de red neuronal artificial conocida como backpropagation. |
| Tasks | |
| Published | 2018-06-13 |
| URL | http://arxiv.org/abs/1806.05298v1 |
| http://arxiv.org/pdf/1806.05298v1.pdf | |
| PWC | https://paperswithcode.com/paper/apuntes-de-redes-neuronales-artificiales |
| Repo | |
| Framework | |
Solutions to problems with deep learning
| Title | Solutions to problems with deep learning |
| Authors | J Gerard Wolff |
| Abstract | Despite the several successes of deep learning systems, there are concerns about their limitations, discussed most recently by Gary Marcus. This paper discusses Marcus’s concerns and some others, together with solutions to several of these problems provided by the “P theory of intelligence” and its realisation in the “SP computer model”. The main advantages of the SP system are: relatively small requirements for data and the ability to learn from a single experience; the ability to model both hierarchical and non-hierarchical structures; strengths in several kinds of reasoning, including commonsense' reasoning; transparency in the representation of knowledge, and the provision of an audit trail for all processing; the likelihood that the SP system could not be fooled into bizarre or eccentric recognition of stimuli, as deep learning systems can be; the SP system provides a robust solution to the problem of catastrophic forgetting’ in deep learning systems; the SP system provides a theoretically-coherent solution to the problems of correcting over- and under-generalisations in learning, and learning correct structures despite errors in data; unlike most research on deep learning, the SP programme of research draws extensively on research on human learning, perception, and cognition; and the SP programme of research has an overarching theory, supported by evidence, something that is largely missing from research on deep learning. In general, the SP system provides a much firmer foundation than deep learning for the development of artificial general intelligence. |
| Tasks | |
| Published | 2018-01-08 |
| URL | http://arxiv.org/abs/1801.05457v1 |
| http://arxiv.org/pdf/1801.05457v1.pdf | |
| PWC | https://paperswithcode.com/paper/solutions-to-problems-with-deep-learning |
| Repo | |
| Framework | |
Leave-One-Out Least Square Monte Carlo Algorithm for Pricing American Options
| Title | Leave-One-Out Least Square Monte Carlo Algorithm for Pricing American Options |
| Authors | Jeechul Woo, Chenru Liu, Jaehyuk Choi |
| Abstract | The least square Monte Carlo (LSM) algorithm proposed by Longstaff and Schwartz [2001] is widely used for pricing American options. The LSM estimator contains undesirable look-ahead bias, and the conventional technique of removing it necessitates doubling simulations. We present the leave-one-out LSM (LOOLSM) algorithm for efficiently eliminating look-ahead bias. We validate the method with several option examples, including the multi-asset cases that the LSM algorithm significantly overvalues. We also obtain the convergence rates of look-ahead bias by measuring it using the LOOLSM method. The analysis and computational evidence support our findings. |
| Tasks | |
| Published | 2018-10-04 |
| URL | https://arxiv.org/abs/1810.02071v2 |
| https://arxiv.org/pdf/1810.02071v2.pdf | |
| PWC | https://paperswithcode.com/paper/an-efficient-approach-for-removing-look-ahead |
| Repo | |
| Framework | |
Time-Aware and Corpus-Specific Entity Relatedness
| Title | Time-Aware and Corpus-Specific Entity Relatedness |
| Authors | Nilamadhaba Mohapatra, Vasileios Iosifidis, Asif Ekbal, Stefan Dietze, Pavlos Fafalios |
| Abstract | Entity relatedness has emerged as an important feature in a plethora of applications such as information retrieval, entity recommendation and entity linking. Given an entity, for instance a person or an organization, entity relatedness measures can be exploited for generating a list of highly-related entities. However, the relation of an entity to some other entity depends on several factors, with time and context being two of the most important ones (where, in our case, context is determined by a particular corpus). For example, the entities related to the International Monetary Fund are different now compared to some years ago, while these entities also may highly differ in the context of a USA news portal compared to a Greek news portal. In this paper, we propose a simple but flexible model for entity relatedness which considers time and entity aware word embeddings by exploiting the underlying corpus. The proposed model does not require external knowledge and is language independent, which makes it widely useful in a variety of applications. |
| Tasks | Entity Linking, Information Retrieval, Word Embeddings |
| Published | 2018-10-23 |
| URL | http://arxiv.org/abs/1810.10004v1 |
| http://arxiv.org/pdf/1810.10004v1.pdf | |
| PWC | https://paperswithcode.com/paper/time-aware-and-corpus-specific-entity |
| Repo | |
| Framework | |
Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning
| Title | Curiosity-driven Exploration for Mapless Navigation with Deep Reinforcement Learning |
| Authors | Oleksii Zhelo, Jingwei Zhang, Lei Tai, Ming Liu, Wolfram Burgard |
| Abstract | This paper investigates exploration strategies of Deep Reinforcement Learning (DRL) methods to learn navigation policies for mobile robots. In particular, we augment the normal external reward for training DRL algorithms with intrinsic reward signals measured by curiosity. We test our approach in a mapless navigation setting, where the autonomous agent is required to navigate without the occupancy map of the environment, to targets whose relative locations can be easily acquired through low-cost solutions (e.g., visible light localization, Wi-Fi signal localization). We validate that the intrinsic motivation is crucial for improving DRL performance in tasks with challenging exploration requirements. Our experimental results show that our proposed method is able to more effectively learn navigation policies, and has better generalization capabilities in previously unseen environments. A video of our experimental results can be found at https://goo.gl/pWbpcF. |
| Tasks | |
| Published | 2018-04-02 |
| URL | http://arxiv.org/abs/1804.00456v2 |
| http://arxiv.org/pdf/1804.00456v2.pdf | |
| PWC | https://paperswithcode.com/paper/curiosity-driven-exploration-for-mapless |
| Repo | |
| Framework | |
Improving Sentence Representations with Consensus Maximisation
| Title | Improving Sentence Representations with Consensus Maximisation |
| Authors | Shuai Tang, Virginia R. de Sa |
| Abstract | Consensus maximisation learning can provide self-supervision when different views are available of the same data. The distributional hypothesis provides another form of useful self-supervision from adjacent sentences which are plentiful in large unlabelled corpora. Motivated by the observation that different learning architectures tend to emphasise different aspects of sentence meaning, we present a new self-supervised learning framework for learning sentence representations which minimises the disagreement between two views of the same sentence where one view encodes the sentence with a recurrent neural network (RNN), and the other view encodes the same sentence with a simple linear model. After learning, the individual views (networks) result in higher quality sentence representations than their single-view learnt counterparts (learnt using only the distributional hypothesis) as judged by performance on standard downstream tasks. An ensemble of both views provides even better generalisation on both supervised and unsupervised downstream tasks. Also, importantly the ensemble of views trained with consensus maximisation between the two different architectures performs better on downstream tasks than an analogous ensemble made from the single-view trained counterparts. |
| Tasks | |
| Published | 2018-10-02 |
| URL | https://arxiv.org/abs/1810.01064v4 |
| https://arxiv.org/pdf/1810.01064v4.pdf | |
| PWC | https://paperswithcode.com/paper/improving-sentence-representations-with-multi |
| Repo | |
| Framework | |
