Paper Group ANR 1220
Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning. Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution. Photometric Transformer Networks and Label Adjustment for Breast Density Prediction. Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis. A Universal Approx …
Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning
Title | Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning |
Authors | Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, Huajun Chen |
Abstract | Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own advantages and difficulties. Rule-based reasoning is accurate and explainable but rule learning with searching over the graph always suffers from efficiency due to huge search space. Embedding-based reasoning is more scalable and efficient as the reasoning is conducted via computation between embeddings, but it has difficulty learning good representations for sparse entities because a good embedding relies heavily on data richness. Based on this observation, in this paper we explore how embedding and rule learning can be combined together and complement each other’s difficulties with their advantages. We propose a novel framework IterE iteratively learning embeddings and rules, in which rules are learned from embeddings with proper pruning strategy and embeddings are learned from existing triples and new triples inferred by rules. Evaluations on embedding qualities of IterE show that rules help improve the quality of sparse entity embeddings and their link prediction results. We also evaluate the efficiency of rule learning and quality of rules from IterE compared with AMIE+, showing that IterE is capable of generating high quality rules more efficiently. Experiments show that iteratively learning embeddings and rules benefit each other during learning and prediction. |
Tasks | Entity Embeddings, Knowledge Graphs, Link Prediction |
Published | 2019-03-21 |
URL | http://arxiv.org/abs/1903.08948v1 |
http://arxiv.org/pdf/1903.08948v1.pdf | |
PWC | https://paperswithcode.com/paper/iteratively-learning-embeddings-and-rules-for |
Repo | |
Framework | |
Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution
Title | Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution |
Authors | Chih-Chung Hsu, Chia-Hsiang Lin |
Abstract | Deep learning-based single image super-resolution enables very fast and high-visual-quality reconstruction. Recently, an enhanced super-resolution based on generative adversarial network (ESRGAN) has achieved excellent performance in terms of both qualitative and quantitative quality of the reconstructed high-resolution image. In this paper, we propose to add one more shortcut between two dense-blocks, as well as add shortcut between two convolution layers inside a dense-block. With this simple strategy of adding more shortcuts in the proposed network, it enables a faster learning process as the gradient information can be back-propagated more easily. Based on the improved ESRGAN, the dual reconstruction is proposed to learn different aspects of the super-resolved image for judiciously enhancing the quality of the reconstructed image. In practice, the super-resolution model is pre-trained solely based on pixel distance, followed by fine-tuning the parameters in the model based on adversarial loss and perceptual loss. Finally, we fuse two different models by weighted-summing their parameters to obtain the final super-resolution model. Experimental results demonstrated that the proposed method achieves excellent performance in the real-world image super-resolution challenge. We have also verified that the proposed dual reconstruction does further improve the quality of the reconstructed image in terms of both PSNR and SSIM. |
Tasks | Image Super-Resolution, Super-Resolution |
Published | 2019-11-20 |
URL | https://arxiv.org/abs/1911.08711v1 |
https://arxiv.org/pdf/1911.08711v1.pdf | |
PWC | https://paperswithcode.com/paper/dual-reconstruction-with-densely-connected |
Repo | |
Framework | |
Photometric Transformer Networks and Label Adjustment for Breast Density Prediction
Title | Photometric Transformer Networks and Label Adjustment for Breast Density Prediction |
Authors | Jaehwan Lee, Donggeon Yoo, Jung Yin Huh, Hyo-Eun Kim |
Abstract | Grading breast density is highly sensitive to normalization settings of digital mammogram as the density is tightly correlated with the distribution of pixel intensity. Also, the grade varies with readers due to uncertain grading criteria. These issues are inherent in the density assessment of digital mammography. They are problematic when designing a computer-aided prediction model for breast density and become worse if the data comes from multiple sites. In this paper, we proposed two novel deep learning techniques for breast density prediction: 1) photometric transformation which adaptively normalizes the input mammograms, and 2) label distillation which adjusts the label by using its output prediction. The photometric transformer network predicts optimal parameters for photometric transformation on the fly, learned jointly with the main prediction network. The label distillation, a type of pseudo-label techniques, is intended to mitigate the grading variation. We experimentally showed that the proposed methods are beneficial in terms of breast density prediction, resulting in significant performance improvement compared to various previous approaches. |
Tasks | |
Published | 2019-05-08 |
URL | https://arxiv.org/abs/1905.02906v1 |
https://arxiv.org/pdf/1905.02906v1.pdf | |
PWC | https://paperswithcode.com/paper/photometric-transformer-networks-and-label |
Repo | |
Framework | |
Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis
Title | Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis |
Authors | Alberto Redondo, David Rios Insua |
Abstract | Malware constitutes a major global risk affecting millions of users each year. Standard algorithms in detection systems perform insufficiently when dealing with malware passed through obfuscation tools. We illustrate this studying in detail an open source metamorphic software, making use of a hybrid framework to obtain the relevant features from binaries. We then provide an improved alternative solution based on adversarial risk analysis which we illustrate describe with an example. |
Tasks | |
Published | 2019-11-09 |
URL | https://arxiv.org/abs/1911.03653v1 |
https://arxiv.org/pdf/1911.03653v1.pdf | |
PWC | https://paperswithcode.com/paper/protecting-from-malware-obfuscation-attacks |
Repo | |
Framework | |
A Universal Approximation Result for Difference of log-sum-exp Neural Networks
Title | A Universal Approximation Result for Difference of log-sum-exp Neural Networks |
Authors | Giuseppe C. Calafiore, Stephane Gaubert, Member, Corrado Possieri |
Abstract | We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node (LSE networks) is a smooth universal approximator of continuous functions over convex, compact sets. By using a logarithmic transform, this class of networks maps to a family of subtraction-free ratios of generalized posynomials, which we also show to be universal approximators of positive functions over log-convex, compact subsets of the positive orthant. The main advantage of Difference-LSE networks with respect to classical feedforward neural networks is that, after a standard training phase, they provide surrogate models for design that possess a specific difference-of-convex-functions form, which makes them optimizable via relatively efficient numerical methods. In particular, by adapting an existing difference-of-convex algorithm to these models, we obtain an algorithm for performing effective optimization-based design. We illustrate the proposed approach by applying it to data-driven design of a diet for a patient with type-2 diabetes. |
Tasks | |
Published | 2019-05-21 |
URL | https://arxiv.org/abs/1905.08503v1 |
https://arxiv.org/pdf/1905.08503v1.pdf | |
PWC | https://paperswithcode.com/paper/a-universal-approximation-result-for |
Repo | |
Framework | |
Doctor of Crosswise: Reducing Over-parametrization in Neural Networks
Title | Doctor of Crosswise: Reducing Over-parametrization in Neural Networks |
Authors | J. D. Curtó, I. C. Zarza, K. Kitani, I. King, M. R. Lyu |
Abstract | Dr. of Crosswise proposes a new architecture to reduce over-parametrization in Neural Networks. It introduces an operand for rapid computation in the framework of Deep Learning that leverages learned weights. The formalism is described in detail providing both an accurate elucidation of the mechanics and the theoretical implications. |
Tasks | |
Published | 2019-05-24 |
URL | https://arxiv.org/abs/1905.10324v2 |
https://arxiv.org/pdf/1905.10324v2.pdf | |
PWC | https://paperswithcode.com/paper/doctor-of-crosswise-reducing-over |
Repo | |
Framework | |
Artistic Domain Generalisation Methods are Limited by their Deep Representations
Title | Artistic Domain Generalisation Methods are Limited by their Deep Representations |
Authors | Padraig Boulton, Peter Hall |
Abstract | The cross-depiction problem refers to the task of recognising visual objects regardless of their depictions; whether photographed, painted, sketched, {\em etc}. In the past, some researchers considered cross-depiction to be domain adaptation (DA). More recent work considers cross-depiction as domain generalisation (DG), in which algorithms extend recognition from one set of domains (such as photographs and coloured artwork) to another (such as sketches). We show that fixing the last layer of AlexNet to random values provides a performance comparable to state of the art DA and DG algorithms, when tested over the PACS benchmark. With support from background literature, our results lead us to conclude that texture alone is insufficient to support generalisation; rather, higher-order representations such as structure and shape are necessary. |
Tasks | Domain Adaptation |
Published | 2019-07-29 |
URL | https://arxiv.org/abs/1907.12622v1 |
https://arxiv.org/pdf/1907.12622v1.pdf | |
PWC | https://paperswithcode.com/paper/artistic-domain-generalisation-methods-are |
Repo | |
Framework | |
Unsupervised Pivot Translation for Distant Languages
Title | Unsupervised Pivot Translation for Distant Languages |
Authors | Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, Tie-Yan Liu |
Abstract | Unsupervised neural machine translation (NMT) has attracted a lot of attention recently. While state-of-the-art methods for unsupervised translation usually perform well between similar languages (e.g., English-German translation), they perform poorly between distant languages, because unsupervised alignment does not work well for distant languages. In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation. We propose a learning to route (LTR) method to choose the translation path between the source and target languages. LTR is trained on language pairs whose best translation path is available and is applied on the unseen language pairs for path selection. Experiments on 20 languages and 294 distant language pairs demonstrate the advantages of the unsupervised pivot translation for distant languages, as well as the effectiveness of the proposed LTR for path selection. Specifically, in the best case, LTR achieves an improvement of 5.58 BLEU points over the conventional direct unsupervised method. |
Tasks | Machine Translation |
Published | 2019-06-06 |
URL | https://arxiv.org/abs/1906.02461v3 |
https://arxiv.org/pdf/1906.02461v3.pdf | |
PWC | https://paperswithcode.com/paper/unsupervised-pivot-translation-for-distant |
Repo | |
Framework | |
Learning Symmetries of Classical Integrable Systems
Title | Learning Symmetries of Classical Integrable Systems |
Authors | Roberto Bondesan, Austen Lamacraft |
Abstract | The solution of problems in physics is often facilitated by a change of variables. In this work we present neural transformations to learn symmetries of Hamiltonian mechanical systems. Maintaining the Hamiltonian structure requires novel network architectures that parametrize symplectic transformations. We demonstrate the utility of these architectures by learning the structure of integrable models. Our work exemplifies the adaptation of neural transformations to a family constrained by more than the condition of invertibility, which we expect to be a common feature of applications of these methods. |
Tasks | |
Published | 2019-06-11 |
URL | https://arxiv.org/abs/1906.04645v1 |
https://arxiv.org/pdf/1906.04645v1.pdf | |
PWC | https://paperswithcode.com/paper/learning-symmetries-of-classical-integrable |
Repo | |
Framework | |
Cross-modal knowledge distillation for action recognition
Title | Cross-modal knowledge distillation for action recognition |
Authors | Fida Mohammad Thoker, Juergen Gall |
Abstract | In this work, we address the problem how a network for action recognition that has been trained on a modality like RGB videos can be adapted to recognize actions for another modality like sequences of 3D human poses. To this end, we extract the knowledge of the trained teacher network for the source modality and transfer it to a small ensemble of student networks for the target modality. For the cross-modal knowledge distillation, we do not require any annotated data. Instead we use pairs of sequences of both modalities as supervision, which are straightforward to acquire. In contrast to previous works for knowledge distillation that use a KL-loss, we show that the cross-entropy loss together with mutual learning of a small ensemble of student networks performs better. In fact, the proposed approach for cross-modal knowledge distillation nearly achieves the accuracy of a student network trained with full supervision. |
Tasks | |
Published | 2019-10-10 |
URL | https://arxiv.org/abs/1910.04641v1 |
https://arxiv.org/pdf/1910.04641v1.pdf | |
PWC | https://paperswithcode.com/paper/cross-modal-knowledge-distillation-for-action |
Repo | |
Framework | |
Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation
Title | Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation |
Authors | Mayur J. Bency, Ahmed H. Qureshi, Michael C. Yip |
Abstract | Fast and efficient path generation is critical for robots operating in complex environments. This motion planning problem is often performed in a robot’s actuation or configuration space, where popular pathfinding methods such as A*, RRT*, get exponentially more computationally expensive to execute as the dimensionality increases or the spaces become more cluttered and complex. On the other hand, if one were to save the entire set of paths connecting all pair of locations in the configuration space a priori, one would run out of memory very quickly. In this work, we introduce a novel way of producing fast and optimal motion plans for static environments by using a stepping neural network approach, called OracleNet. OracleNet uses Recurrent Neural Networks to determine end-to-end trajectories in an iterative manner that implicitly generates optimal motion plans with minimal loss in performance in a compact form. The algorithm is straightforward in implementation while consistently generating near-optimal paths in a single, iterative, end-to-end roll-out. In practice, OracleNet generally has fixed-time execution regardless of the configuration space complexity while outperforming popular pathfinding algorithms in complex environments and higher dimensions |
Tasks | Motion Planning |
Published | 2019-04-25 |
URL | http://arxiv.org/abs/1904.11102v1 |
http://arxiv.org/pdf/1904.11102v1.pdf | |
PWC | https://paperswithcode.com/paper/neural-path-planning-fixed-time-near-optimal |
Repo | |
Framework | |
Learning from Higher-Layer Feature Visualizations
Title | Learning from Higher-Layer Feature Visualizations |
Authors | Konstantinos Nikolaidis, Stein Kristiansen, Vera Goebel, Thomas Plagemann |
Abstract | Driven by the goal to enable sleep apnea monitoring and machine learning-based detection at home with small mobile devices, we investigate whether interpretation-based indirect knowledge transfer can be used to create classifiers with acceptable performance. Interpretation-based indirect knowledge transfer means that a classifier (student) learns from a synthetic dataset based on the knowledge representation from an already trained Deep Network (teacher). We use activation maximization to generate visualizations and create a synthetic dataset to train the student classifier. This approach has the advantage that student classifiers can be trained without access to the original training data. With experiments we investigate the feasibility of interpretation-based indirect knowledge transfer and its limitations. The student achieves an accuracy of 97.8% on MNIST (teacher accuracy: 99.3%) with a similar smaller architecture to that of the teacher. The student classifier achieves an accuracy of 86.1% and 89.5% for a subset of the Apnea-ECG dataset (teacher: 89.5% and 91.1%, respectively). |
Tasks | Transfer Learning |
Published | 2019-03-06 |
URL | http://arxiv.org/abs/1903.02313v1 |
http://arxiv.org/pdf/1903.02313v1.pdf | |
PWC | https://paperswithcode.com/paper/learning-from-higher-layer-feature |
Repo | |
Framework | |
Review of Algorithms for Compressive Sensing of Images
Title | Review of Algorithms for Compressive Sensing of Images |
Authors | Yoni Sher |
Abstract | We provide a comprehensive review of classical algorithms for compressive sensing of images, focused on Total variation methods, with a view to application in LiDAR systems. Our primary focus is providing a full review for beginners in the field, as well as simulating the kind of noise found in real LiDAR systems. To this end, we provide an overview of the theoretical background, a brief discussion of various considerations that come in to play in compressive sensing, and a standardized comparison of off-the-shelf methods, intended as a quick-start guide to choosing algorithms for compressive sensing applications. |
Tasks | Compressive Sensing |
Published | 2019-08-05 |
URL | https://arxiv.org/abs/1908.01642v1 |
https://arxiv.org/pdf/1908.01642v1.pdf | |
PWC | https://paperswithcode.com/paper/review-of-algorithms-for-compressive-sensing |
Repo | |
Framework | |
Image segmentation of liver stage malaria infection with spatial uncertainty sampling
Title | Image segmentation of liver stage malaria infection with spatial uncertainty sampling |
Authors | Ava P. Soleimany, Harini Suresh, Jose Javier Gonzalez Ortiz, Divya Shanmugam, Nil Gural, John Guttag, Sangeeta N. Bhatia |
Abstract | Global eradication of malaria depends on the development of drugs effective against the silent, yet obligate liver stage of the disease. The gold standard in drug development remains microscopic imaging of liver stage parasites in in vitro cell culture models. Image analysis presents a major bottleneck in this pipeline since the parasite has significant variability in size, shape, and density in these models. As with other highly variable datasets, traditional segmentation models have poor generalizability as they rely on hand-crafted features; thus, manual annotation of liver stage malaria images remains standard. To address this need, we develop a convolutional neural network architecture that utilizes spatial dropout sampling for parasite segmentation and epistemic uncertainty estimation in images of liver stage malaria. Our pipeline produces high-precision segmentations nearly identical to expert annotations, generalizes well on a diverse dataset of liver stage malaria parasites, and promotes independence between learned feature maps to model the uncertainty of generated predictions. |
Tasks | Semantic Segmentation |
Published | 2019-11-30 |
URL | https://arxiv.org/abs/1912.00262v1 |
https://arxiv.org/pdf/1912.00262v1.pdf | |
PWC | https://paperswithcode.com/paper/image-segmentation-of-liver-stage-malaria |
Repo | |
Framework | |
PAC Reinforcement Learning without Real-World Feedback
Title | PAC Reinforcement Learning without Real-World Feedback |
Authors | Yuren Zhong, Aniket Anand Deshmukh, Clayton Scott |
Abstract | This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (ROMDP), we formulate a theoretical framework for Sim-to-Real in the situation where feedback in the real world is not available. We establish real-world sample complexity guarantees that are smaller than what is currently known for directly (i.e., without access to simulators) learning a ROMDP with feedback. |
Tasks | |
Published | 2019-09-23 |
URL | https://arxiv.org/abs/1909.10449v3 |
https://arxiv.org/pdf/1909.10449v3.pdf | |
PWC | https://paperswithcode.com/paper/190910449 |
Repo | |
Framework | |