January 27, 2020

2741 words 13 mins read

Paper Group ANR 1220

Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning. Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution. Photometric Transformer Networks and Label Adjustment for Breast Density Prediction. Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis. A Universal Approx …

Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning


Title	Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning
Authors	Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, Huajun Chen
Abstract	Reasoning is essential for the development of large knowledge graphs, especially for completion, which aims to infer new triples based on existing ones. Both rules and embeddings can be used for knowledge graph reasoning and they have their own advantages and difficulties. Rule-based reasoning is accurate and explainable but rule learning with searching over the graph always suffers from efficiency due to huge search space. Embedding-based reasoning is more scalable and efficient as the reasoning is conducted via computation between embeddings, but it has difficulty learning good representations for sparse entities because a good embedding relies heavily on data richness. Based on this observation, in this paper we explore how embedding and rule learning can be combined together and complement each other’s difficulties with their advantages. We propose a novel framework IterE iteratively learning embeddings and rules, in which rules are learned from embeddings with proper pruning strategy and embeddings are learned from existing triples and new triples inferred by rules. Evaluations on embedding qualities of IterE show that rules help improve the quality of sparse entity embeddings and their link prediction results. We also evaluate the efficiency of rule learning and quality of rules from IterE compared with AMIE+, showing that IterE is capable of generating high quality rules more efficiently. Experiments show that iteratively learning embeddings and rules benefit each other during learning and prediction.
Tasks	Entity Embeddings, Knowledge Graphs, Link Prediction
Published	2019-03-21
URL	http://arxiv.org/abs/1903.08948v1
PDF	http://arxiv.org/pdf/1903.08948v1.pdf
PWC	https://paperswithcode.com/paper/iteratively-learning-embeddings-and-rules-for
Repo
Framework

Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution


Title	Dual Reconstruction with Densely Connected Residual Network for Single Image Super-Resolution
Authors	Chih-Chung Hsu, Chia-Hsiang Lin
Abstract	Deep learning-based single image super-resolution enables very fast and high-visual-quality reconstruction. Recently, an enhanced super-resolution based on generative adversarial network (ESRGAN) has achieved excellent performance in terms of both qualitative and quantitative quality of the reconstructed high-resolution image. In this paper, we propose to add one more shortcut between two dense-blocks, as well as add shortcut between two convolution layers inside a dense-block. With this simple strategy of adding more shortcuts in the proposed network, it enables a faster learning process as the gradient information can be back-propagated more easily. Based on the improved ESRGAN, the dual reconstruction is proposed to learn different aspects of the super-resolved image for judiciously enhancing the quality of the reconstructed image. In practice, the super-resolution model is pre-trained solely based on pixel distance, followed by fine-tuning the parameters in the model based on adversarial loss and perceptual loss. Finally, we fuse two different models by weighted-summing their parameters to obtain the final super-resolution model. Experimental results demonstrated that the proposed method achieves excellent performance in the real-world image super-resolution challenge. We have also verified that the proposed dual reconstruction does further improve the quality of the reconstructed image in terms of both PSNR and SSIM.
Tasks	Image Super-Resolution, Super-Resolution
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08711v1
PDF	https://arxiv.org/pdf/1911.08711v1.pdf
PWC	https://paperswithcode.com/paper/dual-reconstruction-with-densely-connected
Repo
Framework

Photometric Transformer Networks and Label Adjustment for Breast Density Prediction


Title	Photometric Transformer Networks and Label Adjustment for Breast Density Prediction
Authors	Jaehwan Lee, Donggeon Yoo, Jung Yin Huh, Hyo-Eun Kim
Abstract	Grading breast density is highly sensitive to normalization settings of digital mammogram as the density is tightly correlated with the distribution of pixel intensity. Also, the grade varies with readers due to uncertain grading criteria. These issues are inherent in the density assessment of digital mammography. They are problematic when designing a computer-aided prediction model for breast density and become worse if the data comes from multiple sites. In this paper, we proposed two novel deep learning techniques for breast density prediction: 1) photometric transformation which adaptively normalizes the input mammograms, and 2) label distillation which adjusts the label by using its output prediction. The photometric transformer network predicts optimal parameters for photometric transformation on the fly, learned jointly with the main prediction network. The label distillation, a type of pseudo-label techniques, is intended to mitigate the grading variation. We experimentally showed that the proposed methods are beneficial in terms of breast density prediction, resulting in significant performance improvement compared to various previous approaches.
Tasks
Published	2019-05-08
URL	https://arxiv.org/abs/1905.02906v1
PDF	https://arxiv.org/pdf/1905.02906v1.pdf
PWC	https://paperswithcode.com/paper/photometric-transformer-networks-and-label
Repo
Framework

Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis


Title	Protecting from Malware Obfuscation Attacks through Adversarial Risk Analysis
Authors	Alberto Redondo, David Rios Insua
Abstract	Malware constitutes a major global risk affecting millions of users each year. Standard algorithms in detection systems perform insufficiently when dealing with malware passed through obfuscation tools. We illustrate this studying in detail an open source metamorphic software, making use of a hybrid framework to obtain the relevant features from binaries. We then provide an improved alternative solution based on adversarial risk analysis which we illustrate describe with an example.
Tasks
Published	2019-11-09
URL	https://arxiv.org/abs/1911.03653v1
PDF	https://arxiv.org/pdf/1911.03653v1.pdf
PWC	https://paperswithcode.com/paper/protecting-from-malware-obfuscation-attacks
Repo
Framework

A Universal Approximation Result for Difference of log-sum-exp Neural Networks


Title	A Universal Approximation Result for Difference of log-sum-exp Neural Networks
Authors	Giuseppe C. Calafiore, Stephane Gaubert, Member, Corrado Possieri
Abstract	We show that a neural network whose output is obtained as the difference of the outputs of two feedforward networks with exponential activation function in the hidden layer and logarithmic activation function in the output node (LSE networks) is a smooth universal approximator of continuous functions over convex, compact sets. By using a logarithmic transform, this class of networks maps to a family of subtraction-free ratios of generalized posynomials, which we also show to be universal approximators of positive functions over log-convex, compact subsets of the positive orthant. The main advantage of Difference-LSE networks with respect to classical feedforward neural networks is that, after a standard training phase, they provide surrogate models for design that possess a specific difference-of-convex-functions form, which makes them optimizable via relatively efficient numerical methods. In particular, by adapting an existing difference-of-convex algorithm to these models, we obtain an algorithm for performing effective optimization-based design. We illustrate the proposed approach by applying it to data-driven design of a diet for a patient with type-2 diabetes.
Tasks
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08503v1
PDF	https://arxiv.org/pdf/1905.08503v1.pdf
PWC	https://paperswithcode.com/paper/a-universal-approximation-result-for
Repo
Framework

Doctor of Crosswise: Reducing Over-parametrization in Neural Networks


Title	Doctor of Crosswise: Reducing Over-parametrization in Neural Networks
Authors	J. D. Curtó, I. C. Zarza, K. Kitani, I. King, M. R. Lyu
Abstract	Dr. of Crosswise proposes a new architecture to reduce over-parametrization in Neural Networks. It introduces an operand for rapid computation in the framework of Deep Learning that leverages learned weights. The formalism is described in detail providing both an accurate elucidation of the mechanics and the theoretical implications.
Tasks
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10324v2
PDF	https://arxiv.org/pdf/1905.10324v2.pdf
PWC	https://paperswithcode.com/paper/doctor-of-crosswise-reducing-over
Repo
Framework

Artistic Domain Generalisation Methods are Limited by their Deep Representations


Title	Artistic Domain Generalisation Methods are Limited by their Deep Representations
Authors	Padraig Boulton, Peter Hall
Abstract	The cross-depiction problem refers to the task of recognising visual objects regardless of their depictions; whether photographed, painted, sketched, {\em etc}. In the past, some researchers considered cross-depiction to be domain adaptation (DA). More recent work considers cross-depiction as domain generalisation (DG), in which algorithms extend recognition from one set of domains (such as photographs and coloured artwork) to another (such as sketches). We show that fixing the last layer of AlexNet to random values provides a performance comparable to state of the art DA and DG algorithms, when tested over the PACS benchmark. With support from background literature, our results lead us to conclude that texture alone is insufficient to support generalisation; rather, higher-order representations such as structure and shape are necessary.
Tasks	Domain Adaptation
Published	2019-07-29
URL	https://arxiv.org/abs/1907.12622v1
PDF	https://arxiv.org/pdf/1907.12622v1.pdf
PWC	https://paperswithcode.com/paper/artistic-domain-generalisation-methods-are
Repo
Framework

Unsupervised Pivot Translation for Distant Languages


Title	Unsupervised Pivot Translation for Distant Languages
Authors	Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, Tie-Yan Liu
Abstract	Unsupervised neural machine translation (NMT) has attracted a lot of attention recently. While state-of-the-art methods for unsupervised translation usually perform well between similar languages (e.g., English-German translation), they perform poorly between distant languages, because unsupervised alignment does not work well for distant languages. In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation. We propose a learning to route (LTR) method to choose the translation path between the source and target languages. LTR is trained on language pairs whose best translation path is available and is applied on the unseen language pairs for path selection. Experiments on 20 languages and 294 distant language pairs demonstrate the advantages of the unsupervised pivot translation for distant languages, as well as the effectiveness of the proposed LTR for path selection. Specifically, in the best case, LTR achieves an improvement of 5.58 BLEU points over the conventional direct unsupervised method.
Tasks	Machine Translation
Published	2019-06-06
URL	https://arxiv.org/abs/1906.02461v3
PDF	https://arxiv.org/pdf/1906.02461v3.pdf
PWC	https://paperswithcode.com/paper/unsupervised-pivot-translation-for-distant
Repo
Framework

Learning Symmetries of Classical Integrable Systems


Title	Learning Symmetries of Classical Integrable Systems
Authors	Roberto Bondesan, Austen Lamacraft
Abstract	The solution of problems in physics is often facilitated by a change of variables. In this work we present neural transformations to learn symmetries of Hamiltonian mechanical systems. Maintaining the Hamiltonian structure requires novel network architectures that parametrize symplectic transformations. We demonstrate the utility of these architectures by learning the structure of integrable models. Our work exemplifies the adaptation of neural transformations to a family constrained by more than the condition of invertibility, which we expect to be a common feature of applications of these methods.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04645v1
PDF	https://arxiv.org/pdf/1906.04645v1.pdf
PWC	https://paperswithcode.com/paper/learning-symmetries-of-classical-integrable
Repo
Framework


Title	Cross-modal knowledge distillation for action recognition
Authors	Fida Mohammad Thoker, Juergen Gall
Abstract	In this work, we address the problem how a network for action recognition that has been trained on a modality like RGB videos can be adapted to recognize actions for another modality like sequences of 3D human poses. To this end, we extract the knowledge of the trained teacher network for the source modality and transfer it to a small ensemble of student networks for the target modality. For the cross-modal knowledge distillation, we do not require any annotated data. Instead we use pairs of sequences of both modalities as supervision, which are straightforward to acquire. In contrast to previous works for knowledge distillation that use a KL-loss, we show that the cross-entropy loss together with mutual learning of a small ensemble of student networks performs better. In fact, the proposed approach for cross-modal knowledge distillation nearly achieves the accuracy of a student network trained with full supervision.
Tasks
Published	2019-10-10
URL	https://arxiv.org/abs/1910.04641v1
PDF	https://arxiv.org/pdf/1910.04641v1.pdf
PWC	https://paperswithcode.com/paper/cross-modal-knowledge-distillation-for-action
Repo
Framework

Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation


Title	Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation
Authors	Mayur J. Bency, Ahmed H. Qureshi, Michael C. Yip
Abstract	Fast and efficient path generation is critical for robots operating in complex environments. This motion planning problem is often performed in a robot’s actuation or configuration space, where popular pathfinding methods such as A, RRT, get exponentially more computationally expensive to execute as the dimensionality increases or the spaces become more cluttered and complex. On the other hand, if one were to save the entire set of paths connecting all pair of locations in the configuration space a priori, one would run out of memory very quickly. In this work, we introduce a novel way of producing fast and optimal motion plans for static environments by using a stepping neural network approach, called OracleNet. OracleNet uses Recurrent Neural Networks to determine end-to-end trajectories in an iterative manner that implicitly generates optimal motion plans with minimal loss in performance in a compact form. The algorithm is straightforward in implementation while consistently generating near-optimal paths in a single, iterative, end-to-end roll-out. In practice, OracleNet generally has fixed-time execution regardless of the configuration space complexity while outperforming popular pathfinding algorithms in complex environments and higher dimensions
Tasks	Motion Planning
Published	2019-04-25
URL	http://arxiv.org/abs/1904.11102v1
PDF	http://arxiv.org/pdf/1904.11102v1.pdf
PWC	https://paperswithcode.com/paper/neural-path-planning-fixed-time-near-optimal
Repo
Framework

Learning from Higher-Layer Feature Visualizations


Title	Learning from Higher-Layer Feature Visualizations
Authors	Konstantinos Nikolaidis, Stein Kristiansen, Vera Goebel, Thomas Plagemann
Abstract	Driven by the goal to enable sleep apnea monitoring and machine learning-based detection at home with small mobile devices, we investigate whether interpretation-based indirect knowledge transfer can be used to create classifiers with acceptable performance. Interpretation-based indirect knowledge transfer means that a classifier (student) learns from a synthetic dataset based on the knowledge representation from an already trained Deep Network (teacher). We use activation maximization to generate visualizations and create a synthetic dataset to train the student classifier. This approach has the advantage that student classifiers can be trained without access to the original training data. With experiments we investigate the feasibility of interpretation-based indirect knowledge transfer and its limitations. The student achieves an accuracy of 97.8% on MNIST (teacher accuracy: 99.3%) with a similar smaller architecture to that of the teacher. The student classifier achieves an accuracy of 86.1% and 89.5% for a subset of the Apnea-ECG dataset (teacher: 89.5% and 91.1%, respectively).
Tasks	Transfer Learning
Published	2019-03-06
URL	http://arxiv.org/abs/1903.02313v1
PDF	http://arxiv.org/pdf/1903.02313v1.pdf
PWC	https://paperswithcode.com/paper/learning-from-higher-layer-feature
Repo
Framework

Review of Algorithms for Compressive Sensing of Images


Title	Review of Algorithms for Compressive Sensing of Images
Authors	Yoni Sher
Abstract	We provide a comprehensive review of classical algorithms for compressive sensing of images, focused on Total variation methods, with a view to application in LiDAR systems. Our primary focus is providing a full review for beginners in the field, as well as simulating the kind of noise found in real LiDAR systems. To this end, we provide an overview of the theoretical background, a brief discussion of various considerations that come in to play in compressive sensing, and a standardized comparison of off-the-shelf methods, intended as a quick-start guide to choosing algorithms for compressive sensing applications.
Tasks	Compressive Sensing
Published	2019-08-05
URL	https://arxiv.org/abs/1908.01642v1
PDF	https://arxiv.org/pdf/1908.01642v1.pdf
PWC	https://paperswithcode.com/paper/review-of-algorithms-for-compressive-sensing
Repo
Framework

Image segmentation of liver stage malaria infection with spatial uncertainty sampling


Title	Image segmentation of liver stage malaria infection with spatial uncertainty sampling
Authors	Ava P. Soleimany, Harini Suresh, Jose Javier Gonzalez Ortiz, Divya Shanmugam, Nil Gural, John Guttag, Sangeeta N. Bhatia
Abstract	Global eradication of malaria depends on the development of drugs effective against the silent, yet obligate liver stage of the disease. The gold standard in drug development remains microscopic imaging of liver stage parasites in in vitro cell culture models. Image analysis presents a major bottleneck in this pipeline since the parasite has significant variability in size, shape, and density in these models. As with other highly variable datasets, traditional segmentation models have poor generalizability as they rely on hand-crafted features; thus, manual annotation of liver stage malaria images remains standard. To address this need, we develop a convolutional neural network architecture that utilizes spatial dropout sampling for parasite segmentation and epistemic uncertainty estimation in images of liver stage malaria. Our pipeline produces high-precision segmentations nearly identical to expert annotations, generalizes well on a diverse dataset of liver stage malaria parasites, and promotes independence between learned feature maps to model the uncertainty of generated predictions.
Tasks	Semantic Segmentation
Published	2019-11-30
URL	https://arxiv.org/abs/1912.00262v1
PDF	https://arxiv.org/pdf/1912.00262v1.pdf
PWC	https://paperswithcode.com/paper/image-segmentation-of-liver-stage-malaria
Repo
Framework

PAC Reinforcement Learning without Real-World Feedback


Title	PAC Reinforcement Learning without Real-World Feedback
Authors	Yuren Zhong, Aniket Anand Deshmukh, Clayton Scott
Abstract	This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (ROMDP), we formulate a theoretical framework for Sim-to-Real in the situation where feedback in the real world is not available. We establish real-world sample complexity guarantees that are smaller than what is currently known for directly (i.e., without access to simulators) learning a ROMDP with feedback.
Tasks
Published	2019-09-23
URL	https://arxiv.org/abs/1909.10449v3
PDF	https://arxiv.org/pdf/1909.10449v3.pdf
PWC	https://paperswithcode.com/paper/190910449
Repo
Framework