January 26, 2020

3035 words 15 mins read

Paper Group ANR 1613

Deep Morphological Hit-or-Miss Transform Neural Network. On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks. Towards an All-Purpose Content-Based Multimedia Information Retrieval System. X-SQL: reinforce schema representation with context. Large scale continuous-time mean-variance portfolio allocation via reinforcement l …

Deep Morphological Hit-or-Miss Transform Neural Network


Title	Deep Morphological Hit-or-Miss Transform Neural Network
Authors	Muhammad Aminul Islam, Bryce Murray, Andrew Buck, Derek T. Anderson, Grant Scott, Mihail Popescu, James Keller
Abstract	Neural networks have demonstrated breakthrough results in numerous application domains. While most architectures are built on the premise of convolution, alternative foundations like morphology are being explored for reasons like interpretability and its connection to the analysis and processing of geometric structures. Herein, we investigate new deep networks based on the morphological hit-or-miss transform. The hit-or-miss takes into account both foreground and background when measuring the fitness of a target shape in an image. We identify limitations of current hit-or-miss definitions, and we formulate an optimization problem to learn the transform. Our analysis shows that convolution, in fact, acts like a hit-miss transform through semantic interpretation of its filter differences. Analogous to the generalized hit-or-miss transform, we also introduce an extension of convolution and show that it outperforms conventional convolution on benchmark data sets. We conducted experiments on synthetic and benchmark data sets, and we show that the direct encoding hit-or-miss transform provides better interpretability on learned shapes consistent with objects whereas our morphologically inspired generalized convolution yields higher classification accuracy.
Tasks
Published	2019-12-04
URL	https://arxiv.org/abs/1912.02259v1
PDF	https://arxiv.org/pdf/1912.02259v1.pdf
PWC	https://paperswithcode.com/paper/deep-morphological-hit-or-miss-transform
Repo
Framework

On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks


Title	On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks
Authors	Peter Langeberg, Emilio Rafael Balda, Arash Behboodi, Rudolf Mathar
Abstract	Recently, there has been an abundance of works on designing Deep Neural Networks (DNNs) that are robust to adversarial examples. In particular, a central question is which features of DNNs influence adversarial robustness and, therefore, can be to used to design robust DNNs. In this work, this problem is studied through the lens of compression which is captured by the low-rank structure of weight matrices. It is first shown that adversarial training tends to promote simultaneously low-rank and sparse structure in the weight matrices of neural networks. This is measured through the notions of effective rank and effective sparsity. In the reverse direction, when the low rank structure is promoted by nuclear norm regularization and combined with sparsity inducing regularizations, neural networks show significantly improved adversarial robustness. The effect of nuclear norm regularization on adversarial robustness is paramount when it is applied to convolutional neural networks. Although still not competing with adversarial training, this result contributes to understanding the key properties of robust classifiers.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10371v1
PDF	http://arxiv.org/pdf/1901.10371v1.pdf
PWC	https://paperswithcode.com/paper/on-the-effect-of-low-rank-weights-on
Repo
Framework

Towards an All-Purpose Content-Based Multimedia Information Retrieval System


Title	Towards an All-Purpose Content-Based Multimedia Information Retrieval System
Authors	Ralph Gasser, Luca Rossetto, Heiko Schuldt
Abstract	The growth of multimedia collections - in terms of size, heterogeneity, and variety of media types - necessitates systems that are able to conjointly deal with several forms of media, especially when it comes to searching for particular objects. However, existing retrieval systems are organized in silos and treat different media types separately. As a consequence, retrieval across media types is either not supported at all or subject to major limitations. In this paper, we present vitrivr, a content-based multimedia information retrieval stack. As opposed to the keyword search approach implemented by most media management systems, vitrivr makes direct use of the object’s content to facilitate different types of similarity search, such as Query-by-Example or Query-by-Sketch, for and, most importantly, across different media types - namely, images, audio, videos, and 3D models. Furthermore, we introduce a new web-based user interface that enables easy-to-use, multimodal retrieval from and browsing in mixed media collections. The effectiveness of vitrivr is shown on the basis of a user study that involves different query and media types. To the best of our knowledge, the full vitrivr stack is unique in that it is the first multimedia retrieval system that seamlessly integrates support for four different types of media. As such, it paves the way towards an all-purpose, content-based multimedia information retrieval system.
Tasks	Information Retrieval
Published	2019-02-11
URL	http://arxiv.org/abs/1902.03878v1
PDF	http://arxiv.org/pdf/1902.03878v1.pdf
PWC	https://paperswithcode.com/paper/towards-an-all-purpose-content-based
Repo
Framework

X-SQL: reinforce schema representation with context


Title	X-SQL: reinforce schema representation with context
Authors	Pengcheng He, Yi Mao, Kaushik Chakrabarti, Weizhu Chen
Abstract	In this work, we present X-SQL, a new network architecture for the problem of parsing natural language to SQL query. X-SQL proposes to enhance the structural schema representation with the contextual output from BERT-style pre-training model, and together with type information to learn a new schema representation for down-stream tasks. We evaluated X-SQL on the WikiSQL dataset and show its new state-of-the-art performance.
Tasks
Published	2019-08-21
URL	https://arxiv.org/abs/1908.08113v1
PDF	https://arxiv.org/pdf/1908.08113v1.pdf
PWC	https://paperswithcode.com/paper/x-sql-reinforce-schema-representation-with
Repo
Framework

Large scale continuous-time mean-variance portfolio allocation via reinforcement learning


Title	Large scale continuous-time mean-variance portfolio allocation via reinforcement learning
Authors	Haoran Wang
Abstract	We propose to solve large scale Markowitz mean-variance (MV) portfolio allocation problem using reinforcement learning (RL). By adopting the recently developed continuous-time exploratory control framework, we formulate the exploratory MV problem in high dimensions. We further show the optimality of a multivariate Gaussian feedback policy, with time-decaying variance, in trading off exploration and exploitation. Based on a provable policy improvement theorem, we devise a scalable and data-efficient RL algorithm and conduct large scale empirical tests using data from the S&P 500 stocks. We found that our method consistently achieves over 10% annualized returns and it outperforms econometric methods and the deep RL method by large margins, for both long and medium terms of investment with monthly and daily trading.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11718v2
PDF	https://arxiv.org/pdf/1907.11718v2.pdf
PWC	https://paperswithcode.com/paper/large-scale-continuous-time-mean-variance
Repo
Framework

Predicting Future Opioid Incidences Today


Title	Predicting Future Opioid Incidences Today
Authors	Sandipan Choudhuri, Kaustav Basu, Kevin Thomas, Arunabha Sen
Abstract	According to the Center of Disease Control (CDC), the Opioid epidemic has claimed more than 72,000 lives in the US in 2017 alone. In spite of various efforts at the local, state and federal level, the impact of the epidemic is becoming progressively worse, as evidenced by the fact that the number of Opioid related deaths increased by 12.5% between 2016 and 2017. Predictive analytics can play an important role in combating the epidemic by providing decision making tools to stakeholders at multiple levels - from health care professionals to policy makers to first responders. Generating Opioid incidence heat maps from past data, aid these stakeholders to visualize the profound impact of the Opioid epidemic. Such post-fact creation of the heat map provides only retrospective information, and as a result, may not be as useful for preventive action in the current or future time-frames. In this paper, we present a novel deep neural architecture, which learns subtle spatio-temporal variations in Opioid incidences data and accurately predicts future heat maps. We evaluated the efficacy of our model on two open source datasets- (i) The Cincinnati Heroin Overdose dataset, and (ii) Connecticut Drug Related Death Dataset.
Tasks	Decision Making
Published	2019-06-20
URL	https://arxiv.org/abs/1906.08891v1
PDF	https://arxiv.org/pdf/1906.08891v1.pdf
PWC	https://paperswithcode.com/paper/predicting-future-opioid-incidences-today
Repo
Framework

FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning


Title	FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning
Authors	Jonas Pfeiffer, Christian M. Meyer, Claudia Schulz, Jan Kiesewetter, Jan Zottmann, Michael Sailer, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych
Abstract	Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data. Diagnosing is an exceptionally difficult skill to obtain but vital for many different professions (e.g., medical doctors, teachers). Previous case simulation systems are limited to multiple-choice questions and thus cannot give constructive individualized feedback on a student’s diagnostic reasoning process. Given initially only limited data, we leverage a (replaceable) NLP model to both support experts in their further data annotation with automatic suggestions, and we provide automatic feedback for students. We argue that because the central model consistently improves, our interactive approach encourages both students and instructors to recurrently use the tool, and thus accelerate the speed of data creation and annotation. We show results from two user studies on diagnostic reasoning in medicine and teacher education and outline how our system can be extended to further use cases.
Tasks
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11254v1
PDF	https://arxiv.org/pdf/1908.11254v1.pdf
PWC	https://paperswithcode.com/paper/famulus-interactive-annotation-and-feedback
Repo
Framework

Modeling a Hidden Dynamical System Using Energy Minimization and Kernel Density Estimates


Title	Modeling a Hidden Dynamical System Using Energy Minimization and Kernel Density Estimates
Authors	Trevor K. Karn, Steven Petrone, Christopher Griffin
Abstract	In this paper we develop a kernel density estimation (KDE) approach to modeling and forecasting recurrent trajectories on a compact manifold. For the purposes of this paper, a trajectory is a sequence of coordinates in a phase space defined by an underlying hidden dynamical system. Our work is inspired by earlier work on the use of KDE to detect shipping anomalies using high-density, high-quality automated information system (AIS) data as well as our own earlier work in trajectory modeling. We focus specifically on the sparse, noisy trajectory reconstruction problem in which the data are (i) sparsely sampled and (ii) subject to an imperfect observer that introduces noise. Under certain regularity assumptions, we show that the constructed estimator minimizes a specific energy function defined over the trajectory as the number of samples obtained grows.
Tasks	Density Estimation
Published	2019-04-08
URL	https://arxiv.org/abs/1904.05172v2
PDF	https://arxiv.org/pdf/1904.05172v2.pdf
PWC	https://paperswithcode.com/paper/modeling-a-hidden-dynamical-system-using
Repo
Framework

Locally Differentially Private Frequency Estimation with Consistency


Title	Locally Differentially Private Frequency Estimation with Consistency
Authors	Tianhao Wang, Milan Lopuhaä-Zwakenberg, Zitao Li, Boris Skoric, Ninghui Li
Abstract	Local Differential Privacy (LDP) protects user privacy from the data collector. LDP protocols have been increasingly deployed in the industry. A basic building block is frequency oracle (FO) protocols, which estimate frequencies of values. While several FO protocols have been proposed, the design goal does not lead to optimal results for answering many queries. In this paper, we show that adding post-processing steps to FO protocols by exploiting the knowledge that all individual frequencies should be non-negative and they sum up to one can lead to significantly better accuracy for a wide range of tasks, including frequencies of individual values, frequencies of the most frequent values, and frequencies of subsets of values. We consider 10 different methods that exploit this knowledge differently. We establish theoretical relationships between some of them and conducted extensive experimental evaluations to understand which methods should be used for different query tasks.
Tasks
Published	2019-05-20
URL	https://arxiv.org/abs/1905.08320v2
PDF	https://arxiv.org/pdf/1905.08320v2.pdf
PWC	https://paperswithcode.com/paper/locally-differentially-private-frequency
Repo
Framework

Extraction of Relevant Images for Boilerplate Removal in Web Browsers


Title	Extraction of Relevant Images for Boilerplate Removal in Web Browsers
Authors	Joy Bose
Abstract	Boilerplate refers to unwanted and repeated parts of a webpage (such as ads or table of contents) that distracts the user from reading the core content of the webpage, such as a news article. Accurate detection and removal of boilerplate content from a webpage can enable the users to have a clutter free view of the webpage or news article. This can be useful in features like reader mode in web browsers. Current implementations of reader mode in web browsers such as Firefox, Chrome and Edge perform reasonably well for textual content in webpages. However, they are mostly heuristic based and not flexible when the webpage content is dynamic. Also they often do not perform well for removing boilerplate content in the form of images and multimedia in webpages. For detection of boilerplate images, one needs to have knowledge of the actual layout of the images in the webpage, which is only possible when the webpage is rendered. In this paper we discuss some of the issues in relevant image extraction. We also present the design of a testing framework to measure accuracy and a classifier to extract relevant images by leveraging a headless browser solution that gives the rendering information for images.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/2001.04338v2
PDF	https://arxiv.org/pdf/2001.04338v2.pdf
PWC	https://paperswithcode.com/paper/extraction-of-relevant-images-for-boilerplate
Repo
Framework

BitSplit-Net: Multi-bit Deep Neural Network with Bitwise Activation Function


Title	BitSplit-Net: Multi-bit Deep Neural Network with Bitwise Activation Function
Authors	Hyungjun Kim, Yulhwa Kim, Sungju Ryu, Jae-Joon Kim
Abstract	Significant computational cost and memory requirements for deep neural networks (DNNs) make it difficult to utilize DNNs in resource-constrained environments. Binary neural network (BNN), which uses binary weights and binary activations, has been gaining interests for its hardware-friendly characteristics and minimal resource requirement. However, BNN usually suffers from accuracy degradation. In this paper, we introduce “BitSplit-Net”, a neural network which maintains the hardware-friendly characteristics of BNN while improving accuracy by using multi-bit precision. In BitSplit-Net, each bit of multi-bit activations propagates independently throughout the network before being merged at the end of the network. Thus, each bit path of the BitSplit-Net resembles BNN and hardware friendly features of BNN, such as bitwise binary activation function, are preserved in our scheme. We demonstrate that the BitSplit version of LeNet-5, VGG-9, AlexNet, and ResNet-18 can be trained to have similar classification accuracy at a lower computational cost compared to conventional multi-bit networks with low bit precision (<= 4-bit). We further evaluate BitSplit-Net on GPU with custom CUDA kernel, showing that BitSplit-Net can achieve better hardware performance in comparison to conventional multi-bit networks.
Tasks
Published	2019-03-23
URL	http://arxiv.org/abs/1903.09807v1
PDF	http://arxiv.org/pdf/1903.09807v1.pdf
PWC	https://paperswithcode.com/paper/bitsplit-net-multi-bit-deep-neural-network
Repo
Framework

Predicting the Voltage Distribution for Low Voltage Networks using Deep Learning


Title	Predicting the Voltage Distribution for Low Voltage Networks using Deep Learning
Authors	Maizura Mokhtar, Valentin Robu, David Flynn, Ciaran Higgins, Jim Whyte, Caroline Loughran, Fiona Fulton
Abstract	The energy landscape for the Low-Voltage (LV) networks are beginning to change; changes resulted from the increase penetration of renewables and/or the predicted increase of electric vehicles charging at home. The previously passive `fit-and-forget' approach to LV network management will be inefficient to ensure its effective operations. A more adaptive approach is required that includes the prediction of risk and capacity of the circuits. Many of the proposed methods require full observability of the networks, motivating the installations of smart meters and advance metering infrastructure in many countries. However, the expectation of` perfect data’ is unrealistic in operational reality. Smart meter (SM) roll-out can have its issues, which may resulted in low-likelihood of full SM coverage for all LV networks. This, together with privacy requirements that limit the availability of high granularity demand power data have resulted in the low uptake of many of the presented methods. To address this issue, Deep Learning Neural Network is proposed to predict the voltage distribution with partial SM coverage. The results show that SM measurements from key locations are sufficient for effective prediction of voltage distribution.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08374v1
PDF	https://arxiv.org/pdf/1906.08374v1.pdf
PWC	https://paperswithcode.com/paper/predicting-the-voltage-distribution-for-low
Repo
Framework

Learning to Reason: Leveraging Neural Networks for Approximate DNF Counting


Title	Learning to Reason: Leveraging Neural Networks for Approximate DNF Counting
Authors	Ralph Abboud, Ismail Ilkan Ceylan, Thomas Lukasiewicz
Abstract	Weighted model counting (WMC) has emerged as a prevalent approach for probabilistic inference. In its most general form, WMC is #P-hard. Weighted DNF counting (weighted #DNF) is a special case, where approximations with probabilistic guarantees are obtained in O(nm), where n denotes the number of variables, and m the number of clauses of the input DNF, but this is not scalable in practice. In this paper, we propose a neural model counting approach for weighted #DNF that combines approximate model counting with deep learning, and accurately approximates model counts in linear time when width is bounded. We conduct experiments to validate our method, and show that our model learns and generalizes very well to large-scale #DNF instances.
Tasks
Published	2019-04-04
URL	https://arxiv.org/abs/1904.02688v5
PDF	https://arxiv.org/pdf/1904.02688v5.pdf
PWC	https://paperswithcode.com/paper/learning-to-reason-leveraging-neural-networks
Repo
Framework

Physical Symmetries Embedded in Neural Networks


Title	Physical Symmetries Embedded in Neural Networks
Authors	M. Mattheakis, P. Protopapas, D. Sondak, M. Di Giovanni, E. Kaxiras
Abstract	Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network.
Tasks
Published	2019-04-18
URL	https://arxiv.org/abs/1904.08991v3
PDF	https://arxiv.org/pdf/1904.08991v3.pdf
PWC	https://paperswithcode.com/paper/physical-symmetries-embedded-in-neural
Repo
Framework

Visualization and Interpretation of Latent Spaces for Controlling Expressive Speech Synthesis through Audio Analysis


Title	Visualization and Interpretation of Latent Spaces for Controlling Expressive Speech Synthesis through Audio Analysis
Authors	Noé Tits, Fengna Wang, Kevin El Haddad, Vincent Pagel, Thierry Dutoit
Abstract	The field of Text-to-Speech has experienced huge improvements last years benefiting from deep learning techniques. Producing realistic speech becomes possible now. As a consequence, the research on the control of the expressiveness, allowing to generate speech in different styles or manners, has attracted increasing attention lately. Systems able to control style have been developed and show impressive results. However the control parameters often consist of latent variables and remain complex to interpret. In this paper, we analyze and compare different latent spaces and obtain an interpretation of their influence on expressive speech. This will enable the possibility to build controllable speech synthesis systems with an understandable behaviour.
Tasks	Latent Variable Models, Learning Network Representations, Speech Synthesis, Text-To-Speech Synthesis
Published	2019-03-27
URL	http://arxiv.org/abs/1903.11570v1
PDF	http://arxiv.org/pdf/1903.11570v1.pdf
PWC	https://paperswithcode.com/paper/visualization-and-interpretation-of-latent
Repo
Framework