January 30, 2020

3141 words 15 mins read

Paper Group ANR 271

Deep 2FBSDEs For Systems With Control Multiplicative Noise. The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning. LS-SVR as a Bayesian RBF network. FortuneTeller: Predicting Microarchitectural Attacks via Unsupervised Deep Learning. Love Me, Love Me, Say (and Write!) that You Love Me: E …

Deep 2FBSDEs For Systems With Control Multiplicative Noise


Title	Deep 2FBSDEs For Systems With Control Multiplicative Noise
Authors	Marcus A Pereira, Ziyi Wang, Tianrong Chen, Emily Reed, Evangelos A Theodorou
Abstract	We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellmanpartial differential equations. Such PDEs arise when one considers stochastic dynamics characterized by uncertainties that are additive and control multiplicative. Stochastic models with the aforementioned characteristics have been used in computational neuroscience, biology, finance and aerospace systems and provide a more accurate representation of actuation than models with additive uncertainty. Previous literature has established the inadequacy of the linear HJB theory and instead rely on a non-linear Feynman-Kac lemma resulting in a second order forward-backward stochastic differential equations representation. However, the proposed solutions that use this representation suffer from compounding errors and computational complexity leading to lack of scalability. In this paper, we propose a deep learning based algorithm that leverages the second order Forward-Backward SDE representation and LSTM based recurrent neural networks to not only solve such Stochastic Optimal Control problems but also overcome the problems faced by previous approaches and scales well to high dimensional systems. The resulting control algorithm is tested on non-linear systems in robotics and biomechanics to demonstrate feasibility and out-performance against previous methods.
Tasks
Published	2019-06-11
URL	https://arxiv.org/abs/1906.04762v4
PDF	https://arxiv.org/pdf/1906.04762v4.pdf
PWC	https://paperswithcode.com/paper/deep-2fbsdes-for-systems-with-control
Repo
Framework

The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning


Title	The Twin-System Approach as One Generic Solution for XAI: An Overview of ANN-CBR Twins for Explaining Deep Learning
Authors	Mark T. Keane, Eoin M. Kenny
Abstract	The notion of twin systems is proposed to address the eXplainable AI (XAI) problem, where an uninterpretable black-box system is mapped to a white-box ‘twin’ that is more interpretable. In this short paper, we overview very recent work that advances a generic solution to the XAI problem, the so called twin system approach. The most popular twinning in the literature is that between an Artificial Neural Networks (ANN ) as a black box and Case Based Reasoning (CBR) system as a white-box, where the latter acts as an interpretable proxy for the former. We outline how recent work reviving this idea has applied it to deep learning methods. Furthermore, we detail the many fruitful directions in which this work may be taken; such as, determining the most (i) accurate feature-weighting methods to be used, (ii) appropriate deployments for explanatory cases, (iii) useful cases of explanatory value to users.
Tasks
Published	2019-05-20
URL	https://arxiv.org/abs/1905.08069v1
PDF	https://arxiv.org/pdf/1905.08069v1.pdf
PWC	https://paperswithcode.com/paper/the-twin-system-approach-as-one-generic
Repo
Framework

LS-SVR as a Bayesian RBF network


Title	LS-SVR as a Bayesian RBF network
Authors	Diego P. P. Mesquita, Luis A. Freitas, João P. P. Gomes, César L. C. Mattos
Abstract	We show theoretical similarities between the Least Squares Support Vector Regression (LS-SVR) model with a Radial Basis Functions (RBF) kernel and maximum a posteriori (MAP) inference on Bayesian RBF networks with a specific Gaussian prior on the regression weights. Although previous works have pointed out similar expressions between those learning approaches, we explicit and formally state the existing correspondences. We empirically demonstrate our result by performing computational experiments with standard regression benchmarks. Our findings open a range of possibilities to improve LS-SVR by borrowing strength from well-established developments in Bayesian methodology.
Tasks
Published	2019-05-01
URL	https://arxiv.org/abs/1905.00332v2
PDF	https://arxiv.org/pdf/1905.00332v2.pdf
PWC	https://paperswithcode.com/paper/ls-svr-as-a-bayesian-rbf-network
Repo
Framework

FortuneTeller: Predicting Microarchitectural Attacks via Unsupervised Deep Learning


Title	FortuneTeller: Predicting Microarchitectural Attacks via Unsupervised Deep Learning
Authors	Berk Gulmezoglu, Ahmad Moghimi, Thomas Eisenbarth, Berk Sunar
Abstract	The growing security threat of microarchitectural attacks underlines the importance of robust security sensors and detection mechanisms at the hardware level. While there are studies on runtime detection of cache attacks, a generic model to consider the broad range of existing and future attacks is missing. Unfortunately, previous approaches only consider either a single attack variant, e.g. Prime+Probe, or specific victim applications such as cryptographic implementations. Furthermore, the state-of-the art anomaly detection methods are based on coarse-grained statistical models, which are not successful to detect anomalies in a large-scale real world systems. Thanks to the memory capability of advanced Recurrent Neural Networks (RNNs) algorithms, both short and long term dependencies can be learned more accurately. Therefore, we propose FortuneTeller, which for the first time leverages the superiority of RNNs to learn complex execution patterns and detects unseen microarchitectural attacks in real world systems. FortuneTeller models benign workload pattern from a microarchitectural standpoint in an unsupervised fashion, and then, it predicts how upcoming benign executions are supposed to behave. Potential attacks and malicious behaviors will be detected automatically, when there is a discrepancy between the predicted execution pattern and the runtime observation. We implement FortuneTeller based on the available hardware performance counters on Intel processors and it is trained with 10 million samples obtained from benign applications. For the first time, the latest attacks such as Meltdown, Spectre, Rowhammer and Zombieload are detected with one trained model and without observing these attacks during the training. We show that FortuneTeller achieves F-score of 0.9970.
Tasks	Anomaly Detection
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03651v1
PDF	https://arxiv.org/pdf/1907.03651v1.pdf
PWC	https://paperswithcode.com/paper/fortuneteller-predicting-microarchitectural
Repo
Framework

Love Me, Love Me, Say (and Write!) that You Love Me: Enriching the WASABI Song Corpus with Lyrics Annotations


Title	Love Me, Love Me, Say (and Write!) that You Love Me: Enriching the WASABI Song Corpus with Lyrics Annotations
Authors	Michael Fell, Elena Cabrio, Elmahdi Korfed, Michel Buffa, Fabien Gandon
Abstract	We present the WASABI Song Corpus, a large corpus of songs enriched with metadata extracted from music databases on the Web, and resulting from the processing of song lyrics and from audio analysis. More specifically, given that lyrics encode an important part of the semantics of a song, we focus here on the description of the methods we proposed to extract relevant information from the lyrics, such as their structure segmentation, their topics, the explicitness of the lyrics content, the salient passages of a song and the emotions conveyed. The creation of the resource is still ongoing: so far, the corpus contains 1.73M songs with lyrics (1.41M unique lyrics) annotated at different levels with the output of the above mentioned methods. Such corpus labels and the provided methods can be exploited by music search engines and music professionals (e.g. journalists, radio presenters) to better handle large collections of lyrics, allowing an intelligent browsing, categorization and segmentation recommendation of songs.
Tasks
Published	2019-12-05
URL	https://arxiv.org/abs/1912.02477v2
PDF	https://arxiv.org/pdf/1912.02477v2.pdf
PWC	https://paperswithcode.com/paper/love-me-love-me-say-and-write-that-you-love
Repo
Framework

Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent


Title	Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent
Authors	Edward Lockhart, Marc Lanctot, Julien Pérolat, Jean-Baptiste Lespiau, Dustin Morrill, Finbarr Timbers, Karl Tuyls
Abstract	In this paper, we present exploitability descent, a new algorithm to compute approximate equilibria in two-player zero-sum extensive-form games with imperfect information, by direct policy optimization against worst-case opponents. We prove that when following this optimization, the exploitability of a player’s strategy converges asymptotically to zero, and hence when both players employ this optimization, the joint policies converge to a Nash equilibrium. Unlike fictitious play (XFP) and counterfactual regret minimization (CFR), our convergence result pertains to the policies being optimized rather than the average policies. Our experiments demonstrate convergence rates comparable to XFP and CFR in four benchmark games in the tabular case. Using function approximation, we find that our algorithm outperforms the tabular version in two of the games, which, to the best of our knowledge, is the first such result in imperfect information games among this class of algorithms.
Tasks
Published	2019-03-13
URL	https://arxiv.org/abs/1903.05614v3
PDF	https://arxiv.org/pdf/1903.05614v3.pdf
PWC	https://paperswithcode.com/paper/computing-approximate-equilibria-in
Repo
Framework

Artificial Intelligence Approaches


Title	Artificial Intelligence Approaches
Authors	Yingjie Hu, Wenwen Li, Dawn Wright, Orhun Aydin, Daniel Wilson, Omar Maher, Mansour Raad
Abstract	Artificial Intelligence (AI) has received tremendous attention from academia, industry, and the general public in recent years. The integration of geography and AI, or GeoAI, provides novel approaches for addressing a variety of problems in the natural environment and our human society. This entry briefly reviews the recent development of AI with a focus on machine learning and deep learning approaches. We discuss the integration of AI with geography and particularly geographic information science, and present a number of GeoAI applications and possible future directions.
Tasks
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10345v1
PDF	https://arxiv.org/pdf/1908.10345v1.pdf
PWC	https://paperswithcode.com/paper/artificial-intelligence-approaches
Repo
Framework


Title	Understanding the Political Ideology of Legislators from Social Media Images
Authors	Nan Xi, Di Ma, Marcus Liou, Zachary C. Steinert-Threlkeld, Jason Anastasopoulos, Jungseock Joo
Abstract	In this paper, we seek to understand how politicians use images to express ideological rhetoric through Facebook images posted by members of the U.S. House and Senate. In the era of social media, politics has become saturated with imagery, a potent and emotionally salient form of political rhetoric which has been used by politicians and political organizations to influence public sentiment and voting behavior for well over a century. To date, however, little is known about how images are used as political rhetoric. Using deep learning techniques to automatically predict Republican or Democratic party affiliation solely from the Facebook photographs of the members of the 114th U.S. Congress, we demonstrate that predicted class probabilities from our model function as an accurate proxy of the political ideology of images along a left-right (liberal-conservative) dimension. After controlling for the gender and race of politicians, our method achieves an accuracy of 59.28% from single photographs and 82.35% when aggregating scores from multiple photographs (up to 150) of the same person. To better understand image content distinguishing liberal from conservative images, we also perform in-depth content analyses of the photographs. Our findings suggest that conservatives tend to use more images supporting status quo political institutions and hierarchy maintenance, featuring individuals from dominant social groups, and displaying greater happiness than liberals.
Tasks
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09594v1
PDF	https://arxiv.org/pdf/1907.09594v1.pdf
PWC	https://paperswithcode.com/paper/understanding-the-political-ideology-of
Repo
Framework

GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems


Title	GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems
Authors	Ankit Raj, Yuqi Li, Yoram Bresler
Abstract	A Generative Adversarial Network (GAN) with generator $G$ trained to model the prior of images has been shown to perform better than sparsity-based regularizers in ill-posed inverse problems. Here, we propose a new method of deploying a GAN-based prior to solve linear inverse problems using projected gradient descent (PGD). Our method learns a network-based projector for use in the PGD algorithm, eliminating expensive computation of the Jacobian of $G$. Experiments show that our approach provides a speed-up of $60\text{-}80\times$ over earlier GAN-based recovery methods along with better accuracy. Our main theoretical result is that if the measurement matrix is moderately conditioned on the manifold range($G$) and the projector is $\delta$-approximate, then the algorithm is guaranteed to reach $O(\delta)$ reconstruction error in $O(log(1/\delta))$ steps in the low noise regime. Additionally, we propose a fast method to design such measurement matrices for a given $G$. Extensive experiments demonstrate the efficacy of this method by requiring $5\text{-}10\times$ fewer measurements than random Gaussian measurement matrices for comparable recovery performance. Because the learning of the GAN and projector is decoupled from the measurement operator, our GAN-based projector and recovery algorithm are applicable without retraining to all linear inverse problems, as confirmed by experiments on compressed sensing, super-resolution, and inpainting.
Tasks	Super-Resolution
Published	2019-02-26
URL	https://arxiv.org/abs/1902.09698v2
PDF	https://arxiv.org/pdf/1902.09698v2.pdf
PWC	https://paperswithcode.com/paper/gan-based-projector-for-faster-recovery-in
Repo
Framework

Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning


Title	Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Authors	Pradeep Dasigi, Nelson F. Liu, Ana Marasović, Noah A. Smith, Matt Gardner
Abstract	Machine comprehension of texts longer than a single sentence often requires coreference resolution. However, most current reading comprehension benchmarks do not contain complex coreferential phenomena and hence fail to evaluate the ability of models to resolve coreference. We present a new crowdsourced dataset containing more than 24K span-selection questions that require resolving coreference among entities in over 4.7K English paragraphs from Wikipedia. Obtaining questions focused on such phenomena is challenging, because it is hard to avoid lexical cues that shortcut complex reasoning. We deal with this issue by using a strong baseline model as an adversary in the crowdsourcing loop, which helps crowdworkers avoid writing questions with exploitable surface cues. We show that state-of-the-art reading comprehension models perform significantly worse than humans on this benchmark—the best model performance is 70.5 F1, while the estimated human performance is 93.4 F1.
Tasks	Coreference Resolution, Reading Comprehension
Published	2019-08-16
URL	https://arxiv.org/abs/1908.05803v2
PDF	https://arxiv.org/pdf/1908.05803v2.pdf
PWC	https://paperswithcode.com/paper/quoref-a-reading-comprehension-dataset-with
Repo
Framework

A literature survey of matrix methods for data science


Title	A literature survey of matrix methods for data science
Authors	Martin Stoll
Abstract	Efficient numerical linear algebra is a core ingredient in many applications across almost all scientific and industrial disciplines. With this survey we want to illustrate that numerical linear algebra has played and is playing a crucial role in enabling and improving data science computations with many new developments being fueled by the availability of data and computing resources.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.07896v1
PDF	https://arxiv.org/pdf/1912.07896v1.pdf
PWC	https://paperswithcode.com/paper/a-literature-survey-of-matrix-methods-for
Repo
Framework

Proppy: A System to Unmask Propaganda in Online News


Title	Proppy: A System to Unmask Propaganda in Online News
Authors	Alberto Barrón-Cedeño, Giovanni Da San Martino, Israa Jaradat, Preslav Nakov
Abstract	We present proppy, the first publicly available real-world, real-time propaganda detection system for online news, which aims at raising awareness, thus potentially limiting the impact of propaganda and helping fight disinformation. The system constantly monitors a number of news sources, deduplicates and clusters the news into events, and organizes the articles about an event on the basis of the likelihood that they contain propagandistic content. The system is trained on known propaganda sources using a variety of stylistic features. The evaluation results on a standard dataset show state-of-the-art results for propaganda detection.
Tasks
Published	2019-12-14
URL	https://arxiv.org/abs/1912.06810v1
PDF	https://arxiv.org/pdf/1912.06810v1.pdf
PWC	https://paperswithcode.com/paper/proppy-a-system-to-unmask-propaganda-in
Repo
Framework

HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision


Title	HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Authors	Zhen Dong, Zhewei Yao, Amir Gholami, Michael Mahoney, Kurt Keutzer
Abstract	Model size and inference speed/power have become a major challenge in the deployment of Neural Networks for many applications. A promising approach to address these problems is quantization. However, uniformly quantizing a model to ultra low precision leads to significant accuracy degradation. A novel solution for this is to use mixed-precision quantization, as some parts of the network may allow lower precision as compared to other layers. However, there is no systematic way to determine the precision of different layers. A brute force approach is not feasible for deep networks, as the search space for mixed-precision is exponential in the number of layers. Another challenge is a similar factorial complexity for determining block-wise fine-tuning order when quantizing the model to a target precision. Here, we introduce Hessian AWare Quantization (HAWQ), a novel second-order quantization method to address these problems. HAWQ allows for the automatic selection of the relative quantization precision of each layer, based on the layer’s Hessian spectrum. Moreover, HAWQ provides a deterministic fine-tuning order for quantizing layers, based on second-order information. We show the results of our method on Cifar-10 using ResNet20, and on ImageNet using Inception-V3, ResNet50 and SqueezeNext models. Comparing HAWQ with state-of-the-art shows that we can achieve similar/better accuracy with $8\times$ activation compression ratio on ResNet20, as compared to DNAS~\cite{wu2018mixed}, and up to $1%$ higher accuracy with up to $14%$ smaller models on ResNet50 and Inception-V3, compared to recently proposed methods of RVQuant~\cite{park2018value} and HAQ~\cite{wang2018haq}. Furthermore, we show that we can quantize SqueezeNext to just 1MB model size while achieving above $68%$ top1 accuracy on ImageNet.
Tasks	Quantization
Published	2019-04-29
URL	http://arxiv.org/abs/1905.03696v1
PDF	http://arxiv.org/pdf/1905.03696v1.pdf
PWC	https://paperswithcode.com/paper/190503696
Repo
Framework

Learning to Synthesize: Robust Phase Retrieval at Low Photon counts


Title	Learning to Synthesize: Robust Phase Retrieval at Low Photon counts
Authors	Mo Deng, Shuai Li, Alexandre Goy, Iksung Kang, George Barbastathis
Abstract	The quality of inverse problem solutions obtained through deep learning [Barbastathis et al, 2019] is limited by the nature of the priors learned from examples presented during the training phase. In the case of quantitative phase retrieval [Sinha et al, 2017, Goy et al, 2019], in particular, spatial frequencies that are underrepresented in the training database, most often at the high band, tend to be suppressed in the reconstruction. Ad hoc solutions have been proposed, such as pre-amplifying the high spatial frequencies in the examples [Li et al, 2018]; however, while that strategy improves resolution, it also leads to high-frequency artifacts as well as low-frequency distortions in the reconstructions. Here, we present a new approach that learns separately how to handle the two frequency bands, low and high; and also learns how to synthesize these two bands into the full-band reconstructions. We show that this “learning to synthesize” (LS) method yields phase reconstructions of high spatial resolution and artifact-free; and it is also resilient to high-noise conditions, e.g. in the case of very low photon flux. In addition to the problem of quantitative phase retrieval, the LS method is applicable, in principle, to any inverse problem where the forward operator treats different frequency bands unevenly, i.e. is ill-posed.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11713v1
PDF	https://arxiv.org/pdf/1907.11713v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-synthesize-robust-phase-retrieval
Repo
Framework

A Survey of Automated Programming Hint Generation – The HINTS Framework


Title	A Survey of Automated Programming Hint Generation – The HINTS Framework
Authors	Jessica McBroom, Irena Koprinska, Kalina Yacef
Abstract	Automated tutoring systems offer the flexibility and scalability necessary to facilitate the provision of high quality and universally accessible programming education. In order to realise the full potential of these systems, recent work has proposed a diverse range of techniques for automatically generating hints to assist students with programming exercises. This paper integrates these apparently disparate approaches into a coherent whole. Specifically, it emphasises that all hint techniques can be understood as a series of simpler components with similar properties. Using this insight, it presents a simple framework for describing such techniques, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework, and it surveys recent work in the context of this framework. It discusses important implications of the survey and framework, including the need to further develop evaluation methods and the importance of considering hint technique components when designing, communicating and evaluating hint systems. Ultimately, this paper is designed to facilitate future opportunities for the development, extension and comparison of automated programming hint techniques in order to maximise their educational potential.
Tasks
Published	2019-08-30
URL	https://arxiv.org/abs/1908.11566v1
PDF	https://arxiv.org/pdf/1908.11566v1.pdf
PWC	https://paperswithcode.com/paper/a-survey-of-automated-programming-hint
Repo
Framework