February 1, 2020

3297 words 16 mins read

Paper Group AWR 240

The Ramanujan Machine: Automatically Generated Conjectures on Fundamental Constants. Ranked List Loss for Deep Metric Learning. Fast Enhanced CT Metal Artifact Reduction using Data Domain Deep Learning. Trouble with the Curve: Predicting Future MLB Players Using Scouting Reports. Explaining Deep Learning-Based Networked Systems. Learning Partial Di …

The Ramanujan Machine: Automatically Generated Conjectures on Fundamental Constants


Title	The Ramanujan Machine: Automatically Generated Conjectures on Fundamental Constants
Authors	Gal Raayoni, George Pisha, Yahel Manor, Uri Mendlovic, Doron Haviv, Yaron Hadad, Ido Kaminer
Abstract	Fundamental mathematical constants like $e$ and $\pi$ are ubiquitous in diverse fields of science, from abstract mathematics and geometry to physics, biology and chemistry. Nevertheless, for centuries new mathematical formulas relating fundamental constants have been scarce and usually discovered sporadically. In this paper we propose a novel and systematic approach that leverages algorithms for deriving mathematical formulas for fundamental constants and help reveal their underlying structure. Our algorithms find dozens of well-known as well as previously unknown continued fraction representations of $\pi$, $e$, and the Riemann zeta function values. Two conjectures produced by our algorithm, along with many others, are: \begin{equation} \frac{e}{e-2} = 4 - \frac{1}{5-\frac{2}{6-\frac{3}{7-\frac{4}{8-\ldots}}}} \quad\quad,\quad\quad \frac{4}{3\pi-8} = 3-\frac{1\cdot1}{6-\frac{2\cdot3}{9-\frac{3\cdot5}{12-\frac{4\cdot 7}{15-\ldots}}}} \end{equation} We present two algorithms that proved useful in finding conjectures: a variant of the Meet-In-The-Middle (MITM) algorithm and a Gradient Descent (GD) tailored to the recurrent structure of continued fractions. Both algorithms are based on matching numerical values and thus they conjecture formulas without providing proofs and without requiring any prior knowledge on any underlaying mathematical structure. This approach is especially attractive for fundamental constants for which no mathematical structure is known, as it reverses the conventional approach of sequential logic in formal proofs. Instead, our work supports a different conceptual approach for research: computer algorithms utilizing numerical data to unveil mathematical structures, thus trying to play the role of intuition of great mathematicians of the past, providing leads to new mathematical research.
Tasks
Published	2019-06-29
URL	https://arxiv.org/abs/1907.00205v3
PDF	https://arxiv.org/pdf/1907.00205v3.pdf
PWC	https://paperswithcode.com/paper/the-ramanujan-machine-automatically-generated
Repo	https://github.com/Omega97/Ramanujan_Machine
Framework	none

Ranked List Loss for Deep Metric Learning


Title	Ranked List Loss for Deep Metric Learning
Authors	Xinshao Wang, Yang Hua, Elyor Kodirov, Guosheng Hu, Romain Garnier, Neil M. Robertson
Abstract	The objective of deep metric learning (DML) is to learn embeddings that can capture semantic similarity information among data points. Existing pairwise or tripletwise loss functions used in DML are known to suffer from slow convergence due to a large proportion of trivial pairs or triplets as the model improves. To improve this, rankingmotivated structured losses are proposed recently to incorporate multiple examples and exploit the structured information among them. They converge faster and achieve state-of-the-art performance. In this work, we present two limitations of existing ranking-motivated structured losses and propose a novel ranked list loss to solve both of them. First, given a query, only a fraction of data points is incorporated to build the similarity structure. Consequently, some useful examples are ignored and the structure is less informative. To address this, we propose to build a setbased similarity structure by exploiting all instances in the gallery. The samples are split into a positive set and a negative set. Our objective is to make the query closer to the positive set than to the negative set by a margin. Second, previous methods aim to pull positive pairs as close as possible in the embedding space. As a result, the intraclass data distribution might be dropped. In contrast, we propose to learn a hypersphere for each class in order to preserve the similarity structure inside it. Our extensive experiments show that the proposed method achieves state-of-the-art performance on three widely used benchmarks.
Tasks	Metric Learning, Semantic Similarity, Semantic Textual Similarity
Published	2019-03-08
URL	https://arxiv.org/abs/1903.03238v5
PDF	https://arxiv.org/pdf/1903.03238v5.pdf
PWC	https://paperswithcode.com/paper/ranked-list-loss-for-deep-metric-learning
Repo	https://github.com/Qidian213/Ranked_Person_ReID
Framework	pytorch

Fast Enhanced CT Metal Artifact Reduction using Data Domain Deep Learning


Title	Fast Enhanced CT Metal Artifact Reduction using Data Domain Deep Learning
Authors	Muhammad Usman Ghani, W. Clem Karl
Abstract	Filtered back projection (FBP) is the most widely used method for image reconstruction in X-ray computed tomography (CT) scanners. The presence of hyper-dense materials in a scene, such as metals, can strongly attenuate X-rays, producing severe streaking artifacts in the reconstruction. These metal artifacts can greatly limit subsequent object delineation and information extraction from the images, restricting their diagnostic value. This problem is particularly acute in the security domain, where there is great heterogeneity in the objects that can appear in a scene, highly accurate decisions must be made quickly. The standard practical approaches to reducing metal artifacts in CT imagery are either simplistic non-adaptive interpolation-based projection data completion methods or direct image post-processing methods. These standard approaches have had limited success. Motivated primarily by security applications, we present a new deep-learning-based metal artifact reduction (MAR) approach that tackles the problem in the projection data domain. We treat the projection data corresponding to metal objects as missing data and train an adversarial deep network to complete the missing data in the projection domain. The subsequent complete projection data is then used with FBP to reconstruct image intended to be free of artifacts. This new approach results in an end-to-end MAR algorithm that is computationally efficient so practical and fits well into existing CT workflows allowing easy adoption in existing scanners. Training deep networks can be challenging, and another contribution of our work is to demonstrate that training data generated using an accurate X-ray simulation can be used to successfully train the deep network when combined with transfer learning using limited real data sets. We demonstrate the effectiveness and potential of our algorithm on simulated and real examples.
Tasks	Computed Tomography (CT), Image Reconstruction, Metal Artifact Reduction, Transfer Learning
Published	2019-04-09
URL	https://arxiv.org/abs/1904.04691v3
PDF	https://arxiv.org/pdf/1904.04691v3.pdf
PWC	https://paperswithcode.com/paper/fast-accurate-ct-metal-artifact-reduction
Repo	https://github.com/mughanibu/DeepMAR
Framework	none

Trouble with the Curve: Predicting Future MLB Players Using Scouting Reports


Title	Trouble with the Curve: Predicting Future MLB Players Using Scouting Reports
Authors	Jacob Danovitch
Abstract	In baseball, a scouting report profiles a player’s characteristics and traits, usually intended for use in player valuation. This work presents a first-of-its-kind dataset of almost 10,000 scouting reports for minor league, international, and draft prospects. Compiled from articles posted to MLB.com and Fangraphs.com, each report consists of a written description of the player, numerical grades for several skills, and unique IDs to reference their profiles on popular resources like MLB.com, FanGraphs, and Baseball-Reference. With this dataset, we employ several deep neural networks to predict if minor league players will make the MLB given their scouting report. We open-source this data to share with the community, and present a web application demonstrating language variations in the reports of successful and unsuccessful prospects.
Tasks
Published	2019-10-21
URL	https://arxiv.org/abs/1910.12622v1
PDF	https://arxiv.org/pdf/1910.12622v1.pdf
PWC	https://paperswithcode.com/paper/trouble-with-the-curve-predicting-future-mlb
Repo	https://github.com/jacobdanovitch/Trouble-With-The-Curve
Framework	pytorch

Explaining Deep Learning-Based Networked Systems


Title	Explaining Deep Learning-Based Networked Systems
Authors	Zili Meng, Minhu Wang, Jiasong Bai, Mingwei Xu, Hongzi Mao, Hongxin Hu
Abstract	While many deep learning (DL)-based networking systems have demonstrated superior performance, the underlying Deep Neural Networks (DNNs) remain blackboxes and stay uninterpretable for network operators. The lack of interpretability makes DL-based networking systems prohibitive to deploy in practice. In this paper, we propose Metis, a framework that provides interpretability for two general categories of networking problems spanning local and global control. Accordingly, Metis introduces two different interpretation methods based on decision tree and hypergraph, where it converts DNN policies to interpretable rule-based controllers and highlight critical components based on analysis over hypergraph. We evaluate Metis over several state-of-the-art DL-based networking systems and show that Metis provides human-readable interpretations while preserving nearly no degradation in performance. We further present four concrete use cases of Metis, showcasing how Metis helps network operators to design, debug, deploy, and ad-hoc adjust DL-based networking systems.
Tasks
Published	2019-10-09
URL	https://arxiv.org/abs/1910.03835v2
PDF	https://arxiv.org/pdf/1910.03835v2.pdf
PWC	https://paperswithcode.com/paper/explaining-deep-learning-based-networked
Repo	https://github.com/TranSys2020/TranSys
Framework	none

Learning Partial Differential Equations from Data Using Neural Networks


Title	Learning Partial Differential Equations from Data Using Neural Networks
Authors	Ali Hasan, João M. Pereira, Robert Ravier, Sina Farsiu, Vahid Tarokh
Abstract	We develop a framework for estimating unknown partial differential equations from noisy data, using a deep learning approach. Given noisy samples of a solution to an unknown PDE, our method interpolates the samples using a neural network, and extracts the PDE by equating derivatives of the neural network approximation. Our method applies to PDEs which are linear combinations of user-defined dictionary functions, and generalizes previous methods that only consider parabolic PDEs. We introduce a regularization scheme that prevents the function approximation from overfitting the data and forces it to be a solution of the underlying PDE. We validate the model on simulated data generated by the known PDEs and added Gaussian noise, and we study our method under different levels of noise. We also compare the error of our method with a Cramer-Rao lower bound for an ordinary differential equation. Our results indicate that our method outperforms other methods in estimating PDEs, especially in the low signal-to-noise regime.
Tasks
Published	2019-10-22
URL	https://arxiv.org/abs/1910.10262v1
PDF	https://arxiv.org/pdf/1910.10262v1.pdf
PWC	https://paperswithcode.com/paper/learning-partial-differential-equations-from
Repo	https://github.com/alluly/pde-estimation
Framework	pytorch

Gemmini: An Agile Systolic Array Generator Enabling Systematic Evaluations of Deep-Learning Architectures


Title	Gemmini: An Agile Systolic Array Generator Enabling Systematic Evaluations of Deep-Learning Architectures
Authors	Hasan Genc, Ameer Haj-Ali, Vighnesh Iyer, Alon Amid, Howard Mao, John Wright, Colin Schmidt, Jerry Zhao, Albert Ou, Max Banister, Yakun Sophia Shao, Borivoje Nikolic, Ion Stoica, Krste Asanovic
Abstract	Advances in deep learning and neural networks have resulted in the rapid development of hardware accelerators that support them. A large majority of ASIC accelerators, however, target a single hardware design point to accelerate the main computational kernels of deep neural networks such as convolutions or matrix multiplication. On the other hand, the spectrum of use-cases for neural network accelerators, ranging from edge devices to cloud, presents a prime opportunity for agile hardware design and generator methodologies. We present Gemmini – an open source and agile systolic array generator enabling systematic evaluations of deep-learning architectures. Gemmini generates a custom ASIC accelerator for matrix multiplication based on a systolic array architecture, complete with additional functions for neural network inference. Gemmini runs with the RISC-V ISA, and is integrated with the Rocket Chip System-on-Chip generator ecosystem, including Rocket in-order cores and BOOM out-of-order cores. Through an elaborate design space exploration case study, this work demonstrates the selection processes of various parameters for the use-case of inference on edge devices. Selected design points achieve two to three orders of magnitude speedup in deep neural network inference compared to the baseline execution on a host processor. Gemmini-generated accelerators were used in the fabrication of test systems-on-chip in TSMC 16nm and Intel 22FFL process technologies.
Tasks
Published	2019-11-22
URL	https://arxiv.org/abs/1911.09925v2
PDF	https://arxiv.org/pdf/1911.09925v2.pdf
PWC	https://paperswithcode.com/paper/gemmini-an-agile-systolic-array-generator
Repo	https://github.com/ucb-bar/chipyard
Framework	none

Understanding Neural Networks via Feature Visualization: A survey


Title	Understanding Neural Networks via Feature Visualization: A survey
Authors	Anh Nguyen, Jason Yosinski, Jeff Clune
Abstract	A neuroscience method to understanding the brain is to find and study the preferred stimuli that highly activate an individual cell or groups of cells. Recent advances in machine learning enable a family of methods to synthesize preferred stimuli that cause a neuron in an artificial or biological brain to fire strongly. Those methods are known as Activation Maximization (AM) or Feature Visualization via Optimization. In this chapter, we (1) review existing AM techniques in the literature; (2) discuss a probabilistic interpretation for AM; and (3) review the applications of AM in debugging and explaining networks.
Tasks
Published	2019-04-18
URL	http://arxiv.org/abs/1904.08939v1
PDF	http://arxiv.org/pdf/1904.08939v1.pdf
PWC	https://paperswithcode.com/paper/understanding-neural-networks-via-feature
Repo	https://github.com/yoshihisa-furusawa/Activation_Maximization
Framework	none

DeepAcid: Classification of macromolecule type based on sequences of amino acids


Title	DeepAcid: Classification of macromolecule type based on sequences of amino acids
Authors	Sarwar Khan
Abstract	The study of the amino acid sequence is vital in life sciences. In this paper, we are using deep learning to solve macromolecule classification problem using amino acids. Deep learning has emerged as a strong and efficient framework that can be applied to a broad spectrum of complex learning problems which were difficult to solve using traditional machine learning techniques in the past. We are using word embedding from NLP to represent the amino acid sequence as vectors. We are using different deep learning model for classification of macromolecules like CNN, LSTM, and GRU. Convolution neural network can extract features from amino acid sequences which are represented by vectors. The extracted features will be feed to a different type of model to train a robust classifier. our results show that Word2vec as embedding combine with VGG-16 has better performance than LSTM and GRU. our approach gets an error rate of 1.5%. Code is available at https://github.com/say2sarwar/DeepAcid
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.03532v1
PDF	https://arxiv.org/pdf/1907.03532v1.pdf
PWC	https://paperswithcode.com/paper/deepacid-classification-of-macromolecule-type
Repo	https://github.com/say2sarwar/DeepAcid
Framework	none

Flat Metric Minimization with Applications in Generative Modeling


Title	Flat Metric Minimization with Applications in Generative Modeling
Authors	Thomas Möllenhoff, Daniel Cremers
Abstract	We take the novel perspective to view data not as a probability distribution but rather as a current. Primarily studied in the field of geometric measure theory, $k$-currents are continuous linear functionals acting on compactly supported smooth differential forms and can be understood as a generalized notion of oriented $k$-dimensional manifold. By moving from distributions (which are $0$-currents) to $k$-currents, we can explicitly orient the data by attaching a $k$-dimensional tangent plane to each sample point. Based on the flat metric which is a fundamental distance between currents, we derive FlatGAN, a formulation in the spirit of generative adversarial networks but generalized to $k$-currents. In our theoretical contribution we prove that the flat metric between a parametrized current and a reference current is Lipschitz continuous in the parameters. In experiments, we show that the proposed shift to $k>0$ leads to interpretable and disentangled latent representations which behave equivariantly to the specified oriented tangent planes.
Tasks
Published	2019-05-12
URL	https://arxiv.org/abs/1905.04730v1
PDF	https://arxiv.org/pdf/1905.04730v1.pdf
PWC	https://paperswithcode.com/paper/flat-metric-minimization-with-applications-in
Repo	https://github.com/moellenh/flatgan
Framework	pytorch

Uncertainty Estimation in One-Stage Object Detection


Title	Uncertainty Estimation in One-Stage Object Detection
Authors	Florian Kraus, Klaus Dietmayer
Abstract	Environment perception is the task for intelligent vehicles on which all subsequent steps rely. A key part of perception is to safely detect other road users such as vehicles, pedestrians, and cyclists. With modern deep learning techniques huge progress was made over the last years in this field. However such deep learning based object detection models cannot predict how certain they are in their predictions, potentially hampering the performance of later steps such as tracking or sensor fusion. We present a viable approaches to estimate uncertainty in an one-stage object detector, while improving the detection performance of the baseline approach. The proposed model is evaluated on a large scale automotive pedestrian dataset. Experimental results show that the uncertainty outputted by our system is coupled with detection accuracy and the occlusion level of pedestrians.
Tasks	Object Detection, Sensor Fusion
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10296v1
PDF	https://arxiv.org/pdf/1905.10296v1.pdf
PWC	https://paperswithcode.com/paper/uncertainty-estimation-in-one-stage-object
Repo	https://github.com/flkraus/bayesian-yolov3
Framework	tf

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises


Title	A Deep Learning Framework for Assessing Physical Rehabilitation Exercises
Authors	Y. Liao, A. Vakanski, M. Xian
Abstract	Computer-aided assessment of physical rehabilitation entails evaluation of patient performance in completing prescribed rehabilitation exercises, based on processing movement data captured with a sensory system. Despite the essential role of rehabilitation assessment toward improved patient outcomes and reduced healthcare costs, existing approaches lack versatility, robustness, and practical relevance. In this paper, we propose a deep learning-based framework for automated assessment of the quality of physical rehabilitation exercises. The main components of the framework are metrics for quantifying movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for generating quality scores of input movements via supervised learning. The proposed performance metric is defined based on the log-likelihood of a Gaussian mixture model, and encodes low-dimensional data representation obtained with a deep autoencoder network. The proposed deep spatio-temporal neural network arranges data into temporal pyramids, and exploits the spatial characteristics of human movements by using sub-networks to process joint displacements of individual body parts. The presented framework is validated using a dataset of ten rehabilitation exercises. The significance of this work is that it is the first that implements deep neural networks for assessment of rehabilitation performance.
Tasks	Dimensionality Reduction
Published	2019-01-29
URL	https://arxiv.org/abs/1901.10435v3
PDF	https://arxiv.org/pdf/1901.10435v3.pdf
PWC	https://paperswithcode.com/paper/a-deep-learning-framework-for-assessing
Repo	https://github.com/avakanski/A-Deep-Learning-Framework-for-Assessing-Physical-Rehabilitation-Exercises
Framework	none

Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning


Title	Percival: Making In-Browser Perceptual Ad Blocking Practical With Deep Learning
Authors	Zain ul Abi Din, Panagiotis Tigas, Samuel T. King, Benjamin Livshits
Abstract	Online advertising has been a long-standing concern for user privacy and overall web experience. Several techniques have been proposed to block ads, mostly based on filter-lists and manually-written rules. While a typical ad blocker relies on manually-curated block lists, these inevitably get out-of-date, thus compromising the ultimate utility of this ad blocking approach. In this paper we present Percival, a browser-embedded, lightweight, deep learning-powered ad blocker. Percival embeds itself within the browser’s image rendering pipeline, which makes it possible to intercept every image obtained during page execution and to perform blocking based on applying machine learning for image classification to flag potential ads. Our implementation inside both Chromium and Brave browsers shows only a minor rendering performance overhead of 4.55%, demonstrating the feasibility of deploying traditionally heavy models (i.e. deep neural networks) inside the critical path of the rendering engine of a browser. We show that our image-based ad blocker can replicate EasyList rules with an accuracy of 96.76%. To show the versatility of the Percival’s approach we present case studies that demonstrate that Percival 1) does surprisingly well on ads in languages other than English; 2) Percival also performs well on blocking first-party Facebook ads, which have presented issues for other ad blockers. Percival proves that image-based perceptual ad blocking is an attractive complement to today’s dominant approach of block lists
Tasks	Image Classification
Published	2019-05-17
URL	https://arxiv.org/abs/1905.07444v2
PDF	https://arxiv.org/pdf/1905.07444v2.pdf
PWC	https://paperswithcode.com/paper/percival-making-in-browser-perceptual-ad
Repo	https://github.com/dxaen/percival
Framework	none

Fashion-AttGAN: Attribute-Aware Fashion Editing with Multi-Objective GAN


Title	Fashion-AttGAN: Attribute-Aware Fashion Editing with Multi-Objective GAN
Authors	Qing Ping, Bing Wu, Wanying Ding, Jiangbo Yuan
Abstract	In this paper, we introduce attribute-aware fashion-editing, a novel task, to the fashion domain. We re-define the overall objectives in AttGAN and propose the Fashion-AttGAN model for this new task. A dataset is constructed for this task with 14,221 and 22 attributes, which has been made publically available. Experimental results show the effectiveness of our Fashion-AttGAN on fashion editing over the original AttGAN.
Tasks
Published	2019-04-16
URL	http://arxiv.org/abs/1904.07460v2
PDF	http://arxiv.org/pdf/1904.07460v2.pdf
PWC	https://paperswithcode.com/paper/fashion-attgan-attribute-aware-fashion
Repo	https://github.com/ChanningPing/Fashion_Attribute_Editing
Framework	tf

Exploratory Combinatorial Optimization with Reinforcement Learning


Title	Exploratory Combinatorial Optimization with Reinforcement Learning
Authors	Thomas D. Barrett, William R. Clements, Jakob N. Foerster, A. I. Lvovsky
Abstract	Many real-world problems can be reduced to combinatorial optimization on a graph, where the subset or ordering of vertices that maximize some objective function must be found. With such tasks often NP-hard and analytically intractable, reinforcement learning (RL) has shown promise as a framework with which efficient heuristic methods to tackle these problems can be learned. Previous works construct the solution subset incrementally, adding one element at a time, however, the irreversible nature of this approach prevents the agent from revising its earlier decisions, which may be necessary given the complexity of the optimization task. We instead propose that the agent should seek to continuously improve the solution by learning to explore at test time. Our approach of exploratory combinatorial optimization (ECO-DQN) is, in principle, applicable to any combinatorial problem that can be defined on a graph. Experimentally, we show our method to produce state-of-the-art RL performance on the Maximum Cut problem. Moreover, because ECO-DQN can start from any arbitrary configuration, it can be combined with other search methods to further improve performance, which we demonstrate using a simple random search.
Tasks	Combinatorial Optimization
Published	2019-09-09
URL	https://arxiv.org/abs/1909.04063v2
PDF	https://arxiv.org/pdf/1909.04063v2.pdf
PWC	https://paperswithcode.com/paper/exploratory-combinatorial-optimization-with
Repo	https://github.com/tomdbar/eco-dqn
Framework	pytorch