February 1, 2020

2902 words 14 mins read

Paper Group AWR 82

Adaptive spline fitting with particle swarm optimization. Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks. A Systematic and Meta-analysis Survey of Whale Optimization Algorithm. Functional Adversarial Attacks. A New Ensemble Adversarial Attack Powered by Long-term Gradient Memories. Student Specialization in Deep Re …

Adaptive spline fitting with particle swarm optimization


Title	Adaptive spline fitting with particle swarm optimization
Authors	Soumya D. Mohanty, Ethan Fahnestock
Abstract	In fitting data with a spline, finding the optimal placement of knots can significantly improve the quality of the fit. However, the challenging high-dimensional and non-convex optimization problem associated with completely free knot placement has been a major roadblock in using this approach. We present a method that uses particle swarm optimization (PSO) combined with model selection to address this challenge. The problem of overfitting due to knot clustering that accompanies free knot placement is mitigated in this method by explicit regularization, resulting in a significantly improved performance on highly noisy data. The principal design choices available in the method are delineated and a statistically rigorous study of their effect on performance is carried out using simulated data and a wide variety of benchmark functions. Our results demonstrate that PSO-based free knot placement leads to a viable and flexible adaptive spline fitting approach that allows the fitting of both smooth and non-smooth functions.
Tasks	Model Selection
Published	2019-07-28
URL	https://arxiv.org/abs/1907.12160v3
PDF	https://arxiv.org/pdf/1907.12160v3.pdf
PWC	https://paperswithcode.com/paper/adaptive-spline-fitting-with-particle-swarm
Repo	https://github.com/mohanty-sd/SHAPES
Framework	none

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks


Title	Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks
Authors	Zhe He, Adrian Spurr, Xucong Zhang, Otmar Hilliges
Abstract	Gaze redirection is the task of changing the gaze to a desired direction for a given monocular eye patch image. Many applications such as videoconferencing, films, games, and generation of training data for gaze estimation require redirecting the gaze, without distorting the appearance of the area surrounding the eye and while producing photo-realistic images. Existing methods lack the ability to generate perceptually plausible images. In this work, we present a novel method to alleviate this problem by leveraging generative adversarial training to synthesize an eye image conditioned on a target gaze direction. Our method ensures perceptual similarity and consistency of synthesized images to the real images. Furthermore, a gaze estimation loss is used to control the gaze direction accurately. To attain high-quality images, we incorporate perceptual and cycle consistency losses into our architecture. In extensive evaluations we show that the proposed method outperforms state-of-the-art approaches in terms of both image quality and redirection precision. Finally, we show that generated images can bring significant improvement for the gaze estimation task if used to augment real training data.
Tasks	Gaze Estimation
Published	2019-03-29
URL	https://arxiv.org/abs/1903.12530v4
PDF	https://arxiv.org/pdf/1903.12530v4.pdf
PWC	https://paperswithcode.com/paper/photo-realistic-monocular-gaze-redirection
Repo	https://github.com/HzDmS/gaze_redirection
Framework	tf

A Systematic and Meta-analysis Survey of Whale Optimization Algorithm


Title	A Systematic and Meta-analysis Survey of Whale Optimization Algorithm
Authors	Hardi M. Mohammed, Shahla U. Umar, Tarik A. Rashid
Abstract	Whale Optimization Algorithm (WOA) is a nature-inspired meta-heuristic optimization algorithm, which was proposed by Mirjalili and Lewis in 2016. This algorithm has shown its ability to solve many problems. Comprehensive surveys have been conducted about some other nature-inspired algorithms, such as ABC, PSO, etc.Nonetheless, no survey search work has been conducted on WOA. Therefore, in this paper, a systematic and meta analysis survey of WOA is conducted to help researchers to use it in different areas or hybridize it with other common algorithms. Thus, WOA is presented in depth in terms of algorithmic backgrounds, its characteristics, limitations, modifications, hybridizations, and applications. Next, WOA performances are presented to solve different problems. Then, the statistical results of WOA modifications and hybridizations are established and compared with the most common optimization algorithms and WOA. The survey’s results indicate that WOA performs better than other common algorithms in terms of convergence speed and balancing between exploration and exploitation. WOA modifications and hybridizations also perform well compared to WOA. In addition, our investigation paves a way to present a new technique by hybridizing both WOA and BAT algorithms. The BAT algorithm is used for the exploration phase, whereas the WOA algorithm is used for the exploitation phase. Finally, statistical results obtained from WOA-BAT are very competitive and better than WOA in 16 benchmarks functions. WOA-BAT also outperforms well in 13 functions from CEC2005 and 7 functions from CEC2019.
Tasks
Published	2019-03-20
URL	http://arxiv.org/abs/1903.08763v3
PDF	http://arxiv.org/pdf/1903.08763v3.pdf
PWC	https://paperswithcode.com/paper/a-systematic-and-meta-analysis-survey-of
Repo	https://github.com/Hardi-Mohammed/WOA-BAT-modification
Framework	none

Functional Adversarial Attacks


Title	Functional Adversarial Attacks
Authors	Cassidy Laidlaw, Soheil Feizi
Abstract	We propose functional adversarial attacks, a novel class of threat models for crafting adversarial examples to fool machine learning models. Unlike a standard $\ell_p$-ball threat model, a functional adversarial threat model allows only a single function to be used to perturb input features to produce an adversarial example. For example, a functional adversarial attack applied on colors of an image can change all red pixels simultaneously to light red. Such global uniform changes in images can be less perceptible than perturbing pixels of the image individually. For simplicity, we refer to functional adversarial attacks on image colors as ReColorAdv, which is the main focus of our experiments. We show that functional threat models can be combined with existing additive ($\ell_p$) threat models to generate stronger threat models that allow both small, individual perturbations and large, uniform changes to an input. Moreover, we prove that such combinations encompass perturbations that would not be allowed in either constituent threat model. In practice, ReColorAdv can significantly reduce the accuracy of a ResNet-32 trained on CIFAR-10. Furthermore, to the best of our knowledge, combining ReColorAdv with other attacks leads to the strongest existing attack even after adversarial training. An implementation of ReColorAdv is available at https://github.com/cassidylaidlaw/ReColorAdv .
Tasks	Adversarial Attack
Published	2019-05-29
URL	https://arxiv.org/abs/1906.00001v2
PDF	https://arxiv.org/pdf/1906.00001v2.pdf
PWC	https://paperswithcode.com/paper/190600001
Repo	https://github.com/cassidylaidlaw/ReColorAdv
Framework	pytorch

A New Ensemble Adversarial Attack Powered by Long-term Gradient Memories


Title	A New Ensemble Adversarial Attack Powered by Long-term Gradient Memories
Authors	Zhaohui Che, Ali Borji, Guangtao Zhai, Suiyi Ling, Jing Li, Patrick Le Callet
Abstract	Deep neural networks are vulnerable to adversarial attacks.
Tasks	Adversarial Attack
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07682v1
PDF	https://arxiv.org/pdf/1911.07682v1.pdf
PWC	https://paperswithcode.com/paper/a-new-ensemble-adversarial-attack-powered-by
Repo	https://github.com/CZHQuality/AAA-Pix2pix
Framework	pytorch

Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension


Title	Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension
Authors	Yuandong Tian
Abstract	To analyze deep ReLU network, we adopt a student-teacher setting in which an over-parameterized student network learns from the output of a fixed teacher network of the same depth, with Stochastic Gradient Descent (SGD). Our contributions are two-fold. First, we prove that when the gradient is small at every training sample, student node \emph{specializes} to teacher nodes in the lowest layer under mild conditions. Second, analysis of noisy recovery and training dynamics in 2-layer network shows that strong teacher nodes (with large fan-out weights) are learned first and subtle teacher nodes are left unlearned until late stage of training. As a result, it could take a long time to converge into these small-gradient critical points. Our analysis shows that over-parameterization is a necessary condition for specialization to happen at the critical points, and helps student nodes cover more teacher nodes with fewer iterations. Both improve generalization. Different from Neural Tangent Kernel and statistical mechanics approach, our approach operates on finite width, mild over-parameterization (as long as there are more student nodes than teacher) and finite input dimension. Experiments justify our finding. The code is released in https://github.com/facebookresearch/luckmatters.
Tasks
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13458v4
PDF	https://arxiv.org/pdf/1909.13458v4.pdf
PWC	https://paperswithcode.com/paper/over-parameterization-as-a-catalyst-for
Repo	https://github.com/facebookresearch/luckmatters
Framework	pytorch

DOM-Q-NET: Grounded RL on Structured Language


Title	DOM-Q-NET: Grounded RL on Structured Language
Authors	Sheng Jia, Jamie Kiros, Jimmy Ba
Abstract	Building agents to interact with the web would allow for significant improvements in knowledge understanding and representation learning. However, web navigation tasks are difficult for current deep reinforcement learning (RL) models due to the large discrete action space and the varying number of actions between the states. In this work, we introduce DOM-Q-NET, a novel architecture for RL-based web navigation to address both of these problems. It parametrizes Q functions with separate networks for different action categories: clicking a DOM element and typing a string input. Our model utilizes a graph neural network to represent the tree-structured HTML of a standard web page. We demonstrate the capabilities of our model on the MiniWoB environment where we can match or outperform existing work without the use of expert demonstrations. Furthermore, we show 2x improvements in sample efficiency when training in the multi-task setting, allowing our model to transfer learned behaviours across tasks.
Tasks	Representation Learning
Published	2019-02-19
URL	http://arxiv.org/abs/1902.07257v1
PDF	http://arxiv.org/pdf/1902.07257v1.pdf
PWC	https://paperswithcode.com/paper/dom-q-net-grounded-rl-on-structured-language
Repo	https://github.com/Sheng-J/DOM-Q-NET
Framework	pytorch

Understanding the Role of Momentum in Stochastic Gradient Methods


Title	Understanding the Role of Momentum in Stochastic Gradient Methods
Authors	Igor Gitman, Hunter Lang, Pengchuan Zhang, Lin Xiao
Abstract	The use of momentum in stochastic gradient methods has become a widespread practice in machine learning. Different variants of momentum, including heavy-ball momentum, Nesterov’s accelerated gradient (NAG), and quasi-hyperbolic momentum (QHM), have demonstrated success on various tasks. Despite these empirical successes, there is a lack of clear understanding of how the momentum parameters affect convergence and various performance measures of different algorithms. In this paper, we use the general formulation of QHM to give a unified analysis of several popular algorithms, covering their asymptotic convergence conditions, stability regions, and properties of their stationary distributions. In addition, by combining the results on convergence rates and stationary distributions, we obtain sometimes counter-intuitive practical guidelines for setting the learning rate and momentum parameters.
Tasks	Stochastic Optimization
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13962v1
PDF	https://arxiv.org/pdf/1910.13962v1.pdf
PWC	https://paperswithcode.com/paper/understanding-the-role-of-momentum-in
Repo	https://github.com/Kipok/understanding-momentum
Framework	pytorch

Simulating extrapolated dynamics with parameterization networks


Title	Simulating extrapolated dynamics with parameterization networks
Authors	James P. L. Tan
Abstract	An artificial neural network architecture, parameterization networks, is proposed for simulating extrapolated dynamics beyond observed data in dynamical systems. Parameterization networks are used to ensure the long term integrity of extrapolated dynamics, while careful tuning of model hyperparameters against validation errors controls overfitting. A parameterization network is demonstrated on the logistic map, where chaos and other nonlinear phenomena consistent with the underlying model can be extrapolated from non-chaotic training time series with good fidelity. The stated results are a lot less fantastical than they appear to be because the neural network is only extrapolating between quadratic return maps. Nonetheless, the results do suggest that successful extrapolation of qualitatively different behaviors requires learning to occur on a level of abstraction where the corresponding behaviors are more similar in nature.
Tasks	Time Series
Published	2019-02-09
URL	http://arxiv.org/abs/1902.03440v1
PDF	http://arxiv.org/pdf/1902.03440v1.pdf
PWC	https://paperswithcode.com/paper/simulating-extrapolated-dynamics-with
Repo	https://github.com/jamespltan/pnn
Framework	none


Title	Data-driven Approach for Quality Evaluation on Knowledge Sharing Platform
Authors	Lu Xu, Jinhai Xiang, Yating Wang, Fuchuan Ni
Abstract	In recent years, voice knowledge sharing and question answering (Q&A) platforms have attracted much attention, which greatly facilitate the knowledge acquisition for people. However, little research has evaluated on the quality evaluation on voice knowledge sharing. This paper presents a data-driven approach to automatically evaluate the quality of a specific Q&A platform (Zhihu Live). Extensive experiments demonstrate the effectiveness of the proposed method. Furthermore, we introduce a dataset of Zhihu Live as an open resource for researchers in related areas. This dataset will facilitate the development of new methods on knowledge sharing services quality evaluation.
Tasks	Question Answering
Published	2019-03-01
URL	http://arxiv.org/abs/1903.00384v1
PDF	http://arxiv.org/pdf/1903.00384v1.pdf
PWC	https://paperswithcode.com/paper/data-driven-approach-for-quality-evaluation
Repo	https://github.com/lucasxlu/ZhihuDataDriven
Framework	none

DeepInSAR: A Deep Learning Framework for SAR Interferometric Phase Restoration and Coherence Estimation


Title	DeepInSAR: A Deep Learning Framework for SAR Interferometric Phase Restoration and Coherence Estimation
Authors	Xinyao Sun, Aaron Zimmer, Subhayan Mukherjee, Navaneeth Kamballur Kottayil, Parwant Ghuman, Irene Cheng
Abstract	Over the past decade, Interferometric Synthetic Aperture Radar (InSAR) has become a successful remote sensing technique. However, during the acquisition step, microwave reflections received at satellite are usually disturbed by strong noise, leading to a noisy single-look complex (SLC) SAR image. The quality of their interferometric phase is even worse. InSAR phase filtering is an ill-posed problem and plays a key role in subsequent processing. However, most of existing methods usually require expert supervision or heavy runtime, which limits the usability and scalability for practical usages such as wide-area monitoring and forecasting. In this work, we propose a deep convolutional neural network (CNN) based model DeepInSAR to intelligently solve both the phase filtering and coherence estimation problems. We demonstrate our DeepInSAR using both simulated and real data. A teacher-student framework is proposed to deal with the issue that there is no ground truth sample for real-world InSAR data. Quantitative and qualitative comparisons show that DeepInSAR achieves comparable or even better results than its stacked-based teacher method on new test datasets but requiring fewer pairs of SLCs as well as outperforms three other established non-stack based methods with less running time and no human supervision.
Tasks
Published	2019-09-06
URL	https://arxiv.org/abs/1909.03120v1
PDF	https://arxiv.org/pdf/1909.03120v1.pdf
PWC	https://paperswithcode.com/paper/deepinsar-a-deep-learning-framework-for-sar
Repo	https://github.com/Lucklyric/InSAR-Simulator
Framework	none

The LogBarrier adversarial attack: making effective use of decision boundary information


Title	The LogBarrier adversarial attack: making effective use of decision boundary information
Authors	Chris Finlay, Aram-Alexandre Pooladian, Adam M. Oberman
Abstract	Adversarial attacks for image classification are small perturbations to images that are designed to cause misclassification by a model. Adversarial attacks formally correspond to an optimization problem: find a minimum norm image perturbation, constrained to cause misclassification. A number of effective attacks have been developed. However, to date, no gradient-based attacks have used best practices from the optimization literature to solve this constrained minimization problem. We design a new untargeted attack, based on these best practices, using the established logarithmic barrier method. On average, our attack distance is similar or better than all state-of-the-art attacks on benchmark datasets (MNIST, CIFAR10, ImageNet-1K). In addition, our method performs significantly better on the most challenging images, those which normally require larger perturbations for misclassification. We employ the LogBarrier attack on several adversarially defended models, and show that it adversarially perturbs all images more efficiently than other attacks: the distance needed to perturb all images is significantly smaller with the LogBarrier attack than with other state-of-the-art attacks.
Tasks	Adversarial Attack, Image Classification
Published	2019-03-25
URL	http://arxiv.org/abs/1903.10396v1
PDF	http://arxiv.org/pdf/1903.10396v1.pdf
PWC	https://paperswithcode.com/paper/the-logbarrier-adversarial-attack-making
Repo	https://github.com/cfinlay/logbarrier
Framework	pytorch

Causality for Machine Learning


Title	Causality for Machine Learning
Authors	Bernhard Schölkopf
Abstract	Graphical causal inference as pioneered by Judea Pearl arose from research on artificial intelligence (AI), and for a long time had little connection to the field of machine learning. This article discusses where links have been and should be established, introducing key concepts along the way. It argues that the hard open problems of machine learning and AI are intrinsically related to causality, and explains how the field is beginning to understand them.
Tasks	Causal Inference
Published	2019-11-24
URL	https://arxiv.org/abs/1911.10500v2
PDF	https://arxiv.org/pdf/1911.10500v2.pdf
PWC	https://paperswithcode.com/paper/causality-for-machine-learning
Repo	https://github.com/Causal-Inference-ZeroToAll/causality4ml
Framework	none

Minimalistic Attacks: How Little it Takes to Fool a Deep Reinforcement Learning Policy


Title	Minimalistic Attacks: How Little it Takes to Fool a Deep Reinforcement Learning Policy
Authors	Xinghua Qu, Zhu Sun, Yew-Soon Ong, Abhishek Gupta, Pengfei Wei
Abstract	Recent studies have revealed that neural network-based policies can be easily fooled by adversarial examples. However, while most prior works analyze the effects of perturbing every pixel of every frame assuming white-box policy access, in this paper we take a more restrictive view towards adversary generation - with the goal of unveiling the limits of a model’s vulnerability. In particular, we explore minimalistic attacks by defining three key settings: (1) black-box policy access: where the attacker only has access to the input (state) and output (action probability) of an RL policy; (2) fractional-state adversary: where only several pixels are perturbed, with the extreme case being a single-pixel adversary; and (3) tactically-chanced attack: where only significant frames are tactically chosen to be attacked. We formulate the adversarial attack by accommodating the three key settings and explore their potency on six Atari games by examining four fully trained state-of-the-art policies. In Breakout, for example, we surprisingly find that: (i) all policies showcase significant performance degradation by merely modifying 0.01% of the input state, and (ii) the policy trained by DQN is totally deceived by perturbation to only 1% frames.
Tasks	Adversarial Attack, Atari Games
Published	2019-11-10
URL	https://arxiv.org/abs/1911.03849v4
PDF	https://arxiv.org/pdf/1911.03849v4.pdf
PWC	https://paperswithcode.com/paper/minimalistic-attacks-how-little-it-takes-to
Repo	https://github.com/RLMA2019/RLMA
Framework	tf

Score-CAM:Improved Visual Explanations Via Score-Weighted Class Activation Mapping


Title	Score-CAM:Improved Visual Explanations Via Score-Weighted Class Activation Mapping
Authors	Haofan Wang, Mengnan Du, Fan Yang, Zijian Zhang
Abstract	Recently, more and more attention has been drawn into the internal mechanism of the convolutional neural network and on what basis does the network make a specific decision. In this paper, we develop a novel post-hoc visual explanation method called Score-CAM based on class activation mapping. Unlike previous class activation mapping based approaches, Score-CAM gets rid of the dependence on gradient by obtaining the weight of each activation map through its forward passing score on target class, the final result is obtained by a linear combination of weights and activation maps. We demonstrate that Score-CAM achieves better visual performance with less noise and has better stability than Grad-CAM and Grad-CAM++. In the experiment, we rethink issues of previous evaluation metrics and propose a representative evaluation approach Energy- Based Pointing Game to measure the quality of the generated saliency maps. Our approach outperforms previous methods on energy-based pointing game and recognition and shows more robustness under adversarial attack.
Tasks	Adversarial Attack
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01279v1
PDF	https://arxiv.org/pdf/1910.01279v1.pdf
PWC	https://paperswithcode.com/paper/score-camimproved-visual-explanations-via
Repo	https://github.com/andreysorokin/scam-net
Framework	none