February 1, 2020

3085 words 15 mins read

Paper Group AWR 168

GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm. Grid R-CNN Plus: Faster and Better. WaveletAE: A Wavelet-enhanced Autoencoder for Wind Turbine Blade Icing Detection. EviDense: a Graph-based Method for Finding Unique High-impact Events with Succinct Keyword-based Descriptions. Memory-Driven Mixed Low Precision Quantization …

GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm


Title	GACNN: Training Deep Convolutional Neural Networks with Genetic Algorithm
Authors	Parsa Esfahanian, Mohammad Akhavan
Abstract	Convolutional Neural Networks (CNNs) have gained a significant attraction in the recent years due to their increasing real-world applications. Their performance is highly dependent to the network structure and the selected optimization method for tuning the network parameters. In this paper, we propose novel yet efficient methods for training convolutional neural networks. The most of current state of the art learning method for CNNs are based on Gradient decent. In contrary to the traditional CNN training methods, we propose to optimize the CNNs using methods based on Genetic Algorithms (GAs). These methods are carried out using three individual GA schemes, Steady-State, Generational, and Elitism. We present new genetic operators for crossover, mutation and also an innovative encoding paradigm of CNNs to chromosomes aiming to reduce the resulting chromosome’s size by a large factor. We compare the effectiveness and scalability of our encoding with the traditional encoding. Furthermore, the performance of individual GA schemes used for training the networks were compared with each other in means of convergence rate and overall accuracy. Finally, our new encoding alongside the superior GA-based training scheme is compared to Backpropagation training with Adam optimization.
Tasks
Published	2019-09-29
URL	https://arxiv.org/abs/1909.13354v1
PDF	https://arxiv.org/pdf/1909.13354v1.pdf
PWC	https://paperswithcode.com/paper/gacnn-training-deep-convolutional-neural
Repo	https://github.com/MatthewZou9419/GACNN
Framework	none

Grid R-CNN Plus: Faster and Better


Title	Grid R-CNN Plus: Faster and Better
Authors	Xin Lu, Buyu Li, Yuxin Yue, Quanquan Li, Junjie Yan
Abstract	Grid R-CNN is a well-performed objection detection framework. It transforms the traditional box offset regression problem into a grid point estimation problem. With the guidance of the grid points, it can obtain high-quality localization results. However, the speed of Grid R-CNN is not so satisfactory. In this technical report we present Grid R-CNN Plus, a better and faster version of Grid R-CNN. We have made several updates that significantly speed up the framework and simultaneously improve the accuracy. On COCO dataset, the Res50-FPN based Grid R-CNN Plus detector achieves an mAP of 40.4%, outperforming the baseline on the same model by 3.0 points with similar inference time. Code is available at https://github.com/STVIR/Grid-R-CNN .
Tasks	Object Detection
Published	2019-06-13
URL	https://arxiv.org/abs/1906.05688v1
PDF	https://arxiv.org/pdf/1906.05688v1.pdf
PWC	https://paperswithcode.com/paper/grid-r-cnn-plus-faster-and-better
Repo	https://github.com/STVIR/Grid-R-CNN
Framework	pytorch

WaveletAE: A Wavelet-enhanced Autoencoder for Wind Turbine Blade Icing Detection


Title	WaveletAE: A Wavelet-enhanced Autoencoder for Wind Turbine Blade Icing Detection
Authors	Binhang Yuan, Chen Wang, Chen Luo, Fei Jiang, Mingsheng Long, Philip S. Yu, Yuan Liu
Abstract	Wind power, as an alternative to burning fossil fuels, is abundant and inexhaustible. To fully utilize wind power, wind farms are usually located in areas of high altitude and facing serious ice conditions, which can lead to serious consequences. Quick detection of blade ice accretion is crucial for the maintenance of wind farms. Unlike traditional methods of installing expensive physical detectors on wind blades, data-driven approaches are increasingly popular for inspecting the wind turbine failures. In this work, we propose a wavelet enhanced autoencoder model (WaveletAE) to identify wind turbine dysfunction by analyzing the multivariate time series monitored by the SCADA system. WaveletAE is enhanced with wavelet detail coefficients to enforce the autoencoder to capture information from multiple scales, and the CNN-LSTM architecture is applied to learn channel-wise and temporal-wise relations. The empirical study shows that the proposed model outperforms other state-of-the-art time series anomaly detection methods for real-world blade icing detection.
Tasks	Anomaly Detection, Time Series, Time Series Classification
Published	2019-02-14
URL	https://arxiv.org/abs/1902.05625v2
PDF	https://arxiv.org/pdf/1902.05625v2.pdf
PWC	https://paperswithcode.com/paper/waveletfcnn-a-deep-time-series-classification
Repo	https://github.com/BinhangYuan/WaveletFCNN
Framework	tf

EviDense: a Graph-based Method for Finding Unique High-impact Events with Succinct Keyword-based Descriptions


Title	EviDense: a Graph-based Method for Finding Unique High-impact Events with Succinct Keyword-based Descriptions
Authors	Oana Balalau, Carlos Castillo, Mauro Sozio
Abstract	Despite the significant efforts made by the research community in recent years, automatically acquiring valuable information about high impact-events from social media remains challenging. We present EviDense, a graph-based approach for finding high-impact events (such as disaster events) in social media. One of the challenges we address in our work is to provide for each event a succinct keyword-based description, containing the most relevant information about it, such as what happened, the location, as well as its timeframe. We evaluate our approach on a large collection of tweets posted over a period of 19 months, using a crowdsourcing platform. Our evaluation shows that our method outperforms state-of-the-art approaches for the same problem, in terms of having higher precision, lower number of duplicates, and presenting a keyword-based description that is succinct and informative. We further improve the results of our algorithm by incorporating news from mainstream media. A preliminary version of this work was presented as a 4-pages short paper at ICWSM 2018.
Tasks
Published	2019-12-05
URL	https://arxiv.org/abs/1912.02484v1
PDF	https://arxiv.org/pdf/1912.02484v1.pdf
PWC	https://paperswithcode.com/paper/evidense-a-graph-based-method-for-finding
Repo	https://github.com/nyxpho/evidense
Framework	none

Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers


Title	Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers
Authors	Manuele Rusci, Alessandro Capotondi, Luca Benini
Abstract	This paper presents a novel end-to-end methodology for enabling the deployment of low-error deep networks on microcontrollers. To fit the memory and computational limitations of resource-constrained edge-devices, we exploit mixed low-bitwidth compression, featuring 8, 4 or 2-bit uniform quantization, and we model the inference graph with integer-only operations. Our approach aims at determining the minimum bit precision of every activation and weight tensor given the memory constraints of a device. This is achieved through a rule-based iterative procedure, which cuts the number of bits of the most memory-demanding layers, aiming at meeting the memory constraints. After a quantization-aware retraining step, the fake-quantized graph is converted into an inference integer-only model by inserting the Integer Channel-Normalization (ICN) layers, which introduce a negligible loss as demonstrated on INT4 MobilenetV1 models. We report the latency-accuracy evaluation of mixed-precision MobilenetV1 family networks on a STM32H7 microcontroller. Our experimental results demonstrate an end-to-end deployment of an integer-only Mobilenet network with Top1 accuracy of 68% on a device with only 2MB of FLASH memory and 512kB of RAM, improving by 8% the Top1 accuracy with respect to previously published 8 bit implementations for microcontrollers.
Tasks	Quantization
Published	2019-05-30
URL	https://arxiv.org/abs/1905.13082v1
PDF	https://arxiv.org/pdf/1905.13082v1.pdf
PWC	https://paperswithcode.com/paper/memory-driven-mixed-low-precision
Repo	https://github.com/mrusci/training-mixed-precision-quantized-networks
Framework	pytorch

Join Query Optimization with Deep Reinforcement Learning Algorithms


Title	Join Query Optimization with Deep Reinforcement Learning Algorithms
Authors	Jonas Heitz, Kurt Stockinger
Abstract	Join query optimization is a complex task and is central to the performance of query processing. In fact it belongs to the class of NP-hard problems. Traditional query optimizers use dynamic programming (DP) methods combined with a set of rules and restrictions to avoid exhaustive enumeration of all possible join orders. However, DP methods are very resource intensive. Moreover, given simplifying assumptions of attribute independence, traditional query optimizers rely on erroneous cost estimations, which can lead to suboptimal query plans. Recent success of deep reinforcement learning (DRL) creates new opportunities for the field of query optimization to tackle the above-mentioned problems. In this paper, we present our DRL-based Fully Observed Optimizer (FOOP) which is a generic query optimization framework that enables plugging in different machine learning algorithms. The main idea of FOOP is to use a data-adaptive learning query optimizer that avoids exhaustive enumerations of join orders and is thus significantly faster than traditional approaches based on dynamic programming. In particular, we evaluate various DRL-algorithms and show that Proximal Policy Optimization significantly outperforms Q-learning based algorithms. Finally we demonstrate how ensemble learning techniques combined with DRL can further improve the query optimizer.
Tasks	Q-Learning
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11689v1
PDF	https://arxiv.org/pdf/1911.11689v1.pdf
PWC	https://paperswithcode.com/paper/join-query-optimization-with-deep
Repo	https://github.com/heitzjon/mt-join-query-optimization-with-drl
Framework	none

PPGNet: Learning Point-Pair Graph for Line Segment Detection


Title	PPGNet: Learning Point-Pair Graph for Line Segment Detection
Authors	Ziheng Zhang, Zhengxin Li, Ning Bi, Jia Zheng, Jinlei Wang, Kun Huang, Weixin Luo, Yanyu Xu, Shenghua Gao
Abstract	In this paper, we present a novel framework to detect line segments in man-made environments. Specifically, we propose to describe junctions, line segments and relationships between them with a simple graph, which is more structured and informative than end-point representation used in existing line segment detection methods. In order to extract a line segment graph from an image, we further introduce the PPGNet, a convolutional neural network that directly infers a graph from an image. We evaluate our method on published benchmarks including York Urban and Wireframe datasets. The results demonstrate that our method achieves satisfactory performance and generalizes well on all the benchmarks. The source code of our work is available at \url{https://github.com/svip-lab/PPGNet}.
Tasks	Line Segment Detection
Published	2019-05-09
URL	https://arxiv.org/abs/1905.03415v2
PDF	https://arxiv.org/pdf/1905.03415v2.pdf
PWC	https://paperswithcode.com/paper/190503415
Repo	https://github.com/svip-lab/PPGNet
Framework	pytorch

Attention-based Dropout Layer for Weakly Supervised Object Localization


Title	Attention-based Dropout Layer for Weakly Supervised Object Localization
Authors	Junsuk Choe, Hyunjung Shim
Abstract	Weakly Supervised Object Localization (WSOL) techniques learn the object location only using image-level labels, without location annotations. A common limitation for these techniques is that they cover only the most discriminative part of the object, not the entire object. To address this problem, we propose an Attention-based Dropout Layer (ADL), which utilizes the self-attention mechanism to process the feature maps of the model. The proposed method is composed of two key components: 1) hiding the most discriminative part from the model for capturing the integral extent of object, and 2) highlighting the informative region for improving the recognition power of the model. Based on extensive experiments, we demonstrate that the proposed method is effective to improve the accuracy of WSOL, achieving a new state-of-the-art localization accuracy in CUB-200-2011 dataset. We also show that the proposed method is much more efficient in terms of both parameter and computation overheads than existing techniques.
Tasks	Object Localization, Weakly-Supervised Object Localization
Published	2019-08-27
URL	https://arxiv.org/abs/1908.10028v1
PDF	https://arxiv.org/pdf/1908.10028v1.pdf
PWC	https://paperswithcode.com/paper/attention-based-dropout-layer-for-weakly-1
Repo	https://github.com/junsukchoe/ADL
Framework	pytorch

Adversarial Attacks on Time Series


Title	Adversarial Attacks on Time Series
Authors	Fazle Karim, Somshubra Majumdar, Houshang Darabi
Abstract	Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-Nearest Neighbor Dynamic Time Warping (1-NN ) DTW, a Fully Connected Network and a Fully Convolutional Network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. To the best of our knowledge, such an attack on time series classification models has never been done before. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples and to consider model robustness as an evaluative metric.
Tasks	Time Series, Time Series Classification
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10755v2
PDF	http://arxiv.org/pdf/1902.10755v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-attacks-on-time-series
Repo	https://github.com/titu1994/Adversarial-Attacks-Time-Series
Framework	tf

Confidence-based Graph Convolutional Networks for Semi-Supervised Learning


Title	Confidence-based Graph Convolutional Networks for Semi-Supervised Learning
Authors	Shikhar Vashishth, Prateek Yadav, Manik Bhandari, Partha Talukdar
Abstract	Predicting properties of nodes in a graph is an important problem with applications in a variety of domains. Graph-based Semi-Supervised Learning (SSL) methods aim to address this problem by labeling a small subset of the nodes as seeds and then utilizing the graph structure to predict label scores for the rest of the nodes in the graph. Recently, Graph Convolutional Networks (GCNs) have achieved impressive performance on the graph-based SSL task. In addition to label scores, it is also desirable to have confidence scores associated with them. Unfortunately, confidence estimation in the context of GCN has not been previously explored. We fill this important gap in this paper and propose ConfGCN, which estimates labels scores along with their confidences jointly in GCN-based setting. ConfGCN uses these estimated confidences to determine the influence of one node on another during neighborhood aggregation, thereby acquiring anisotropic capabilities. Through extensive analysis and experiments on standard benchmarks, we find that ConfGCN is able to outperform state-of-the-art baselines. We have made ConfGCN’s source code available to encourage reproducible research.
Tasks
Published	2019-01-24
URL	http://arxiv.org/abs/1901.08255v2
PDF	http://arxiv.org/pdf/1901.08255v2.pdf
PWC	https://paperswithcode.com/paper/confidence-based-graph-convolutional-networks
Repo	https://github.com/malllabiisc/ConfGCN
Framework	tf

Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals


Title	Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals
Authors	Thomas Moreau, Alexandre Gramfort
Abstract	Convolutional dictionary learning (CDL) estimates shift invariant basis adapted to multidimensional data. CDL has proven useful for image denoising or inpainting, as well as for pattern discovery on multivariate signals. As estimated patterns can be positioned anywhere in signals or images, optimization techniques face the difficulty of working in extremely high dimensions with millions of pixels or time samples, contrarily to standard patch-based dictionary learning. To address this optimization problem, this work proposes a distributed and asynchronous algorithm, employing locally greedy coordinate descent and an asynchronous locking mechanism that does not require a central server. This algorithm can be used to distribute the computation on a number of workers which scales linearly with the encoded signal’s size. Experiments confirm the scaling properties which allows us to learn patterns on large scales images from the Hubble Space Telescope.
Tasks	Denoising, Dictionary Learning, Image Denoising
Published	2019-01-26
URL	http://arxiv.org/abs/1901.09235v1
PDF	http://arxiv.org/pdf/1901.09235v1.pdf
PWC	https://paperswithcode.com/paper/distributed-convolutional-dictionary-learning
Repo	https://github.com/tommoral/dicodile
Framework	none

FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction


Title	FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
Authors	Tongwen Huang, Zhiqi Zhang, Junlin Zhang
Abstract	Advertising and feed ranking are essential to many Internet companies such as Facebook and Sina Weibo. Among many real-world advertising and feed ranking systems, click through rate (CTR) prediction plays a central role. There are many proposed models in this field such as logistic regression, tree based models, factorization machine based models and deep learning based CTR models. However, many current works calculate the feature interactions in a simple way such as Hadamard product and inner product and they care less about the importance of features. In this paper, a new model named FiBiNET as an abbreviation for Feature Importance and Bilinear feature Interaction NETwork is proposed to dynamically learn the feature importance and fine-grained feature interactions. On the one hand, the FiBiNET can dynamically learn the importance of features via the Squeeze-Excitation network (SENET) mechanism; on the other hand, it is able to effectively learn the feature interactions via bilinear function. We conduct extensive experiments on two real-world datasets and show that our shallow model outperforms other shallow models such as factorization machine(FM) and field-aware factorization machine(FFM). In order to improve performance further, we combine a classical deep neural network(DNN) component with the shallow model to be a deep model. The deep FiBiNET consistently outperforms the other state-of-the-art deep models such as DeepFM and extreme deep factorization machine(XdeepFM).
Tasks	Click-Through Rate Prediction, Feature Importance
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09433v1
PDF	https://arxiv.org/pdf/1905.09433v1.pdf
PWC	https://paperswithcode.com/paper/fibinet-combining-feature-importance-and
Repo	https://github.com/shenweichen/DeepCTR-PyTorch
Framework	pytorch

Identifying Mislabeled Instances in Classification Datasets


Title	Identifying Mislabeled Instances in Classification Datasets
Authors	Nicolas Michael Müller, Karla Markert
Abstract	A key requirement for supervised machine learning is labeled training data, which is created by annotating unlabeled data with the appropriate class. Because this process can in many cases not be done by machines, labeling needs to be performed by human domain experts. This process tends to be expensive both in time and money, and is prone to errors. Additionally, reviewing an entire labeled dataset manually is often prohibitively costly, so many real world datasets contain mislabeled instances. To address this issue, we present in this paper a non-parametric end-to-end pipeline to find mislabeled instances in numerical, image and natural language datasets. We evaluate our system quantitatively by adding a small number of label noise to 29 datasets, and show that we find mislabeled instances with an average precision of more than 0.84 when reviewing our system’s top 1% recommendation. We then apply our system to publicly available datasets and find mislabeled instances in CIFAR-100, Fashion-MNIST, and others. Finally, we publish the code and an applicable implementation of our approach.
Tasks
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05283v1
PDF	https://arxiv.org/pdf/1912.05283v1.pdf
PWC	https://paperswithcode.com/paper/identifying-mislabeled-instances-in
Repo	https://github.com/mueller91/labelfix
Framework	tf

Style Transfer by Relaxed Optimal Transport and Self-Similarity


Title	Style Transfer by Relaxed Optimal Transport and Self-Similarity
Authors	Nicholas Kolkin, Jason Salavon, Greg Shakhnarovich
Abstract	Style transfer algorithms strive to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm. We extend our method to allow user-specified point-to-point or region-to-region control over visual similarity between the style image and the output. Such guidance can be used to either achieve a particular visual effect or correct errors made by unconstrained style transfer. In order to quantitatively compare our method to prior work, we conduct a large-scale user study designed to assess the style-content tradeoff across settings in style transfer algorithms. Our results indicate that for any desired level of content preservation, our method provides higher quality stylization than prior work. Code is available at https://github.com/nkolkin13/STROTSS
Tasks	Style Transfer
Published	2019-04-29
URL	https://arxiv.org/abs/1904.12785v2
PDF	https://arxiv.org/pdf/1904.12785v2.pdf
PWC	https://paperswithcode.com/paper/190412785
Repo	https://github.com/nkolkin13/STROTSS
Framework	pytorch


Title	SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation
Authors	Daniel Gordon, Abhishek Kadian, Devi Parikh, Judy Hoffman, Dhruv Batra
Abstract	We propose SplitNet, a method for decoupling visual perception and policy learning. By incorporating auxiliary tasks and selective learning of portions of the model, we explicitly decompose the learning objectives for visual navigation into perceiving the world and acting on that perception. We show dramatic improvements over baseline models on transferring between simulators, an encouraging step towards Sim2Real. Additionally, SplitNet generalizes better to unseen environments from the same simulator and transfers faster and more effectively to novel embodied navigation tasks. Further, given only a small sample from a target domain, SplitNet can match the performance of traditional end-to-end pipelines which receive the entire dataset. Code is available https://github.com/facebookresearch/splitnet
Tasks	Visual Navigation
Published	2019-05-18
URL	https://arxiv.org/abs/1905.07512v3
PDF	https://arxiv.org/pdf/1905.07512v3.pdf
PWC	https://paperswithcode.com/paper/splitnet-sim2sim-and-task2task-transfer-for
Repo	https://github.com/facebookresearch/splitnet
Framework	pytorch