January 29, 2020

3206 words 16 mins read

Paper Group ANR 682

Not Enough Data? Deep Learning to the Rescue!. Approximate LSTMs for Time-Constrained Inference: Enabling Fast Reaction in Self-Driving Cars. FeCaffe: FPGA-enabled Caffe with OpenCL for Deep Learning Training and Inference on Intel Stratix 10. Massively Parallel Benders Decomposition for Correlation Clustering. Locally Linear Image Structural Embed …

Not Enough Data? Deep Learning to the Rescue!


Title	Not Enough Data? Deep Learning to the Rescue!
Authors	Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, Naama Zwerdling
Abstract	Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-tuning a state-of-the-art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers’ performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-the-art techniques for data augmentation, specifically those applicable to text classification tasks with little data.
Tasks	Data Augmentation, Language Modelling, Text Classification, Text Generation
Published	2019-11-08
URL	https://arxiv.org/abs/1911.03118v2
PDF	https://arxiv.org/pdf/1911.03118v2.pdf
PWC	https://paperswithcode.com/paper/not-enough-data-deep-learning-to-the-rescue
Repo
Framework

Approximate LSTMs for Time-Constrained Inference: Enabling Fast Reaction in Self-Driving Cars


Title	Approximate LSTMs for Time-Constrained Inference: Enabling Fast Reaction in Self-Driving Cars
Authors	Alexandros Kouris, Stylianos I. Venieris, Michail Rizakis, Christos-Savvas Bouganis
Abstract	The need to recognise long-term dependencies in sequential data such as video streams has made Long Short-Term Memory (LSTM) networks a prominent Artificial Intelligence model for many emerging applications. However, the high computational and memory demands of LSTMs introduce challenges in their deployment on latency-critical systems such as self-driving cars which are equipped with limited computational resources on-board. In this paper, we introduce a progressive inference computing scheme that combines model pruning and computation restructuring leading to the best possible approximation of the result given the available latency budget of the target application. The proposed methodology enables mission-critical systems to make informed decisions even in early stages of the computation, based on approximate LSTM inference, meeting their specifications on safety and robustness. Our experiments on a state-of-the-art driving model for autonomous vehicle navigation demonstrate that the proposed approach can yield outputs with similar quality of result compared to a faithful LSTM baseline, up to 415x faster (198x on average, 76x geo. mean).
Tasks	Autonomous Navigation, Self-Driving Cars
Published	2019-05-02
URL	https://arxiv.org/abs/1905.00689v2
PDF	https://arxiv.org/pdf/1905.00689v2.pdf
PWC	https://paperswithcode.com/paper/approximate-lstms-for-time-constrained
Repo
Framework

FeCaffe: FPGA-enabled Caffe with OpenCL for Deep Learning Training and Inference on Intel Stratix 10


Title	FeCaffe: FPGA-enabled Caffe with OpenCL for Deep Learning Training and Inference on Intel Stratix 10
Authors	Ke He, Bo Liu, Yu Zhang, Andrew Ling, Dian Gu
Abstract	Deep learning and Convolutional Neural Network (CNN) have becoming increasingly more popular and important in both academic and industrial areas in recent years cause they are able to provide better accuracy and result in classification, detection and recognition areas, compared to traditional approaches. Currently, there are many popular frameworks in the market for deep learning development, such as Caffe, TensorFlow, Pytorch, and most of frameworks natively support CPU and consider GPU as the mainline accelerator by default. FPGA device, viewed as a potential heterogeneous platform, still cannot provide a comprehensive support for CNN development in popular frameworks, in particular to the training phase. In this paper, we firstly propose the FeCaffe, i.e. FPGA-enabled Caffe, a hierarchical software and hardware design methodology based on the Caffe to enable FPGA to support mainline deep learning development features, e.g. training and inference with Caffe. Furthermore, we provide some benchmarks with FeCaffe by taking some classical CNN networks as examples, and further analysis of kernel execution time in details accordingly. Finally, some optimization directions including FPGA kernel design, system pipeline, network architecture, user case application and heterogeneous platform levels, have been proposed gradually to improve FeCaffe performance and efficiency. The result demonstrates the proposed FeCaffe is capable of supporting almost full features during CNN network training and inference respectively with high degree of design flexibility, expansibility and reusability for deep learning development. Compared to prior studies, our architecture can support more network and training settings, and current configuration can achieve 6.4x and 8.4x average execution time improvement for forward and backward respectively for LeNet.
Tasks
Published	2019-11-18
URL	https://arxiv.org/abs/1911.08905v1
PDF	https://arxiv.org/pdf/1911.08905v1.pdf
PWC	https://paperswithcode.com/paper/fecaffe-fpga-enabled-caffe-with-opencl-for
Repo
Framework

Massively Parallel Benders Decomposition for Correlation Clustering


Title	Massively Parallel Benders Decomposition for Correlation Clustering
Authors	Margret Keuper, Jovita Lukasik, Maneesh Singh, Julian Yarkony
Abstract	We tackle the problem of graph partitioning for image segmentation using correlation clustering (CC), which we treat as an integer linear program (ILP). We reformulate optimization in the ILP so as to admit efficient optimization via Benders decomposition, a classic technique from operations research. Our Benders decomposition formulation has many subproblems, each associated with a node in the CC instance’s graph, which are solved in parallel. Each Benders subproblem enforces the cycle inequalities corresponding to the negative weight edges attached to its corresponding node in the CC instance. We generate Magnanti-Wong Benders rows in addition to standard Benders rows, to accelerate optimization. Our Benders decomposition approach provides a promising new avenue to accelerate optimization for CC, and allows for massive parallelization.
Tasks	graph partitioning, Semantic Segmentation
Published	2019-02-15
URL	https://arxiv.org/abs/1902.05659v2
PDF	https://arxiv.org/pdf/1902.05659v2.pdf
PWC	https://paperswithcode.com/paper/massively-parallel-benders-decomposition-for
Repo
Framework

Locally Linear Image Structural Embedding for Image Structure Manifold Learning


Title	Locally Linear Image Structural Embedding for Image Structure Manifold Learning
Authors	Benyamin Ghojogh, Fakhri Karray, Mark Crowley
Abstract	Most of existing manifold learning methods rely on Mean Squared Error (MSE) or $\ell_2$ norm. However, for the problem of image quality assessment, these are not promising measure. In this paper, we introduce the concept of an image structure manifold which captures image structure features and discriminates image distortions. We propose a new manifold learning method, Locally Linear Image Structural Embedding (LLISE), and kernel LLISE for learning this manifold. The LLISE is inspired by Locally Linear Embedding (LLE) but uses SSIM rather than MSE. This paper builds a bridge between manifold learning and image fidelity assessment and it can open a new area for future investigations.
Tasks	Image Quality Assessment
Published	2019-08-25
URL	https://arxiv.org/abs/1908.09288v1
PDF	https://arxiv.org/pdf/1908.09288v1.pdf
PWC	https://paperswithcode.com/paper/locally-linear-image-structural-embedding-for
Repo
Framework

Algorithm Selection for Image Quality Assessment


Title	Algorithm Selection for Image Quality Assessment
Authors	Markus Wagner, Hanhe Lin, Shujun Li, Dietmar Saupe
Abstract	Subjective perceptual image quality can be assessed in lab studies by human observers. Objective image quality assessment (IQA) refers to algorithms for estimation of the mean subjective quality ratings. Many such methods have been proposed, both for blind IQA in which no original reference image is available as well as for the full-reference case. We compared 8 state-of-the-art algorithms for blind IQA and showed that an oracle, able to predict the best performing method for any given input image, yields a hybrid method that could outperform even the best single existing method by a large margin. In this contribution we address the research question whether established methods to learn such an oracle can improve blind IQA. We applied AutoFolio, a state-of-the-art system that trains an algorithm selector to choose a well-performing algorithm for a given instance. We also trained deep neural networks to predict the best method. Our results did not give a positive answer, algorithm selection did not yield a significant improvement over the single best method. Looking into the results in depth, we observed that the noise in images may have played a role in why our trained classifiers could not predict the oracle. This motivates the consideration of noisiness in IQA methods, a property that has so far not been observed and that opens up several interesting new research questions and applications.
Tasks	Image Quality Assessment
Published	2019-08-19
URL	https://arxiv.org/abs/1908.06911v1
PDF	https://arxiv.org/pdf/1908.06911v1.pdf
PWC	https://paperswithcode.com/paper/algorithm-selection-for-image-quality
Repo
Framework

No-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement


Title	No-Reference Light Field Image Quality Assessment Based on Spatial-Angular Measurement
Authors	Likun Shi, Wei Zhou, Zhibo Chen
Abstract	Light field image quality assessment (LFI-QA) is a significant and challenging research problem. It helps to better guide light field acquisition, processing and applications. However, only a few objective models have been proposed and none of them completely consider intrinsic factors affecting the LFI quality. In this paper, we propose a No-Reference Light Field image Quality Assessment (NR-LFQA) scheme, where the main idea is to quantify the LFI quality degradation through evaluating the spatial quality and angular consistency. We first measure the spatial quality deterioration by capturing the naturalness distribution of the light field cyclopean image array, which is formed when human observes the LFI. Then, as a transformed representation of LFI, the Epipolar Plane Image (EPI) contains the slopes of lines and involves the angular information. Therefore, EPI is utilized to extract the global and local features from LFI to measure angular consistency degradation. Specifically, the distribution of gradient direction map of EPI is proposed to measure the global angular consistency distortion in the LFI. We further propose the weighted local binary pattern to capture the characteristics of local angular consistency degradation. Extensive experimental results on four publicly available LFI quality datasets demonstrate that the proposed method outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.
Tasks	Image Quality Assessment
Published	2019-08-17
URL	https://arxiv.org/abs/1908.06280v1
PDF	https://arxiv.org/pdf/1908.06280v1.pdf
PWC	https://paperswithcode.com/paper/no-reference-light-field-image-quality
Repo
Framework

A comprehensive evaluation of full-reference image quality assessment algorithms on KADID-10k


Title	A comprehensive evaluation of full-reference image quality assessment algorithms on KADID-10k
Authors	Domonkos Varga
Abstract	Significant progress has been made in the past decade for full-reference image quality assessment (FR-IQA). However, new large scale image quality databases have been released for evaluating image quality assessment algorithms. In this study, our goal is to give a comprehensive evaluation of state-of-the-art FR-IQA metrics using the recently published KADID-10k database which is largest available one at the moment. Our evaluation results and the associated discussions is very helpful to obtain a clear understanding about the status of state-of-the-art FR-IQA metrics.
Tasks	Image Quality Assessment
Published	2019-07-03
URL	https://arxiv.org/abs/1907.02096v1
PDF	https://arxiv.org/pdf/1907.02096v1.pdf
PWC	https://paperswithcode.com/paper/a-comprehensive-evaluation-of-full-reference
Repo
Framework

Dynamic Prediction of ICU Mortality Risk Using Domain Adaptation


Title	Dynamic Prediction of ICU Mortality Risk Using Domain Adaptation
Authors	Tiago Alves, Alberto Laender, Adriano Veloso, Nivio Ziviani
Abstract	Early recognition of risky trajectories during an Intensive Care Unit (ICU) stay is one of the key steps towards improving patient survival. Learning trajectories from physiological signals continuously measured during an ICU stay requires learning time-series features that are robust and discriminative across diverse patient populations. Patients within different ICU populations (referred here as domains) vary by age, conditions and interventions. Thus, mortality prediction models using patient data from a particular ICU population may perform suboptimally in other populations because the features used to train such models have different distributions across the groups. In this paper, we explore domain adaptation strategies in order to learn mortality prediction models that extract and transfer complex temporal features from multivariate time-series ICU data. Features are extracted in a way that the state of the patient in a certain time depends on the previous state. This enables dynamic predictions and creates a mortality risk space that describes the risk of a patient at a particular time. Experiments based on cross-ICU populations reveals that our model outperforms all considered baselines. Gains in terms of AUC range from 4% to 8% for early predictions when compared with a recent state-of-the-art representative for ICU mortality prediction. In particular, models for the Cardiac ICU population achieve AUC numbers as high as 0.88, showing excellent clinical utility for early mortality prediction. Finally, we present an explanation of factors contributing to the possible ICU outcomes, so that our models can be used to complement clinical reasoning.
Tasks	Domain Adaptation, Mortality Prediction, Time Series
Published	2019-12-20
URL	https://arxiv.org/abs/1912.10080v1
PDF	https://arxiv.org/pdf/1912.10080v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-prediction-of-icu-mortality-risk
Repo
Framework

Ludii as a Competition Platform


Title	Ludii as a Competition Platform
Authors	Matthew Stephenson, Éric Piette, Dennis J. N. J. Soemers, Cameron Browne
Abstract	Ludii is a general game system being developed as part of the ERC-funded Digital Ludeme Project (DLP). While its primary aim is to model, play, and analyse the full range of traditional strategy games, Ludii also has the potential to support a wide range of AI research topics and competitions. This paper describes some of the future competitions and challenges that we intend to run using the Ludii system, highlighting some of its most important aspects that can potentially lead to many algorithm improvements and new avenues of research. We compare and contrast our proposed competition motivations, goals and frameworks against those of existing general game playing competitions, addressing the strengths and weaknesses of each platform.
Tasks
Published	2019-06-29
URL	https://arxiv.org/abs/1907.00246v1
PDF	https://arxiv.org/pdf/1907.00246v1.pdf
PWC	https://paperswithcode.com/paper/ludii-as-a-competition-platform
Repo
Framework

CompactNet: Platform-Aware Automatic Optimization for Convolutional Neural Networks


Title	CompactNet: Platform-Aware Automatic Optimization for Convolutional Neural Networks
Authors	Weicheng Li, Rui Wang, Zhongzhi Luan, Di Huang, Zidong Du, Yunji Chen, Depei Qian
Abstract	Convolutional Neural Network (CNN) based Deep Learning (DL) has achieved great progress in many real-life applications. Meanwhile, due to the complex model structures against strict latency and memory restriction, the implementation of CNN models on the resource-limited platforms is becoming more challenging. This work proposes a solution, called CompactNet\footnote{Project URL: \url{https://github.com/CompactNet/CompactNet}}, which automatically optimizes a pre-trained CNN model on a specific resource-limited platform given a specific target of inference speedup. Guided by a simulator of the target platform, CompactNet progressively trims a pre-trained network by removing certain redundant filters until the target speedup is reached and generates an optimal platform-specific model while maintaining the accuracy. We evaluate our work on two platforms of a mobile ARM CPU and a machine learning accelerator NPU (Cambricon-1A ISA) on a Huawei Mate10 smartphone. For the state-of-the-art slim CNN model made for the embedded platform, MobileNetV2, CompactNet achieves up to a 1.8x kernel computation speedup with equal or even higher accuracy for image classification tasks on the Cifar-10 dataset.
Tasks	Image Classification
Published	2019-05-28
URL	https://arxiv.org/abs/1905.11669v1
PDF	https://arxiv.org/pdf/1905.11669v1.pdf
PWC	https://paperswithcode.com/paper/compactnet-platform-aware-automatic
Repo
Framework

Representation Transfer for Differentially Private Drug Sensitivity Prediction


Title	Representation Transfer for Differentially Private Drug Sensitivity Prediction
Authors	Teppo Niinimäki, Mikko Heikkilä, Antti Honkela, Samuel Kaski
Abstract	Motivation: Human genomic datasets often contain sensitive information that limits use and sharing of the data. In particular, simple anonymisation strategies fail to provide sufficient level of protection for genomic data, because the data are inherently identifiable. Differentially private machine learning can help by guaranteeing that the published results do not leak too much information about any individual data point. Recent research has reached promising results on differentially private drug sensitivity prediction using gene expression data. Differentially private learning with genomic data is challenging because it is more difficult to guarantee the privacy in high dimensions. Dimensionality reduction can help, but if the dimension reduction mapping is learned from the data, then it needs to be differentially private too, which can carry a significant privacy cost. Furthermore, the selection of any hyperparameters (such as the target dimensionality) needs to also avoid leaking private information. Results: We study an approach that uses a large public dataset of similar type to learn a compact representation for differentially private learning. We compare three representation learning methods: variational autoencoders, PCA and random projection. We solve two machine learning tasks on gene expression of cancer cell lines: cancer type classification, and drug sensitivity prediction. The experiments demonstrate significant benefit from all representation learning methods with variational autoencoders providing the most accurate predictions most often. Our results significantly improve over previous state-of-the-art in accuracy of differentially private drug sensitivity prediction.
Tasks	Dimensionality Reduction, Representation Learning
Published	2019-01-29
URL	http://arxiv.org/abs/1901.10227v1
PDF	http://arxiv.org/pdf/1901.10227v1.pdf
PWC	https://paperswithcode.com/paper/representation-transfer-for-differentially
Repo
Framework

A Convergence Analysis of Nonlinearly Constrained ADMM in Deep Learning


Title	A Convergence Analysis of Nonlinearly Constrained ADMM in Deep Learning
Authors	Jinshan Zeng, Shao-Bo Lin, Yuan Yao
Abstract	Efficient training of deep neural networks (DNNs) is a challenge due to the associated highly nonconvex optimization. The alternating direction method of multipliers (ADMM) has attracted rising attention in deep learning for its potential of distributed computing. However, it remains an open problem to establish the convergence of ADMM in DNN training due to the nonlinear constraints involved. In this paper, we provide an answer to this problem by establishing the convergence of some nonlinearly constrained ADMM for DNNs with smooth activations. To be specific, we establish the global convergence to a Karush-Kuhn-Tucker (KKT) point at a ${\cal O}(1/k)$ rate. To achieve this goal, the key development lies in a new local linear approximation technique which enables us to overcome the hurdle of nonlinear constraints in ADMM for DNNs.
Tasks
Published	2019-02-06
URL	http://arxiv.org/abs/1902.02060v1
PDF	http://arxiv.org/pdf/1902.02060v1.pdf
PWC	https://paperswithcode.com/paper/a-convergence-analysis-of-nonlinearly
Repo
Framework

Evaluating Usage of Images for App Classification


Title	Evaluating Usage of Images for App Classification
Authors	Kushal Singla, Niloy Mukherjee, Hari Manassery Koduvely, Joy Bose
Abstract	App classification is useful in a number of applications such as adding apps to an app store or building a user model based on the installed apps. Presently there are a number of existing methods to classify apps based on a given taxonomy on the basis of their text metadata. However, text based methods for app classification may not work in all cases, such as when the text descriptions are in a different language, or missing, or inadequate to classify the app. One solution in such cases is to utilize the app images to supplement the text description. In this paper, we evaluate a number of approaches in which app images can be used to classify the apps. In one approach, we use Optical character recognition (OCR) to extract text from images, which is then used to supplement the text description of the app. In another, we use pic2vec to convert the app images into vectors, then train an SVM to classify the vectors to the correct app label. In another, we use the captionbot.ai tool to generate natural language descriptions from the app images. Finally, we use a method to detect and label objects in the app images and use a voting technique to determine the category of the app based on all the images. We compare the performance of our image-based techniques to classify a number of apps in our dataset. We use a text based SVM app classifier as our base and obtained an improved classification accuracy of 96% for some classes when app images are added.
Tasks	Optical Character Recognition
Published	2019-12-16
URL	https://arxiv.org/abs/1912.12144v1
PDF	https://arxiv.org/pdf/1912.12144v1.pdf
PWC	https://paperswithcode.com/paper/evaluating-usage-of-images-for-app
Repo
Framework

Wasserstein Index Generation Model: Automatic Generation of Time-series Index with Application to Economic Policy Uncertainty


Title	Wasserstein Index Generation Model: Automatic Generation of Time-series Index with Application to Economic Policy Uncertainty
Authors	Fangzhou Xie
Abstract	I propose a novel method, the Wasserstein Index Generation model (WIG), to generate a public sentiment index automatically. To test the model`s effectiveness, an application to generate Economic Policy Uncertainty (EPU) index is showcased. \|
Tasks	Time Series
Published	2019-08-12
URL	https://arxiv.org/abs/1908.04369v4
PDF	https://arxiv.org/pdf/1908.04369v4.pdf
PWC	https://paperswithcode.com/paper/wasserstein-index-generation-model-automatic
Repo
Framework