January 30, 2020

3212 words 16 mins read

Paper Group ANR 336

Improving Differentially Private Models with Active Learning. The Design and Implementation of a Scalable DL Benchmarking Platform. Multi-Domain Translation by Learning Uncoupled Autoencoders. Robust, Extensible, and Fast: Teamed Classifiers for Vehicle Tracking and Vehicle Re-ID in Multi-Camera Networks. Efficient Feature Selection of Power Qualit …

Improving Differentially Private Models with Active Learning


Title	Improving Differentially Private Models with Active Learning
Authors	Zhengli Zhao, Nicolas Papernot, Sameer Singh, Neoklis Polyzotis, Augustus Odena
Abstract	Broad adoption of machine learning techniques has increased privacy concerns for models trained on sensitive data such as medical records. Existing techniques for training differentially private (DP) models give rigorous privacy guarantees, but applying these techniques to neural networks can severely degrade model performance. This performance reduction is an obstacle to deploying private models in the real world. In this work, we improve the performance of DP models by fine-tuning them through active learning on public data. We introduce two new techniques - DIVERSEPUBLIC and NEARPRIVATE - for doing this fine-tuning in a privacy-aware way. For the MNIST and SVHN datasets, these techniques improve state-of-the-art accuracy for DP models while retaining privacy guarantees.
Tasks	Active Learning
Published	2019-10-02
URL	https://arxiv.org/abs/1910.01177v1
PDF	https://arxiv.org/pdf/1910.01177v1.pdf
PWC	https://paperswithcode.com/paper/improving-differentially-private-models-with
Repo
Framework

The Design and Implementation of a Scalable DL Benchmarking Platform


Title	The Design and Implementation of a Scalable DL Benchmarking Platform
Authors	Cheng Li, Abdul Dakkak, Jinjun Xiong, Wen-mei Hwu
Abstract	The current Deep Learning (DL) landscape is fast-paced and is rife with non-uniform models, hardware/software (HW/SW) stacks, but lacks a DL benchmarking platform to facilitate evaluation and comparison of DL innovations, be it models, frameworks, libraries, or hardware. Due to the lack of a benchmarking platform, the current practice of evaluating the benefits of proposed DL innovations is both arduous and error-prone - stifling the adoption of the innovations. In this work, we first identify $10$ design features which are desirable within a DL benchmarking platform. These features include: performing the evaluation in a consistent, reproducible, and scalable manner, being framework and hardware agnostic, supporting real-world benchmarking workloads, providing in-depth model execution inspection across the HW/SW stack levels, etc. We then propose MLModelScope, a DL benchmarking platform design that realizes the $10$ objectives. MLModelScope proposes a specification to define DL model evaluations and techniques to provision the evaluation workflow using the user-specified HW/SW stack. MLModelScope defines abstractions for frameworks and supports board range of DL models and evaluation scenarios. We implement MLModelScope as an open-source project with support for all major frameworks and hardware architectures. Through MLModelScope’s evaluation and automated analysis workflows, we performed case-study analyses of $37$ models across $4$ systems and show how model, hardware, and framework selection affects model accuracy and performance under different benchmarking scenarios. We further demonstrated how MLModelScope’s tracing capability gives a holistic view of model execution and helps pinpoint bottlenecks.
Tasks
Published	2019-11-19
URL	https://arxiv.org/abs/1911.08031v1
PDF	https://arxiv.org/pdf/1911.08031v1.pdf
PWC	https://paperswithcode.com/paper/the-design-and-implementation-of-a-scalable
Repo
Framework

Multi-Domain Translation by Learning Uncoupled Autoencoders


Title	Multi-Domain Translation by Learning Uncoupled Autoencoders
Authors	Karren D. Yang, Caroline Uhler
Abstract	Multi-domain translation seeks to learn a probabilistic coupling between marginal distributions that reflects the correspondence between different domains. We assume that data from different domains are generated from a shared latent representation based on a structural equation model. Under this assumption, we show that the problem of computing a probabilistic coupling between marginals is equivalent to learning multiple uncoupled autoencoders that embed to a given shared latent distribution. In addition, we propose a new framework and algorithm for multi-domain translation based on learning the shared latent distribution and training autoencoders under distributional constraints. A key practical advantage of our framework is that new autoencoders (i.e., new domains) can be added sequentially to the model without retraining on the other domains, which we demonstrate experimentally on image as well as genomics datasets.
Tasks
Published	2019-02-09
URL	http://arxiv.org/abs/1902.03515v1
PDF	http://arxiv.org/pdf/1902.03515v1.pdf
PWC	https://paperswithcode.com/paper/multi-domain-translation-by-learning
Repo
Framework

Robust, Extensible, and Fast: Teamed Classifiers for Vehicle Tracking and Vehicle Re-ID in Multi-Camera Networks


Title	Robust, Extensible, and Fast: Teamed Classifiers for Vehicle Tracking and Vehicle Re-ID in Multi-Camera Networks
Authors	Abhijit Suprem, Rodrigo Alves Lima, Bruno Padilha, Joao Eduardo Ferreira, Calton Pu
Abstract	As camera networks have become more ubiquitous over the past decade, the research interest in video management has shifted to analytics on multi-camera networks. This includes performing tasks such as object detection, attribute identification, and vehicle/person tracking across different cameras without overlap. Current frameworks for management are designed for multi-camera networks in a closed dataset environment where there is limited variability in cameras and characteristics of the surveillance environment are well known. Furthermore, current frameworks are designed for offline analytics with guidance from human operators for forensic applications. This paper presents a teamed classifier framework for video analytics in heterogeneous many-camera networks with adversarial conditions such as multi-scale, multi-resolution cameras capturing the environment with varying occlusion, blur, and orientations. We describe an implementation for vehicle tracking and vehicle re-identification (re-id), where we implement a zero-shot learning (ZSL) system that performs automated tracking of all vehicles all the time. Our evaluations on VeRi-776 and Cars196 show the teamed classifier framework is robust to adversarial conditions, extensible to changing video characteristics such as new vehicle types/brands and new cameras, and offers real-time performance compared to current offline video analytics approaches.
Tasks	Object Detection, Vehicle Re-Identification, Zero-Shot Learning
Published	2019-12-09
URL	https://arxiv.org/abs/1912.04423v2
PDF	https://arxiv.org/pdf/1912.04423v2.pdf
PWC	https://paperswithcode.com/paper/robust-extensible-and-fast-teamed-classifiers
Repo
Framework

Efficient Feature Selection of Power Quality Events using Two Dimensional (2D) Particle Swarms


Title	Efficient Feature Selection of Power Quality Events using Two Dimensional (2D) Particle Swarms
Authors	Faizal Hafiz, Akshya Swain, Chirag Naik, Nitish Patel
Abstract	A novel two-dimensional (2D) learning framework has been proposed to address the feature selection problem in Power Quality (PQ) events. Unlike the existing feature selection approaches, the proposed 2D learning explicitly incorporates the information about the subset cardinality (i.e., the number of features) as an additional learning dimension to effectively guide the search process. The efficacy of this approach has been demonstrated considering fourteen distinct classes of PQ events which conform to the IEEE Standard 1159. The search performance of the 2D learning approach has been compared to the other six well-known feature selection wrappers by considering two induction algorithms: Naive Bayes (NB) and k-Nearest Neighbors (k-NN). Further, the robustness of the selected/reduced feature subsets has been investigated considering seven different levels of noise. The results of this investigation convincingly demonstrate that the proposed 2D learning can identify significantly better and robust feature subsets for PQ events.
Tasks	Feature Selection
Published	2019-04-15
URL	http://arxiv.org/abs/1904.06972v1
PDF	http://arxiv.org/pdf/1904.06972v1.pdf
PWC	https://paperswithcode.com/paper/efficient-feature-selection-of-power-quality
Repo
Framework

Combining Acoustics, Content and Interaction Features to Find Hot Spots in Meetings


Title	Combining Acoustics, Content and Interaction Features to Find Hot Spots in Meetings
Authors	Dave Makhervaks, William Hinthorn, Dimitrios Dimitriadis, Andreas Stolcke
Abstract	Involvement hot spots have been proposed as a useful concept for meeting analysis and studied off and on for over 15 years. These are regions of meetings that are marked by high participant involvement, as judged by human annotators. However, prior work was either not conducted in a formal machine learning setting, or focused on only a subset of possible meeting features or downstream applications (such as summarization). In this paper we investigate to what extent various acoustic, linguistic and pragmatic aspects of the meetings, both in isolation and jointly, can help detect hot spots. In this context, the openSMILE toolkit is to used to extract features based on acoustic-prosodic cues, BERT word embeddings are used for encoding the lexical content, and a variety of statistics based on speech activity are used to describe the verbal interaction among participants. In experiments on the annotated ICSI meeting corpus, we find that the lexical model is the most informative, with incremental contributions from interaction and acoustic-prosodic model components.
Tasks	Word Embeddings
Published	2019-10-24
URL	https://arxiv.org/abs/1910.10869v2
PDF	https://arxiv.org/pdf/1910.10869v2.pdf
PWC	https://paperswithcode.com/paper/combining-acoustics-content-and-interaction
Repo
Framework

CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery


Title	CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery
Authors	Gongjie Zhang, Shijian Lu, Wei Zhang
Abstract	Accurate and robust detection of multi-class objects in optical remote sensing images is essential to many real-world applications such as urban planning, traffic control, searching and rescuing, etc. However, state-of-the-art object detection techniques designed for images captured using ground-level sensors usually experience a sharp performance drop when directly applied to remote sensing images, largely due to the object appearance differences in remote sensing images in term of sparse texture, low contrast, arbitrary orientations, large scale variations, etc. This paper presents a novel object detection network (CAD-Net) that exploits attention-modulated features as well as global and local contexts to address the new challenges in detecting objects from remote sensing images. The proposed CAD-Net learns global and local contexts of objects by capturing their correlations with the global scene (at scene-level) and the local neighboring objects or features (at object-level), respectively. In addition, it designs a spatial-and-scale-aware attention module that guides the network to focus on more informative regions and features as well as more appropriate feature scales. Experiments over two publicly available object detection datasets for remote sensing images demonstrate that the proposed CAD-Net achieves superior detection performance. The implementation codes will be made publicly available for facilitating future researches.
Tasks	Object Detection
Published	2019-03-03
URL	http://arxiv.org/abs/1903.00857v1
PDF	http://arxiv.org/pdf/1903.00857v1.pdf
PWC	https://paperswithcode.com/paper/cad-net-a-context-aware-detection-network-for
Repo
Framework

Large-Scale Noun Compound Interpretation Using Bootstrapping and the Web as a Corpus


Title	Large-Scale Noun Compound Interpretation Using Bootstrapping and the Web as a Corpus
Authors	Su Nam Kim, Preslav Nakov
Abstract	Responding to the need for semantic lexical resources in natural language processing applications, we examine methods to acquire noun compounds (NCs), e.g., “orange juice”, together with suitable fine-grained semantic interpretations, e.g., “squeezed from”, which are directly usable as paraphrases. We employ bootstrapping and web statistics, and utilize the relationship between NCs and paraphrasing patterns to jointly extract NCs and such patterns in multiple alternating iterations. In evaluation, we found that having one compound noun fixed yields both a higher number of semantically interpreted NCs and improved accuracy due to stronger semantic restrictions.
Tasks
Published	2019-11-27
URL	https://arxiv.org/abs/1911.12085v1
PDF	https://arxiv.org/pdf/1911.12085v1.pdf
PWC	https://paperswithcode.com/paper/large-scale-noun-compound-interpretation
Repo
Framework

AGRR-2019: A Corpus for Gapping Resolution in Russian


Title	AGRR-2019: A Corpus for Gapping Resolution in Russian
Authors	Maria Ponomareva, Kira Droganova, Ivan Smurov, Tatiana Shavrina
Abstract	This paper provides a comprehensive overview of the gapping dataset for Russian that consists of 7.5k sentences with gapping (as well as 15k relevant negative sentences) and comprises data from various genres: news, fiction, social media and technical texts. The dataset was prepared for the Automatic Gapping Resolution Shared Task for Russian (AGRR-2019) - a competition aimed at stimulating the development of NLP tools and methods for processing of ellipsis. In this paper, we pay special attention to the gapping resolution methods that were introduced within the shared task as well as an alternative test set that illustrates that our corpus is a diverse and representative subset of Russian language gapping sufficient for effective utilization of machine learning techniques.
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04099v1
PDF	https://arxiv.org/pdf/1906.04099v1.pdf
PWC	https://paperswithcode.com/paper/agrr-2019-a-corpus-for-gapping-resolution-in
Repo
Framework

k is the Magic Number – Inferring the Number of Clusters Through Nonparametric Concentration Inequalities


Title	k is the Magic Number – Inferring the Number of Clusters Through Nonparametric Concentration Inequalities
Authors	Sibylle Hess, Wouter Duivesteijn
Abstract	Most convex and nonconvex clustering algorithms come with one crucial parameter: the $k$ in $k$-means. To this day, there is not one generally accepted way to accurately determine this parameter. Popular methods are simple yet theoretically unfounded, such as searching for an elbow in the curve of a given cost measure. In contrast, statistically founded methods often make strict assumptions over the data distribution or come with their own optimization scheme for the clustering objective. This limits either the set of applicable datasets or clustering algorithms. In this paper, we strive to determine the number of clusters by answering a simple question: given two clusters, is it likely that they jointly stem from a single distribution? To this end, we propose a bound on the probability that two clusters originate from the distribution of the unified cluster, specified only by the sample mean and variance. Our method is applicable as a simple wrapper to the result of any clustering method minimizing the objective of $k$-means, which includes Gaussian mixtures and Spectral Clustering. We focus in our experimental evaluation on an application for nonconvex clustering and demonstrate the suitability of our theoretical results. Our \textsc{SpecialK} clustering algorithm automatically determines the appropriate value for $k$, without requiring any data transformation or projection, and without assumptions on the data distribution. Additionally, it is capable to decide that the data consists of only a single cluster, which many existing algorithms cannot.
Tasks
Published	2019-07-04
URL	https://arxiv.org/abs/1907.02343v1
PDF	https://arxiv.org/pdf/1907.02343v1.pdf
PWC	https://paperswithcode.com/paper/k-is-the-magic-number-inferring-the-number-of
Repo
Framework

Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification


Title	Stripe-based and Attribute-aware Network: A Two-Branch Deep Model for Vehicle Re-identification
Authors	Jingjing Qian, Wei Jiang, Hao Luo, Hongyan Yu
Abstract	Vehicle re-identification (Re-ID) has been attracting increasing interest in the field of computer vision due to the growing utilization of surveillance cameras in public security. However, vehicle Re-ID still suffers a similarity challenge despite the efforts made to solve this problem. This challenge involves distinguishing different instances with nearly identical appearances. In this paper, we propose a novel two-branch stripe-based and attribute-aware deep convolutional neural network (SAN) to learn the efficient feature embedding for vehicle Re-ID task. The two-branch neural network, consisting of stripe-based branch and attribute-aware branches, can adaptively extract the discriminative features from the visual appearance of vehicles. A horizontal average pooling and dimension-reduced convolutional layers are inserted into the stripe-based branch to achieve part-level features. Meanwhile, the attribute-aware branch extracts the global feature under the supervision of vehicle attribute labels to separate the similar vehicle identities with different attribute annotations. Finally, the part-level and global features are concatenated together to form the final descriptor of the input image for vehicle Re-ID. The final descriptor not only can separate vehicles with different attributes but also distinguish vehicle identities with the same attributes. The extensive experiments on both VehicleID and VeRi databases show that the proposed SAN method outperforms other state-of-the-art vehicle Re-ID approaches.
Tasks	Vehicle Re-Identification
Published	2019-10-12
URL	https://arxiv.org/abs/1910.05549v1
PDF	https://arxiv.org/pdf/1910.05549v1.pdf
PWC	https://paperswithcode.com/paper/stripe-based-and-attribute-aware-network-a
Repo
Framework

Keyframe-based Direct Thermal-Inertial Odometry


Title	Keyframe-based Direct Thermal-Inertial Odometry
Authors	Shehryar Khattak, Christos Papachristos, Kostas Alexis
Abstract	This paper proposes an approach for fusing direct radiometric data from a thermal camera with inertial measurements to extend the robotic capabilities of aerial robots for navigation in GPS-denied and visually degraded environments in the conditions of darkness and in the presence of airborne obscurants such as dust, fog and smoke. An optimization based approach is developed that jointly minimizes the re-projection error of 3D landmarks and inertial measurement errors. The developed solution is extensively verified against both ground-truth in an indoor laboratory setting, as well as inside an underground mine under severely visually degraded conditions.
Tasks
Published	2019-03-03
URL	http://arxiv.org/abs/1903.00798v1
PDF	http://arxiv.org/pdf/1903.00798v1.pdf
PWC	https://paperswithcode.com/paper/keyframe-based-direct-thermal-inertial
Repo
Framework


Title	Privacy-Preserving Blockchain Based Federated Learning with Differential Data Sharing
Authors	Anudit Nagar
Abstract	For the modern world where data is becoming one of the most valuable assets, robust data privacy policies rooted in the fundamental infrastructure of networks and applications are becoming an even bigger necessity to secure sensitive user data. In due course with the ever-evolving nature of newer statistical techniques infringing user privacy, machine learning models with algorithms built with respect for user privacy can offer a dynamically adaptive solution to preserve user privacy against the exponentially increasing multidimensional relationships that datasets create. Using these privacy aware ML Models at the core of a Federated Learning Ecosystem can enable the entire network to learn from data in a decentralized manner. By harnessing the ever-increasing computational power of mobile devices, increasing network reliability and IoT devices revolutionizing the smart devices industry, and combining it with a secure and scalable, global learning session backed by a blockchain network with the ability to ensure on-device privacy, we allow any Internet enabled device to participate and contribute data to a global privacy preserving, data sharing network with blockchain technology even allowing the network to reward quality work. This network architecture can also be built on top of existing blockchain networks like Ethereum and Hyperledger, this lets even small startups build enterprise ready decentralized solutions allowing anyone to learn from data across different departments of a company, all the way to thousands of devices participating in a global synchronized learning network.
Tasks
Published	2019-12-10
URL	https://arxiv.org/abs/1912.04859v1
PDF	https://arxiv.org/pdf/1912.04859v1.pdf
PWC	https://paperswithcode.com/paper/privacy-preserving-blockchain-based-federated
Repo
Framework

Unsupervised Scene Adaptation with Memory Regularization in vivo


Title	Unsupervised Scene Adaptation with Memory Regularization in vivo
Authors	Zhedong Zheng, Yi Yang
Abstract	We consider the unsupervised scene adaptation problem of learning from both labeled source data and unlabeled target data. Existing methods focus on minoring the inter-domain gap between the source and target domains. However, the intra-domain knowledge and inherent uncertainty learned by the network are under-explored. In this paper, we propose an orthogonal method, called memory regularization in vivo to exploit the intra-domain knowledge and regularize the model training. Specifically, we refer to the segmentation model itself as the memory module, and minor the discrepancy of the two classifiers, i.e., the primary classifier and the auxiliary classifier, to reduce the prediction inconsistency. Without extra parameters, the proposed method is complementary to the most existing domain adaptation methods and could generally improve the performance of existing methods. Albeit simple, we verify the effectiveness of memory regularization on two synthetic-to-real benchmarks: GTA5 -> Cityscapes and SYNTHIA -> Cityscapes, yielding +11.1% and +11.3% mIoU improvement over the baseline model, respectively. Besides, a similar +12.0% mIoU improvement is observed on the cross-city benchmark: Cityscapes -> Oxford RobotCar.
Tasks	Domain Adaptation, Semantic Segmentation
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11164v2
PDF	https://arxiv.org/pdf/1912.11164v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-scene-adaptation-with-memory
Repo
Framework

Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured Meta Learning


Title	Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured Meta Learning
Authors	Sameeksha Katoch, Kowshik Thopalli, Jayaraman J. Thiagarajan, Pavan Turaga, Andreas Spanias
Abstract	Exploiting known semantic relationships between fine-grained tasks is critical to the success of recent model agnostic approaches. These approaches often rely on meta-optimization to make a model robust to systematic task or domain shifts. However, in practice, the performance of these methods can suffer, when there are no coherent semantic relationships between the tasks (or domains). We present Invenio, a structured meta-learning algorithm to infer semantic similarities between a given set of tasks and to provide insights into the complexity of transferring knowledge between different tasks. In contrast to existing techniques such as Task2Vec and Taskonomy, which measure similarities between pre-trained models, our approach employs a novel self-supervised learning strategy to discover these relationships in the training loop and at the same time utilizes them to update task-specific models in the meta-update step. Using challenging task and domain databases, under few-shot learning settings, we show that Invenio can discover intricate dependencies between tasks or domains, and can provide significant gains over existing approaches in terms of generalization performance. The learned semantic structure between tasks/domains from Invenio is interpretable and can be used to construct meaningful priors for tasks or domains.
Tasks	Few-Shot Learning, Meta-Learning
Published	2019-11-24
URL	https://arxiv.org/abs/1911.10600v2
PDF	https://arxiv.org/pdf/1911.10600v2.pdf
PWC	https://paperswithcode.com/paper/invenio-discovering-hidden-relationships
Repo
Framework