January 28, 2020

3229 words 16 mins read

Paper Group ANR 1038

Multi-loss-aware Channel Pruning of Deep Networks. Semantic Segmentation of Seismic Images. The CM Algorithm for the Maximum Mutual Information Classifications of Unseen Instances. Learning Erdős-Rényi Graphs under Partial Observations: Concentration or Sparsity?. Global Temporal Representation based CNNs for Infrared Action Recognition. Data-Depen …

Multi-loss-aware Channel Pruning of Deep Networks


Title	Multi-loss-aware Channel Pruning of Deep Networks
Authors	Yiming Hu, Siyang Sun, Jianquan Li, Jiagang Zhu, Xingang Wang, Qingyi Gu
Abstract	Channel pruning, which seeks to reduce the model size by removing redundant channels, is a popular solution for deep networks compression. Existing channel pruning methods usually conduct layer-wise channel selection by directly minimizing the reconstruction error of feature maps between the baseline model and the pruned one. However, they ignore the feature and semantic distributions within feature maps and real contribution of channels to the overall performance. In this paper, we propose a new channel pruning method by explicitly using both intermediate outputs of the baseline model and the classification loss of the pruned model to supervise layer-wise channel selection. Particularly, we introduce an additional loss to encode the differences in the feature and semantic distributions within feature maps between the baseline model and the pruned one. By considering the reconstruction error, the additional loss and the classification loss at the same time, our approach can significantly improve the performance of the pruned model. Comprehensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method.
Tasks
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10364v1
PDF	http://arxiv.org/pdf/1902.10364v1.pdf
PWC	https://paperswithcode.com/paper/multi-loss-aware-channel-pruning-of-deep
Repo
Framework

Semantic Segmentation of Seismic Images


Title	Semantic Segmentation of Seismic Images
Authors	Daniel Civitarese, Daniela Szwarcman, Emilio Vital Brazil, Bianca Zadrozny
Abstract	Almost all work to understand Earth’s subsurface on a large scale relies on the interpretation of seismic surveys by experts who segment the survey (usually a cube) into layers; a process that is very time demanding. In this paper, we present a new deep neural network architecture specially designed to semantically segment seismic images with a minimal amount of training data. To achieve this, we make use of a transposed residual unit that replaces the traditional dilated convolution for the decode block. Also, instead of using a predefined shape for up-scaling, our network learns all the steps to upscale the features from the encoder. We train our neural network using the Penobscot 3D dataset; a real seismic dataset acquired offshore Nova Scotia, Canada. We compare our approach with two well-known deep neural network topologies: Fully Convolutional Network and U-Net. In our experiments, we show that our approach can achieve more than 99 percent of the mean intersection over union (mIOU) metric, outperforming the existing topologies. Moreover, our qualitative results show that the obtained model can produce masks very close to human interpretation with very little discontinuity.
Tasks	Semantic Segmentation
Published	2019-05-10
URL	https://arxiv.org/abs/1905.04307v1
PDF	https://arxiv.org/pdf/1905.04307v1.pdf
PWC	https://paperswithcode.com/paper/190504307
Repo
Framework

The CM Algorithm for the Maximum Mutual Information Classifications of Unseen Instances


Title	The CM Algorithm for the Maximum Mutual Information Classifications of Unseen Instances
Authors	Chenguang Lu
Abstract	The Maximum Mutual Information (MMI) criterion is different from the Least Error Rate (LER) criterion. It can reduce failing to report small probability events. This paper introduces the Channels Matching (CM) algorithm for the MMI classifications of unseen instances. It also introduces some semantic information methods, which base the CM algorithm. In the CM algorithm, label learning is to let the semantic channel match the Shannon channel (Matching I) whereas classifying is to let the Shannon channel match the semantic channel (Matching II). We can achieve the MMI classifications by repeating Matching I and II. For low-dimensional feature spaces, we only use parameters to construct n likelihood functions for n different classes (rather than to construct partitioning boundaries as gradient descent) and expresses the boundaries by numerical values. Without searching in parameter spaces, the computation of the CM algorithm for low-dimensional feature spaces is very simple and fast. Using a two-dimensional example, we test the speed and reliability of the CM algorithm by different initial partitions. For most initial partitions, two iterations can make the mutual information surpass 99% of the convergent MMI. The analysis indicates that for high-dimensional feature spaces, we may combine the CM algorithm with neural networks to improve the MMI classifications for faster and more reliable convergence.
Tasks
Published	2019-01-28
URL	http://arxiv.org/abs/1901.09902v1
PDF	http://arxiv.org/pdf/1901.09902v1.pdf
PWC	https://paperswithcode.com/paper/the-cm-algorithm-for-the-maximum-mutual
Repo
Framework

Learning Erdős-Rényi Graphs under Partial Observations: Concentration or Sparsity?


Title	Learning Erdős-Rényi Graphs under Partial Observations: Concentration or Sparsity?
Authors	Vincenzo Matta, Augusto Santos, Ali H. Sayed
Abstract	This work examines the problem of graph learning over a diffusion network when data can be collected from a limited portion of the network (partial observability). While most works in the literature rely on a degree of sparsity to provide guarantees of consistent graph recovery, our analysis moves away from this condition and includes the demanding setting of dense connectivity. We ascertain that suitable estimators of the combination matrix (i.e., the matrix that quantifies the pairwise interaction between nodes) possess an identifiability gap that enables the discrimination between connected and disconnected nodes. Fundamental conditions are established under which the subgraph of monitored nodes can be recovered, with high probability as the network size increases, through universal clustering algorithms. This claim is proved for three matrix estimators: i) the Granger estimator that adapts to the partial observability setting the solution that is optimal under full observability ; ii) the one-lag correlation matrix; and iii) the residual estimator based on the difference between two consecutive time samples. Comparison among the estimators is performed through illustrative examples that reveal how estimators that are not optimal in the full observability regime can outperform the Granger estimator in the partial observability regime. The analysis reveals that the fundamental property enabling consistent graph learning is the statistical concentration of node degrees, rather than the sparsity of connections.
Tasks
Published	2019-04-05
URL	http://arxiv.org/abs/1904.02963v1
PDF	http://arxiv.org/pdf/1904.02963v1.pdf
PWC	https://paperswithcode.com/paper/learning-erdos-renyi-graphs-under-partial
Repo
Framework

Global Temporal Representation based CNNs for Infrared Action Recognition


Title	Global Temporal Representation based CNNs for Infrared Action Recognition
Authors	Yang Liu, Zhaoyang Lu, Jing Li, Tao Yang, Chao Yao
Abstract	Infrared human action recognition has many advantages, i.e., it is insensitive to illumination change, appearance variability, and shadows. Existing methods for infrared action recognition are either based on spatial or local temporal information, however, the global temporal information, which can better describe the movements of body parts across the whole video, is not considered. In this letter, we propose a novel global temporal representation named optical-flow stacked difference image (OFSDI) and extract robust and discriminative feature from the infrared action data by considering the local, global, and spatial temporal information together. Due to the small size of the infrared action dataset, we first apply convolutional neural networks on local, spatial, and global temporal stream respectively to obtain efficient convolutional feature maps from the raw data rather than train a classifier directly. Then these convolutional feature maps are aggregated into effective descriptors named three-stream trajectory-pooled deep-convolutional descriptors by trajectory-constrained pooling. Furthermore, we improve the robustness of these features by using the locality-constrained linear coding (LLC) method. With these features, a linear support vector machine (SVM) is adopted to classify the action data in our scheme. We conduct the experiments on infrared action recognition datasets InfAR and NTU RGB+D. The experimental results show that the proposed approach outperforms the representative state-of-the-art handcrafted features and deep learning features based methods for the infrared action recognition.
Tasks	Optical Flow Estimation, Temporal Action Localization
Published	2019-09-18
URL	https://arxiv.org/abs/1909.08287v1
PDF	https://arxiv.org/pdf/1909.08287v1.pdf
PWC	https://paperswithcode.com/paper/global-temporal-representation-based-cnns-for
Repo
Framework

Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models


Title	Data-Dependent Differentially Private Parameter Learning for Directed Graphical Models
Authors	Amrita Roy Chowdhury, Theodoros Rekatsinas, Somesh Jha
Abstract	Directed graphical models (DGMs) are a class of probabilistic models that are widely used for predictive analysis in sensitive domains, such as medical diagnostics. In this paper we present an algorithm for differentially private learning of the parameters of a DGM with a publicly known graph structure over fully observed data. Our solution optimizes for the utility of inference queries over the DGM and \textit{adds noise that is customized to the properties of the private input dataset and the graph structure of the DGM}. To the best of our knowledge, this is the first explicit data-dependent privacy budget allocation algorithm for DGMs. We compare our algorithm with a standard data-independent approach over a diverse suite of DGM benchmarks and demonstrate that our solution requires a privacy budget that is $3\times$ smaller to obtain the same or higher utility.
Tasks
Published	2019-05-30
URL	https://arxiv.org/abs/1905.12813v2
PDF	https://arxiv.org/pdf/1905.12813v2.pdf
PWC	https://paperswithcode.com/paper/data-dependent-differentially-private
Repo
Framework

Fairness in Clustering with Multiple Sensitive Attributes


Title	Fairness in Clustering with Multiple Sensitive Attributes
Authors	Savitha Sam Abraham, Deepak P, Sowmya S Sundaram
Abstract	A clustering may be considered as fair on pre-specified sensitive attributes if the proportions of sensitive attribute groups in each cluster reflect that in the dataset. In this paper, we consider the task of fair clustering for scenarios involving multiple multi-valued or numeric sensitive attributes. We propose a fair clustering method, \textit{FairKM} (Fair K-Means), that is inspired by the popular K-Means clustering formulation. We outline a computational notion of fairness which is used along with a cluster coherence objective, to yield the FairKM clustering method. We empirically evaluate our approach, wherein we quantify both the quality and fairness of clusters, over real-world datasets. Our experimental evaluation illustrates that the clusters generated by FairKM fare significantly better on both clustering quality and fair representation of sensitive attribute groups compared to the clusters from a state-of-the-art baseline fair clustering method.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05113v2
PDF	https://arxiv.org/pdf/1910.05113v2.pdf
PWC	https://paperswithcode.com/paper/fairness-in-clustering-with-multiple
Repo
Framework

Differentiable Reasoning on Large Knowledge Bases and Natural Language


Title	Differentiable Reasoning on Large Knowledge Bases and Natural Language
Authors	Pasquale Minervini, Matko Bošnjak, Tim Rocktäschel, Sebastian Riedel, Edward Grefenstette
Abstract	Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering. General neural architectures that jointly learn representations and transformations of text are very data-inefficient, and it is hard to analyse their reasoning process. These issues are addressed by end-to-end differentiable reasoning systems such as Neural Theorem Provers (NTPs), although they can only be used with small-scale symbolic KBs. In this paper we first propose Greedy NTPs (GNTPs), an extension to NTPs addressing their complexity and scalability limitations, thus making them applicable to real-world datasets. This result is achieved by dynamically constructing the computation graph of NTPs and including only the most promising proof paths during inference, thus obtaining orders of magnitude more efficient models. Then, we propose a novel approach for jointly reasoning over KBs and textual mentions, by embedding logic facts and natural language sentences in a shared embedding space. We show that GNTPs perform on par with NTPs at a fraction of their cost while achieving competitive link prediction results on large datasets, providing explanations for predictions, and inducing interpretable models. Source code, datasets, and supplementary material are available online at https://github.com/uclnlp/gntp.
Tasks	Link Prediction, Question Answering, Reading Comprehension
Published	2019-12-17
URL	https://arxiv.org/abs/1912.10824v1
PDF	https://arxiv.org/pdf/1912.10824v1.pdf
PWC	https://paperswithcode.com/paper/differentiable-reasoning-on-large-knowledge
Repo
Framework

A language processing algorithm for predicting tactical solutions to an operational planning problem under uncertainty


Title	A language processing algorithm for predicting tactical solutions to an operational planning problem under uncertainty
Authors	Emma Frejinger, Eric Larsen
Abstract	This paper is devoted to the prediction of solutions to a stochastic discrete optimization problem. Through an application, we illustrate how we can use a state-of-the-art neural machine translation (NMT) algorithm to predict the solutions by defining appropriate vocabularies, syntaxes and constraints. We attend to applications where the predictions need to be computed in very short computing time – in the order of milliseconds or less. The results show that with minimal adaptations to the model architecture and hyperparameter tuning, the NMT algorithm can produce accurate solutions within the computing time budget. While these predictions are slightly less accurate than approximate stochastic programming solutions (sample average approximation), they can be computed faster and with less variability.
Tasks	Machine Translation
Published	2019-10-18
URL	https://arxiv.org/abs/1910.08216v1
PDF	https://arxiv.org/pdf/1910.08216v1.pdf
PWC	https://paperswithcode.com/paper/a-language-processing-algorithm-for
Repo
Framework

Deep Learning vs. Traditional Computer Vision


Title	Deep Learning vs. Traditional Computer Vision
Authors	Niall O’ Mahony, Sean Campbell, Anderson Carvalho, Suman Harapanahalli, Gustavo Velasco-Hernandez, Lenka Krpalkova, Daniel Riordan, Joseph Walsh
Abstract	Deep Learning has pushed the limits of what was possible in the domain of Digital Image Processing. However, that is not to say that the traditional computer vision techniques which had been undergoing progressive development in years prior to the rise of DL have become obsolete. This paper will analyse the benefits and drawbacks of each approach. The aim of this paper is to promote a discussion on whether knowledge of classical computer vision techniques should be maintained. The paper will also explore how the two sides of computer vision can be combined. Several recent hybrid methodologies are reviewed which have demonstrated the ability to improve computer vision performance and to tackle problems not suited to Deep Learning. For example, combining traditional computer vision techniques with Deep Learning has been popular in emerging domains such as Panoramic Vision and 3D vision for which Deep Learning models have not yet been fully optimised
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1910.13796v1
PDF	https://arxiv.org/pdf/1910.13796v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-vs-traditional-computer-vision
Repo
Framework

Silas: High Performance, Explainable and Verifiable Machine Learning


Title	Silas: High Performance, Explainable and Verifiable Machine Learning
Authors	Hadrien Bride, Zhe Hou, Jie Dong, Jin Song Dong, Ali Mirjalili
Abstract	This paper introduces a new classification tool named Silas, which is built to provide a more transparent and dependable data analytics service. A focus of Silas is on providing a formal foundation of decision trees in order to support logical analysis and verification of learned prediction models. This paper describes the distinct features of Silas: The Model Audit module formally verifies the prediction model against user specifications, the Enforcement Learning module trains prediction models that are guaranteed correct, the Model Insight and Prediction Insight modules reason about the prediction model and explain the decision-making of predictions. We also discuss implementation details ranging from programming paradigm to memory management that help achieve high-performance computation.
Tasks	Decision Making
Published	2019-10-03
URL	https://arxiv.org/abs/1910.01382v1
PDF	https://arxiv.org/pdf/1910.01382v1.pdf
PWC	https://paperswithcode.com/paper/silas-high-performance-explainable-and
Repo
Framework

Universal Policies to Learn Them All


Title	Universal Policies to Learn Them All
Authors	Hassam Ullah Sheikh, Ladislau Bölöni
Abstract	We explore a collaborative and cooperative multi-agent reinforcement learning setting where a team of reinforcement learning agents attempt to solve a single cooperative task in a multi-scenario setting. We propose a novel multi-agent reinforcement learning algorithm inspired by universal value function approximators that not only generalizes over state space but also over a set of different scenarios. Additionally, to prove our claim, we are introducing a challenging 2D multi-agent urban security environment where the learning agents are trying to protect a person from nearby bystanders in a variety of scenarios. Our study shows that state-of-the-art multi-agent reinforcement learning algorithms fail to generalize a single task over multiple scenarios while our proposed solution works equally well as scenario-dependent policies.
Tasks	Multi-agent Reinforcement Learning
Published	2019-08-24
URL	https://arxiv.org/abs/1908.09184v1
PDF	https://arxiv.org/pdf/1908.09184v1.pdf
PWC	https://paperswithcode.com/paper/universal-policies-to-learn-them-all
Repo
Framework

Average Individual Fairness: Algorithms, Generalization and Experiments


Title	Average Individual Fairness: Algorithms, Generalization and Experiments
Authors	Michael Kearns, Aaron Roth, Saeed Sharifi-Malvajerdi
Abstract	We propose a new family of fairness definitions for classification problems that combine some of the best properties of both statistical and individual notions of fairness. We posit not only a distribution over individuals, but also a distribution over (or collection of) classification tasks. We then ask that standard statistics (such as error or false positive/negative rates) be (approximately) equalized across individuals, where the rate is defined as an expectation over the classification tasks. Because we are no longer averaging over coarse groups (such as race or gender), this is a semantically meaningful individual-level constraint. Given a sample of individuals and classification problems, we design an oracle-efficient algorithm (i.e. one that is given access to any standard, fairness-free learning heuristic) for the fair empirical risk minimization task. We also show that given sufficiently many samples, the ERM solution generalizes in two directions: both to new individuals, and to new classification tasks, drawn from their corresponding distributions. Finally we implement our algorithm and empirically verify its effectiveness.
Tasks
Published	2019-05-25
URL	https://arxiv.org/abs/1905.10607v2
PDF	https://arxiv.org/pdf/1905.10607v2.pdf
PWC	https://paperswithcode.com/paper/average-individual-fairness-algorithms
Repo
Framework

Language in Our Time: An Empirical Analysis of Hashtags


Title	Language in Our Time: An Empirical Analysis of Hashtags
Authors	Yang Zhang
Abstract	Hashtags in online social networks have gained tremendous popularity during the past five years. The resulting large quantity of data has provided a new lens into modern society. Previously, researchers mainly rely on data collected from Twitter to study either a certain type of hashtags or a certain property of hashtags. In this paper, we perform the first large-scale empirical analysis of hashtags shared on Instagram, the major platform for hashtag-sharing. We study hashtags from three different dimensions including the temporal-spatial dimension, the semantic dimension, and the social dimension. Extensive experiments performed on three large-scale datasets with more than 7 million hashtags in total provide a series of interesting observations. First, we show that the temporal patterns of hashtags can be categorized into four different clusters, and people tend to share fewer hashtags at certain places and more hashtags at others. Second, we observe that a non-negligible proportion of hashtags exhibit large semantic displacement. We demonstrate hashtags that are more uniformly shared among users, as quantified by the proposed hashtag entropy, are less prone to semantic displacement. In the end, we propose a bipartite graph embedding model to summarize users’ hashtag profiles, and rely on these profiles to perform friendship prediction. Evaluation results show that our approach achieves an effective prediction with AUC (area under the ROC curve) above 0.8 which demonstrates the strong social signals possessed in hashtags.
Tasks	Graph Embedding
Published	2019-05-11
URL	https://arxiv.org/abs/1905.04590v1
PDF	https://arxiv.org/pdf/1905.04590v1.pdf
PWC	https://paperswithcode.com/paper/language-in-our-time-an-empirical-analysis-of
Repo
Framework

Automatic estimation of heading date of paddy rice using deep learning


Title	Automatic estimation of heading date of paddy rice using deep learning
Authors	Sai Vikas Desai, Vineeth N Balasubramanian, Tokihiro Fukatsu, Seishi Ninomiya, Wei Guo
Abstract	Accurate estimation of heading date of paddy rice greatly helps the breeders to understand the adaptability of different crop varieties in a given location. The heading date also plays a vital role in determining grain yield for research experiments. Visual examination of the crop is laborious and time consuming. Therefore, quick and precise estimation of heading date of paddy rice is highly essential. In this work, we propose a simple pipeline to detect regions containing flowering panicles from ground level RGB images of paddy rice. Given a fixed region size for an image, the number of regions containing flowering panicles is directly proportional to the number of flowering panicles present. Consequently, we use the flowering panicle region counts to estimate the heading date of the crop. The method is based on image classification using Convolutional Neural Networks (CNNs). We evaluated the performance of our algorithm on five time series image sequences of three different varieties of rice crops. When compared to the previous work on this dataset, the accuracy and general versatility of the method has been improved and heading date has been estimated with a mean absolute error of less than 1 day.
Tasks	Image Classification, Time Series
Published	2019-06-19
URL	https://arxiv.org/abs/1906.07917v1
PDF	https://arxiv.org/pdf/1906.07917v1.pdf
PWC	https://paperswithcode.com/paper/automatic-estimation-of-heading-date-of-paddy
Repo
Framework