October 17, 2019

3456 words 17 mins read

Paper Group ANR 712

Hierarchical Transfer Convolutional Neural Networks for Image Classification. Weighted Bilinear Coding over Salient Body Parts for Person Re-identification. Diagnosing Convolutional Neural Networks using their Spectral Response. Word sense induction using word embeddings and community detection in complex networks. Better Fixed-Arity Unbiased Black …

Hierarchical Transfer Convolutional Neural Networks for Image Classification


Title	Hierarchical Transfer Convolutional Neural Networks for Image Classification
Authors	Xishuang Dong, Hsiang-Huang Wu, Yuzhong Yan, Lijun Qian
Abstract	In this paper, we address the issue of how to enhance the generalization performance of convolutional neural networks (CNN) in the early learning stage for image classification. This is motivated by real-time applications that require the generalization performance of CNN to be satisfactory within limited training time. In order to achieve this, a novel hierarchical transfer CNN framework is proposed. It consists of a group of shallow CNNs and a cloud CNN, where the shallow CNNs are trained firstly and then the first layers of the trained shallow CNNs are used to initialize the first layer of the cloud CNN. This method will boost the generalization performance of the cloud CNN significantly, especially during the early stage of training. Experiments using CIFAR-10 and ImageNet datasets are performed to examine the proposed method. Results demonstrate the improvement of testing accuracy is 12% on average and as much as 20% for the CIFAR-10 case while 5% testing accuracy improvement for the ImageNet case during the early stage of learning. It is also shown that universal improvements of testing accuracy are obtained across different settings of dropout and number of shallow CNNs.
Tasks	Image Classification
Published	2018-03-30
URL	http://arxiv.org/abs/1804.00021v2
PDF	http://arxiv.org/pdf/1804.00021v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-transfer-convolutional-neural
Repo
Framework

Weighted Bilinear Coding over Salient Body Parts for Person Re-identification


Title	Weighted Bilinear Coding over Salient Body Parts for Person Re-identification
Authors	Zhigang Chang, Qin Zhou, Heng Fan, Hang Su, Hua Yang, Shibao Zheng, Haibin Ling
Abstract	Deep convolutional neural networks (CNNs) have demonstrated dominant performance in person re-identification (Re-ID). Existing CNN based methods utilize global average pooling (GAP) to aggregate intermediate convolutional features for Re-ID. However, this strategy only considers the first-order statistics of local features and treats local features at different locations equally important, leading to sub-optimal feature representation. To deal with these issues, we propose a novel weighted bilinear coding (WBC) framework for local feature aggregation in CNN networks to pursue more representative and discriminative feature representations, which can adapt to other state-of-the-art methods and improve their performance. In specific, bilinear coding is used to encode the channel-wise feature correlations to capture richer feature interactions. Meanwhile, a weighting scheme is applied on the bilinear coding to adaptively adjust the weights of local features at different locations based on their importance in recognition, further improving the discriminability of feature aggregation. To handle the spatial misalignment issue, we use a salient part net (spatial attention module) to derive salient body parts, and apply the WBC model on each part. The final representation, formed by concatenating the WBC encoded features of each part, is both discriminative and resistant to spatial misalignment. Experiments on three benchmarks including Market-1501, DukeMTMC-reID and CUHK03 evidence the favorable performance of our method against other outstanding methods.
Tasks	Person Re-Identification
Published	2018-03-22
URL	https://arxiv.org/abs/1803.08580v3
PDF	https://arxiv.org/pdf/1803.08580v3.pdf
PWC	https://paperswithcode.com/paper/weighted-bilinear-coding-over-salient-body
Repo
Framework

Diagnosing Convolutional Neural Networks using their Spectral Response


Title	Diagnosing Convolutional Neural Networks using their Spectral Response
Authors	Victor Stamatescu, Mark D. McDonnell
Abstract	Convolutional Neural Networks (CNNs) are a class of artificial neural networks whose computational blocks use convolution, together with other linear and non-linear operations, to perform classification or regression. This paper explores the spectral response of CNNs and its potential use in diagnosing problems with their training. We measure the gain of CNNs trained for image classification on ImageNet and observe that the best models are also the most sensitive to perturbations of their input. Further, we perform experiments on MNIST and CIFAR-10 to find that the gain rises as the network learns and then saturates as the network converges. Moreover, we find that strong gain fluctuations can point to overfitting and learning problems caused by a poor choice of learning rate. We argue that the gain of CNNs can act as a diagnostic tool and potential replacement for the validation loss when hold-out validation data are not available.
Tasks	Image Classification
Published	2018-10-08
URL	http://arxiv.org/abs/1810.03241v1
PDF	http://arxiv.org/pdf/1810.03241v1.pdf
PWC	https://paperswithcode.com/paper/diagnosing-convolutional-neural-networks
Repo
Framework

Word sense induction using word embeddings and community detection in complex networks


Title	Word sense induction using word embeddings and community detection in complex networks
Authors	Edilson A. Corrêa Jr., Diego R. Amancio
Abstract	Word Sense Induction (WSI) is the ability to automatically induce word senses from corpora. The WSI task was first proposed to overcome the limitations of manually annotated corpus that are required in word sense disambiguation systems. Even though several works have been proposed to induce word senses, existing systems are still very limited in the sense that they make use of structured, domain-specific knowledge sources. In this paper, we devise a method that leverages recent findings in word embeddings research to generate context embeddings, which are embeddings containing information about the semantical context of a word. In order to induce senses, we modeled the set of ambiguous words as a complex network. In the generated network, two instances (nodes) are connected if the respective context embeddings are similar. Upon using well-established community detection methods to cluster the obtained context embeddings, we found that the proposed method yields excellent performance for the WSI task. Our method outperformed competing algorithms and baselines, in a completely unsupervised manner and without the need of any additional structured knowledge source.
Tasks	Community Detection, Word Embeddings, Word Sense Disambiguation, Word Sense Induction
Published	2018-03-22
URL	http://arxiv.org/abs/1803.08476v1
PDF	http://arxiv.org/pdf/1803.08476v1.pdf
PWC	https://paperswithcode.com/paper/word-sense-induction-using-word-embeddings
Repo
Framework

Better Fixed-Arity Unbiased Black-Box Algorithms


Title	Better Fixed-Arity Unbiased Black-Box Algorithms
Authors	Nina Bulanova, Maxim Buzdalov
Abstract	In their GECCO’12 paper, Doerr and Doerr proved that the $k$-ary unbiased black-box complexity of OneMax on $n$ bits is $O(n/k)$ for $2\le k\le O(\log n)$. We propose an alternative strategy for achieving this unbiased black-box complexity when $3\le k\le\log_2 n$. While it is based on the same idea of block-wise optimization, it uses $k$-ary unbiased operators in a different way. For each block of size $2^{k-1}-1$ we set up, in $O(k)$ queries, a virtual coordinate system, which enables us to use an arbitrary unrestricted algorithm to optimize this block. This is possible because this coordinate system introduces a bijection between unrestricted queries and a subset of $k$-ary unbiased operators. We note that this technique does not depend on OneMax being solved and can be used in more general contexts. This together constitutes an algorithm which is conceptually simpler than the one by Doerr and Doerr, and at the same time achieves better constant factors in the asymptotic notation. Our algorithm works in $(2+o(1))\cdot n/(k-1)$, where $o(1)$ relates to $k$. Our experimental evaluation of this algorithm shows its efficiency already for $3\le k\le6$.
Tasks
Published	2018-04-15
URL	http://arxiv.org/abs/1804.05443v2
PDF	http://arxiv.org/pdf/1804.05443v2.pdf
PWC	https://paperswithcode.com/paper/better-fixed-arity-unbiased-black-box
Repo
Framework

Meta-Learner with Linear Nulling


Title	Meta-Learner with Linear Nulling
Authors	Sung Whan Yoon, Jun Seo, Jaekyun Moon
Abstract	We propose a meta-learning algorithm utilizing a linear transformer that carries out null-space projection of neural network outputs. The main idea is to construct an alternative classification space such that the error signals during few-shot learning are quickly zero-forced on that space so that reliable classification on low data is possible. The final decision on a query is obtained utilizing a null-space-projected distance measure between the network output and reference vectors, both of which have been trained in the initial learning phase. Among the known methods with a given model size, our meta-learner achieves the best or near-best image classification accuracies with Omniglot and miniImageNet datasets.
Tasks	Few-Shot Learning, Image Classification, Meta-Learning, Omniglot
Published	2018-06-04
URL	http://arxiv.org/abs/1806.01010v3
PDF	http://arxiv.org/pdf/1806.01010v3.pdf
PWC	https://paperswithcode.com/paper/meta-learner-with-linear-nulling
Repo
Framework

Efficiently Learning Mixtures of Mallows Models


Title	Efficiently Learning Mixtures of Mallows Models
Authors	Allen Liu, Ankur Moitra
Abstract	Mixtures of Mallows models are a popular generative model for ranking data coming from a heterogeneous population. They have a variety of applications including social choice, recommendation systems and natural language processing. Here we give the first polynomial time algorithm for provably learning the parameters of a mixture of Mallows models with any constant number of components. Prior to our work, only the two component case had been settled. Our analysis revolves around a determinantal identity of Zagier which was proven in the context of mathematical physics, which we use to show polynomial identifiability and ultimately to construct test functions to peel off one component at a time. To complement our upper bounds, we show information-theoretic lower bounds on the sample complexity as well as lower bounds against restricted families of algorithms that make only local queries. Together, these results demonstrate various impediments to improving the dependence on the number of components. They also motivate the study of learning mixtures of Mallows models from the perspective of beyond worst-case analysis. In this direction, we show that when the scaling parameters of the Mallows models have separation, there are much faster learning algorithms.
Tasks	Recommendation Systems
Published	2018-08-17
URL	http://arxiv.org/abs/1808.05731v1
PDF	http://arxiv.org/pdf/1808.05731v1.pdf
PWC	https://paperswithcode.com/paper/efficiently-learning-mixtures-of-mallows
Repo
Framework

Reconstruction of Simulation-Based Physical Field by Reconstruction Neural Network Method


Title	Reconstruction of Simulation-Based Physical Field by Reconstruction Neural Network Method
Authors	Yu Li, Hu Wang, Kangjia Mo, Tao Zeng
Abstract	A variety of modeling techniques have been developed in the past decade to reduce the computational expense and improve the accuracy of modeling. In this study, a new framework of modeling is suggested. Compared with other popular methods, a distinctive characteristic is “from image based model to analysis based model (e.g. stress, strain, and deformation)". In such a framework, a reconstruction neural network (ReConNN) model designed for simulation-based physical field’s reconstruction is proposed. The ReConNN contains two submodels that are convolutional neural network (CNN) and generative adversarial net-work (GAN). The CNN is employed to construct the mapping between contour images of physical field and objective function. Subsequently, the GAN is utilized to generate more images which are similar to the existing contour images. Finally, Lagrange polynomial is applied to complete the reconstruction. However, the existing CNN models are commonly applied to the classification tasks, which seem to be difficult to handle with regression tasks of images. Meanwhile, the existing GAN architectures are insufficient to generate high-accuracy “pseudo contour images”. Therefore, a ReConNN model based on a Convolution in Convolution (CIC) and a Convolutional AutoEncoder based on Wasserstein Generative Adversarial Network (WGAN-CAE) is suggested. To evaluate the performance of the proposed model representatively, a classical topology optimization procedure is considered. Then the ReConNN is utilized to the reconstruction of heat transfer process of a pin fin heat sink. It demonstrates that the proposed ReConNN model is proved to be a potential capability to reconstruct physical field for multidisciplinary, such as structural optimization.
Tasks
Published	2018-04-19
URL	http://arxiv.org/abs/1805.00528v3
PDF	http://arxiv.org/pdf/1805.00528v3.pdf
PWC	https://paperswithcode.com/paper/reconstruction-of-simulation-based-physical
Repo
Framework

A Generalized Active Learning Approach for Unsupervised Anomaly Detection


Title	A Generalized Active Learning Approach for Unsupervised Anomaly Detection
Authors	Tiago Pimentel, Marianne Monteiro, Juliano Viana, Adriano Veloso, Nivio Ziviani
Abstract	This work formalizes the new framework for anomaly detection, called active anomaly detection. This framework has, in practice, the same cost of unsupervised anomaly detection but with the possibility of much better results. We show that unsupervised anomaly detection is an undecidable problem and that a prior needs to be assumed for the anomalies probability distribution in order to have performance guarantees. Finally, we also present a new layer that can be attached to any deep learning model designed for unsupervised anomaly detection to transform it into an active anomaly detection method, presenting results on both synthetic and real anomaly detection datasets.
Tasks	Active Learning, Anomaly Detection, Unsupervised Anomaly Detection
Published	2018-05-23
URL	http://arxiv.org/abs/1805.09411v1
PDF	http://arxiv.org/pdf/1805.09411v1.pdf
PWC	https://paperswithcode.com/paper/a-generalized-active-learning-approach-for
Repo
Framework

COSINE: Compressive Network Embedding on Large-scale Information Networks


Title	COSINE: Compressive Network Embedding on Large-scale Information Networks
Authors	Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun, Zhichong Fang, Bo Zhang, Leyu Lin
Abstract	There is recently a surge in approaches that learn low-dimensional embeddings of nodes in networks. As there are many large-scale real-world networks, it’s inefficient for existing approaches to store amounts of parameters in memory and update them edge after edge. With the knowledge that nodes having similar neighborhood will be close to each other in embedding space, we propose COSINE (COmpresSIve NE) algorithm which reduces the memory footprint and accelerates the training process by parameters sharing among similar nodes. COSINE applies graph partitioning algorithms to networks and builds parameter sharing dependency of nodes based on the result of partitioning. With parameters sharing among similar nodes, COSINE injects prior knowledge about higher structural information into training process which makes network embedding more efficient and effective. COSINE can be applied to any embedding lookup method and learn high-quality embeddings with limited memory and shorter training time. We conduct experiments of multi-label classification and link prediction, where baselines and our model have the same memory usage. Experimental results show that COSINE gives baselines up to 23% increase on classification and up to 25% increase on link prediction. Moreover, time of all representation learning methods using COSINE decreases from 30% to 70%.
Tasks	graph partitioning, Link Prediction, Multi-Label Classification, Network Embedding, Representation Learning
Published	2018-12-21
URL	http://arxiv.org/abs/1812.08972v1
PDF	http://arxiv.org/pdf/1812.08972v1.pdf
PWC	https://paperswithcode.com/paper/cosine-compressive-network-embedding-on-large
Repo
Framework

Writer-Aware CNN for Parsimonious HMM-Based Offline Handwritten Chinese Text Recognition


Title	Writer-Aware CNN for Parsimonious HMM-Based Offline Handwritten Chinese Text Recognition
Authors	Zi-Rui Wang, Jun Du, Jia-Ming Wang
Abstract	Recently, the hybrid convolutional neural network hidden Markov model (CNN-HMM) has been introduced for offline handwritten Chinese text recognition (HCTR) and has achieved state-of-the-art performance. However, modeling each of the large vocabulary of Chinese characters with a uniform and fixed number of hidden states requires high memory and computational costs and makes the tens of thousands of HMM state classes confusing. Another key issue of CNN-HMM for HCTR is the diversified writing style, which leads to model strain and a significant performance decline for specific writers. To address these issues, we propose a writer-aware CNN based on parsimonious HMM (WCNN-PHMM). First, PHMM is designed using a data-driven state-tying algorithm to greatly reduce the total number of HMM states, which not only yields a compact CNN by state sharing of the same or similar radicals among different Chinese characters but also improves the recognition accuracy due to the more accurate modeling of tied states and the lower confusion among them. Second, WCNN integrates each convolutional layer with one adaptive layer fed by a writer-dependent vector, namely, the writer code, to extract the irrelevant variability in writer information to improve recognition performance. The parameters of writer-adaptive layers are jointly optimized with other network parameters in the training stage, while a multiple-pass decoding strategy is adopted to learn the writer code and generate recognition results. Validated on the ICDAR 2013 competition of CASIA-HWDB database, the more compact WCNN-PHMM of a 7360-class vocabulary can achieve a relative character error rate (CER) reduction of 16.6% over the conventional CNN-HMM without considering language modeling. By adopting a powerful hybrid language model (N-gram language model and recurrent neural network language model), the CER of WCNN-PHMM is reduced to 3.17%.
Tasks	Handwritten Chinese Text Recognition, Language Modelling
Published	2018-12-24
URL	https://arxiv.org/abs/1812.09809v2
PDF	https://arxiv.org/pdf/1812.09809v2.pdf
PWC	https://paperswithcode.com/paper/writer-aware-cnn-for-parsimonious-hmm-based
Repo
Framework

A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes


Title	A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processes
Authors	Jialin Song, Yuxin Chen, Yisong Yue
Abstract	How can we efficiently gather information to optimize an unknown function, when presented with multiple, mutually dependent information sources with different costs? For example, when optimizing a robotic system, intelligently trading off computer simulations and real robot testings can lead to significant savings. Existing methods, such as multi-fidelity GP-UCB or Entropy Search-based approaches, either make simplistic assumptions on the interaction among different fidelities or use simple heuristics that lack theoretical guarantees. In this paper, we study multi-fidelity Bayesian optimization with complex structural dependencies among multiple outputs, and propose MF-MI-Greedy, a principled algorithmic framework for addressing this problem. In particular, we model different fidelities using additive Gaussian processes based on shared latent structures with the target function. Then we use cost-sensitive mutual information gain for efficient Bayesian global optimization. We propose a simple notion of regret which incorporates the cost of different fidelities, and prove that MF-MI-Greedy achieves low regret. We demonstrate the strong empirical performance of our algorithm on both synthetic and real-world datasets.
Tasks	Gaussian Processes
Published	2018-11-02
URL	http://arxiv.org/abs/1811.00755v1
PDF	http://arxiv.org/pdf/1811.00755v1.pdf
PWC	https://paperswithcode.com/paper/a-general-framework-for-multi-fidelity
Repo
Framework

Formal Verification of Neural Network Controlled Autonomous Systems


Title	Formal Verification of Neural Network Controlled Autonomous Systems
Authors	Xiaowu Sun, Haitham Khedr, Yasser Shoukry
Abstract	In this paper, we consider the problem of formally verifying the safety of an autonomous robot equipped with a Neural Network (NN) controller that processes LiDAR images to produce control actions. Given a workspace that is characterized by a set of polytopic obstacles, our objective is to compute the set of safe initial conditions such that a robot trajectory starting from these initial conditions is guaranteed to avoid the obstacles. Our approach is to construct a finite state abstraction of the system and use standard reachability analysis over the finite state abstraction to compute the set of the safe initial states. The first technical problem in computing the finite state abstraction is to mathematically model the imaging function that maps the robot position to the LiDAR image. To that end, we introduce the notion of imaging-adapted sets as partitions of the workspace in which the imaging function is guaranteed to be affine. We develop a polynomial-time algorithm to partition the workspace into imaging-adapted sets along with computing the corresponding affine imaging functions. Given this workspace partitioning, a discrete-time linear dynamics of the robot, and a pre-trained NN controller with Rectified Linear Unit (ReLU) nonlinearity, the second technical challenge is to analyze the behavior of the neural network. To that end, we utilize a Satisfiability Modulo Convex (SMC) encoding to enumerate all the possible segments of different ReLUs. SMC solvers then use a Boolean satisfiability solver and a convex programming solver and decompose the problem into smaller subproblems. To accelerate this process, we develop a pre-processing algorithm that could rapidly prune the space feasible ReLU segments. Finally, we demonstrate the efficiency of the proposed algorithms using numerical simulations with increasing complexity of the neural network controller.
Tasks
Published	2018-10-31
URL	http://arxiv.org/abs/1810.13072v1
PDF	http://arxiv.org/pdf/1810.13072v1.pdf
PWC	https://paperswithcode.com/paper/formal-verification-of-neural-network
Repo
Framework

Intentional Control of Type I Error over Unconscious Data Distortion: a Neyman-Pearson Approach to Text Classification


Title	Intentional Control of Type I Error over Unconscious Data Distortion: a Neyman-Pearson Approach to Text Classification
Authors	Lucy Xia, Richard Zhao, Yanhui Wu, Xin Tong
Abstract	Digital texts have become an increasingly important source of data for social studies. However, textual data from open platforms are vulnerable to manipulation (e.g., censorship and information inflation), often leading to bias in subsequent empirical analysis. This paper investigates the problem of data distortion in text classification when controlling type I error (a relevant textual message is classified as irrelevant) is the priority. The default classical classification paradigm that minimizes the overall classification error can yield an undesirably large type I error, and data distortion exacerbates this situation. As a solution, we propose the Neyman-Pearson (NP) classification paradigm which minimizes type II error under a user-specified type I error constraint. Theoretically, we show that while the classical oracle (i.e., optimal classifier) cannot be recovered under unknown data distortion even if one has the entire post-distortion population, the NP oracle is unaffected by data distortion and can be recovered under the same condition. Empirically, we illustrate the advantage of NP classification methods in a case study that classifies posts about strikes and corruption published on a leading Chinese blogging platform.
Tasks	Text Classification
Published	2018-02-07
URL	http://arxiv.org/abs/1802.02558v2
PDF	http://arxiv.org/pdf/1802.02558v2.pdf
PWC	https://paperswithcode.com/paper/intentional-control-of-type-i-error-over
Repo
Framework

The Global Optimization Geometry of Shallow Linear Neural Networks


Title	The Global Optimization Geometry of Shallow Linear Neural Networks
Authors	Zhihui Zhu, Daniel Soudry, Yonina C. Eldar, Michael B. Wakin
Abstract	We examine the squared error loss landscape of shallow linear neural networks. We show—with significantly milder assumptions than previous works—that the corresponding optimization problems have benign geometric properties: there are no spurious local minima and the Hessian at every saddle point has at least one negative eigenvalue. This means that at every saddle point there is a directional negative curvature which algorithms can utilize to further decrease the objective value. These geometric properties imply that many local search algorithms (such as the gradient descent which is widely utilized for training neural networks) can provably solve the training problem with global convergence.
Tasks
Published	2018-05-13
URL	http://arxiv.org/abs/1805.04938v2
PDF	http://arxiv.org/pdf/1805.04938v2.pdf
PWC	https://paperswithcode.com/paper/the-global-optimization-geometry-of-shallow
Repo
Framework