April 2, 2020

3069 words 15 mins read

Paper Group ANR 321

Progressive Local Filter Pruning for Image Retrieval Acceleration. On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks. A classification for the performance of online SGD for high-dimensional inference. Connecting Dualities and Machine Learning. Learning to be Global Optimizer. End-to-End Face Parsing via Interlinked …

Progressive Local Filter Pruning for Image Retrieval Acceleration


Title	Progressive Local Filter Pruning for Image Retrieval Acceleration
Authors	Xiaodong Wang, Zhedong Zheng, Yang He, Fei Yan, Zhiqiang Zeng, Yi Yang
Abstract	This paper focuses on network pruning for image retrieval acceleration. Prevailing image retrieval works target at the discriminative feature learning, while little attention is paid to how to accelerate the model inference, which should be taken into consideration in real-world practice. The challenge of pruning image retrieval models is that the middle-level feature should be preserved as much as possible. Such different requirements of the retrieval and classification model make the traditional pruning methods not that suitable for our task. To solve the problem, we propose a new Progressive Local Filter Pruning (PLFP) method for image retrieval acceleration. Specifically, layer by layer, we analyze the local geometric properties of each filter and select the one that can be replaced by the neighbors. Then we progressively prune the filter by gradually changing the filter weights. In this way, the representation ability of the model is preserved. To verify this, we evaluate our method on two widely-used image retrieval datasets,i.e., Oxford5k and Paris6K, and one person re-identification dataset,i.e., Market-1501. The proposed method arrives with superior performance to the conventional pruning methods, suggesting the effectiveness of the proposed method for image retrieval.
Tasks	Image Retrieval, Network Pruning, Person Re-Identification
Published	2020-01-24
URL	https://arxiv.org/abs/2001.08878v1
PDF	https://arxiv.org/pdf/2001.08878v1.pdf
PWC	https://paperswithcode.com/paper/progressive-local-filter-pruning-for-image
Repo
Framework

On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks


Title	On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks
Authors	Michela Paganini, Jessica Forde
Abstract	We examine how recently documented, fundamental phenomena in deep learning models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze differences in the connectivity structure and learning dynamics of pruned models found through a set of common iterative pruning techniques, to address questions of uniqueness of trainable, high-sparsity sub-networks, and their dependence on the chosen pruning method. In convolutional layers, we document the emergence of structure induced by magnitude-based unstructured pruning in conjunction with weight rewinding that resembles the effects of structured pruning. We also show empirical evidence that weight stability can be automatically achieved through apposite pruning techniques.
Tasks	Network Pruning
Published	2020-01-14
URL	https://arxiv.org/abs/2001.05050v1
PDF	https://arxiv.org/pdf/2001.05050v1.pdf
PWC	https://paperswithcode.com/paper/on-iterative-neural-network-pruning-1
Repo
Framework

A classification for the performance of online SGD for high-dimensional inference


Title	A classification for the performance of online SGD for high-dimensional inference
Authors	Gerard Ben Arous, Reza Gheissari, Aukosh Jagannath
Abstract	Stochastic gradient descent (SGD) is a popular algorithm for optimization problems arising in high-dimensional inference tasks. Here one produces an estimator of an unknown parameter from a large number of independent samples of data by iteratively optimizing a loss function. This loss function is high-dimensional, random, and often complex. We study here the performance of the simplest version of SGD, namely online SGD, in the initial “search” phase, where the algorithm is far from a trust region and the loss landscape is highly non-convex. To this end, we investigate the performance of online SGD at attaining a “better than random” correlation with the unknown parameter, i.e, achieving weak recovery. Our contribution is a classification of the difficulty of typical instances of this task for online SGD in terms of the number of samples required as the dimension diverges. This classification depends only on an intrinsic property of the population loss, which we call the information exponent. Using the information exponent, we find that there are three distinct regimes—the easy, critical, and difficult regimes—where one requires linear, quasilinear, and polynomially many samples (in the dimension) respectively to achieve weak recovery. We illustrate our approach by applying it to a wide variety of estimation tasks such as parameter estimation for generalized linear models, two-component Gaussian mixture models, phase retrieval, and spiked matrix and tensor models, as well as supervised learning for single-layer networks with general activation functions. In this latter case, our results translate into a classification of the difficulty of this task in terms of the Hermite decomposition of the activation function.
Tasks
Published	2020-03-23
URL	https://arxiv.org/abs/2003.10409v1
PDF	https://arxiv.org/pdf/2003.10409v1.pdf
PWC	https://paperswithcode.com/paper/a-classification-for-the-performance-of
Repo
Framework

Connecting Dualities and Machine Learning


Title	Connecting Dualities and Machine Learning
Authors	Philip Betzler, Sven Krippendorf
Abstract	Dualities are widely used in quantum field theories and string theory to obtain correlation functions at high accuracy. Here we present examples where dual data representations are useful in supervised classification, linking machine learning and typical tasks in theoretical physics. We then discuss how such beneficial representations can be enforced in the latent dimension of neural networks. We find that additional contributions to the loss based on feature separation, feature matching with respect to desired representations, and a good performance on a `simple’ correlation function can lead to known and unknown dual representations. This is the first proof of concept that computers can find dualities. We discuss how our examples, based on discrete Fourier transformation and Ising models, connect to other dualities in theoretical physics, for instance Seiberg duality. \|
Tasks
Published	2020-02-12
URL	https://arxiv.org/abs/2002.05169v1
PDF	https://arxiv.org/pdf/2002.05169v1.pdf
PWC	https://paperswithcode.com/paper/connecting-dualities-and-machine-learning
Repo
Framework

Learning to be Global Optimizer


Title	Learning to be Global Optimizer
Authors	Haotian Zhang, Jianyong Sun, Zongben Xu
Abstract	The advancement of artificial intelligence has cast a new light on the development of optimization algorithm. This paper proposes to learn a two-phase (including a minimization phase and an escaping phase) global optimization algorithm for smooth non-convex functions. For the minimization phase, a model-driven deep learning method is developed to learn the update rule of descent direction, which is formalized as a nonlinear combination of historical information, for convex functions. We prove that the resultant algorithm with the proposed adaptive direction guarantees convergence for convex functions. Empirical study shows that the learned algorithm significantly outperforms some well-known classical optimization algorithms, such as gradient descent, conjugate descent and BFGS, and performs well on ill-posed functions. The escaping phase from local optimum is modeled as a Markov decision process with a fixed escaping policy. We further propose to learn an optimal escaping policy by reinforcement learning. The effectiveness of the escaping policies is verified by optimizing synthesized functions and training a deep neural network for CIFAR image classification. The learned two-phase global optimization algorithm demonstrates a promising global search capability on some benchmark functions and machine learning tasks.
Tasks	Image Classification
Published	2020-03-10
URL	https://arxiv.org/abs/2003.04521v1
PDF	https://arxiv.org/pdf/2003.04521v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-be-global-optimizer
Repo
Framework

End-to-End Face Parsing via Interlinked Convolutional Neural Networks


Title	End-to-End Face Parsing via Interlinked Convolutional Neural Networks
Authors	Zi Yin, Valentin Yiu, Xiaolin Hu, Liang Tang
Abstract	Face parsing is an important computer vision task that requires accurate pixel segmentation of facial parts (such as eyes, nose, mouth, etc.), providing a basis for further face analysis, modification, and other applications. In this paper, we introduce a simple, end-to-end face parsing framework: STN-aided iCNN (STN-iCNN), which extends interlinked Convolutional Neural Network (iCNN) by adding a Spatial Transformer Network (STN) between the two isolated stages. The STN-iCNN uses the STN to provide a trainable connection to the original two-stage iCNN pipe-line, making end-to-end joint training possible. Moreover, as a by-product, STN also provides more precise cropped parts than the original cropper. Due to the two advantages, our approach significantly improves the accuracy of the original model.
Tasks
Published	2020-02-12
URL	https://arxiv.org/abs/2002.04831v1
PDF	https://arxiv.org/pdf/2002.04831v1.pdf
PWC	https://paperswithcode.com/paper/end-to-end-face-parsing-via-interlinked
Repo
Framework

Legion: Best-First Concolic Testing


Title	Legion: Best-First Concolic Testing
Authors	Dongge Liu, Gidon Ernst, Toby Murray, Benjamin I. P. Rubinstein
Abstract	Legion is a grey-box concolic tool that aims to balance the complementary nature of fuzzing and symbolic execution to achieve the best of both worlds. It proposes a variation of Monte Carlo tree search (MCTS) that formulates program exploration as sequential decisionmaking under uncertainty guided by the best-first search strategy. It relies on approximate path-preserving fuzzing, a novel instance of constrained random testing, which quickly generates many diverse inputs that likely target program parts of interest. In Test-Comp 2020, the prototype performed within 90% of the best score in 9 of 22 categories.
Tasks
Published	2020-02-15
URL	https://arxiv.org/abs/2002.06311v1
PDF	https://arxiv.org/pdf/2002.06311v1.pdf
PWC	https://paperswithcode.com/paper/legion-best-first-concolic-testing
Repo
Framework

An Internal Clock Based Space-time Neural Network for Motion Speed Recognition


Title	An Internal Clock Based Space-time Neural Network for Motion Speed Recognition
Authors	Junwen Luo, Jiaoyan Chen
Abstract	In this work we present a novel internal clock based space-time neural network for motion speed recognition. The developed system has a spike train encoder, a Spiking Neural Network (SNN) with internal clocking behaviors, a pattern transformation block and a Network Dynamic Dependent Plasticity (NDDP) learning block. The core principle is that the developed SNN will automatically tune its network pattern frequency (internal clock frequency) to recognize human motions in a speed domain. We employed both cartoons and real-world videos as training benchmarks, results demonstrate that our system can not only recognize motions with considerable speed differences (e.g. run, walk, jump, wonder(think) and standstill), but also motions with subtle speed gaps such as run and fast walk. The inference accuracy can be up to 83.3% (cartoon videos) and 75% (real-world videos). Meanwhile, the system only requires six video datasets in the learning stage and with up to 42 training trials. Hardware performance estimation indicates that the training time is 0.84-4.35s and power consumption is 33.26-201mW (based on an ARM Cortex M4 processor). Therefore, our system takes unique learning advantages of the requirement of the small dataset, quick learning and low power performance, which shows great potentials for edge or scalable AI-based applications.
Tasks
Published	2020-01-28
URL	https://arxiv.org/abs/2001.10159v1
PDF	https://arxiv.org/pdf/2001.10159v1.pdf
PWC	https://paperswithcode.com/paper/an-internal-clock-based-space-time-neural
Repo
Framework

A Model of Fast Concept Inference with Object-Factorized Cognitive Programs


Title	A Model of Fast Concept Inference with Object-Factorized Cognitive Programs
Authors	Daniel P. Sawyer, Miguel Lázaro-Gredilla, Dileep George
Abstract	The ability of humans to quickly identify general concepts from a handful of images has proven difficult to emulate with robots. Recently, a computer architecture was developed that allows robots to mimic some aspects of this human ability by modeling concepts as cognitive programs using an instruction set of primitive cognitive functions. This allowed a robot to emulate human imagination by simulating candidate programs in a world model before generalizing to the physical world. However, this model used a naive search algorithm that required 30 minutes to discover a single concept, and became intractable for programs with more than 20 instructions. To circumvent this bottleneck, we present an algorithm that emulates the human cognitive heuristics of object factorization and sub-goaling, allowing human-level inference speed, improving accuracy, and making the output more explainable.
Tasks
Published	2020-02-10
URL	https://arxiv.org/abs/2002.04021v1
PDF	https://arxiv.org/pdf/2002.04021v1.pdf
PWC	https://paperswithcode.com/paper/a-model-of-fast-concept-inference-with-object
Repo
Framework

Quantum Cognitive Triad. Semantic geometry of context representation


Title	Quantum Cognitive Triad. Semantic geometry of context representation
Authors	Ilya A. Surov
Abstract	The paper describes an algorithm for cognitive representation of triples of related behavioral contexts two of which correspond to mutually exclusive states of some binary situational factor while uncertainty of this factor is the third context. The contexts are mapped to vector states in the two-dimensional quantum Hilbert space describing a dichotomic decision alternative in relation to which the contexts are subjectively recognized. The obtained triad of quantum cognitive representations functions as a minimal carrier of semantic relations between the contexts, which are quantified by phase relations between the corresponding quantum representation states. The described quantum model of subjective semantics supports interpretable vector calculus which is geometrically visualized in the Bloch sphere view of quantum cognitive states.
Tasks
Published	2020-02-22
URL	https://arxiv.org/abs/2002.11195v1
PDF	https://arxiv.org/pdf/2002.11195v1.pdf
PWC	https://paperswithcode.com/paper/quantum-cognitive-triad-semantic-geometry-of
Repo
Framework

Grab: Fast and Accurate Sensor Processing for Cashier-Free Shopping


Title	Grab: Fast and Accurate Sensor Processing for Cashier-Free Shopping
Authors	Xiaochen Liu, Yurong Jiang, Kyu-Han Kim, Ramesh Govindan
Abstract	Cashier-free shopping systems like Amazon Go improve shopping experience, but can require significant store redesign. In this paper, we propose Grab, a practical system that leverages existing infrastructure and devices to enable cashier-free shopping. Grab needs to accurately identify and track customers, and associate each shopper with items he or she retrieves from shelves. To do this, it uses a keypoint-based pose tracker as a building block for identification and tracking, develops robust feature-based face trackers, and algorithms for associating and tracking arm movements. It also uses a probabilistic framework to fuse readings from camera, weight and RFID sensors in order to accurately assess which shopper picks up which item. In experiments from a pilot deployment in a retail store, Grab can achieve over 90% precision and recall even when 40% of shopping actions are designed to confuse the system. Moreover, Grab has optimizations that help reduce investment in computing infrastructure four-fold.
Tasks
Published	2020-01-04
URL	https://arxiv.org/abs/2001.01033v1
PDF	https://arxiv.org/pdf/2001.01033v1.pdf
PWC	https://paperswithcode.com/paper/grab-fast-and-accurate-sensor-processing-for
Repo
Framework

Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem


Title	Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem
Authors	Matthias De Lange, Xu Jia, Sarah Parisot, Ales Leonardis, Gregory Slabaugh, Tinne Tuytelaars
Abstract	This work investigates the task of unsupervised model personalization, adapted to continually evolving, unlabeled local user images. We consider the practical scenario where a high capacity server interacts with a myriad of resource-limited edge devices, imposing strong requirements on scalability and local data privacy. We aim to address this challenge within the continual learning paradigm and provide a novel Dual User-Adaptation framework (DUA) to explore the problem. This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device, with desirable properties regarding scalability and privacy constraints. First, on the server, we introduce incremental learning of task-specific expert models, subsequently aggregated using a concealed unsupervised user prior. Aggregation avoids retraining, whereas the user prior conceals sensitive raw user data, and grants unsupervised adaptation. Second, local user-adaptation incorporates a domain adaptation point of view, adapting regularizing batch normalization parameters to the user data. We explore various empirical user configurations with different priors in categories and a tenfold of transforms for MIT Indoor Scene recognition, and classify numbers in a combined MNIST and SVHN setup. Extensive experiments yield promising results for data-driven local adaptation and elicit user priors for server adaptation to depend on the model rather than user data. Hence, although user-adaptation remains a challenging open problem, the DUA framework formalizes a principled foundation for personalizing both on server and user device, while maintaining privacy and scalability.
Tasks	Continual Learning, Domain Adaptation, Scene Recognition
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13296v1
PDF	https://arxiv.org/pdf/2003.13296v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-model-personalization-while
Repo
Framework

A Generalized Flow for B2B Sales Predictive Modeling: An Azure Machine Learning Approach


Title	A Generalized Flow for B2B Sales Predictive Modeling: An Azure Machine Learning Approach
Authors	Alireza Rezazadeh
Abstract	Predicting sales opportunities outcome is a core to successful business management and revenue forecasting. Conventionally, this prediction has relied mostly on subjective human evaluations in the process of business to business (B2B) sales decision making. Here, we proposed a practical Machine Learning (ML) workflow to empower B2B sales outcome (win/lose) prediction within a cloud-based computing platform: Microsoft Azure Machine Learning Service (Azure ML). This workflow consists of two pipelines: 1) an ML pipeline that trains probabilistic predictive models in parallel on the closed sales opportunities data enhanced with an extensive feature engineering procedure for automated selection and parameterization of an optimal ML model and 2) a Prediction pipeline that uses the optimal ML model to estimate the likelihood of winning new sales opportunities as well as predicting their outcome using optimized decision boundaries. The performance of the proposed workflow was evaluated on a real sales dataset of a B2B consulting firm.
Tasks	Decision Making, Feature Engineering
Published	2020-02-04
URL	https://arxiv.org/abs/2002.01441v1
PDF	https://arxiv.org/pdf/2002.01441v1.pdf
PWC	https://paperswithcode.com/paper/a-generalized-flow-for-b2b-sales-predictive
Repo
Framework

Towards Stable and Comprehensive Domain Alignment: Max-Margin Domain-Adversarial Training


Title	Towards Stable and Comprehensive Domain Alignment: Max-Margin Domain-Adversarial Training
Authors	Jianfei Yang, Han Zou, Yuxun Zhou, Lihua Xie
Abstract	Domain adaptation tackles the problem of transferring knowledge from a label-rich source domain to a label-scarce or even unlabeled target domain. Recently domain-adversarial training (DAT) has shown promising capacity to learn a domain-invariant feature space by reversing the gradient propagation of a domain classifier. However, DAT is still vulnerable in several aspects including (1) training instability due to the overwhelming discriminative ability of the domain classifier in adversarial training, (2) restrictive feature-level alignment, and (3) lack of interpretability or systematic explanation of the learned feature space. In this paper, we propose a novel Max-margin Domain-Adversarial Training (MDAT) by designing an Adversarial Reconstruction Network (ARN). The proposed MDAT stabilizes the gradient reversing in ARN by replacing the domain classifier with a reconstruction network, and in this manner ARN conducts both feature-level and pixel-level domain alignment without involving extra network structures. Furthermore, ARN demonstrates strong robustness to a wide range of hyper-parameters settings, greatly alleviating the task of model selection. Extensive empirical results validate that our approach outperforms other state-of-the-art domain alignment methods. Moreover, reconstructing adapted features reveals the domain-invariant feature space which conforms with our intuition.
Tasks	Domain Adaptation, Model Selection
Published	2020-03-30
URL	https://arxiv.org/abs/2003.13249v1
PDF	https://arxiv.org/pdf/2003.13249v1.pdf
PWC	https://paperswithcode.com/paper/towards-stable-and-comprehensive-domain-1
Repo
Framework

Generating Chinese Poetry from Images via Concrete and Abstract Information


Title	Generating Chinese Poetry from Images via Concrete and Abstract Information
Authors	Yusen Liu, Dayiheng Liu, Jiancheng Lv, Yongsheng Sang
Abstract	In recent years, the automatic generation of classical Chinese poetry has made great progress. Besides focusing on improving the quality of the generated poetry, there is a new topic about generating poetry from an image. However, the existing methods for this topic still have the problem of topic drift and semantic inconsistency, and the image-poem pairs dataset is hard to be built when training these models. In this paper, we extract and integrate the Concrete and Abstract information from images to address those issues. We proposed an infilling-based Chinese poetry generation model which can infill the Concrete keywords into each line of poems in an explicit way, and an abstract information embedding to integrate the Abstract information into generated poems. In addition, we use non-parallel data during training and construct separate image datasets and poem datasets to train the different components in our framework. Both automatic and human evaluation results show that our approach can generate poems which have better consistency with images without losing the quality.
Tasks
Published	2020-03-24
URL	https://arxiv.org/abs/2003.10773v1
PDF	https://arxiv.org/pdf/2003.10773v1.pdf
PWC	https://paperswithcode.com/paper/generating-chinese-poetry-from-images-via
Repo
Framework