January 30, 2020

3646 words 18 mins read

Paper Group ANR 224

High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework. Toward Learning a Unified Many-to-Many Mapping for Diverse Image Translation. Edge-Aware Deep Image Deblurring. Efficient Neural Network Approaches for Leather Defect Classification. Enhanced Convolutional Neural Tangent Kernels. Breast Cancer Diagnosis by Higher-Orde …

High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework


Title	High Dimensional M-Estimation with Missing Outcomes: A Semi-Parametric Framework
Authors	Abhishek Chakrabortty, Jiarui Lu, T. Tony Cai, Hongzhe Li
Abstract	We consider high dimensional $M$-estimation in settings where the response $Y$ is possibly missing at random and the covariates $\mathbf{X} \in \mathbb{R}^p$ can be high dimensional compared to the sample size $n$. The parameter of interest $\boldsymbol{\theta}_0 \in \mathbb{R}^d$ is defined as the minimizer of the risk of a convex loss, under a fully non-parametric model, and $\boldsymbol{\theta}_0$ itself is high dimensional which is a key distinction from existing works. Standard high dimensional regression and series estimation with possibly misspecified models and missing $Y$ are included as special cases, as well as their counterparts in causal inference using ‘potential outcomes’. Assuming $\boldsymbol{\theta}_0$ is $s$-sparse ($s \ll n$), we propose an $L_1$-regularized debiased and doubly robust (DDR) estimator of $\boldsymbol{\theta}_0$ based on a high dimensional adaptation of the traditional double robust (DR) estimator’s construction. Under mild tail assumptions and arbitrarily chosen (working) models for the propensity score (PS) and the outcome regression (OR) estimators, satisfying only some high-level conditions, we establish finite sample performance bounds for the DDR estimator showing its (optimal) $L_2$ error rate to be $\sqrt{s (\log d)/ n}$ when both models are correct, and its consistency and DR properties when only one of them is correct. Further, when both the models are correct, we propose a desparsified version of our DDR estimator that satisfies an asymptotic linear expansion and facilitates inference on low dimensional components of $\boldsymbol{\theta}_0$. Finally, we discuss various of choices of high dimensional parametric/semi-parametric working models for the PS and OR estimators. All results are validated via detailed simulations.
Tasks	Causal Inference
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11345v1
PDF	https://arxiv.org/pdf/1911.11345v1.pdf
PWC	https://paperswithcode.com/paper/high-dimensional-m-estimation-with-missing
Repo
Framework

Toward Learning a Unified Many-to-Many Mapping for Diverse Image Translation


Title	Toward Learning a Unified Many-to-Many Mapping for Diverse Image Translation
Authors	Wenju Xu, Shawn Keshmiri, Guanghui Wang
Abstract	Image-to-image translation, which translates input images to a different domain with a learned one-to-one mapping, has achieved impressive success in recent years. The success of translation mainly relies on the network architecture to reserve the structural information while modify the appearance slightly at the pixel level through adversarial training. Although these networks are able to learn the mapping, the translated images are predictable without exclusion. It is more desirable to diversify them using image-to-image translation by introducing uncertainties, i.e., the generated images hold potential for variations in colors and textures in addition to the general similarity to the input images, and this happens in both the target and source domains. To this end, we propose a novel generative adversarial network (GAN) based model, InjectionGAN, to learn a many-to-many mapping. In this model, the input image is combined with latent variables, which comprise of domain-specific attribute and unspecific random variations. The domain-specific attribute indicates the target domain of the translation, while the unspecific random variations introduce uncertainty into the model. A unified framework is proposed to regroup these two parts and obtain diverse generations in each domain. Extensive experiments demonstrate that the diverse generations have high quality for the challenging image-to-image translation tasks where no pairing information of the training dataset exits. Both quantitative and qualitative results prove the superior performance of InjectionGAN over the state-of-the-art approaches.
Tasks	Image-to-Image Translation
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08766v1
PDF	https://arxiv.org/pdf/1905.08766v1.pdf
PWC	https://paperswithcode.com/paper/toward-learning-a-unified-many-to-many
Repo
Framework

Edge-Aware Deep Image Deblurring


Title	Edge-Aware Deep Image Deblurring
Authors	Zhichao Fu, Yingbin Zheng, Hao Ye, Yu Kong, Jing Yang, Liang He
Abstract	Image deblurring is a fundamental and challenging low-level vision problem. Previous vision research indicates that edge structure in natural scenes is one of the most important factors to estimate the abilities of human visual perception. In this paper, we resort to human visual demands of sharp edges and propose a two-phase edge-aware deep network to improve deep image deblurring. An edge detection convolutional subnet is designed in the first phase and a residual fully convolutional deblur subnet is then used for generating deblur results. The introduction of the edge-aware network enables our model with the specific capacity of enhancing images with sharp edges. We successfully apply our framework on standard benchmarks and promising results are achieved by our proposed deblur model.
Tasks	Deblurring, Edge Detection
Published	2019-07-04
URL	https://arxiv.org/abs/1907.02282v1
PDF	https://arxiv.org/pdf/1907.02282v1.pdf
PWC	https://paperswithcode.com/paper/edge-aware-deep-image-deblurring
Repo
Framework

Efficient Neural Network Approaches for Leather Defect Classification


Title	Efficient Neural Network Approaches for Leather Defect Classification
Authors	Sze-Teng Liong, Y. S. Gan, Kun-Hong Liu, Tran Quang Binh, Cong Tue Le, Chien An Wu, Cheng-Yan Yang, Yen-Chang Huang
Abstract	Genuine leather, such as the hides of cows, crocodiles, lizards and goats usually contain natural and artificial defects, like holes, fly bites, tick marks, veining, cuts, wrinkles and others. A traditional solution to identify the defects is by manual defect inspection, which involves skilled experts. It is time consuming and may incur a high error rate and results in low productivity. This paper presents a series of automatic image processing processes to perform the classification of leather defects by adopting deep learning approaches. Particularly, the leather images are first partitioned into small patches,then it undergoes a pre-processing technique, namely the Canny edge detection to enhance defect visualization. Next, artificial neural network (ANN) and convolutional neural network (CNN) are employed to extract the rich image features. The best classification result achieved is 80.3 %, evaluated on a data set that consists of 2000 samples. In addition, the performance metrics such as confusion matrix and Receiver Operating Characteristic (ROC) are reported to demonstrate the efficiency of the method proposed.
Tasks	Edge Detection
Published	2019-06-15
URL	https://arxiv.org/abs/1906.06446v1
PDF	https://arxiv.org/pdf/1906.06446v1.pdf
PWC	https://paperswithcode.com/paper/efficient-neural-network-approaches-for
Repo
Framework

Enhanced Convolutional Neural Tangent Kernels


Title	Enhanced Convolutional Neural Tangent Kernels
Authors	Zhiyuan Li, Ruosong Wang, Dingli Yu, Simon S. Du, Wei Hu, Ruslan Salakhutdinov, Sanjeev Arora
Abstract	Recent research shows that for training with $\ell_2$ loss, convolutional neural networks (CNNs) whose width (number of channels in convolutional layers) goes to infinity correspond to regression with respect to the CNN Gaussian Process kernel (CNN-GP) if only the last layer is trained, and correspond to regression with respect to the Convolutional Neural Tangent Kernel (CNTK) if all layers are trained. An exact algorithm to compute CNTK (Arora et al., 2019) yielded the finding that classification accuracy of CNTK on CIFAR-10 is within 6-7% of that of that of the corresponding CNN architecture (best figure being around 78%) which is interesting performance for a fixed kernel. Here we show how to significantly enhance the performance of these kernels using two ideas. (1) Modifying the kernel using a new operation called Local Average Pooling (LAP) which preserves efficient computability of the kernel and inherits the spirit of standard data augmentation using pixel shifts. Earlier papers were unable to incorporate naive data augmentation because of the quadratic training cost of kernel regression. This idea is inspired by Global Average Pooling (GAP), which we show for CNN-GP and CNTK is equivalent to full translation data augmentation. (2) Representing the input image using a pre-processing technique proposed by Coates et al. (2011), which uses a single convolutional layer composed of random image patches. On CIFAR-10, the resulting kernel, CNN-GP with LAP and horizontal flip data augmentation, achieves 89% accuracy, matching the performance of AlexNet (Krizhevsky et al., 2012). Note that this is the best such result we know of for a classifier that is not a trained neural network. Similar improvements are obtained for Fashion-MNIST.
Tasks	Data Augmentation
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00809v1
PDF	https://arxiv.org/pdf/1911.00809v1.pdf
PWC	https://paperswithcode.com/paper/enhanced-convolutional-neural-tangent-kernels-1
Repo
Framework

Breast Cancer Diagnosis by Higher-Order Probabilistic Perceptrons


Title	Breast Cancer Diagnosis by Higher-Order Probabilistic Perceptrons
Authors	Aditya Cowsik, John W. Clark
Abstract	A two-layer neural network model that systematically includes correlations among input variables to arbitrary order and is designed to implement Bayes inference has been adapted to classify breast cancer tumors as malignant or benign, assigning a probability for either outcome. The inputs to the network represent measured characteristics of cell nuclei imaged in Fine Needle Aspiration biopsies. The present machine-learning approach to diagnosis (known as HOPP, for higher-order probabilistic perceptron) is tested on the much-studied, open-access Breast Cancer Wisconsin (Diagnosis) Data Set of Wolberg et al. This set lists, for each tumor, measured physical parameters of the cell nuclei of each sample. The HOPP model can identify the key factors – input features and their combinations – most relevant for reliable diagnosis. HOPP networks were trained on 90% of the examples in the Wisconsin database, and tested on the remaining 10%. Referred to ensembles of 300 networks, selected randomly for cross-validation, accuracy of classification for the test sets of up to 97% was readily achieved, with standard deviation around 2%, together with average Matthews correlation coefficients reaching 0.94 indicating excellent predictive performance. Demonstrably, the HOPP is capable of matching the predictive power attained by other advanced machine-learning algorithms applied to this much-studied database, over several decades. Analysis shows that in this special problem, which is almost linearly separable, the effects of irreducible correlations among the measured features of the Wisconsin database are of relatively minor importance, as the Naive Bayes approximation can itself yield predictive accuracy approaching 95%. The advantages of the HOPP algorithm will be more clearly revealed in application to more challenging machine-learning problems.
Tasks
Published	2019-12-15
URL	https://arxiv.org/abs/1912.06969v1
PDF	https://arxiv.org/pdf/1912.06969v1.pdf
PWC	https://paperswithcode.com/paper/breast-cancer-diagnosis-by-higher-order
Repo
Framework

Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning


Title	Semisupervised Adversarial Neural Networks for Cyber Security Transfer Learning
Authors	Casey Kneale, Kolia Sadeghi
Abstract	On the path to establishing a global cybersecurity framework where each enterprise shares information about malicious behavior, an important question arises. How can a machine learning representation characterizing a cyber attack on one network be used to detect similar attacks on other enterprise networks if each networks has wildly different distributions of benign and malicious traffic? We address this issue by comparing the results of naively transferring a model across network domains and using CORrelation ALignment, to our novel adversarial Siamese neural network. Our proposed model learns attack representations that are more invariant to each network’s particularities via an adversarial approach. It uses a simple ranking loss that prioritizes the labeling of the most egregious malicious events correctly over average accuracy. This is appropriate for driving an alert triage workflow wherein an analyst only has time to inspect the top few events ranked highest by the model. In terms of accuracy, the other approaches fail completely to detect any malicious events when models were trained on one dataset are evaluated on another for the first 100 events. While, the method presented here retrieves sizable proportions of malicious events, at the expense of some training instabilities due in adversarial modeling. We evaluate these approaches using 2 publicly available networking datasets, and suggest areas for future research.
Tasks	Transfer Learning
Published	2019-07-25
URL	https://arxiv.org/abs/1907.11129v1
PDF	https://arxiv.org/pdf/1907.11129v1.pdf
PWC	https://paperswithcode.com/paper/semisupervised-adversarial-neural-networks
Repo
Framework


Title	Classification of dry age-related macular degeneration and diabetic macular edema from optical coherence tomography images using dictionary learning
Authors	Elahe Mousavi, Rahele Kafieh, Hossein Rabbani
Abstract	Age-related Macular Degeneration (AMD) and Diabetic Macular Edema (DME) are the major causes of vision loss in developed countries. Alteration of retinal layer structure and appearance of exudate are the most significant signs of these diseases. With the aim of automatic classification of DME, AMD and normal subjects from Optical Coherence Tomography (OCT) images, we proposed a classification algorithm. The two important issues intended in this approach are, not utilizing retinal layer segmentation which by itself is a challenging task and attempting to identify diseases in their early stages, where the signs of diseases appear in a small fraction of B-Scans. We used a histogram of oriented gradients (HOG) feature descriptor to well characterize the distribution of local intensity gradients and edge directions. In order to capture the structure of extracted features, we employed different dictionary learning-based classifiers. Our dataset consists of 45 subjects: 15 patients with AMD, 15 patients with DME and 15 normal subjects. The proposed classifier leads to an accuracy of 95.13%, 100.00%, and 100.00% for DME, AMD, and normal OCT images, respectively, only by considering the 4% of all B-Scans of a volume which outperforms the state of the art methods.
Tasks	Dictionary Learning
Published	2019-03-16
URL	http://arxiv.org/abs/1903.06909v1
PDF	http://arxiv.org/pdf/1903.06909v1.pdf
PWC	https://paperswithcode.com/paper/classification-of-dry-age-related-macular
Repo
Framework

Label-Noise Robust Multi-Domain Image-to-Image Translation


Title	Label-Noise Robust Multi-Domain Image-to-Image Translation
Authors	Takuhiro Kaneko, Tatsuya Harada
Abstract	Multi-domain image-to-image translation is a problem where the goal is to learn mappings among multiple domains. This problem is challenging in terms of scalability because it requires the learning of numerous mappings, the number of which increases proportional to the number of domains. However, generative adversarial networks (GANs) have emerged recently as a powerful framework for this problem. In particular, label-conditional extensions (e.g., StarGAN) have become a promising solution owing to their ability to address this problem using only a single unified model. Nonetheless, a limitation is that they rely on the availability of large-scale clean-labeled data, which are often laborious or impractical to collect in a real-world scenario. To overcome this limitation, we propose a novel model called the label-noise robust image-to-image translation model (RMIT) that can learn a clean label conditional generator even when noisy labeled data are only available. In particular, we propose a novel loss called the virtual cycle consistency loss that is able to regularize cyclic reconstruction independently of noisy labeled data, as well as we introduce advanced techniques to boost the performance in practice. Our experimental results demonstrate that RMIT is useful for obtaining label-noise robustness in various settings including synthetic and real-world noise.
Tasks	Image-to-Image Translation
Published	2019-05-06
URL	https://arxiv.org/abs/1905.02185v1
PDF	https://arxiv.org/pdf/1905.02185v1.pdf
PWC	https://paperswithcode.com/paper/label-noise-robust-multi-domain-image-to
Repo
Framework

A Rule for Gradient Estimator Selection, with an Application to Variational Inference


Title	A Rule for Gradient Estimator Selection, with an Application to Variational Inference
Authors	Tomas Geffner, Justin Domke
Abstract	Stochastic gradient descent (SGD) is the workhorse of modern machine learning. Sometimes, there are many different potential gradient estimators that can be used. When so, choosing the one with the best tradeoff between cost and variance is important. This paper analyzes the convergence rates of SGD as a function of time, rather than iterations. This results in a simple rule to select the estimator that leads to the best optimization convergence guarantee. This choice is the same for different variants of SGD, and with different assumptions about the objective (e.g. convexity or smoothness). Inspired by this principle, we propose a technique to automatically select an estimator when a finite pool of estimators is given. Then, we extend to infinite pools of estimators, where each one is indexed by control variate weights. This is enabled by a reduction to a mixed-integer quadratic program. Empirically, automatically choosing an estimator performs comparably to the best estimator chosen with hindsight.
Tasks
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01894v1
PDF	https://arxiv.org/pdf/1911.01894v1.pdf
PWC	https://paperswithcode.com/paper/a-rule-for-gradient-estimator-selection-with
Repo
Framework

Confident Kernel Sparse Coding and Dictionary Learning


Title	Confident Kernel Sparse Coding and Dictionary Learning
Authors	Babak Hosseini, Barbara Hammer
Abstract	In recent years, kernel-based sparse coding (K-SRC) has received particular attention due to its efficient representation of nonlinear data structures in the feature space. Nevertheless, the existing K-SRC methods suffer from the lack of consistency between their training and test optimization frameworks. In this work, we propose a novel confident K-SRC and dictionary learning algorithm (CKSC) which focuses on the discriminative reconstruction of the data based on its representation in the kernel space. CKSC focuses on reconstructing each data sample via weighted contributions which are confident in its corresponding class of data. We employ novel discriminative terms to apply this scheme to both training and test frameworks in our algorithm. This specific design increases the consistency of these optimization frameworks and improves the discriminative performance in the recall phase. In addition, CKSC directly employs the supervised information in its dictionary learning framework to enhance the discriminative structure of the dictionary. For empirical evaluations, we implement our CKSC algorithm on multivariate time-series benchmarks such as DynTex++ and UTKinect. Our claims regarding the superior performance of the proposed algorithm are justified throughout comparing its classification results to the state-of-the-art K-SRC algorithms.
Tasks	Dictionary Learning, Time Series
Published	2019-03-12
URL	http://arxiv.org/abs/1903.05219v1
PDF	http://arxiv.org/pdf/1903.05219v1.pdf
PWC	https://paperswithcode.com/paper/confident-kernel-sparse-coding-and-dictionary
Repo
Framework

Pain Detection with fNIRS-Measured Brain Signals: A Personalized Machine Learning Approach Using the Wavelet Transform and Bayesian Hierarchical Modeling with Dirichlet Process Priors


Title	Pain Detection with fNIRS-Measured Brain Signals: A Personalized Machine Learning Approach Using the Wavelet Transform and Bayesian Hierarchical Modeling with Dirichlet Process Priors
Authors	Daniel Lopez-Martinez, Ke Peng, Arielle Lee, David Borsook, Rosalind Picard
Abstract	Currently self-report pain ratings are the gold standard in clinical pain assessment. However, the development of objective automatic measures of pain could substantially aid pain diagnosis and therapy. Recent neuroimaging studies have shown the potential of functional near-infrared spectroscopy (fNIRS) for pain detection. This is a brain-imaging technique that provides non-invasive, long-term measurements of cortical hemoglobin concentration changes. In this study, we focused on fNIRS signals acquired exclusively from the prefrontal cortex, which can be accessed unobtrusively, and derived an algorithm for the detection of the presence of pain using Bayesian hierarchical modelling with wavelet features. This approach allows personalization of the inference process by accounting for inter-participant variability in pain responses. Our work highlights the importance of adopting a personalized approach and supports the use of fNIRS for pain assessment.
Tasks
Published	2019-07-30
URL	https://arxiv.org/abs/1907.12830v1
PDF	https://arxiv.org/pdf/1907.12830v1.pdf
PWC	https://paperswithcode.com/paper/pain-detection-with-fnirs-measured-brain
Repo
Framework

Analysis Dictionary Learning: An Efficient and Discriminative Solution


Title	Analysis Dictionary Learning: An Efficient and Discriminative Solution
Authors	Wen Tang, Ashkan Panahi, Hamid Krim, Liyi Dai
Abstract	Discriminative Dictionary Learning (DL) methods have been widely advocated for image classification problems. To further sharpen their discriminative capabilities, most state-of-the-art DL methods have additional constraints included in the learning stages. These various constraints, however, lead to additional computational complexity. We hence propose an efficient Discriminative Convolutional Analysis Dictionary Learning (DCADL) method, as a lower cost Discriminative DL framework, to both characterize the image structures and refine the interclass structure representations. The proposed DCADL jointly learns a convolutional analysis dictionary and a universal classifier, while greatly reducing the time complexity in both training and testing phases, and achieving a competitive accuracy, thus demonstrating great performance in many experiments with standard databases.
Tasks	Dictionary Learning, Image Classification
Published	2019-03-07
URL	http://arxiv.org/abs/1903.03058v1
PDF	http://arxiv.org/pdf/1903.03058v1.pdf
PWC	https://paperswithcode.com/paper/analysis-dictionary-learning-an-efficient-and
Repo
Framework

Bidirectional RNN-based Few-shot Training for Detecting Multi-stage Attack


Title	Bidirectional RNN-based Few-shot Training for Detecting Multi-stage Attack
Authors	Di Zhao, Jiqiang Liu, Jialin Wang, Wenjia Niu, Endong Tong, Tong Chen, Gang Li
Abstract	“Feint Attack”, as a new type of APT attack, has become the focus of attention. It adopts a multi-stage attacks mode which can be concluded as a combination of virtual attacks and real attacks. Under the cover of virtual attacks, real attacks can achieve the real purpose of the attacker, as a result, it often caused huge losses inadvertently. However, to our knowledge, all previous works use common methods such as Causal-Correlation or Cased-based to detect outdated multi-stage attacks. Few attentions have been paid to detect the “Feint Attack”, because the difficulty of detection lies in the diversification of the concept of “Feint Attack” and the lack of professional datasets, many detection methods ignore the semantic relationship in the attack. Aiming at the existing challenge, this paper explores a new method to solve the problem. In the attack scenario, the fuzzy clustering method based on attribute similarity is used to mine multi-stage attack chains. Then we use a few-shot deep learning algorithm (SMOTE&CNN-SVM) and bidirectional Recurrent Neural Network model (Bi-RNN) to obtain the “Feint Attack” chains. “Feint Attack” is simulated by the real attack inserted in the normal causal attack chain, and the addition of the real attack destroys the causal relationship of the original attack chain. So we used Bi-RNN coding to obtain the hidden feature of “Feint Attack” chain. In the end, our method achieved the goal to detect the “Feint Attack” accurately by using the LLDoS1.0 and LLDoS2.0 of DARPA2000 and CICIDS2017 of Canadian Institute for Cybersecurity.
Tasks
Published	2019-05-09
URL	https://arxiv.org/abs/1905.03454v1
PDF	https://arxiv.org/pdf/1905.03454v1.pdf
PWC	https://paperswithcode.com/paper/190503454
Repo
Framework

Towards Instance-level Image-to-Image Translation


Title	Towards Instance-level Image-to-Image Translation
Authors	Zhiqiang Shen, Mingyang Huang, Jianping Shi, Xiangyang Xue, Thomas Huang
Abstract	Unpaired Image-to-image Translation is a new rising and challenging vision problem that aims to learn a mapping between unaligned image pairs in diverse domains. Recent advances in this field like MUNIT and DRIT mainly focus on disentangling content and style/attribute from a given image first, then directly adopting the global style to guide the model to synthesize new domain images. However, this kind of approaches severely incurs contradiction if the target domain images are content-rich with multiple discrepant objects. In this paper, we present a simple yet effective instance-aware image-to-image translation approach (INIT), which employs the fine-grained local (instance) and global styles to the target image spatially. The proposed INIT exhibits three import advantages: (1) the instance-level objective loss can help learn a more accurate reconstruction and incorporate diverse attributes of objects; (2) the styles used for target domain of local/global areas are from corresponding spatial regions in source domain, which intuitively is a more reasonable mapping; (3) the joint training process can benefit both fine and coarse granularity and incorporates instance information to improve the quality of global translation. We also collect a large-scale benchmark for the new instance-level translation task. We observe that our synthetic images can even benefit real-world vision tasks like generic object detection.
Tasks	Image-to-Image Translation, Object Detection
Published	2019-05-05
URL	https://arxiv.org/abs/1905.01744v1
PDF	https://arxiv.org/pdf/1905.01744v1.pdf
PWC	https://paperswithcode.com/paper/towards-instance-level-image-to-image
Repo
Framework