January 31, 2020

3021 words 15 mins read

Paper Group ANR 52

Fundamental Entropic Laws and $\mathcal{L}_p$ Limitations of Feedback Systems: Implications for Machine-Learning-in-the-Loop Control. Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons. Region-Adaptive Dense Network for Efficient Motion Deblurring. Automatic detection and diagnosis of sacroiliitis in CT scans as incidenta …

Fundamental Entropic Laws and $\mathcal{L}_p$ Limitations of Feedback Systems: Implications for Machine-Learning-in-the-Loop Control


Title	Fundamental Entropic Laws and $\mathcal{L}_p$ Limitations of Feedback Systems: Implications for Machine-Learning-in-the-Loop Control
Authors	Song Fang, Quanyan Zhu
Abstract	In this paper, we study the fundamental performance limitations for generic feedback systems in which both the controller and the plant may be arbitrarily causal while the disturbance can be with any distributions. In particular, we obtain fundamental $\mathcal{L}_p$ bounds based on the entropic laws that are inherent in any feedback systems. We also examine the implications of the generic bounds for machine-learning-in-the-loop control; in other words, fundamental limits in general exist to what machine learning elements in feedback loops can achieve.
Tasks
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05541v2
PDF	https://arxiv.org/pdf/1912.05541v2.pdf
PWC	https://paperswithcode.com/paper/fundamental-entropic-laws-and-mathcall_p
Repo
Framework

Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons


Title	Convolutional Neural Networks Utilizing Multifunctional Spin-Hall MTJ Neurons
Authors	Andrew W. Stephan, Steven J. Koester
Abstract	We propose a new network architecture for standard spin-Hall magnetic tunnel junction-based spintronic neurons that allows them to compute multiple critical convolutional neural network functionalities simultaneously and in parallel, saving space and time. An approximation to the Rectified Linear Unit transfer function and the local pooling function are computed simultaneously with the convolution operation itself. A proof-of-concept simulation is performed on the MNIST dataset, achieving up to 98% accuracy at a cost of less than 1 nJ for all convolution, activation and pooling operations combined. The simulations are remarkably robust to thermal noise, performing well even with very small magnetic layers.
Tasks
Published	2019-05-09
URL	https://arxiv.org/abs/1905.03812v1
PDF	https://arxiv.org/pdf/1905.03812v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-neural-networks-utilizing
Repo
Framework

Region-Adaptive Dense Network for Efficient Motion Deblurring


Title	Region-Adaptive Dense Network for Efficient Motion Deblurring
Authors	Kuldeep Purohit, A. N. Rajagopalan
Abstract	In this paper, we address the problem of dynamic scene deblurring in the presence of motion blur. Restoration of images affected by severe blur necessitates a network design with a large receptive field, which existing networks attempt to achieve through simple increment in the number of generic convolution layers, kernel-size, or the scales at which the image is processed. However, these techniques ignore the non-uniform nature of blur, and they come at the expense of an increase in model size and inference time. We present a new architecture composed of region adaptive dense deformable modules that implicitly discover the spatially varying shifts responsible for non-uniform blur in the input image and learn to modulate the filters. This capability is complemented by a self-attentive module which captures non-local spatial relationships among the intermediate features and enhances the spatially-varying processing capability. We incorporate these modules into a densely connected encoder-decoder design which utilizes pre-trained Densenet filters to further improve the performance. Our network facilitates interpretable modeling of the spatially-varying deblurring process while dispensing with multi-scale processing and large filters entirely. Extensive comparisons with prior art on benchmark dynamic scene deblurring datasets clearly demonstrate the superiority of the proposed networks via significant improvements in accuracy and speed, enabling almost real-time deblurring.
Tasks	Deblurring
Published	2019-03-25
URL	https://arxiv.org/abs/1903.11394v3
PDF	https://arxiv.org/pdf/1903.11394v3.pdf
PWC	https://paperswithcode.com/paper/spatially-adaptive-residual-networks-for
Repo
Framework

Automatic detection and diagnosis of sacroiliitis in CT scans as incidental findings


Title	Automatic detection and diagnosis of sacroiliitis in CT scans as incidental findings
Authors	Yigal Shenkman, Bilal Qutteineh, Leo Joskowicz, Adi Szeskin, Yusef Azraq, Arnaldo Mayer, Iris Eshed
Abstract	Early diagnosis of sacroiliitis may lead to preventive treatment which can significantly improve the patient’s quality of life in the long run. Oftentimes, a CT scan of the lower back or abdomen is acquired for suspected back pain. However, since the differences between a healthy and an inflamed sacroiliac joint in the early stages are subtle, the condition may be missed. We have developed a new automatic algorithm for the diagnosis and grading of sacroiliitis CT scans as incidental findings, for patients who underwent CT scanning as part of their lower back pain workout. The method is based on supervised machine and deep learning techniques. The input is a CT scan that includes the patient’s pelvis. The output is a diagnosis for each sacroiliac joint. The algorithm consists of four steps: 1) computation of an initial region of interest (ROI) that includes the pelvic joints region using heuristics and a U-Net classifier; 2) refinement of the ROI to detect both sacroiliac joints using a four-tree random forest; 3) individual sacroiliitis grading of each sacroiliac joint in each CT slice with a custom slice CNN classifier, and; 4) sacroiliitis diagnosis and grading by combining the individual slice grades using a random forest. Experimental results on 484 sacroiliac joints yield a binary and a 3-class case classification accuracy of 91.9% and 86%, a sensitivity of 95% and 82%, and an Area-Under-the-Curve of 0.97 and 0.57, respectively. Automatic computer-based analysis of CT scans has the potential of being a useful method for the diagnosis and grading of sacroiliitis as an incidental finding.
Tasks
Published	2019-08-14
URL	https://arxiv.org/abs/1908.05663v1
PDF	https://arxiv.org/pdf/1908.05663v1.pdf
PWC	https://paperswithcode.com/paper/automatic-detection-and-diagnosis-of
Repo
Framework

Learning Effective Loss Functions Efficiently


Title	Learning Effective Loss Functions Efficiently
Authors	Matthew Streeter
Abstract	We consider the problem of learning a loss function which, when minimized over a training dataset, yields a model that approximately minimizes a validation error metric. Though learning an optimal loss function is NP-hard, we present an anytime algorithm that is asymptotically optimal in the worst case, and is provably efficient in an idealized “easy” case. Experimentally, we show that this algorithm can be used to tune loss function hyperparameters orders of magnitude faster than state-of-the-art alternatives. We also show that our algorithm can be used to learn novel and effective loss functions on-the-fly during training.
Tasks
Published	2019-06-28
URL	https://arxiv.org/abs/1907.00103v1
PDF	https://arxiv.org/pdf/1907.00103v1.pdf
PWC	https://paperswithcode.com/paper/learning-effective-loss-functions-efficiently
Repo
Framework

SGD Learns One-Layer Networks in WGANs


Title	SGD Learns One-Layer Networks in WGANs
Authors	Qi Lei, Jason D. Lee, Alexandros G. Dimakis, Constantinos Daskalakis
Abstract	Generative adversarial networks (GANs) are a widely used framework for learning generative models. Wasserstein GANs (WGANs), one of the most successful variants of GANs, require solving a minmax optimization problem to global optimality, but are in practice successfully trained using stochastic gradient descent-ascent. In this paper, we show that, when the generator is a one-layer network, stochastic gradient descent-ascent converges to a global solution with polynomial time and sample complexity.
Tasks
Published	2019-10-15
URL	https://arxiv.org/abs/1910.07030v1
PDF	https://arxiv.org/pdf/1910.07030v1.pdf
PWC	https://paperswithcode.com/paper/sgd-learns-one-layer-networks-in-wgans-1
Repo
Framework

Verification and Validation of Computer Models for Diagnosing Breast Cancer Based on Machine Learning for Medical Data Analysis


Title	Verification and Validation of Computer Models for Diagnosing Breast Cancer Based on Machine Learning for Medical Data Analysis
Authors	Vladislav Levshinskii, Maxim Polyakov, Alexander Losev, Alexander Khoperskov
Abstract	The method of microwave radiometry is one of the areas of medical diagnosis of breast cancer. It is based on analysis of the spatial distribution of internal and surface tissue temperatures, which are measured in the microwave (RTM) and infrared (IR) ranges. Complex mathematical and computer models describing complex physical and biological processes within biotissue increase the efficiency of this method. Physical and biological processes are related to temperature dynamics and microwave electromagnetic radiation. Verification and validation of the numerical model is a key challenge to ensure consistency with medical big data. These data are obtained by medical measurements of patients. We present an original approach to verification and validation of simulation models of physical processes in biological tissues. Our approach is based on deep analysis of medical data and we use machine learning algorithms. We have achieved impressive success for the model of dynamics of thermal processes in a breast with cancer foci. This method allows us to carry out a significant refinement of almost all parameters of the mathematical model in order to achieve the maximum possible adequacy.
Tasks	Medical Diagnosis
Published	2019-09-21
URL	https://arxiv.org/abs/1910.02779v1
PDF	https://arxiv.org/pdf/1910.02779v1.pdf
PWC	https://paperswithcode.com/paper/verification-and-validation-of-computer
Repo
Framework

Unsupervised Domain-Specific Deblurring via Disentangled Representations


Title	Unsupervised Domain-Specific Deblurring via Disentangled Representations
Authors	Boyu Lu, Jun-Cheng Chen, Rama Chellappa
Abstract	Image deblurring aims to restore the latent sharp images from the corresponding blurred ones. In this paper, we present an unsupervised method for domain-specific single-image deblurring based on disentangled representations. The disentanglement is achieved by splitting the content and blur features in a blurred image using content encoders and blur encoders. We enforce a KL divergence loss to regularize the distribution range of extracted blur attributes such that little content information is contained. Meanwhile, to handle the unpaired training data, a blurring branch and the cycle-consistency loss are added to guarantee that the content structures of the deblurred results match the original images. We also add an adversarial loss on deblurred results to generate visually realistic images and a perceptual loss to further mitigate the artifacts. We perform extensive experiments on the tasks of face and text deblurring using both synthetic datasets and real images, and achieve improved results compared to recent state-of-the-art deblurring methods.
Tasks	Deblurring
Published	2019-03-05
URL	https://arxiv.org/abs/1903.01594v2
PDF	https://arxiv.org/pdf/1903.01594v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-domain-specific-deblurring-via
Repo
Framework

Extreme Channel Prior Embedded Network for Dynamic Scene Deblurring


Title	Extreme Channel Prior Embedded Network for Dynamic Scene Deblurring
Authors	Jianrui Cai, Wangmeng Zuo, Lei Zhang
Abstract	Recent years have witnessed the significant progress on convolutional neural networks (CNNs) in dynamic scene deblurring. While CNN models are generally learned by the reconstruction loss defined on training data, incorporating suitable image priors as well as regularization terms into the network architecture could boost the deblurring performance. In this work, we propose an Extreme Channel Prior embedded Network (ECPeNet) to plug the extreme channel priors (i.e., priors on dark and bright channels) into a network architecture for effective dynamic scene deblurring. A novel trainable extreme channel prior embedded layer (ECPeL) is developed to aggregate both extreme channel and blurry image representations, and sparse regularization is introduced to regularize the ECPeNet model learning. Furthermore, we present an effective multi-scale network architecture that works in both coarse-to-fine and fine-to-coarse manners for better exploiting information flow across scales. Experimental results on GoPro and Kohler datasets show that our proposed ECPeNet performs favorably against state-of-the-art deep image deblurring methods in terms of both quantitative metrics and visual quality.
Tasks	Deblurring
Published	2019-03-02
URL	http://arxiv.org/abs/1903.00763v1
PDF	http://arxiv.org/pdf/1903.00763v1.pdf
PWC	https://paperswithcode.com/paper/extreme-channel-prior-embedded-network-for
Repo
Framework

An Acceleration Framework for High Resolution Image Synthesis


Title	An Acceleration Framework for High Resolution Image Synthesis
Authors	Jinlin Liu, Yuan Yao, Jianqiang Ren
Abstract	Synthesis of high resolution images using Generative Adversarial Networks (GANs) is challenging, which usually requires numbers of high-end graphic cards with large memory and long time of training. In this paper, we propose a two-stage framework to accelerate the training process of synthesizing high resolution images. High resolution images are first transformed to small codes via the trained encoder and decoder networks. The code in latent space is times smaller than the original high resolution images. Then, we train a code generation network to learn the distribution of the latent codes. In this way, the generator only learns to generate small latent codes instead of large images. Finally, we decode the generated latent codes to image space via the decoder networks so as to output the synthesized high resolution images. Experimental results show that the proposed method accelerates the training process significantly and increases the quality of the generated samples. The proposed acceleration framework makes it possible to generate high resolution images using less training time with limited hardware resource. After using the proposed acceleration method, it takes only 3 days to train a 1024 *1024 image generator on Celeba-HQ dataset using just one NVIDIA P100 graphic card.
Tasks	Code Generation, Image Generation
Published	2019-09-09
URL	https://arxiv.org/abs/1909.03611v1
PDF	https://arxiv.org/pdf/1909.03611v1.pdf
PWC	https://paperswithcode.com/paper/an-acceleration-framework-for-high-resolution
Repo
Framework

A Matrix–free Likelihood Method for Exploratory Factor Analysis of High-dimensional Gaussian Data


Title	A Matrix–free Likelihood Method for Exploratory Factor Analysis of High-dimensional Gaussian Data
Authors	Fan Dai, Somak Dutta, Ranjan Maitra
Abstract	This paper proposes a novel profile likelihood method for estimating the covariance parameters in exploratory factor analysis of high-dimensional Gaussian datasets with fewer observations than number of variables. An implicitly restarted Lanczos algorithm and a limited-memory quasi-Newton method are implemented to develop a matrix-free framework for likelihood maximization. Simulation results show that our method is substantially faster than the expectation-maximization solution without sacrificing accuracy. Our method is applied to fit factor models on data from suicide attempters, suicide ideators and a control group.
Tasks
Published	2019-07-27
URL	https://arxiv.org/abs/1907.11970v2
PDF	https://arxiv.org/pdf/1907.11970v2.pdf
PWC	https://paperswithcode.com/paper/a-matrix-free-likelihood-method-for
Repo
Framework

Bridging the ELBO and MMD


Title	Bridging the ELBO and MMD
Authors	Talip Ucar
Abstract	One of the challenges in training generative models such as the variational auto encoder (VAE) is avoiding posterior collapse. When the generator has too much capacity, it is prone to ignoring latent code. This problem is exacerbated when the dataset is small, and the latent dimension is high. The root of the problem is the ELBO objective, specifically the Kullback-Leibler (KL) divergence term in objective function \citep{zhao2019infovae}. This paper proposes a new objective function to replace the KL term with one that emulates the maximum mean discrepancy (MMD) objective. It also introduces a new technique, named latent clipping, that is used to control distance between samples in latent space. A probabilistic autoencoder model, named $\mu$-VAE, is designed and trained on MNIST and MNIST Fashion datasets, using the new objective function and is shown to outperform models trained with ELBO and $\beta$-VAE objective. The $\mu$-VAE is less prone to posterior collapse, and can generate reconstructions and new samples in good quality. Latent representations learned by $\mu$-VAE are shown to be good and can be used for downstream tasks such as classification.
Tasks
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13181v1
PDF	https://arxiv.org/pdf/1910.13181v1.pdf
PWC	https://paperswithcode.com/paper/191013181
Repo
Framework

Multi-Adapter RGBT Tracking


Title	Multi-Adapter RGBT Tracking
Authors	Chenglong Li, Andong Lu, Aihua Zheng, Zhengzheng Tu, Jin Tang
Abstract	The task of RGBT tracking aims to take the complementary advantages from visible spectrum and thermal infrared data to achieve robust visual tracking, and receives more and more attention in recent years. Existing works focus on modality-specific information integration by introducing modality weights to achieve adaptive fusion or learning robust feature representations of different modalities. Although these methods could effectively deploy the modality-specific properties, they ignore the potential values of modality-shared cues as well as instance-aware information, which are crucial for effective fusion of different modalities in RGBT tracking. In this paper, we propose a novel Multi-Adapter convolutional Network (MANet) to jointly perform modality-shared, modality-specific and instance-aware feature learning in an end-to-end trained deep framework for RGBT tracking. We design three kinds of adapters within our network. In a specific, the generality adapter is to extract shared object representations, the modality adapter aims at encoding modality-specific information to deploy their complementary advantages, and the instance adapter is to model the appearance properties and temporal variations of a certain object. Moreover, to reduce computational complexity for real-time demand of visual tracking, we design a parallel structure of generic adapter and modality adapter. Extensive experiments on two RGBT tracking benchmark datasets demonstrate the outstanding performance of the proposed tracker against other state-of-the-art RGB and RGBT tracking algorithms.
Tasks	Visual Tracking
Published	2019-07-17
URL	https://arxiv.org/abs/1907.07485v1
PDF	https://arxiv.org/pdf/1907.07485v1.pdf
PWC	https://paperswithcode.com/paper/multi-adapter-rgbt-tracking
Repo
Framework


Title	Unifying Ensemble Methods for Q-learning via Social Choice Theory
Authors	Rishav Chourasia, Adish Singla
Abstract	Ensemble methods have been widely applied in Reinforcement Learning (RL) in order to enhance stability, increase convergence speed, and improve exploration. These methods typically work by employing an aggregation mechanism over actions of different RL algorithms. We show that a variety of these methods can be unified by drawing parallels from committee voting rules in Social Choice Theory. We map the problem of designing an action aggregation mechanism in an ensemble method to a voting problem which, under different voting rules, yield popular ensemble-based RL algorithms like Majority Voting Q-learning or Bootstrapped Q-learning. Our unification framework, in turn, allows us to design new ensemble-RL algorithms with better performance. For instance, we map two diversity-centered committee voting rules, namely Single Non-Transferable Voting Rule and Chamberlin-Courant Rule, into new RL algorithms that demonstrate excellent exploratory behavior in our experiments.
Tasks	Q-Learning
Published	2019-02-27
URL	https://arxiv.org/abs/1902.10646v2
PDF	https://arxiv.org/pdf/1902.10646v2.pdf
PWC	https://paperswithcode.com/paper/unifying-ensemble-methods-for-q-learning-via
Repo
Framework

Finding a latent k-simplex in O(k . nnz(data)) time via Subset Smoothing


Title	Finding a latent k-simplex in O(k . nnz(data)) time via Subset Smoothing
Authors	Chiranjib Bhattacharyya, Ravindran Kannan
Abstract	In this paper we show that a large class of Latent variable models, such as Mixed Membership Stochastic Block(MMSB) Models, Topic Models, and Adversarial Clustering, can be unified through a geometric perspective, replacing model specific assumptions and algorithms for individual models. The geometric perspective leads to the formulation: \emph{find a latent $k-$ polytope $K$ in ${\bf R}^d$ given $n$ data points, each obtained by perturbing a latent point in $K$}. This problem does not seem to have been considered in the literature. The most important contribution of this paper is to show that the latent $k-$polytope problem admits an efficient algorithm under deterministic assumptions which naturally hold in Latent variable models considered in this paper. ur algorithm runs in time $O^(k; \mbox{nnz})$ matching the best running time of algorithms in special cases considered here and is better when the data is sparse, as is the case in applications. An important novelty of the algorithm is the introduction of \emph{subset smoothed polytope}, $K'$, the convex hull of the ${n\choose \delta n}$ points obtained by averaging all $\delta n$ subsets of the data points, for a given $\delta \in (0,1)$. We show that $K$ and $K'$ are close in Hausdroff distance. Among the consequences of our algorithm are the following: (a) MMSB Models and Topic Models: the first quasi-input-sparsity time algorithm for parameter estimation for $k \in O^(1)$, (b) Adversarial Clustering: In $k-$means, if, an adversary is allowed to move many data points from each cluster an arbitrary amount towards the convex hull of the centers of other clusters, our algorithm still estimates cluster centers well.
Tasks	Community Detection, Latent Variable Models, Topic Models
Published	2019-04-14
URL	https://arxiv.org/abs/1904.06738v4
PDF	https://arxiv.org/pdf/1904.06738v4.pdf
PWC	https://paperswithcode.com/paper/finding-a-latent-k-simplex-in-ok-nnzdata-time
Repo
Framework