October 18, 2019

3225 words 16 mins read

Paper Group ANR 605

Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy. A Layer Decomposition-Recomposition Framework for Neuron Pruning towards Accurate Lightweight Networks. Trust Region Policy Optimization for POMDPs. Multi-View Inpainting for RGB-D Sequence. Nth Absolute Root Mean Error. On Optimizing Deep Convolutional Neural N …

Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy


Title	Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy
Authors	Firat Ozdemir, Zixuan Peng, Christine Tanner, Philipp Fuernstahl, Orcun Goksel
Abstract	Segmentation is essential for medical image analysis tasks such as intervention planning, therapy guidance, diagnosis, treatment decisions. Deep learning is becoming increasingly prominent for segmentation, where the lack of annotations, however, often becomes the main limitation. Due to privacy concerns and ethical considerations, most medical datasets are created, curated, and allow access only locally. Furthermore, current deep learning methods are often suboptimal in translating anatomical knowledge between different medical imaging modalities. Active learning can be used to select an informed set of image samples to request for manual annotation, in order to best utilize the limited annotation time of clinical experts for optimal outcomes, which we focus on in this work. Our contributions herein are two fold: (1) we enforce domain-representativeness of selected samples using a proposed penalization scheme to maximize information at the network abstraction layer, and (2) we propose a Borda-count based sample querying scheme for selecting samples for segmentation. Comparative experiments with baseline approaches show that the samples queried with our proposed method, where both above contributions are combined, result in significantly improved segmentation performance for this active learning task.
Tasks	Active Learning
Published	2018-07-18
URL	http://arxiv.org/abs/1807.06962v1
PDF	http://arxiv.org/pdf/1807.06962v1.pdf
PWC	https://paperswithcode.com/paper/active-learning-for-segmentation-by
Repo
Framework

A Layer Decomposition-Recomposition Framework for Neuron Pruning towards Accurate Lightweight Networks


Title	A Layer Decomposition-Recomposition Framework for Neuron Pruning towards Accurate Lightweight Networks
Authors	Weijie Chen, Yuan Zhang, Di Xie, Shiliang Pu
Abstract	Neuron pruning is an efficient method to compress the network into a slimmer one for reducing the computational cost and storage overhead. Most of state-of-the-art results are obtained in a layer-by-layer optimization mode. It discards the unimportant input neurons and uses the survived ones to reconstruct the output neurons approaching to the original ones in a layer-by-layer manner. However, an unnoticed problem arises that the information loss is accumulated as layer increases since the survived neurons still do not encode the entire information as before. A better alternative is to propagate the entire useful information to reconstruct the pruned layer instead of directly discarding the less important neurons. To this end, we propose a novel Layer Decomposition-Recomposition Framework (LDRF) for neuron pruning, by which each layer’s output information is recovered in an embedding space and then propagated to reconstruct the following pruned layers with useful information preserved. We mainly conduct our experiments on ILSVRC-12 benchmark with VGG-16 and ResNet-50. What should be emphasized is that our results before end-to-end fine-tuning are significantly superior owing to the information-preserving property of our proposed framework.With end-to-end fine-tuning, we achieve state-of-the-art results of 5.13x and 3x speed-up with only 0.5% and 0.65% top-5 accuracy drop respectively, which outperform the existing neuron pruning methods.
Tasks
Published	2018-12-17
URL	http://arxiv.org/abs/1812.06611v1
PDF	http://arxiv.org/pdf/1812.06611v1.pdf
PWC	https://paperswithcode.com/paper/a-layer-decomposition-recomposition-framework
Repo
Framework

Trust Region Policy Optimization for POMDPs


Title	Trust Region Policy Optimization for POMDPs
Authors	Kamyar Azizzadenesheli, Manish Kumar Bera, Animashree Anandkumar
Abstract	We propose Generalized Trust Region Policy Optimization (GTRPO), a policy gradient Reinforcement Learning (RL) algorithm for both Markov decision processes (MDP) and Partially Observable Markov Decision Processes (POMDP). Policy gradient is a class of model-free RL methods. Previous policy gradient methods are guaranteed to converge only when the underlying model is an MDP and the policy is run for an infinite horizon. We relax these assumptions to episodic settings and to partially observable models with memory-less policies. For the latter class, GTRPO uses a variant of the Q-function with only three consecutive observations for each policy updates, and hence, is computationally efficient. We theoretically show that the policy updates in GTRPO monotonically improve the expected cumulative return and hence, GTRPO has convergence guarantees.
Tasks	Decision Making, Policy Gradient Methods
Published	2018-10-18
URL	http://arxiv.org/abs/1810.07900v2
PDF	http://arxiv.org/pdf/1810.07900v2.pdf
PWC	https://paperswithcode.com/paper/trust-region-policy-optimization-for-pomdps
Repo
Framework

Multi-View Inpainting for RGB-D Sequence


Title	Multi-View Inpainting for RGB-D Sequence
Authors	Feiran Li, Gustavo Alfonso Garcia Ricardez, Jun Takamatsu, Tsukasa Ogasawara
Abstract	In this work we propose a novel approach to remove undesired objects from RGB-D sequences captured with freely moving cameras, which enables static 3D reconstruction. Our method jointly uses existing information from multiple frames as well as generates new one via inpainting techniques. We use balanced rules to select source frames; local homography based image warping method for alignment and Markov random field (MRF) based approach for combining existing information. For the left holes, we employ exemplar based multi-view inpainting method to deal with the color image and coherently use it as guidance to complete the depth correspondence. Experiments show that our approach is qualified for removing the undesired objects and inpainting the holes.
Tasks	3D Reconstruction
Published	2018-11-22
URL	http://arxiv.org/abs/1811.09012v1
PDF	http://arxiv.org/pdf/1811.09012v1.pdf
PWC	https://paperswithcode.com/paper/multi-view-inpainting-for-rgb-d-sequence
Repo
Framework

Nth Absolute Root Mean Error


Title	Nth Absolute Root Mean Error
Authors	Siddhartha Dhar Choudhury, Shashank Pandey
Abstract	Neural network training process takes long time when the size of training data is huge, without the large set of training values the neural network is unable to learn features. This dilemma between time and size of data is often solved using fast GPUs, but we present a better solution for a subset of those problems. To reduce the time for training a regression model using neural network we introduce a loss function called Nth Absolute Root Mean Error (NARME). It helps to train regression models much faster compared to other existing loss functions. Experiments show that in most use cases NARME reduces the required number of epochs to almost one-tenth of that required by other commonly used loss functions, and also achieves great accuracy in the small amount of time in which it was trained.
Tasks
Published	2018-09-30
URL	http://arxiv.org/abs/1810.00421v2
PDF	http://arxiv.org/pdf/1810.00421v2.pdf
PWC	https://paperswithcode.com/paper/nth-absolute-root-mean-error
Repo
Framework

On Optimizing Deep Convolutional Neural Networks by Evolutionary Computing


Title	On Optimizing Deep Convolutional Neural Networks by Evolutionary Computing
Authors	M. U. B. Dias, D. D. N. De Silva, S. Fernando
Abstract	Optimization for deep networks is currently a very active area of research. As neural networks become deeper, the ability in manually optimizing the network becomes harder. Mini-batch normalization, identification of effective respective fields, momentum updates, introduction of residual blocks, learning rate adoption, etc. have been proposed to speed up the rate of convergent in manual training process while keeping the higher accuracy level. However, the problem of finding optimal topological structure for a given problem is becoming a challenging task need to be addressed immediately. Few researchers have attempted to optimize the network structure using evolutionary computing approaches. Among them, few have successfully evolved networks with reinforcement learning and long-short-term memory. A very few has applied evolutionary programming into deep convolution neural networks. These attempts are mainly evolved the network structure and then subsequently optimized the hyper-parameters of the network. However, a mechanism to evolve the deep network structure under the techniques currently being practiced in manual process is still absent. Incorporation of such techniques into chromosomes level of evolutionary computing, certainly can take us to better topological deep structures. The paper concludes by identifying the gap between evolutionary based deep neural networks and deep neural networks. Further, it proposes some insights for optimizing deep neural networks using evolutionary computing techniques.
Tasks
Published	2018-08-06
URL	http://arxiv.org/abs/1808.01766v1
PDF	http://arxiv.org/pdf/1808.01766v1.pdf
PWC	https://paperswithcode.com/paper/on-optimizing-deep-convolutional-neural
Repo
Framework

On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs


Title	On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs
Authors	Sandipan Banerjee, Walter J. Scheirer, Kevin W. Bowyer, Patrick J. Flynn
Abstract	We propose a multi-scale GAN model to hallucinate realistic context (forehead, hair, neck, clothes) and background pixels automatically from a single input face mask. Instead of swapping a face on to an existing picture, our model directly generates realistic context and background pixels based on the features of the provided face mask. Unlike face inpainting algorithms, it can generate realistic hallucinations even for a large number of missing pixels. Our model is composed of a cascaded network of GAN blocks, each tasked with hallucination of missing pixels at a particular resolution while guiding the synthesis process of the next GAN block. The hallucinated full face image is made photo-realistic by using a combination of reconstruction, perceptual, adversarial and identity preserving losses at each block of the network. With a set of extensive experiments, we demonstrate the effectiveness of our model in hallucinating context and background pixels from face masks varying in facial pose, expression and lighting, collected from multiple datasets subject disjoint with our training data. We also compare our method with two popular face swapping and face completion methods in terms of visual quality and recognition performance. Additionally, we analyze our cascaded pipeline and compare it with the recently proposed progressive growing of GANs.
Tasks	Face Swapping, Facial Inpainting
Published	2018-11-17
URL	https://arxiv.org/abs/1811.07104v3
PDF	https://arxiv.org/pdf/1811.07104v3.pdf
PWC	https://paperswithcode.com/paper/on-hallucinating-context-and-background
Repo
Framework

Role Semantics for Better Models of Implicit Discourse Relations


Title	Role Semantics for Better Models of Implicit Discourse Relations
Authors	Michael Roth
Abstract	Predicting the structure of a discourse is challenging because relations between discourse segments are often implicit and thus hard to distinguish computationally. I extend previous work to classify implicit discourse relations by introducing a novel set of features on the level of semantic roles. My results demonstrate that such features are helpful, yielding results competitive with other feature-rich approaches on the PDTB. My main contribution is an analysis of improvements that can be traced back to role-based features, providing insights into why and when role semantics is helpful.
Tasks
Published	2018-08-24
URL	http://arxiv.org/abs/1808.08047v1
PDF	http://arxiv.org/pdf/1808.08047v1.pdf
PWC	https://paperswithcode.com/paper/role-semantics-for-better-models-of-implicit
Repo
Framework

Deploying Customized Data Representation and Approximate Computing in Machine Learning Applications


Title	Deploying Customized Data Representation and Approximate Computing in Machine Learning Applications
Authors	Mahdi Nazemi, Massoud Pedram
Abstract	Major advancements in building general-purpose and customized hardware have been one of the key enablers of versatility and pervasiveness of machine learning models such as deep neural networks. To sustain this ubiquitous deployment of machine learning models and cope with their computational and storage complexity, several solutions such as low-precision representation of model parameters using fixed-point representation and deploying approximate arithmetic operations have been employed. Studying the potency of such solutions in different applications requires integrating them into existing machine learning frameworks for high-level simulations as well as implementing them in hardware to analyze their effects on power/energy dissipation, throughput, and chip area. Lop is a library for design space exploration that bridges the gap between machine learning and efficient hardware realization. It comprises a Python module, which can be integrated with some of the existing machine learning frameworks and implements various customizable data representations including fixed-point and floating-point as well as approximate arithmetic operations.Furthermore, it includes a highly-parameterized Scala module, which allows synthesizing hardware based on the said data representations and arithmetic operations. Lop allows researchers and designers to quickly compare quality of their models using various data representations and arithmetic operations in Python and contrast the hardware cost of viable representations by synthesizing them on their target platforms (e.g., FPGA or ASIC). To the best of our knowledge, Lop is the first library that allows both software simulation and hardware realization using customized data representations and approximate computing techniques.
Tasks
Published	2018-06-03
URL	http://arxiv.org/abs/1806.00875v1
PDF	http://arxiv.org/pdf/1806.00875v1.pdf
PWC	https://paperswithcode.com/paper/deploying-customized-data-representation-and
Repo
Framework

Artificial Quantum Neural Network: quantum neurons, logical elements and tests of convolutional nets


Title	Artificial Quantum Neural Network: quantum neurons, logical elements and tests of convolutional nets
Authors	V. I. Dorozhinsky, O. V. Pavlovsky
Abstract	We consider a model of an artificial neural network that uses quantum-mechanical particles in a two-humped potential as a neuron. To simulate such a quantum-mechanical system the Monte-Carlo integration method is used. A form of the self-potential of a particle and two potentials (exciting and inhibiting) interaction are proposed. The possibility of implementing the simplest logical elements, (such as AND, OR and NOT) based on introduced quantum particles is shown. Further we show implementation of a simplest convolutional network. Finally we construct a network that recognizes handwritten symbols, which shows that in the case of simple architectures, it is possible to transfer weights from a classical network to a quantum one.
Tasks
Published	2018-06-25
URL	http://arxiv.org/abs/1806.09664v1
PDF	http://arxiv.org/pdf/1806.09664v1.pdf
PWC	https://paperswithcode.com/paper/artificial-quantum-neural-network-quantum
Repo
Framework

Hybrid deep neural networks for all-cause Mortality Prediction from LDCT Images


Title	Hybrid deep neural networks for all-cause Mortality Prediction from LDCT Images
Authors	Pingkun Yan, Hengtao Guo, Ge Wang, Ruben De Man, Mannudeep K. Kalra
Abstract	Known for its high morbidity and mortality rates, lung cancer poses a significant threat to human health and well-being. However, the same population is also at high risk for other deadly diseases, such as cardiovascular disease. Since Low-Dose CT (LDCT) has been shown to significantly improve the lung cancer diagnosis accuracy, it will be very useful for clinical practice to predict the all-cause mortality for lung cancer patients to take corresponding actions. In this paper, we propose a deep learning based method, which takes both chest LDCT image patches and coronary artery calcification risk scores as input, for direct prediction of mortality risk of lung cancer subjects. The proposed method is called Hybrid Risk Network (HyRiskNet) for mortality risk prediction, which is an end-to-end framework utilizing hybrid imaging features, instead of completely relying on automatic feature extraction. Our work demonstrates the feasibility of using deep learning techniques for all-cause lung cancer mortality prediction from chest LDCT images. The experimental results show that the proposed HyRiskNet can achieve superior performance compared with the neural networks with only image input and with other traditional semi-automatic scoring methods. The study also indicates that radiologist defined features can well complement convolutional neural networks for more comprehensive feature extraction.
Tasks	Lung Cancer Diagnosis, Mortality Prediction
Published	2018-10-19
URL	http://arxiv.org/abs/1810.08503v1
PDF	http://arxiv.org/pdf/1810.08503v1.pdf
PWC	https://paperswithcode.com/paper/hybrid-deep-neural-networks-for-all-cause
Repo
Framework

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification


Title	Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification
Authors	Yinhao Zhu, Nicholas Zabaras
Abstract	We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein’s method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification benchmark problems including flow in heterogeneous media defined in terms of limited data-driven permeability realizations. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to $4,225$ where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.
Tasks	Bayesian Inference, Gaussian Processes
Published	2018-01-21
URL	http://arxiv.org/abs/1801.06879v1
PDF	http://arxiv.org/pdf/1801.06879v1.pdf
PWC	https://paperswithcode.com/paper/bayesian-deep-convolutional-encoder-decoder
Repo
Framework

Scene Categorization from Contours: Medial Axis Based Salience Measures


Title	Scene Categorization from Contours: Medial Axis Based Salience Measures
Authors	Morteza Rezanejad, Gabriel Downs, John Wilder, Dirk B. Walther, Allan Jepson, Sven Dickinson, Kaleem Siddiqi
Abstract	The computer vision community has witnessed recent advances in scene categorization from images, with the state-of-the art systems now achieving impressive recognition rates on challenging benchmarks such as the Places365 dataset. Such systems have been trained on photographs which include color, texture and shading cues. The geometry of shapes and surfaces, as conveyed by scene contours, is not explicitly considered for this task. Remarkably, humans can accurately recognize natural scenes from line drawings, which consist solely of contour-based shape cues. Here we report the first computer vision study on scene categorization of line drawings derived from popular databases including an artist scene database, MIT67, and Places365. Specifically, we use off-the-shelf pre-trained CNNs to perform scene classification given only contour information as input and find performance levels well above chance. We also show that medial-axis based contour salience methods can be used to select more informative subsets of contour pixels and that the variation in CNN classification performance on various choices for these subsets is qualitatively similar to that observed in human performance. Moreover, when the salience measures are used to weight the contours, as opposed to pruning them, we find that these weights boost our CNN performance above that for unweighted contour input. That is, the medial axis based salience weights appear to add useful information that is not available when CNNs are trained to use contours alone.
Tasks	Scene Classification
Published	2018-11-26
URL	http://arxiv.org/abs/1811.10524v1
PDF	http://arxiv.org/pdf/1811.10524v1.pdf
PWC	https://paperswithcode.com/paper/scene-categorization-from-contours-medial
Repo
Framework

Size-Noise Tradeoffs in Generative Networks


Title	Size-Noise Tradeoffs in Generative Networks
Authors	Bolton Bailey, Matus Telgarsky
Abstract	This paper investigates the ability of generative networks to convert their input noise distributions into other distributions. Firstly, we demonstrate a construction that allows ReLU networks to increase the dimensionality of their noise distribution by implementing a “space-filling” function based on iterated tent maps. We show this construction is optimal by analyzing the number of affine pieces in functions computed by multivariate ReLU networks. Secondly, we provide efficient ways (using polylog $(1/\epsilon)$ nodes) for networks to pass between univariate uniform and normal distributions, using a Taylor series approximation and a binary search gadget for computing function inverses. Lastly, we indicate how high dimensional distributions can be efficiently transformed into low dimensional distributions.
Tasks
Published	2018-10-26
URL	http://arxiv.org/abs/1810.11158v1
PDF	http://arxiv.org/pdf/1810.11158v1.pdf
PWC	https://paperswithcode.com/paper/size-noise-tradeoffs-in-generative-networks
Repo
Framework

Provably Correct Automatic Subdifferentiation for Qualified Programs


Title	Provably Correct Automatic Subdifferentiation for Qualified Programs
Authors	Sham Kakade, Jason D. Lee
Abstract	The Cheap Gradient Principle (Griewank 2008) — the computational cost of computing the gradient of a scalar-valued function is nearly the same (often within a factor of $5$) as that of simply computing the function itself — is of central importance in optimization; it allows us to quickly obtain (high dimensional) gradients of scalar loss functions which are subsequently used in black box gradient-based optimization procedures. The current state of affairs is markedly different with regards to computing subderivatives: widely used ML libraries, including TensorFlow and PyTorch, do not correctly compute (generalized) subderivatives even on simple examples. This work considers the question: is there a Cheap Subgradient Principle? Our main result shows that, under certain restrictions on our library of nonsmooth functions (standard in nonlinear programming), provably correct generalized subderivatives can be computed at a computational cost that is within a (dimension-free) factor of $6$ of the cost of computing the scalar function itself.
Tasks
Published	2018-09-23
URL	http://arxiv.org/abs/1809.08530v2
PDF	http://arxiv.org/pdf/1809.08530v2.pdf
PWC	https://paperswithcode.com/paper/provably-correct-automatic-subdifferentiation
Repo
Framework