October 19, 2019

3414 words 17 mins read

Paper Group ANR 281

Calculating the Midsagittal Plane for Symmetrical Bilateral Shapes: Applications to Clinical Facial Surgical Planning. Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics. Semidefinite relaxations for certifying robustness to adversarial examples. DRUNET: A Dilated-Residual U-Net Deep Learning Net …

Calculating the Midsagittal Plane for Symmetrical Bilateral Shapes: Applications to Clinical Facial Surgical Planning


Title	Calculating the Midsagittal Plane for Symmetrical Bilateral Shapes: Applications to Clinical Facial Surgical Planning
Authors	Aarti Jajoo, Matthew Nicol, Jaime Gateno, Ken-Chung Chen, Zhen Tang, Tasadduk Chowdhury, Jainfu Li, Steve Goufang Shen, James J. Xia
Abstract	It is difficult to estimate the midsagittal plane of human subjects with craniomaxillofacial (CMF) deformities. We have developed a LAndmark GEometric Routine (LAGER), which automatically estimates a midsagittal plane for such subjects. The LAGER algorithm was based on the assumption that the optimal midsagittal plane of a patient with a deformity is the premorbid midsagittal plane of the patient (i.e. hypothetically normal without deformity). The LAGER algorithm consists of three steps. The first step quantifies the asymmetry of the landmarks using a Euclidean distance matrix analysis and ranks the landmarks according to their degree of asymmetry. The second step uses a recursive algorithm to drop outlier landmarks. The third step inputs the remaining landmarks into an optimization algorithm to determine an optimal midsaggital plane. We validate LAGER on 20 synthetic models mimicking the skulls of real patients with CMF deformities. The results indicated that all the LAGER algorithm-generated midsagittal planes met clinical criteria. Thus it can be used clinically to determine the midsagittal plane for patients with CMF deformities.
Tasks
Published	2018-03-11
URL	http://arxiv.org/abs/1803.05853v1
PDF	http://arxiv.org/pdf/1803.05853v1.pdf
PWC	https://paperswithcode.com/paper/calculating-the-midsagittal-plane-for
Repo
Framework

Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics


Title	Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics
Authors	Jeroen van Baar, Alan Sullivan, Radu Cordorel, Devesh Jha, Diego Romeres, Daniel Nikovski
Abstract	Learning robot tasks or controllers using deep reinforcement learning has been proven effective in simulations. Learning in simulation has several advantages. For example, one can fully control the simulated environment, including halting motions while performing computations. Another advantage when robots are involved, is that the amount of time a robot is occupied learning a task—rather than being productive—can be reduced by transferring the learned task to the real robot. Transfer learning requires some amount of fine-tuning on the real robot. For tasks which involve complex (non-linear) dynamics, the fine-tuning itself may take a substantial amount of time. In order to reduce the amount of fine-tuning we propose to learn robustified controllers in simulation. Robustified controllers are learned by exploiting the ability to change simulation parameters (both appearance and dynamics) for successive training episodes. An additional benefit for this approach is that it alleviates the precise determination of physics parameters for the simulator, which is a non-trivial task. We demonstrate our proposed approach on a real setup in which a robot aims to solve a maze game, which involves complex dynamics due to static friction and potentially large accelerations. We show that the amount of fine-tuning in transfer learning for a robustified controller is substantially reduced compared to a non-robustified controller.
Tasks	Transfer Learning
Published	2018-09-13
URL	http://arxiv.org/abs/1809.04720v2
PDF	http://arxiv.org/pdf/1809.04720v2.pdf
PWC	https://paperswithcode.com/paper/sim-to-real-transfer-learning-using
Repo
Framework

Semidefinite relaxations for certifying robustness to adversarial examples


Title	Semidefinite relaxations for certifying robustness to adversarial examples
Authors	Aditi Raghunathan, Jacob Steinhardt, Percy Liang
Abstract	Despite their impressive performance on diverse tasks, neural networks fail catastrophically in the presence of adversarial inputs—imperceptibly but adversarially perturbed versions of natural inputs. We have witnessed an arms race between defenders who attempt to train robust networks and attackers who try to construct adversarial examples. One promise of ending the arms race is developing certified defenses, ones which are provably robust against all attackers in some family. These certified defenses are based on convex relaxations which construct an upper bound on the worst case loss over all attackers in the family. Previous relaxations are loose on networks that are not trained against the respective relaxation. In this paper, we propose a new semidefinite relaxation for certifying robustness that applies to arbitrary ReLU networks. We show that our proposed relaxation is tighter than previous relaxations and produces meaningful robustness guarantees on three different “foreign networks” whose training objectives are agnostic to our proposed relaxation.
Tasks
Published	2018-11-02
URL	http://arxiv.org/abs/1811.01057v1
PDF	http://arxiv.org/pdf/1811.01057v1.pdf
PWC	https://paperswithcode.com/paper/semidefinite-relaxations-for-certifying
Repo
Framework

DRUNET: A Dilated-Residual U-Net Deep Learning Network to Digitally Stain Optic Nerve Head Tissues in Optical Coherence Tomography Images


Title	DRUNET: A Dilated-Residual U-Net Deep Learning Network to Digitally Stain Optic Nerve Head Tissues in Optical Coherence Tomography Images
Authors	Sripad Krishna Devalla, Prajwal K. Renukanand, Bharathwaj K. Sreedhar, Shamira Perera, Jean-Martial Mari, Khai Sing Chin, Tin A. Tun, Nicholas G. Strouthidis, Tin Aung, Alexandre H. Thiery, Michael J. A. Girard
Abstract	Given that the neural and connective tissues of the optic nerve head (ONH) exhibit complex morphological changes with the development and progression of glaucoma, their simultaneous isolation from optical coherence tomography (OCT) images may be of great interest for the clinical diagnosis and management of this pathology. A deep learning algorithm was designed and trained to digitally stain (i.e. highlight) 6 ONH tissue layers by capturing both the local (tissue texture) and contextual information (spatial arrangement of tissues). The overall dice coefficient (mean of all tissues) was $0.91 \pm 0.05$ when assessed against manual segmentations performed by an expert observer. We offer here a robust segmentation framework that could be extended for the automated parametric study of the ONH tissues.
Tasks
Published	2018-03-01
URL	http://arxiv.org/abs/1803.00232v1
PDF	http://arxiv.org/pdf/1803.00232v1.pdf
PWC	https://paperswithcode.com/paper/drunet-a-dilated-residual-u-net-deep-learning
Repo
Framework

A Convergence Theory for Deep Learning via Over-Parameterization


Title	A Convergence Theory for Deep Learning via Over-Parameterization
Authors	Zeyuan Allen-Zhu, Yuanzhi Li, Zhao Song
Abstract	Deep neural networks (DNNs) have demonstrated dominating performance in many fields; since AlexNet, networks used in practice are going wider and deeper. On the theoretical side, a long line of works has been focusing on training neural networks with one hidden layer. The theory of multi-layer networks remains largely unsettled. In this work, we prove why stochastic gradient descent (SGD) can find $\textit{global minima}$ on the training objective of DNNs in $\textit{polynomial time}$. We only make two assumptions: the inputs are non-degenerate and the network is over-parameterized. The latter means the network width is sufficiently large: $\textit{polynomial}$ in $L$, the number of layers and in $n$, the number of samples. Our key technique is to derive that, in a sufficiently large neighborhood of the random initialization, the optimization landscape is almost-convex and semi-smooth even with ReLU activations. This implies an equivalence between over-parameterized neural networks and neural tangent kernel (NTK) in the finite (and polynomial) width setting. As concrete examples, starting from randomly initialized weights, we prove that SGD can attain 100% training accuracy in classification tasks, or minimize regression loss in linear convergence speed, with running time polynomial in $n,L$. Our theory applies to the widely-used but non-smooth ReLU activation, and to any smooth and possibly non-convex loss functions. In terms of network architectures, our theory at least applies to fully-connected neural networks, convolutional neural networks (CNN), and residual neural networks (ResNet).
Tasks
Published	2018-11-09
URL	https://arxiv.org/abs/1811.03962v5
PDF	https://arxiv.org/pdf/1811.03962v5.pdf
PWC	https://paperswithcode.com/paper/a-convergence-theory-for-deep-learning-via
Repo
Framework

Rectification from Radially-Distorted Scales


Title	Rectification from Radially-Distorted Scales
Authors	James Pritts, Zuzana Kukelova, Viktor Larsson, Ondrej Chum
Abstract	This paper introduces the first minimal solvers that jointly estimate lens distortion and affine rectification from repetitions of rigidly transformed coplanar local features. The proposed solvers incorporate lens distortion into the camera model and extend accurate rectification to wide-angle images that contain nearly any type of coplanar repeated content. We demonstrate a principled approach to generating stable minimal solvers by the Grobner basis method, which is accomplished by sampling feasible monomial bases to maximize numerical stability. Synthetic and real-image experiments confirm that the solvers give accurate rectifications from noisy measurements when used in a RANSAC-based estimator. The proposed solvers demonstrate superior robustness to noise compared to the state-of-the-art. The solvers work on scenes without straight lines and, in general, relax the strong assumptions on scene content made by the state-of-the-art. Accurate rectifications on imagery that was taken with narrow focal length to near fish-eye lenses demonstrate the wide applicability of the proposed method. The method is fully automated, and the code is publicly available at https://github.com/prittjam/repeats.
Tasks
Published	2018-07-16
URL	http://arxiv.org/abs/1807.06110v4
PDF	http://arxiv.org/pdf/1807.06110v4.pdf
PWC	https://paperswithcode.com/paper/rectification-from-radially-distorted-scales
Repo
Framework

Three-Dimensional Radiotherapy Dose Prediction on Head and Neck Cancer Patients with a Hierarchically Densely Connected U-net Deep Learning Architecture


Title	Three-Dimensional Radiotherapy Dose Prediction on Head and Neck Cancer Patients with a Hierarchically Densely Connected U-net Deep Learning Architecture
Authors	Dan Nguyen, Xun Jia, David Sher, Mu-Han Lin, Zohaib Iqbal, Hui Liu, Steve Jiang
Abstract	The treatment planning process for patients with head and neck (H&N) cancer is regarded as one of the most complicated due to large target volume, multiple prescription dose levels, and many radiation-sensitive critical structures near the target. Treatment planning for this site requires a high level of human expertise and a tremendous amount of effort to produce personalized high quality plans, taking as long as a week, which deteriorates the chances of tumor control and patient survival. To solve this problem, we propose to investigate a deep learning-based dose prediction model, Hierarchically Densely Connected U-net, based on two highly popular network architectures: U-net and DenseNet. We find that this new architecture is able to accurately and efficiently predict the dose distribution, outperforming the other two models, the Standard U-net and DenseNet, in homogeneity, dose conformity, and dose coverage on the test data. Averaging across all organs at risk, our proposed model is capable of predicting the organ-at-risk max dose within 6.3% and mean dose within 5.1% of the prescription dose on the test data. The other models, the Standard U-net and DenseNet, performed worse, having an averaged organ-at-risk max dose prediction error of 8.2% and 9.3%, respectively, and averaged mean dose prediction error of 6.4% and 6.8%, respectively. In addition, our proposed model used 12 times less trainable parameters than the Standard U-net, and predicted the patient dose 4 times faster than DenseNet.
Tasks
Published	2018-05-25
URL	http://arxiv.org/abs/1805.10397v3
PDF	http://arxiv.org/pdf/1805.10397v3.pdf
PWC	https://paperswithcode.com/paper/three-dimensional-radiotherapy-dose
Repo
Framework

Exploration versus exploitation in reinforcement learning: a stochastic control approach


Title	Exploration versus exploitation in reinforcement learning: a stochastic control approach
Authors	Haoran Wang, Thaleia Zariphopoulou, Xunyu Zhou
Abstract	We consider reinforcement learning (RL) in continuous time and study the problem of achieving the best trade-off between exploration of a black box environment and exploitation of current knowledge. We propose an entropy-regularized reward function involving the differential entropy of the distributions of actions, and motivate and devise an exploratory formulation for the feature dynamics that captures repetitive learning under exploration. The resulting optimization problem is a revitalization of the classical relaxed stochastic control. We carry out a complete analysis of the problem in the linear–quadratic (LQ) setting and deduce that the optimal feedback control distribution for balancing exploitation and exploration is Gaussian. This in turn interprets and justifies the widely adopted Gaussian exploration in RL, beyond its simplicity for sampling. Moreover, the exploitation and exploration are captured, respectively and mutual-exclusively, by the mean and variance of the Gaussian distribution. We also find that a more random environment contains more learning opportunities in the sense that less exploration is needed. We characterize the cost of exploration, which, for the LQ case, is shown to be proportional to the entropy regularization weight and inversely proportional to the discount rate. Finally, as the weight of exploration decays to zero, we prove the convergence of the solution of the entropy-regularized LQ problem to the one of the classical LQ problem.
Tasks
Published	2018-12-04
URL	http://arxiv.org/abs/1812.01552v3
PDF	http://arxiv.org/pdf/1812.01552v3.pdf
PWC	https://paperswithcode.com/paper/exploration-versus-exploitation-in
Repo
Framework

Modelling contextuality by probabilistic programs with hypergraph semantics


Title	Modelling contextuality by probabilistic programs with hypergraph semantics
Authors	Peter D. Bruza
Abstract	Models of a phenomenon are often developed by examining it under different experimental conditions, or measurement contexts. The resultant probabilistic models assume that the underlying random variables, which define a measurable set of outcomes, can be defined independent of the measurement context. The phenomenon is deemed contextual when this assumption fails. Contextuality is an important issue in quantum physics. However, there has been growing speculation that it manifests outside the quantum realm with human cognition being a particularly prominent area of investigation. This article contributes the foundations of a probabilistic programming language that allows convenient exploration of contextuality in wide range of applications relevant to cognitive science and artificial intelligence. Specific syntax is proposed to allow the specification of “measurement contexts”. Each such context delivers a partial model of the phenomenon based on the associated experimental condition described by the measurement context. The probabilistic program is translated into a hypergraph in a modular way. Recent theoretical results from the field of quantum physics show that contextuality can be equated with the possibility of constructing a probabilistic model on the resulting hypergraph. The use of hypergraphs opens the door for a theoretically succinct and efficient computational semantics sensitive to modelling both contextual and non-contextual phenomena. Finally, this article raises awareness of contextuality beyond quantum physics and to contribute formal methods to detect its presence by means of hypergraph semantics.
Tasks	Probabilistic Programming
Published	2018-01-31
URL	http://arxiv.org/abs/1802.00690v1
PDF	http://arxiv.org/pdf/1802.00690v1.pdf
PWC	https://paperswithcode.com/paper/modelling-contextuality-by-probabilistic
Repo
Framework

Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving


Title	Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
Authors	Jiaolong Xu, Peng Wang, Heng Yang, Antonio M. López
Abstract	Autonomous driving has harsh requirements of small model size and energy efficiency, in order to enable the embedded system to achieve real-time on-board object detection. Recent deep convolutional neural network based object detectors have achieved state-of-the-art accuracy. However, such models are trained with numerous parameters and their high computational costs and large storage prohibit the deployment to memory and computation resource limited systems. Low-precision neural networks are popular techniques for reducing the computation requirements and memory footprint. Among them, binary weight neural network (BWN) is the extreme case which quantizes the float-point into just $1$ bit. BWNs are difficult to train and suffer from accuracy deprecation due to the extreme low-bit representation. To address this problem, we propose a knowledge transfer (KT) method to aid the training of BWN using a full-precision teacher network. We built DarkNet- and MobileNet-based binary weight YOLO-v2 detectors and conduct experiments on KITTI benchmark for car, pedestrian and cyclist detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the model size of DarkNet-YOLO from 257 MB to 8.8 MB and MobileNet-YOLO from 193 MB to 7.9 MB.
Tasks	Autonomous Driving, Object Detection, Transfer Learning
Published	2018-04-17
URL	https://arxiv.org/abs/1804.06332v2
PDF	https://arxiv.org/pdf/1804.06332v2.pdf
PWC	https://paperswithcode.com/paper/training-a-binary-weight-object-detector-by
Repo
Framework

A Generative Modeling Approach to Limited Channel ECG Classification


Title	A Generative Modeling Approach to Limited Channel ECG Classification
Authors	Deepta Rajan, Jayaraman J. Thiagarajan
Abstract	Processing temporal sequences is central to a variety of applications in health care, and in particular multi-channel Electrocardiogram (ECG) is a highly prevalent diagnostic modality that relies on robust sequence modeling. While Recurrent Neural Networks (RNNs) have led to significant advances in automated diagnosis with time-series data, they perform poorly when models are trained using a limited set of channels. A crucial limitation of existing solutions is that they rely solely on discriminative models, which tend to generalize poorly in such scenarios. In order to combat this limitation, we develop a generative modeling approach to limited channel ECG classification. This approach first uses a Seq2Seq model to implicitly generate the missing channel information, and then uses the latent representation to perform the actual supervisory task. This decoupling enables the use of unsupervised data and also provides highly robust metric spaces for subsequent discriminative learning. Our experiments with the Physionet dataset clearly evidence the effectiveness of our approach over standard RNNs in disease prediction.
Tasks	Disease Prediction, ECG Classification, Time Series
Published	2018-02-18
URL	http://arxiv.org/abs/1802.06458v3
PDF	http://arxiv.org/pdf/1802.06458v3.pdf
PWC	https://paperswithcode.com/paper/a-generative-modeling-approach-to-limited
Repo
Framework

Simplified Active Calibration


Title	Simplified Active Calibration
Authors	Mehdi Faraji, Anup Basu
Abstract	We present a new mathematical formulation to estimate the intrinsic parameters of a camera in active or robotic platforms. We show that the focal lengths can be estimated using only one point correspondence that relates images taken before and after a degenerate rotation of the camera. The estimated focal lengths are then treated as known parameters to obtain a linear set of equations to calculate the principal point. Assuming that the principal point is close to the image center, the accuracy of the linear equations are increased by integrating the image center into the formulation. We extensively evaluate the formulations on a simulated camera, 3D scenes and real-world images. Our error analysis over simulated and real images indicates that the proposed Simplified Active Calibration method estimates the parameters of a camera with low error rates that can be used as an initial guess for further non-linear refinement procedures. Simplified Active Calibration can be employed in real-time environments for automatic calibrations given the proposed closed-form solutions.
Tasks	Calibration
Published	2018-06-29
URL	http://arxiv.org/abs/1806.11468v2
PDF	http://arxiv.org/pdf/1806.11468v2.pdf
PWC	https://paperswithcode.com/paper/simplified-active-calibration
Repo
Framework

Enhancing the Regularization Effect of Weight Pruning in Artificial Neural Networks


Title	Enhancing the Regularization Effect of Weight Pruning in Artificial Neural Networks
Authors	Brian Bartoldson, Adrian Barbu, Gordon Erlebacher
Abstract	Artificial neural networks (ANNs) may not be worth their computational/memory costs when used in mobile phones or embedded devices. Parameter-pruning algorithms combat these costs, with some algorithms capable of removing over 90% of an ANN’s weights without harming the ANN’s performance. Removing weights from an ANN is a form of regularization, but existing pruning algorithms do not significantly improve generalization error. We show that pruning ANNs can improve generalization if pruning targets large weights instead of small weights. Applying our pruning algorithm to an ANN leads to a higher image classification accuracy on CIFAR-10 data than applying the popular regularizer dropout. The pruning couples this higher accuracy with an 85% reduction of the ANN’s parameter count.
Tasks	Image Classification
Published	2018-05-04
URL	http://arxiv.org/abs/1805.01930v1
PDF	http://arxiv.org/pdf/1805.01930v1.pdf
PWC	https://paperswithcode.com/paper/enhancing-the-regularization-effect-of-weight
Repo
Framework

A Two-Dimensional (2-D) Learning Framework for Particle Swarm based Feature Selection


Title	A Two-Dimensional (2-D) Learning Framework for Particle Swarm based Feature Selection
Authors	Faizal Hafiz, Akshya Swain, Nitish Patel, Chirag Naik
Abstract	This paper proposes a new generalized two dimensional learning approach for particle swarm based feature selection. The core idea of the proposed approach is to include the information about the subset cardinality into the learning framework by extending the dimension of the velocity. The 2D-learning framework retains all the key features of the original PSO, despite the extra learning dimension. Most of the popular variants of PSO can easily be adapted into this 2D learning framework for feature selection problems. The efficacy of the proposed learning approach has been evaluated considering several benchmark data and two induction algorithms: Naive-Bayes and k-Nearest Neighbor. The results of the comparative investigation including the time-complexity analysis with GA, ACO and five other PSO variants illustrate that the proposed 2D learning approach gives feature subset with relatively smaller cardinality and better classification performance with shorter run times.
Tasks	Feature Selection
Published	2018-08-03
URL	http://arxiv.org/abs/1808.01150v1
PDF	http://arxiv.org/pdf/1808.01150v1.pdf
PWC	https://paperswithcode.com/paper/a-two-dimensional-2-d-learning-framework-for
Repo
Framework

SECaps: A Sequence Enhanced Capsule Model for Charge Prediction


Title	SECaps: A Sequence Enhanced Capsule Model for Charge Prediction
Authors	Congqing He, Li Peng, Yuquan Le, Jiawei He, Xiangyu Zhu
Abstract	Automatic charge prediction aims to predict appropriate final charges according to the fact descriptions for a given criminal case. Automatic charge prediction plays a critical role in assisting judges and lawyers to improve the efficiency of legal decisions, and thus has received much attention. Nevertheless, most existing works on automatic charge prediction perform adequately on high-frequency charges but are not yet capable of predicting few-shot charges with limited cases. In this paper, we propose a Sequence Enhanced Capsule model, dubbed as SECaps model, to relieve this problem. Specifically, following the work of capsule networks, we propose the seq-caps layer, which considers sequence information and spatial information of legal texts simultaneously. Then we design a attention residual unit, which provides auxiliary information for charge prediction. In addition, our SECaps model introduces focal loss, which relieves the problem of imbalanced charges. Comparing the state-of-the-art methods, our SECaps model obtains 4.5% and 6.4% absolutely considerable improvements under Macro F1 in Criminal-S and Criminal-L respectively. The experimental results consistently demonstrate the superiorities and competitiveness of our proposed model.
Tasks
Published	2018-10-10
URL	https://arxiv.org/abs/1810.04465v2
PDF	https://arxiv.org/pdf/1810.04465v2.pdf
PWC	https://paperswithcode.com/paper/secaps-a-sequence-enhanced-capsule-model-for
Repo
Framework