January 30, 2020

3098 words 15 mins read

Paper Group ANR 462

Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization. Filter Bank Regularization of Convolutional Neural Networks. Understanding Deep Neural Network Predictions for Medical Imaging Applications. Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin. Enriching Visual with Ve …

Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization


Title	Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization
Authors	Michael Weylandt, John Nagorski, Genevera I. Allen
Abstract	Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling visualizations. To address these impediments, we introduce Algorithmic Regularization, an innovative technique for obtaining high-quality estimates of regularization paths using an iterative one-step approximation scheme. We justify our approach with a novel theoretical result, guaranteeing global convergence of the approximate path to the exact solution under easily-checked non-data-dependent assumptions. The application of algorithmic regularization to convex clustering yields the Convex Clustering via Algorithmic Regularization Paths (CARP) algorithm for computing the clustering solution path. On example data sets from genomics and text analysis, CARP delivers over a 100-fold speed-up over existing methods, while attaining a finer approximation grid than standard methods. Furthermore, CARP enables improved visualization of clustering solutions: the fine solution grid returned by CARP can be used to construct a convex clustering-based dendrogram, as well as forming the basis of a dynamic path-wise visualization based on modern web technologies. Our methods are implemented in the open-source R package clustRviz, available at https://github.com/DataSlingers/clustRviz.
Tasks
Published	2019-01-06
URL	https://arxiv.org/abs/1901.01477v4
PDF	https://arxiv.org/pdf/1901.01477v4.pdf
PWC	https://paperswithcode.com/paper/dynamic-visualization-and-fast-computation
Repo
Framework

Filter Bank Regularization of Convolutional Neural Networks


Title	Filter Bank Regularization of Convolutional Neural Networks
Authors	Seyed Mehdi Ayyoubzadeh, Xiaolin Wu
Abstract	Regularization techniques are widely used to improve the generality, robustness, and efficiency of deep convolutional neural networks (DCNNs). In this paper, we propose a novel approach of regulating DCNN convolutional kernels by a structured filter bank. Comparing with the existing regularization methods, such as $\ell_1$ or $\ell_2$ minimization of DCNN kernel weights and the kernel orthogonality, which ignore sample correlations within a kernel, the use of filter bank in regularization of DCNNs can mold the DCNN kernels to common spatial structures and features (e.g., edges or textures of various orientations and frequencies) of natural images. On the other hand, unlike directly making DCNN kernels fixed filters, the filter bank regularization still allows the freedom of optimizing DCNN weights via deep learning. This new DCNN design strategy aims to combine the best of two worlds: the inclusion of structural image priors of traditional filter banks to improve the robustness and generality of DCNN solutions and the capability of modern deep learning to model complex non-linear functions hidden in training data. Experimental results on object recognition tasks show that the proposed regularization approach guides DCNNs to faster convergence and better generalization than existing regularization methods of weight decay and kernel orthogonality.
Tasks	Object Recognition
Published	2019-07-25
URL	https://arxiv.org/abs/1907.11110v3
PDF	https://arxiv.org/pdf/1907.11110v3.pdf
PWC	https://paperswithcode.com/paper/filter-bank-regularization-of-convolutional
Repo
Framework

Understanding Deep Neural Network Predictions for Medical Imaging Applications


Title	Understanding Deep Neural Network Predictions for Medical Imaging Applications
Authors	Barath Narayanan Narayanan, Manawaduge Supun De Silva, Russell C. Hardie, Nathan K. Kueterman, Redha Ali
Abstract	Computer-aided detection has been a research area attracting great interest in the past decade. Machine learning algorithms have been utilized extensively for this application as they provide a valuable second opinion to the doctors. Despite several machine learning models being available for medical imaging applications, not many have been implemented in the real-world due to the uninterpretable nature of the decisions made by the network. In this paper, we investigate the results provided by deep neural networks for the detection of malaria, diabetic retinopathy, brain tumor, and tuberculosis in different imaging modalities. We visualize the class activation mappings for all the applications in order to enhance the understanding of these networks. This type of visualization, along with the corresponding network performance metrics, would aid the data science experts in better understanding of their models as well as assisting doctors in their decision-making process.
Tasks	Decision Making
Published	2019-12-20
URL	https://arxiv.org/abs/1912.09621v1
PDF	https://arxiv.org/pdf/1912.09621v1.pdf
PWC	https://paperswithcode.com/paper/understanding-deep-neural-network-predictions
Repo
Framework

Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin


Title	Understanding Adversarial Robustness: The Trade-off between Minimum and Average Margin
Authors	Kaiwen Wu, Yaoliang Yu
Abstract	Deep models, while being extremely versatile and accurate, are vulnerable to adversarial attacks: slight perturbations that are imperceptible to humans can completely flip the prediction of deep models. Many attack and defense mechanisms have been proposed, although a satisfying solution still largely remains elusive. In this work, we give strong evidence that during training, deep models maximize the minimum margin in order to achieve high accuracy, but at the same time decrease the \emph{average} margin hence hurting robustness. Our empirical results highlight an intrinsic trade-off between accuracy and robustness for current deep model training. To further address this issue, we propose a new regularizer to explicitly promote average margin, and we verify through extensive experiments that it does lead to better robustness. Our regularized objective remains Fisher-consistent, hence asymptotically can still recover the Bayes optimal classifier.
Tasks
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11780v1
PDF	https://arxiv.org/pdf/1907.11780v1.pdf
PWC	https://paperswithcode.com/paper/understanding-adversarial-robustness-the
Repo
Framework

Enriching Visual with Verbal Explanations for Relational Concepts – Combining LIME with Aleph


Title	Enriching Visual with Verbal Explanations for Relational Concepts – Combining LIME with Aleph
Authors	Johannes Rabold, Hannah Deininger, Michael Siebers, Ute Schmid
Abstract	With the increasing number of deep learning applications, there is a growing demand for explanations. Visual explanations provide information about which parts of an image are relevant for a classifier’s decision. However, highlighting of image parts (e.g., an eye) cannot capture the relevance of a specific feature value for a class (e.g., that the eye is wide open). Furthermore, highlighting cannot convey whether the classification depends on the mere presence of parts or on a specific spatial relation between them. Consequently, we present an approach that is capable of explaining a classifier’s decision in terms of logic rules obtained by the Inductive Logic Programming system Aleph. The examples and the background knowledge needed for Aleph are based on the explanation generation method LIME. We demonstrate our approach with images of a blocksworld domain. First, we show that our approach is capable of identifying a single relation as important explanatory construct. Afterwards, we present the more complex relational concept of towers. Finally, we show how the generated relational rules can be explicitly related with the input image, resulting in richer explanations.
Tasks
Published	2019-10-04
URL	https://arxiv.org/abs/1910.01837v1
PDF	https://arxiv.org/pdf/1910.01837v1.pdf
PWC	https://paperswithcode.com/paper/enriching-visual-with-verbal-explanations-for
Repo
Framework

Enhancement Mask for Hippocampus Detection and Segmentation


Title	Enhancement Mask for Hippocampus Detection and Segmentation
Authors	Dengsheng Chen, Wenxi Liu, You Huang, Tong Tong, Yuanlong Yu
Abstract	Detection and segmentation of the hippocampal structures in volumetric brain images is a challenging problem in the area of medical imaging. In this paper, we propose a two-stage 3D fully convolutional neural network that efficiently detects and segments the hippocampal structures. In particular, our approach first localizes the hippocampus from the whole volumetric image while obtaining a proposal for a rough segmentation. After localization, we apply the proposal as an enhancement mask to extract the fine structure of the hippocampus. The proposed method has been evaluated on a public dataset and compares with state-of-the-art approaches. Results indicate the effectiveness of the proposed method, which yields mean Dice Similarity Coefficients (i.e. DSC) of $0.897$ and $0.900$ for the left and right hippocampus, respectively. Furthermore, extensive experiments manifest that the proposed enhancement mask layer has remarkable benefits for accelerating training process and obtaining more accurate segmentation results.
Tasks
Published	2019-02-12
URL	http://arxiv.org/abs/1902.04244v1
PDF	http://arxiv.org/pdf/1902.04244v1.pdf
PWC	https://paperswithcode.com/paper/enhancement-mask-for-hippocampus-detection
Repo
Framework

Real-Time Correlation Tracking via Joint Model Compression and Transfer


Title	Real-Time Correlation Tracking via Joint Model Compression and Transfer
Authors	Ning Wang, Wengang Zhou, Yibing Song, Chao Ma, Houqiang Li
Abstract	Correlation filters (CF) have received considerable attention in visual tracking because of their computational efficiency. Leveraging deep features via off-the-shelf CNN models (e.g., VGG), CF trackers achieve state-of-the-art performance while consuming a large number of computing resources. This limits deep CF trackers to be deployed to many mobile platforms on which only a single-core CPU is available. In this paper, we propose to jointly compress and transfer off-the-shelf CNN models within a knowledge distillation framework. We formulate a CNN model pretrained from the image classification task as a teacher network, and distill this teacher network into a lightweight student network as the feature extractor to speed up CF trackers. In the distillation process, we propose a fidelity loss to enable the student network to maintain the representation capability of the teacher network. Meanwhile, we design a tracking loss to adapt the objective of the student network from object recognition to visual tracking. The distillation process is performed offline on multiple layers and adaptively updates the student network using a background-aware online learning scheme. Extensive experiments on five challenging datasets demonstrate that the lightweight student network accelerates the speed of state-of-the-art deep CF trackers to real-time on a single-core CPU while maintaining almost the same tracking accuracy.
Tasks	Image Classification, Model Compression, Object Recognition, Visual Tracking
Published	2019-07-23
URL	https://arxiv.org/abs/1907.09831v1
PDF	https://arxiv.org/pdf/1907.09831v1.pdf
PWC	https://paperswithcode.com/paper/real-time-correlation-tracking-via-joint
Repo
Framework

Recurrent Connections Aid Occluded Object Recognition by Discounting Occluders


Title	Recurrent Connections Aid Occluded Object Recognition by Discounting Occluders
Authors	Markus Roland Ernst, Jochen Triesch, Thomas Burwick
Abstract	Recurrent connections in the visual cortex are thought to aid object recognition when part of the stimulus is occluded. Here we investigate if and how recurrent connections in artificial neural networks similarly aid object recognition. We systematically test and compare architectures comprised of bottom-up (B), lateral (L) and top-down (T) connections. Performance is evaluated on a novel stereoscopic occluded object recognition dataset. The task consists of recognizing one target digit occluded by multiple occluder digits in a pseudo-3D environment. We find that recurrent models perform significantly better than their feedforward counterparts, which were matched in parametric complexity. Furthermore, we analyze how the network’s representation of the stimuli evolves over time due to recurrent connections. We show that the recurrent connections tend to move the network’s representation of an occluded digit towards its un-occluded version. Our results suggest that both the brain and artificial neural networks can exploit recurrent connectivity to aid occluded object recognition.
Tasks	Object Recognition
Published	2019-07-20
URL	https://arxiv.org/abs/1907.08831v2
PDF	https://arxiv.org/pdf/1907.08831v2.pdf
PWC	https://paperswithcode.com/paper/recurrent-connections-aid-occluded-object
Repo
Framework

Learn-By-Calibrating: Using Calibration as a Training Objective


Title	Learn-By-Calibrating: Using Calibration as a Training Objective
Authors	Jayaraman J. Thiagarajan, Bindya Venkatesh, Deepta Rajan
Abstract	Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not explicitly measure the uncertainties. This is conceptually similar to heteroscedastic neural networks that produce variance estimates for each prediction, with the key difference that we do not place a Gaussian prior on the predictions. We propose a novel algorithm that performs simultaneous interval estimation for different calibration levels and effectively leverages the intervals to refine the mean estimates. Our results show that, our approach is consistently superior to existing regularization strategies in deep regression models. Finally, we propose to augment partial dependence plots, a model-agnostic interpretability tool, with expected prediction intervals to reveal interesting dependencies between data and the target.
Tasks	Calibration
Published	2019-10-30
URL	https://arxiv.org/abs/1910.14175v1
PDF	https://arxiv.org/pdf/1910.14175v1.pdf
PWC	https://paperswithcode.com/paper/learn-by-calibrating-using-calibration-as-a
Repo
Framework

Effective and Efficient Dropout for Deep Convolutional Neural Networks


Title	Effective and Efficient Dropout for Deep Convolutional Neural Networks
Authors	Shaofeng Cai, Yao Shu, Wei Wang, Meihui Zhang, Gang Chen, Beng Chin Ooi
Abstract	Machine-learning-based data-driven applications have become ubiquitous, e.g., health-care analysis and database system optimization. Big training data and large (deep) models are crucial for good performance. Dropout has been widely used as an efficient regularization technique to prevent large models from overfitting. However, many recent works show that dropout does not bring much performance improvement for deep convolutional neural networks (CNNs), a popular deep learning model for data-driven applications. In this paper, we revisit the problem and investigate its failure. We attribute the failure to the conflict between the conventional dropout and the batch normalization operation after it. We propose to adjust the order of the dropout operations to address the conflict; and further, other structurally more suited dropout variants are also examined and introduced for more efficient and effective regularization for CNNs. These dropout variants can be easily integrated into the building blocks of CNNs implemented by existing deep learning libraries, e.g., Apache Singa, to provide effective regularization for CNNs. Extensive experiments on benchmark datasets CIFAR, SVHN and ImageNet are conducted to compare the existing building blocks and the proposed building blocks with the proposed customizable dropout methods. The results confirm the superiority of our building blocks due to the regularization and implicit model ensemble effect of dropout. In particular, we improve over state-of-the-art CNNs with significantly better performance of 3.17%, 16.15%, 1.44%, 21.68% error rate on CIFAR-10, CIFAR-100, SVHN and ImageNet respectively.
Tasks
Published	2019-04-06
URL	https://arxiv.org/abs/1904.03392v3
PDF	https://arxiv.org/pdf/1904.03392v3.pdf
PWC	https://paperswithcode.com/paper/effective-and-efficient-dropout-for-deep
Repo
Framework

Region-Manipulated Fusion Networks for Pancreatitis Recognition


Title	Region-Manipulated Fusion Networks for Pancreatitis Recognition
Authors	Jian Wang, Xiaoyao Li, Xiangbo Shu, Weiqin Li
Abstract	This work first attempts to automatically recognize pancreatitis on CT scan images. However, different form the traditional object recognition, such pancreatitis recognition is challenging due to the fine-grained and non-rigid appearance variability of the local diseased regions. To this end, we propose a customized Region-Manipulated Fusion Networks (RMFN) to capture the key characteristics of local lesion for pancreatitis recognition. Specifically, to effectively highlight the imperceptible lesion regions, a novel region-manipulated scheme in RMFN is proposed to force the lesion regions while weaken the non-lesion regions by ceaselessly aggregating the multi-scale local information onto feature maps. The proposed scheme can be flexibly equipped into the existing neural networks, such as AlexNet and VGG. To evaluate the performance of the propose method, a real CT image database about pancreatitis is collected from hospitals \footnote{The database is available later}. And experimental results on such database well demonstrate the effectiveness of the proposed method for pancreatitis recognition.
Tasks	Object Recognition
Published	2019-07-03
URL	https://arxiv.org/abs/1907.01744v1
PDF	https://arxiv.org/pdf/1907.01744v1.pdf
PWC	https://paperswithcode.com/paper/region-manipulated-fusion-networks-for
Repo
Framework

Strategies for Structuring Story Generation


Title	Strategies for Structuring Story Generation
Authors	Angela Fan, Mike Lewis, Yann Dauphin
Abstract	Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right. We explore coarse-to-fine models for creating narrative texts of several hundred words, and introduce new models which decompose stories by abstracting over actions and entities. The model first generates the predicate-argument structure of the text, where different mentions of the same entity are marked with placeholder tokens. It then generates a surface realization of the predicate-argument structure, and finally replaces the entity placeholders with context-sensitive names and references. Human judges prefer the stories from our models to a wide range of previous approaches to hierarchical text generation. Extensive analysis shows that our methods can help improve the diversity and coherence of events and entities in generated stories.
Tasks	Text Generation
Published	2019-02-04
URL	https://arxiv.org/abs/1902.01109v2
PDF	https://arxiv.org/pdf/1902.01109v2.pdf
PWC	https://paperswithcode.com/paper/strategies-for-structuring-story-generation
Repo
Framework

Quaternion Product Units for Deep Learning on 3D Rotation Groups


Title	Quaternion Product Units for Deep Learning on 3D Rotation Groups
Authors	Xuan Zhang, Shaofei Qin, Yi Xu, Hongteng Xu
Abstract	We propose a novel quaternion product unit (QPU) to represent data on 3D rotation groups. The QPU leverages quaternion algebra and the law of 3D rotation group, representing 3D rotation data as quaternions and merging them via a weighted chain of Hamilton products. We prove that the representations derived by the proposed QPU can be disentangled into “rotation-invariant” features and “rotation-equivariant” features, respectively, which supports the rationality and the efficiency of the QPU in theory. We design quaternion neural networks based on our QPUs and make our models compatible with existing deep learning models. Experiments on both synthetic and real-world data show that the proposed QPU is beneficial for the learning tasks requiring rotation robustness.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.07791v1
PDF	https://arxiv.org/pdf/1912.07791v1.pdf
PWC	https://paperswithcode.com/paper/quaternion-product-units-for-deep-learning-on
Repo
Framework

Theoretical Investigation of Composite Neural Network


Title	Theoretical Investigation of Composite Neural Network
Authors	Ming-Chuan Yang, Meng Chang Chen
Abstract	This work theoretically investigates the performance of a composite neural network. A composite neural network is a rooted directed acyclic graph combining a set of pre-trained and non-instantiated neural network models, where a pre-trained neural network model is well-crafted for a specific task and targeted to approximate a specific function with instantiated weights. The advantages of adopting such a pre-trained model in a composite neural network are two folds. One is to benefit from other’s intelligence and diligence, and the other is saving the efforts in data preparation and resources and time in training. However, the overall performance of composite neural network is still not clear. In this work, we prove that a composite neural network, with high probability, performs better than any of its pre-trained components under certain assumptions. In addition, if an extra pre-trained component is added to a composite network, with high probability the overall performance will be improved. In the empirical evaluations, distinctively different applications support the above findings.
Tasks
Published	2019-10-18
URL	https://arxiv.org/abs/1910.09351v2
PDF	https://arxiv.org/pdf/1910.09351v2.pdf
PWC	https://paperswithcode.com/paper/theoretical-investigation-of-composite-neural
Repo
Framework

Neural Image Captioning


Title	Neural Image Captioning
Authors	Elaina Tan, Lakshay Sharma
Abstract	In recent years, the biggest advances in major Computer Vision tasks, such as object recognition, handwritten-digit identification, facial recognition, and many others., have all come through the use of Convolutional Neural Networks (CNNs). Similarly, in the domain of Natural Language Processing, Recurrent Neural Networks (RNNs), and Long Short Term Memory networks (LSTMs) in particular, have been crucial to some of the biggest breakthroughs in performance for tasks such as machine translation, part-of-speech tagging, sentiment analysis, and many others. These individual advances have greatly benefited tasks even at the intersection of NLP and Computer Vision, and inspired by this success, we studied some existing neural image captioning models that have proven to work well. In this work, we study some existing captioning models that provide near state-of-the-art performances, and try to enhance one such model. We also present a simple image captioning model that makes use of a CNN, an LSTM, and the beam search1 algorithm, and study its performance based on various qualitative and quantitative metrics.
Tasks	Image Captioning, Machine Translation, Object Recognition, Part-Of-Speech Tagging, Sentiment Analysis
Published	2019-07-02
URL	https://arxiv.org/abs/1907.02065v1
PDF	https://arxiv.org/pdf/1907.02065v1.pdf
PWC	https://paperswithcode.com/paper/neural-image-captioning
Repo
Framework