October 17, 2019

3114 words 15 mins read

Paper Group ANR 958

Knowledge Distillation with Feature Maps for Image Classification. Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks. Dynamic Feature Scaling for K-Nearest Neighbor Algorithm. Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations. An Empirical Assessment of the Complexity an …

Knowledge Distillation with Feature Maps for Image Classification


Title	Knowledge Distillation with Feature Maps for Image Classification
Authors	Wei-Chun Chen, Chia-Che Chang, Chien-Yu Lu, Che-Rung Lee
Abstract	The model reduction problem that eases the computation costs and latency of complex deep learning architectures has received an increasing number of investigations owing to its importance in model deployment. One promising method is knowledge distillation (KD), which creates a fast-to-execute student model to mimic a large teacher network. In this paper, we propose a method, called KDFM (Knowledge Distillation with Feature Maps), which improves the effectiveness of KD by learning the feature maps from the teacher network. Two major techniques used in KDFM are shared classifier and generative adversarial network. Experimental results show that KDFM can use a four layers CNN to mimic DenseNet-40 and use MobileNet to mimic DenseNet-100. Both student networks have less than 1% accuracy loss comparing to their teacher models for CIFAR-100 datasets. The student networks are 2-6 times faster than their teacher models for inference, and the model size of MobileNet is less than half of DenseNet-100’s.
Tasks	Image Classification
Published	2018-12-03
URL	http://arxiv.org/abs/1812.00660v1
PDF	http://arxiv.org/pdf/1812.00660v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-distillation-with-feature-maps-for
Repo
Framework

Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks


Title	Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
Authors	Pantelis R. Vlachas, Wonmin Byeon, Zhong Y. Wan, Themistoklis P. Sapsis, Petros Koumoutsakos
Abstract	We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.
Tasks	Gaussian Processes, Time Series
Published	2018-02-21
URL	https://arxiv.org/abs/1802.07486v5
PDF	https://arxiv.org/pdf/1802.07486v5.pdf
PWC	https://paperswithcode.com/paper/data-driven-forecasting-of-high-dimensional
Repo
Framework

Dynamic Feature Scaling for K-Nearest Neighbor Algorithm


Title	Dynamic Feature Scaling for K-Nearest Neighbor Algorithm
Authors	Chandrasekaran Anirudh Bhardwaj, Megha Mishra, Kalyani Desikan
Abstract	Nearest Neighbors Algorithm is a Lazy Learning Algorithm, in which the algorithm tries to approximate the predictions with the help of similar existing vectors in the training dataset. The predictions made by the K-Nearest Neighbors algorithm is based on averaging the target values of the spatial neighbors. The selection process for neighbors in the Hermitian space is done with the help of distance metrics such as Euclidean distance, Minkowski distance, Mahalanobis distance etc. A majority of the metrics such as Euclidean distance are scale variant, meaning that the results could vary for different range of values used for the features. Standard techniques used for the normalization of scaling factors are feature scaling method such as Z-score normalization technique, Min-Max scaling etc. Scaling methods uniformly assign equal weights to all the features, which might result in a non-ideal situation. This paper proposes a novel method to assign weights to individual feature with the help of out of bag errors obtained from constructing multiple decision tree models.
Tasks
Published	2018-11-13
URL	http://arxiv.org/abs/1811.05062v1
PDF	http://arxiv.org/pdf/1811.05062v1.pdf
PWC	https://paperswithcode.com/paper/dynamic-feature-scaling-for-k-nearest
Repo
Framework

Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations


Title	Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations
Authors	Ting Chen, Martin Renqiang Min, Yizhou Sun
Abstract	Conventional embedding methods directly associate each symbol with a continuous embedding vector, which is equivalent to applying a linear transformation based on a “one-hot” encoding of the discrete symbols. Despite its simplicity, such approach yields the number of parameters that grows linearly with the vocabulary size and can lead to overfitting. In this work, we propose a much more compact K-way D-dimensional discrete encoding scheme to replace the “one-hot” encoding. In the proposed “KD encoding”, each symbol is represented by a $D$-dimensional code with a cardinality of $K$, and the final symbol embedding vector is generated by composing the code embedding vectors. To end-to-end learn semantically meaningful codes, we derive a relaxed discrete optimization approach based on stochastic gradient descent, which can be generally applied to any differentiable computational graph with an embedding layer. In our experiments with various applications from natural language processing to graph convolutional networks, the total size of the embedding layer can be reduced up to 98% while achieving similar or better performance.
Tasks
Published	2018-06-21
URL	http://arxiv.org/abs/1806.09464v1
PDF	http://arxiv.org/pdf/1806.09464v1.pdf
PWC	https://paperswithcode.com/paper/learning-k-way-d-dimensional-discrete-codes
Repo
Framework


Title	An Empirical Assessment of the Complexity and Realism of Synthetic Social Contact Networks
Authors	Kiran Karra, Samarth Swarup, Justus Graham
Abstract	We use multiple measures of graph complexity to evaluate the realism of synthetically-generated networks of human activity, in comparison with several stylized network models as well as a collection of empirical networks from the literature. The synthetic networks are generated by integrating data about human populations from several sources, including the Census, transportation surveys, and geographical data. The resulting networks represent an approximation of daily or weekly human interaction. Our results indicate that the synthetically generated graphs according to our methodology are closer to the real world graphs, as measured across multiple structural measures, than a range of stylized graphs generated using common network models from the literature.
Tasks
Published	2018-10-06
URL	http://arxiv.org/abs/1811.07746v2
PDF	http://arxiv.org/pdf/1811.07746v2.pdf
PWC	https://paperswithcode.com/paper/an-empirical-assessment-of-the-complexity-and
Repo
Framework

Multi-Scale Convolutional-Stack Aggregation for Robust White Matter Hyperintensities Segmentation


Title	Multi-Scale Convolutional-Stack Aggregation for Robust White Matter Hyperintensities Segmentation
Authors	Hongwei Li, Jianguo Zhang, Mark Muehlau, Jan Kirschke, Bjoern Menze
Abstract	Segmentation of both large and small white matter hyperintensities/lesions in brain MR images is a challenging task which has drawn much attention in recent years. We propose a multi-scale aggregation model framework to deal with volume-varied lesions. Firstly, we present a specifically-designed network for small lesion segmentation called Stack-Net, in which multiple convolutional layers are connected, aiming to preserve rich local spatial information of small lesions before the sub-sampling layer. Secondly, we aggregate multi-scale Stack-Nets with different receptive fields to learn multi-scale contextual information of both large and small lesions. Our model is evaluated on recent MICCAI WMH Challenge Dataset and outperforms the state-of-the-art on lesion recall and lesion F1-score under 5-fold cross validation. In addition, we further test our pre-trained models on a Multiple Sclerosis lesion dataset with 30 subjects under cross-center evaluation. Results show that the aggregation model is effective in learning multi-scale spatial information.It claimed the first place on the hidden test set after independent evaluation by the challenge organizer. In addition, we further test our pre-trained models on a Multiple Sclerosis lesion dataset with 30 subjects under cross-center evaluation. Results show that the aggregation model is effective in learning multi-scale spatial information.
Tasks	Lesion Segmentation
Published	2018-07-13
URL	http://arxiv.org/abs/1807.05153v3
PDF	http://arxiv.org/pdf/1807.05153v3.pdf
PWC	https://paperswithcode.com/paper/multi-scale-convolutional-stack-aggregation
Repo
Framework

Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST


Title	Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST
Authors	Jinzheng Cai, Youbao Tang, Le Lu, Adam P. Harrison, Ke Yan, Jing Xiao, Lin Yang, Ronald M. Summers
Abstract	Volumetric lesion segmentation from computed tomography (CT) images is a powerful means to precisely assess multiple time-point lesion/tumor changes. However, because manual 3D segmentation is prohibitively time consuming, current practices rely on an imprecise surrogate called response evaluation criteria in solid tumors (RECIST). Despite their coarseness, RECIST markers are commonly found in current hospital picture and archiving systems (PACS), meaning they can provide a potentially powerful, yet extraordinarily challenging, source of weak supervision for full 3D segmentation. Toward this end, we introduce a convolutional neural network (CNN) based weakly supervised slice-propagated segmentation (WSSS) method to 1) generate the initial lesion segmentation on the axial RECIST-slice; 2) learn the data distribution on RECIST-slices; 3) extrapolate to segment the whole lesion slice by slice to finally obtain a volumetric segmentation. To validate the proposed method, we first test its performance on a fully annotated lymph node dataset, where WSSS performs comparably to its fully supervised counterparts. We then test on a comprehensive lesion dataset with 32,735 RECIST marks, where we report a mean Dice score of 92% on RECIST-marked slices and 76% on the entire 3D volumes.
Tasks	Computed Tomography (CT), Lesion Segmentation
Published	2018-07-02
URL	http://arxiv.org/abs/1807.01172v1
PDF	http://arxiv.org/pdf/1807.01172v1.pdf
PWC	https://paperswithcode.com/paper/accurate-weakly-supervised-deep-lesion-1
Repo
Framework

SkinNet: A Deep Learning Framework for Skin Lesion Segmentation


Title	SkinNet: A Deep Learning Framework for Skin Lesion Segmentation
Authors	Sulaiman Vesal, Nishant Ravikumar, Andreas Maier
Abstract	There has been a steady increase in the incidence of skin cancer worldwide, with a high rate of mortality. Early detection and segmentation of skin lesions are crucial for timely diagnosis and treatment, necessary to improve the survival rate of patients. However, skin lesion segmentation is a challenging task due to the low contrast of lesions and their high similarity in terms of appearance, to healthy tissue. This underlines the need for an accurate and automatic approach for skin lesion segmentation. To tackle this issue, we propose a convolutional neural network (CNN) called SkinNet. The proposed CNN is a modified version of U-Net. We compared the performance of our approach with other state-of-the-art techniques, using the ISBI 2017 challenge dataset. Our approach outperformed the others in terms of the Dice coefficient, Jaccard index and sensitivity, evaluated on the held-out challenge test data set, across 5-fold cross validation experiments. SkinNet achieved an average value of 85.10, 76.67 and 93.0%, for the DC, JI, and SE, respectively.
Tasks	Lesion Segmentation
Published	2018-06-25
URL	http://arxiv.org/abs/1806.09522v1
PDF	http://arxiv.org/pdf/1806.09522v1.pdf
PWC	https://paperswithcode.com/paper/skinnet-a-deep-learning-framework-for-skin
Repo
Framework

Learning to Describe Phrases with Local and Global Contexts


Title	Learning to Describe Phrases with Local and Global Contexts
Authors	Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, Masaru Kitsuregawa
Abstract	When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities. If we humans cannot figure out the meaning of those expressions from the immediate local context, we consult dictionaries for definitions or search documents or the web to find other global context to help in interpretation. Can machines help us do this work? Which type of context is more important for machines to solve the problem? To answer these questions, we undertake a task of describing a given phrase in natural language based on its local and global contexts. To solve this task, we propose a neural description model that consists of two context encoders and a description decoder. In contrast to the existing methods for non-standard English explanation [Ni+ 2017] and definition generation [Noraset+ 2017; Gadetsky+ 2018], our model appropriately takes important clues from both local and global contexts. Experimental results on three existing datasets (including WordNet, Oxford and Urban Dictionaries) and a dataset newly created from Wikipedia demonstrate the effectiveness of our method over previous work.
Tasks	Reading Comprehension
Published	2018-11-01
URL	http://arxiv.org/abs/1811.00266v2
PDF	http://arxiv.org/pdf/1811.00266v2.pdf
PWC	https://paperswithcode.com/paper/learning-to-describe-phrases-with-local-and
Repo
Framework

LDW-SCSA: Logistic Dynamic Weight based Sine Cosine Search Algorithm for Numerical Functions Optimization


Title	LDW-SCSA: Logistic Dynamic Weight based Sine Cosine Search Algorithm for Numerical Functions Optimization
Authors	Turker Tuncer
Abstract	Particle swarm optimization (PSO) and Sine Cosine algorithm (SCA) have been widely used optimization methods but these methods have some disadvantages such as trapped local optimum point. In order to solve this problem and obtain more successful results than others, a novel logistic dynamic weight based sine cosine search algorithm (LDW-SCSA) is presented in this paper. In the LDW-SCSA method, logistic map is used as dynamic weight generator. Logistic map is one of the famous and widely used chaotic map in the literature. Search process of SCA is modified in the LDW-SCSA. To evaluate performance of the LDW-SCSA, the widely used numerical benchmark functions were utilized as test suite and other swarm optimization methods were used to obtain the comparison results. Superior performances of the LDW-SCSA are proved success of this method.
Tasks
Published	2018-09-09
URL	http://arxiv.org/abs/1809.03055v1
PDF	http://arxiv.org/pdf/1809.03055v1.pdf
PWC	https://paperswithcode.com/paper/ldw-scsa-logistic-dynamic-weight-based-sine
Repo
Framework

SLSDeep: Skin Lesion Segmentation Based on Dilated Residual and Pyramid Pooling Networks


Title	SLSDeep: Skin Lesion Segmentation Based on Dilated Residual and Pyramid Pooling Networks
Authors	Md. Mostafa Kamal Sarker, Hatem A. Rashwan, Farhan Akram, Syeda Furruka Banu, Adel Saleh, Vivek Kumar Singh, Forhad U H Chowdhury, Saddam Abdulwahab, Santiago Romani, Petia Radeva, Domenec Puig
Abstract	Skin lesion segmentation (SLS) in dermoscopic images is a crucial task for automated diagnosis of melanoma. In this paper, we present a robust deep learning SLS model, so-called SLSDeep, which is represented as an encoder-decoder network. The encoder network is constructed by dilated residual layers, in turn, a pyramid pooling network followed by three convolution layers is used for the decoder. Unlike the traditional methods employing a cross-entropy loss, we investigated a loss function by combining both Negative Log Likelihood (NLL) and End Point Error (EPE) to accurately segment the melanoma regions with sharp boundaries. The robustness of the proposed model was evaluated on two public databases: ISBI 2016 and 2017 for skin lesion analysis towards melanoma detection challenge. The proposed model outperforms the state-of-the-art methods in terms of segmentation accuracy. Moreover, it is capable to segment more than $100$ images of size 384x384 per second on a recent GPU.
Tasks	Lesion Segmentation
Published	2018-05-25
URL	http://arxiv.org/abs/1805.10241v2
PDF	http://arxiv.org/pdf/1805.10241v2.pdf
PWC	https://paperswithcode.com/paper/slsdeep-skin-lesion-segmentation-based-on
Repo
Framework

Evaluation of CNN-based Single-Image Depth Estimation Methods


Title	Evaluation of CNN-based Single-Image Depth Estimation Methods
Authors	Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, Marco Körner
Abstract	While an increasing interest in deep models for single-image depth estimation methods can be observed, established schemes for their evaluation are still limited. We propose a set of novel quality criteria, allowing for a more detailed analysis by focusing on specific characteristics of depth maps. In particular, we address the preservation of edges and planar regions, depth consistency, and absolute distance accuracy. In order to employ these metrics to evaluate and compare state-of-the-art single-image depth estimation approaches, we provide a new high-quality RGB-D dataset. We used a DSLR camera together with a laser scanner to acquire high-resolution images and highly accurate depth maps. Experimental results show the validity of our proposed evaluation protocol.
Tasks	Depth Estimation
Published	2018-05-03
URL	http://arxiv.org/abs/1805.01328v1
PDF	http://arxiv.org/pdf/1805.01328v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-cnn-based-single-image-depth
Repo
Framework

DFNet: Semantic Segmentation on Panoramic Images with Dynamic Loss Weights and Residual Fusion Block


Title	DFNet: Semantic Segmentation on Panoramic Images with Dynamic Loss Weights and Residual Fusion Block
Authors	Wei Jiang, Yan Wu
Abstract	For the self-driving and automatic parking, perception is the basic and critical technique, moreover, the detection of lane markings and parking slots is an important part of visual perception. In this paper, we use the semantic segmentation method to segment the area and classify the class of lane makings and parking slots on panoramic surround view (PSV) dataset. We propose the DFNet and make two main contributions, one is dynamic loss weights, and the other is residual fusion block (RFB). Dynamic loss weights are varying from classes, calculated according to the pixel number of each class in a batch. RFB is composed of two convolutional layers, one pooling layer, and a fusion layer to combine the feature maps by pixel multiplication. We evaluate our method on PSV dataset, and achieve an advanced result.
Tasks	Semantic Segmentation
Published	2018-06-11
URL	http://arxiv.org/abs/1806.07226v1
PDF	http://arxiv.org/pdf/1806.07226v1.pdf
PWC	https://paperswithcode.com/paper/dfnet-semantic-segmentation-on-panoramic
Repo
Framework

Connectionist Recommendation in the Wild: On the utility and scrutability of neural networks for personalized course guidance


Title	Connectionist Recommendation in the Wild: On the utility and scrutability of neural networks for personalized course guidance
Authors	Zachary A. Pardos, Zihao Fan, Weijie Jiang
Abstract	The aggregate behaviors of users can collectively encode deep semantic information about the objects with which they interact. In this paper, we demonstrate novel ways in which the synthesis of these data can illuminate the terrain of users’ environment and support them in their decision making and wayfinding. A novel application of Recurrent Neural Networks and skip-gram models, approaches popularized by their application to modeling language, are brought to bear on student university enrollment sequences to create vector representations of courses and map out traversals across them. We present demonstrations of how scrutability from these neural networks can be gained and how the combination of these techniques can be seen as an evolution of content tagging and a means for a recommender to balance user preferences inferred from data with those explicitly specified. From validation of the models to the development of a UI, we discuss additional requisite functionality informed by the results of a usability study leading to the ultimate deployment of the system at a university.
Tasks	Decision Making
Published	2018-03-26
URL	http://arxiv.org/abs/1803.09535v3
PDF	http://arxiv.org/pdf/1803.09535v3.pdf
PWC	https://paperswithcode.com/paper/connectionist-recommendation-in-the-wild-on
Repo
Framework

The Effects of Image Pre- and Post-Processing, Wavelet Decomposition, and Local Binary Patterns on U-Nets for Skin Lesion Segmentation


Title	The Effects of Image Pre- and Post-Processing, Wavelet Decomposition, and Local Binary Patterns on U-Nets for Skin Lesion Segmentation
Authors	Sara Ross-Howe, H. R. Tizhoosh
Abstract	Skin cancer is a widespread, global, and potentially deadly disease, which over the last three decades has afflicted more lives in the USA than all other forms of cancer combined. There have been a lot of promising recent works utilizing deep network architectures, such as FCNs, U-Nets, and ResNets, for developing automated skin lesion segmentation. This paper investigates various pre- and post-processing techniques for improving the performance of U-Nets as measured by the Jaccard Index. The dataset provided as part of the “2017 ISBI Challenges on Skin Lesion Analysis Towards Melanoma Detection” was used for this evaluation and the performance of the finalist competitors was the standard for comparison. The pre-processing techniques employed in the proposed system included contrast enhancement, artifact removal, and vignette correction. More advanced image transformations, such as local binary patterns and wavelet decomposition, were also employed to augment the raw grayscale images used as network input features. While the performance of the proposed system fell short of the winners of the challenge, it was determined that using wavelet decomposition as an early transformation step improved the overall performance of the system over pre- and post-processing steps alone.
Tasks	Lesion Segmentation
Published	2018-04-30
URL	http://arxiv.org/abs/1805.05239v1
PDF	http://arxiv.org/pdf/1805.05239v1.pdf
PWC	https://paperswithcode.com/paper/the-effects-of-image-pre-and-post-processing
Repo
Framework