October 17, 2019

3406 words 16 mins read

Paper Group ANR 773

A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM. Vision-Based Gait Analysis for Senior Care. Impact of Intervals on the Emotional Effect in Western Music. Concise Fuzzy Representation of Big Graphs: a Dimensionality Reduction Approach. Multiple Abnormality Detection for Automatic Medical Image Diagnosis Using …

A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM


Title	A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
Authors	Shaokai Ye, Tianyun Zhang, Kaiqi Zhang, Jiayu Li, Jiaming Xie, Yun Liang, Sijia Liu, Xue Lin, Yanzhi Wang
Abstract	Many model compression techniques of Deep Neural Networks (DNNs) have been investigated, including weight pruning, weight clustering and quantization, etc. Weight pruning leverages the redundancy in the number of weights in DNNs, while weight clustering/quantization leverages the redundancy in the number of bit representations of weights. They can be effectively combined in order to exploit the maximum degree of redundancy. However, there lacks a systematic investigation in literature towards this direction. In this paper, we fill this void and develop a unified, systematic framework of DNN weight pruning and clustering/quantization using Alternating Direction Method of Multipliers (ADMM), a powerful technique in optimization theory to deal with non-convex optimization problems. Both DNN weight pruning and clustering/quantization, as well as their combinations, can be solved in a unified manner. For further performance improvement in this framework, we adopt multiple techniques including iterative weight quantization and retraining, joint weight clustering training and centroid updating, weight clustering retraining, etc. The proposed framework achieves significant improvements both in individual weight pruning and clustering/quantization problems, as well as their combinations. For weight pruning alone, we achieve 167x weight reduction in LeNet-5, 24.7x in AlexNet, and 23.4x in VGGNet, without any accuracy loss. For the combination of DNN weight pruning and clustering/quantization, we achieve 1,910x and 210x storage reduction of weight data on LeNet-5 and AlexNet, respectively, without accuracy loss. Our codes and models are released at the link http://bit.ly/2D3F0np
Tasks	Model Compression, Quantization
Published	2018-11-05
URL	http://arxiv.org/abs/1811.01907v1
PDF	http://arxiv.org/pdf/1811.01907v1.pdf
PWC	https://paperswithcode.com/paper/a-unified-framework-of-dnn-weight-pruning-and
Repo
Framework

Vision-Based Gait Analysis for Senior Care


Title	Vision-Based Gait Analysis for Senior Care
Authors	David Xue, Anin Sayana, Evan Darke, Kelly Shen, Jun-Ting Hsieh, Zelun Luo, Li-Jia Li, N. Lance Downing, Arnold Milstein, Li Fei-Fei
Abstract	As the senior population rapidly increases, it is challenging yet crucial to provide effective long-term care for seniors who live at home or in senior care facilities. Smart senior homes, which have gained widespread interest in the healthcare community, have been proposed to improve the well-being of seniors living independently. In particular, non-intrusive, cost-effective sensors placed in these senior homes enable gait characterization, which can provide clinically relevant information including mobility level and early neurodegenerative disease risk. In this paper, we present a method to perform gait analysis from a single camera placed within the home. We show that we can accurately calculate various gait parameters, demonstrating the potential for our system to monitor the long-term gait of seniors and thus aid clinicians in understanding a patient’s medical profile.
Tasks
Published	2018-12-01
URL	http://arxiv.org/abs/1812.00169v1
PDF	http://arxiv.org/pdf/1812.00169v1.pdf
PWC	https://paperswithcode.com/paper/vision-based-gait-analysis-for-senior-care
Repo
Framework

Impact of Intervals on the Emotional Effect in Western Music


Title	Impact of Intervals on the Emotional Effect in Western Music
Authors	Cengiz Kaygusuz, Julian Zuluaga
Abstract	Every art form ultimately aims to invoke an emotional response over the audience, and music is no different. While the precise perception of music is a highly subjective topic, there is an agreement in the “feeling” of a piece of music in broad terms. Based on this observation, in this study, we aimed to determine the emotional feeling associated with short passages of music; specifically by analyzing the melodic aspects. We have used the dataset put together by Eerola et. al. which is comprised of labeled short passages of film music. Our initial survey of the dataset indicated that other than “happy” and “sad” labels do not possess a melodic structure. We transcribed the main melody of the happy and sad tracks and used the intervals between the notes to classify them. Our experiments have shown that treating a melody as a bag-of-intervals do not possess any predictive power whatsoever, whereas counting intervals with respect to the key of the melody yielded a classifier with 85% accuracy.
Tasks
Published	2018-12-10
URL	http://arxiv.org/abs/1812.04723v1
PDF	http://arxiv.org/pdf/1812.04723v1.pdf
PWC	https://paperswithcode.com/paper/impact-of-intervals-on-the-emotional-effect
Repo
Framework

Concise Fuzzy Representation of Big Graphs: a Dimensionality Reduction Approach


Title	Concise Fuzzy Representation of Big Graphs: a Dimensionality Reduction Approach
Authors	Faisal N. Abu-Khzam, Rana H. Mouawi
Abstract	The enormous amount of data to be represented using large graphs exceeds in some cases the resources of a conventional computer. Edges in particular can take up a considerable amount of memory as compared to the number of nodes. However, rigorous edge storage might not always be essential to be able to draw the needed conclusions. A similar problem takes records with many variables and attempts to extract the most discernible features. It is said that the “dimension” of this data is reduced. Following an approach with the same objective in mind, we can map a graph representation to a k-dimensional space and answer queries of neighboring nodes by measuring Euclidean distances. The accuracy of our answers would decrease but would be compensated for by fuzzy logic which gives an idea about the likelihood of error. This method allows for reasonable representation in memory while maintaining a fair amount of useful information. Promising preliminary results are obtained and reported by testing the proposed approach on a number of Facebook graphs.
Tasks	Dimensionality Reduction
Published	2018-03-08
URL	http://arxiv.org/abs/1803.03114v1
PDF	http://arxiv.org/pdf/1803.03114v1.pdf
PWC	https://paperswithcode.com/paper/concise-fuzzy-representation-of-big-graphs-a
Repo
Framework

Multiple Abnormality Detection for Automatic Medical Image Diagnosis Using Bifurcated Convolutional Neural Network


Title	Multiple Abnormality Detection for Automatic Medical Image Diagnosis Using Bifurcated Convolutional Neural Network
Authors	Mohsen Hajabdollahi, Reza Esfandiarpoor, Elyas Sabeti, Nader Karimi, Kayvan Najarian, S. M. Reza Soroushmehr, Shadrokh Samavi
Abstract	Automating classification and segmentation process of abnormal regions in different body organs has a crucial role in most of medical imaging applications such as funduscopy, endoscopy, and dermoscopy. Detecting multiple abnormalities in each type of images is necessary for better and more accurate diagnosis procedure and medical decisions. In recent years portable medical imaging devices such as capsule endoscopy and digital dermatoscope have been introduced and made the diagnosis procedure easier and more efficient. However, these portable devices have constrained power resources and limited computational capability. To address this problem, we propose a bifurcated structure for convolutional neural networks performing both classification and segmentation of multiple abnormalities simultaneously. The proposed network is first trained by each abnormality separately. Then the network is trained using all abnormalities. In order to reduce the computational complexity, the network is redesigned to share some features which are common among all abnormalities. Later, these shared features are used in different settings (directions) to segment and classify the abnormal region of the image. Finally, results of the classification and segmentation directions are fused to obtain the classified segmentation map. Proposed framework is simulated using four frequent gastrointestinal abnormalities as well as three dermoscopic lesions and for evaluation of the proposed framework the results are compared with the corresponding ground truth map. Properties of the bifurcated network like low complexity and resource sharing make it suitable to be implemented as a part of portable medical imaging devices.
Tasks	Anomaly Detection
Published	2018-09-16
URL	http://arxiv.org/abs/1809.05831v2
PDF	http://arxiv.org/pdf/1809.05831v2.pdf
PWC	https://paperswithcode.com/paper/multiple-abnormality-detection-for-automatic
Repo
Framework

Improving Deep Binary Embedding Networks by Order-aware Reweighting of Triplets


Title	Improving Deep Binary Embedding Networks by Order-aware Reweighting of Triplets
Authors	Jikai Chen, Hanjiang Lai, Libing Geng, Yan Pan
Abstract	In this paper, we focus on triplet-based deep binary embedding networks for image retrieval task. The triplet loss has been shown to be most effective for the ranking problem. However, most of the previous works treat the triplets equally or select the hard triplets based on the loss. Such strategies do not consider the order relations, which is important for retrieval task. To this end, we propose an order-aware reweighting method to effectively train the triplet-based deep networks, which up-weights the important triplets and down-weights the uninformative triplets. First, we present the order-aware weighting factors to indicate the importance of the triplets, which depend on the rank order of binary codes. Then, we reshape the triplet loss to the squared triplet loss such that the loss function will put more weights on the important triplets. Extensive evaluations on four benchmark datasets show that the proposed method achieves significant performance compared with the state-of-the-art baselines.
Tasks	Image Retrieval
Published	2018-04-17
URL	http://arxiv.org/abs/1804.06061v1
PDF	http://arxiv.org/pdf/1804.06061v1.pdf
PWC	https://paperswithcode.com/paper/improving-deep-binary-embedding-networks-by
Repo
Framework

Drug response prediction by ensemble learning and drug-induced gene expression signatures


Title	Drug response prediction by ensemble learning and drug-induced gene expression signatures
Authors	Mehmet Tan, Ozan Fırat Özgül, Batuhan Bardak, Işıksu Ekşioğlu, Suna Sabuncuoğlu
Abstract	Chemotherapeutic response of cancer cells to a given compound is one of the most fundamental information one requires to design anti-cancer drugs. Recent advances in producing large drug screens against cancer cell lines provided an opportunity to apply machine learning methods for this purpose. In addition to cytotoxicity databases, considerable amount of drug-induced gene expression data has also become publicly available. Following this, several methods that exploit omics data were proposed to predict drug activity on cancer cells. However, due to the complexity of cancer drug mechanisms, none of the existing methods are perfect. One possible direction, therefore, is to combine the strengths of both the methods and the databases for improved performance. We demonstrate that integrating a large number of predictions by the proposed method improves the performance for this task. The predictors in the ensemble differ in several aspects such as the method itself, the number of tasks method considers (multi-task vs. single-task) and the subset of data considered (sub-sampling). We show that all these different aspects contribute to the success of the final ensemble. In addition, we attempt to use the drug screen data together with two novel signatures produced from the drug-induced gene expression profiles of cancer cell lines. Finally, we evaluate the method predictions by in vitro experiments in addition to the tests on data sets.The predictions of the methods, the signatures and the software are available from \url{http://mtan.etu.edu.tr/drug-response-prediction/}.
Tasks
Published	2018-02-11
URL	http://arxiv.org/abs/1802.03800v3
PDF	http://arxiv.org/pdf/1802.03800v3.pdf
PWC	https://paperswithcode.com/paper/drug-response-prediction-by-ensemble-learning
Repo
Framework


Title	Network-Clustered Multi-Modal Bug Localization
Authors	Thong Hoang, Richard J. Oentaryo, Tien-Duy B. Le, David Lo
Abstract	Developers often spend much effort and resources to debug a program. To help the developers debug, numerous information retrieval (IR)-based and spectrum-based bug localization techniques have been devised. IR-based techniques process textual information in bug reports, while spectrum-based techniques process program spectra (i.e., a record of which program elements are executed for each test case). While both techniques ultimately generate a ranked list of program elements that likely contain a bug, they only consider one source of information–either bug reports or program spectra–which is not optimal. In light of this deficiency, this paper presents a new approach dubbed Network-clustered Multi-modal Bug Localization (NetML), which utilizes multi-modal information from both bug reports and program spectra to localize bugs. NetML facilitates an effective bug localization by carrying out a joint optimization of bug localization error and clustering of both bug reports and program elements (i.e., methods). The clustering is achieved through the incorporation of network Lasso regularization, which incentivizes the model parameters of similar bug reports and similar program elements to be close together. To estimate the model parameters of both bug reports and methods, NetML employs an adaptive learning procedure based on Newton method that updates the parameters on a per-feature basis. Extensive experiments on 355 real bugs from seven software systems have been conducted to benchmark NetML against various state-of-the-art localization methods. The results show that NetML surpasses the best-performing baseline by 31.82%, 22.35%, 19.72%, and 19.24%, in terms of the number of bugs successfully localized when a developer inspects the top 1, 5, and 10 methods and Mean Average Precision (MAP), respectively.
Tasks	Information Retrieval
Published	2018-02-27
URL	http://arxiv.org/abs/1802.09729v1
PDF	http://arxiv.org/pdf/1802.09729v1.pdf
PWC	https://paperswithcode.com/paper/network-clustered-multi-modal-bug
Repo
Framework

Adaptive Gaussian process surrogates for Bayesian inference


Title	Adaptive Gaussian process surrogates for Bayesian inference
Authors	Timur Takhtaganov, Juliane Müller
Abstract	We present an adaptive approach to the construction of Gaussian process surrogates for Bayesian inference with expensive-to-evaluate forward models. Our method relies on the fully Bayesian approach to training Gaussian process models and utilizes the expected improvement idea from Bayesian global optimization. We adaptively construct training designs by maximizing the expected improvement in fit of the Gaussian process model to the noisy observational data. Numerical experiments on model problems with synthetic data demonstrate the effectiveness of the obtained adaptive designs compared to the fixed non-adaptive designs in terms of accurate posterior estimation at a fraction of the cost of inference with forward models.
Tasks	Bayesian Inference
Published	2018-09-27
URL	http://arxiv.org/abs/1809.10784v1
PDF	http://arxiv.org/pdf/1809.10784v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-gaussian-process-surrogates-for
Repo
Framework

Sentiment Adaptive End-to-End Dialog Systems


Title	Sentiment Adaptive End-to-End Dialog Systems
Authors	Weiyan Shi, Zhou Yu
Abstract	End-to-end learning framework is useful for building dialog systems for its simplicity in training and efficiency in model updating. However, current end-to-end approaches only consider user semantic inputs in learning and under-utilize other user information. Therefore, we propose to include user sentiment obtained through multimodal information (acoustic, dialogic and textual), in the end-to-end learning framework to make systems more user-adaptive and effective. We incorporated user sentiment information in both supervised and reinforcement learning settings. In both settings, adding sentiment information reduced the dialog length and improved the task success rate on a bus information search task. This work is the first attempt to incorporate multimodal user information in the adaptive end-to-end dialog system training framework and attained state-of-the-art performance.
Tasks
Published	2018-04-28
URL	https://arxiv.org/abs/1804.10731v3
PDF	https://arxiv.org/pdf/1804.10731v3.pdf
PWC	https://paperswithcode.com/paper/sentiment-adaptive-end-to-end-dialog-systems
Repo
Framework

Adaptive Selection of Deep Learning Models on Embedded Systems


Title	Adaptive Selection of Deep Learning Models on Embedded Systems
Authors	Ben Taylor, Vicent Sanz Marco, Willy Wolff, Yehia Elkhatib, Zheng Wang
Abstract	The recent ground-breaking advances in deep learning networks ( DNNs ) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-limited embedded devices. Offloading the computation into the cloud is often infeasible due to privacy concerns, high latency, or the lack of connectivity. As such, there is a critical need to find a way to effectively execute the DNN models locally on the devices. This paper presents an adaptive scheme to determine which DNN model to use for a given input, by considering the desired accuracy and inference time. Our approach employs machine learning to develop a predictive model to quickly select a pre-trained DNN to use for a given input and the optimization constraint. We achieve this by first training off-line a predictive model, and then use the learnt model to select a DNN model to use for new, unseen inputs. We apply our approach to the image classification task and evaluate it on a Jetson TX2 embedded deep learning platform using the ImageNet ILSVRC 2012 validation dataset. We consider a range of influential DNN models. Experimental results show that our approach achieves a 7.52% improvement in inference accuracy, and a 1.8x reduction in inference time over the most-capable single DNN model.
Tasks	Image Classification
Published	2018-05-11
URL	http://arxiv.org/abs/1805.04252v1
PDF	http://arxiv.org/pdf/1805.04252v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-selection-of-deep-learning-models-on
Repo
Framework

Spectral-Pruning: Compressing deep neural network via spectral analysis


Title	Spectral-Pruning: Compressing deep neural network via spectral analysis
Authors	Taiji Suzuki, Hiroshi Abe, Tomoya Murata, Shingo Horiuchi, Kotaro Ito, Tokuma Wachi, So Hirai, Masatoshi Yukishima, Tomoaki Nishimura
Abstract	The model size of deep neural network is getting larger and larger to realize superior performance in complicated tasks. This makes it difficult to implement deep neural network in small edge-computing devices. To overcome this problem, model compression methods have been gathering much attention. However, there have been only few theoretical back-grounds that explain what kind of quantity determines the compression ability. To resolve this issue, we develop a new theoretical frame-work for model compression, and propose a new method called {\it Spectral-Pruning} based on the theory. Our theoretical analysis is based on the observation such that the eigenvalues of the covariance matrix of the output from nodes in the internal layers often shows rapid decay. We define “degree of freedom” to quantify an intrinsic dimensionality of the model by using the eigenvalue distribution and show that the compression ability is essentially controlled by this quantity. Along with this, we give a generalization error bound of the compressed model. Our proposed method is applicable to wide range of models, unlike the existing methods, e.g., ones possess complicated branches as implemented in SegNet and ResNet. Our method makes use of both “input” and “output” in each layer and is easy to implement. We apply our method to several datasets to justify our theoretical analyses and show that the proposed method achieves the state-of-the-art performance.
Tasks	Model Compression
Published	2018-08-26
URL	http://arxiv.org/abs/1808.08558v1
PDF	http://arxiv.org/pdf/1808.08558v1.pdf
PWC	https://paperswithcode.com/paper/spectral-pruning-compressing-deep-neural
Repo
Framework

Does the brain represent words? An evaluation of brain decoding studies of language understanding


Title	Does the brain represent words? An evaluation of brain decoding studies of language understanding
Authors	Jon Gauthier, Anna Ivanova
Abstract	Language decoding studies have identified word representations which can be used to predict brain activity in response to novel words and sentences (Anderson et al., 2016; Pereira et al., 2018). The unspoken assumption of these studies is that, during processing, linguistic information is transformed into some shared semantic space, and those semantic representations are then used for a variety of linguistic and non-linguistic tasks. We claim that current studies vastly underdetermine the content of these representations, the algorithms which the brain deploys to produce and consume them, and the computational tasks which they are designed to solve. We illustrate this indeterminacy with an extension of the sentence-decoding experiment of Pereira et al. (2018), showing how standard evaluations fail to distinguish between language processing models which deploy different mechanisms and which are optimized to solve very different tasks. We conclude by suggesting changes to the brain decoding paradigm which can support stronger claims of neural representation.
Tasks	Brain Decoding
Published	2018-06-02
URL	http://arxiv.org/abs/1806.00591v1
PDF	http://arxiv.org/pdf/1806.00591v1.pdf
PWC	https://paperswithcode.com/paper/does-the-brain-represent-words-an-evaluation
Repo
Framework

Is your Statement Purposeless? Predicting Computer Science Graduation Admission Acceptance based on Statement Of Purpose


Title	Is your Statement Purposeless? Predicting Computer Science Graduation Admission Acceptance based on Statement Of Purpose
Authors	Diptesh Kanojia, Nikhil Wani, Pushpak Bhattacharyya
Abstract	We present a quantitative, data-driven machine learning approach to mitigate the problem of unpredictability of Computer Science Graduate School Admissions. In this paper, we discuss the possibility of a system which may help prospective applicants evaluate their Statement of Purpose (SOP) based on our system output. We, then, identify feature sets which can be used to train a predictive model. We train a model over fifty manually verified SOPs for which it uses an SVM classifier and achieves the highest accuracy of 92% with 10-fold cross-validation. We also perform experiments to establish that Word Embedding based features and Document Similarity-based features outperform other identified feature combinations. We plan to deploy our application as a web service and release it as a FOSS service.
Tasks
Published	2018-10-09
URL	http://arxiv.org/abs/1810.04502v1
PDF	http://arxiv.org/pdf/1810.04502v1.pdf
PWC	https://paperswithcode.com/paper/is-your-statement-purposeless-predicting
Repo
Framework

Background subtraction using the factored 3-way restricted Boltzmann machines


Title	Background subtraction using the factored 3-way restricted Boltzmann machines
Authors	Soonam Lee, Daekeun Kim
Abstract	In this paper, we proposed a method for reconstructing the 3D model based on continuous sensory input. The robot can draw on extremely large data from the real world using various sensors. However, the sensory inputs are usually too noisy and high-dimensional data. It is very difficult and time consuming for robot to process using such raw data when the robot tries to construct 3D model. Hence, there needs to be a method that can extract useful information from such sensory inputs. To address this problem our method utilizes the concept of Object Semantic Hierarchy (OSH). Different from the previous work that used this hierarchy framework, we extract the motion information using the Deep Belief Network technique instead of applying classical computer vision approaches. We have trained on two large sets of random dot images (10,000) which are translated and rotated, respectively, and have successfully extracted several bases that explain the translation and rotation motion. Based on this translation and rotation bases, background subtraction have become possible using Object Semantic Hierarchy.
Tasks
Published	2018-02-05
URL	http://arxiv.org/abs/1802.01522v1
PDF	http://arxiv.org/pdf/1802.01522v1.pdf
PWC	https://paperswithcode.com/paper/background-subtraction-using-the-factored-3
Repo
Framework