Paper Group ANR 241
Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability. GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking. Approach for Semi-automatic Construction of Anti-infective Drug Ontology Based on Entity Linking. An FPGA Implementation of a Time Delay Reservoir Using S …
Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability
| Title | Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability |
| Authors | Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez |
| Abstract | Image fusion combines data from different heterogeneous sources to obtain more precise information about an underlying scene. Hyperspectral-multispectral (HS-MS) image fusion is currently attracting great interest in remote sensing since it allows the generation of high spatial resolution HS images, circumventing the main limitation of this imaging modality. Existing HS-MS fusion algorithms, however, neglect the spectral variability often existing between images acquired at different time instants. This time difference causes variations in spectral signatures of the underlying constituent materials due to different acquisition and seasonal conditions. This paper introduces a novel HS-MS image fusion strategy that combines an unmixing-based formulation with an explicit parametric model for typical spectral variability between the two images. Simulations with synthetic and real data show that the proposed strategy leads to a significant performance improvement under spectral variability and state-of-the-art performance otherwise. |
| Tasks | Super-Resolution |
| Published | 2018-08-30 |
| URL | https://arxiv.org/abs/1808.10072v2 |
| https://arxiv.org/pdf/1808.10072v2.pdf | |
| PWC | https://paperswithcode.com/paper/super-resolution-for-hyperspectral-and |
| Repo | |
| Framework | |
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
| Title | GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking |
| Authors | Patrick H. Chen, Si Si, Yang Li, Ciprian Chelba, Cho-jui Hsieh |
| Abstract | Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses. As a case study, a state-of-the-art neural language model usually consists of one or more recurrent layers sandwiched between an embedding layer used for representing input tokens and a softmax layer for generating output tokens. For problems with a very large vocabulary size, the embedding and the softmax matrices can account for more than half of the model size. For instance, the bigLSTM model achieves state-of- the-art performance on the One-Billion-Word (OBW) dataset with around 800k vocabulary, and its word embedding and softmax matrices use more than 6GBytes space, and are responsible for over 90% of the model parameters. In this paper, we propose GroupReduce, a novel compression method for neural language models, based on vocabulary-partition (block) based low-rank matrix approximation and the inherent frequency distribution of tokens (the power-law distribution of words). The experimental results show our method can significantly outperform traditional compression methods such as low-rank approximation and pruning. On the OBW dataset, our method achieved 6.6 times compression rate for the embedding and softmax matrices, and when combined with quantization, our method can achieve 26 times compression rate, which translates to a factor of 12.8 times compression for the entire model with very little degradation in perplexity. |
| Tasks | Language Modelling, Model Compression, Quantization |
| Published | 2018-06-18 |
| URL | http://arxiv.org/abs/1806.06950v1 |
| http://arxiv.org/pdf/1806.06950v1.pdf | |
| PWC | https://paperswithcode.com/paper/groupreduce-block-wise-low-rank-approximation |
| Repo | |
| Framework | |
Approach for Semi-automatic Construction of Anti-infective Drug Ontology Based on Entity Linking
| Title | Approach for Semi-automatic Construction of Anti-infective Drug Ontology Based on Entity Linking |
| Authors | Ying Shen, Yang Deng, Kaiqi Yuan, Li Liu, Yong Liu |
| Abstract | Ontology can be used for the interpretation of natural language. To construct an anti-infective drug ontology, one needs to design and deploy a methodological step to carry out the entity discovery and linking. Medical synonym resources have been an important part of medical natural language processing (NLP). However, there are problems such as low precision and low recall rate. In this study, an NLP approach is adopted to generate candidate entities. Open ontology is analyzed to extract semantic relations. Six-word vector features and word-level features are selected to perform the entity linking. The extraction results of synonyms with a single feature and different combinations of features are studied. Experiments show that our selected features have achieved a precision rate of 86.77%, a recall rate of 89.03% and an F1 score of 87.89%. This paper finally presents the structure of the proposed ontology and its relevant statistical data. |
| Tasks | Entity Linking |
| Published | 2018-12-05 |
| URL | http://arxiv.org/abs/1812.01887v1 |
| http://arxiv.org/pdf/1812.01887v1.pdf | |
| PWC | https://paperswithcode.com/paper/approach-for-semi-automatic-construction-of |
| Repo | |
| Framework | |
An FPGA Implementation of a Time Delay Reservoir Using Stochastic Logic
| Title | An FPGA Implementation of a Time Delay Reservoir Using Stochastic Logic |
| Authors | Lisa Loomis, Nathan McDonald, Cory Merkel |
| Abstract | This paper presents and demonstrates a stochastic logic time delay reservoir design in FPGA hardware. The reservoir network approach is analyzed using a number of metrics, such as kernel quality, generalization rank, performance on simple benchmarks, and is also compared to a deterministic design. A novel re-seeding method is introduced to reduce the adverse effects of stochastic noise, which may also be implemented in other stochastic logic reservoir computing designs, such as echo state networks. Benchmark results indicate that the proposed design performs well on noise-tolerant classification problems, but more work needs to be done to improve the stochastic logic time delay reservoirs robustness for regression problems. In addition, we show that the stochastic design can significantly reduce area cost if the conversion between binary and stochastic representations implemented efficiently. |
| Tasks | |
| Published | 2018-09-12 |
| URL | http://arxiv.org/abs/1809.05407v1 |
| http://arxiv.org/pdf/1809.05407v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-fpga-implementation-of-a-time-delay |
| Repo | |
| Framework | |
Tips, guidelines and tools for managing multi-label datasets: the mldr.datasets R package and the Cometa data repository
| Title | Tips, guidelines and tools for managing multi-label datasets: the mldr.datasets R package and the Cometa data repository |
| Authors | Francisco Charte, Antonio J. Rivera, David Charte, María J. del Jesus, Francisco Herrera |
| Abstract | New proposals in the field of multi-label learning algorithms have been growing in number steadily over the last few years. The experimentation associated with each of them always goes through the same phases: selection of datasets, partitioning, training, analysis of results and, finally, comparison with existing methods. This last step is often hampered since it involves using exactly the same datasets, partitioned in the same way and using the same validation strategy. In this paper we present a set of tools whose objective is to facilitate the management of multi-label datasets, aiming to standardize the experimentation procedure. The two main tools are an R package, mldr.datasets, and a web repository with datasets, Cometa. Together, these tools will simplify the collection of datasets, their partitioning, documentation and export to multiple formats, among other functions. Some tips, recommendations and guidelines for a good experimental analysis of multi-label methods are also presented. |
| Tasks | Multi-Label Learning |
| Published | 2018-02-10 |
| URL | http://arxiv.org/abs/1802.03568v1 |
| http://arxiv.org/pdf/1802.03568v1.pdf | |
| PWC | https://paperswithcode.com/paper/tips-guidelines-and-tools-for-managing-multi |
| Repo | |
| Framework | |
Feature Trajectory Dynamic Time Warping for Clustering of Speech Segments
| Title | Feature Trajectory Dynamic Time Warping for Clustering of Speech Segments |
| Authors | Lerato Lerato, Thomas Niesler |
| Abstract | Dynamic time warping (DTW) can be used to compute the similarity between two sequences of generally differing length. We propose a modification to DTW that performs individual and independent pairwise alignment of feature trajectories. The modified technique, termed feature trajectory dynamic time warping (FTDTW), is applied as a similarity measure in the agglomerative hierarchical clustering of speech segments. Experiments using MFCC and PLP parametrisations extracted from TIMIT and from the Spoken Arabic Digit Dataset (SADD) show consistent and statistically significant improvements in the quality of the resulting clusters in terms of F-measure and normalised mutual information (NMI). |
| Tasks | |
| Published | 2018-10-30 |
| URL | http://arxiv.org/abs/1810.12722v1 |
| http://arxiv.org/pdf/1810.12722v1.pdf | |
| PWC | https://paperswithcode.com/paper/feature-trajectory-dynamic-time-warping-for |
| Repo | |
| Framework | |
SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners
| Title | SCSP: Spectral Clustering Filter Pruning with Soft Self-adaption Manners |
| Authors | Huiyuan Zhuo, Xuelin Qian, Yanwei Fu, Heng Yang, Xiangyang Xue |
| Abstract | Deep Convolutional Neural Networks (CNN) has achieved significant success in computer vision field. However, the high computational cost of the deep complex models prevents the deployment on edge devices with limited memory and computational resource. In this paper, we proposed a novel filter pruning for convolutional neural networks compression, namely spectral clustering filter pruning with soft self-adaption manners (SCSP). We first apply spectral clustering on filters layer by layer to explore their intrinsic connections and only count on efficient groups. By self-adaption manners, the pruning operations can be done in few epochs to let the network gradually choose meaningful groups. According to this strategy, we not only achieve model compression while keeping considerable performance, but also find a novel angle to interpret the model compression process. |
| Tasks | Model Compression |
| Published | 2018-06-14 |
| URL | http://arxiv.org/abs/1806.05320v1 |
| http://arxiv.org/pdf/1806.05320v1.pdf | |
| PWC | https://paperswithcode.com/paper/scsp-spectral-clustering-filter-pruning-with |
| Repo | |
| Framework | |
Attributes’ Importance for Zero-Shot Pose-Classification Based on Wearable Sensors
| Title | Attributes’ Importance for Zero-Shot Pose-Classification Based on Wearable Sensors |
| Authors | Hiroki Ohashi, Mohammad Al-Naser, Sheraz Ahmed, Katsuyuki Nakamura, Takuto Sato, Andreas Dengel |
| Abstract | This paper presents a simple yet effective method for improving the performance of zero-shot learning (ZSL). ZSL classifies instances of unseen classes, from which no training data is available, by utilizing the attributes of the classes. Conventional ZSL methods have equally dealt with all the available attributes, but this sometimes causes misclassification. This is because an attribute that is effective for classifying instances of one class is not always effective for another class. In this case, a metric of classifying the latter class can be undesirably influenced by the irrelevant attribute. This paper solves this problem by taking the importance of each attribute for each class into account when calculating the metric. In addition to the proposal of this new method, this paper also contributes by providing a dataset for pose classification based on wearable sensors, named HDPoseDS. It contains 22 classes of poses performed by 10 subjects with 31 IMU sensors across full body. To the best of our knowledge, it is the richest wearable-sensor dataset especially in terms of sensor density, and thus it is suitable for studying zero-shot pose/action recognition. The presented method was evaluated on HDPoseDS and outperformed relative improvement of 5.9% in comparison to the best baseline method. |
| Tasks | Temporal Action Localization, Zero-Shot Learning |
| Published | 2018-08-02 |
| URL | http://arxiv.org/abs/1808.01358v1 |
| http://arxiv.org/pdf/1808.01358v1.pdf | |
| PWC | https://paperswithcode.com/paper/attributes-importance-for-zero-shot-pose |
| Repo | |
| Framework | |
Binary Space Partitioning as Intrinsic Reward
| Title | Binary Space Partitioning as Intrinsic Reward |
| Authors | Wojciech Skaba |
| Abstract | An autonomous agent embodied in a humanoid robot, in order to learn from the overwhelming flow of raw and noisy sensory, has to effectively reduce the high spatial-temporal data dimensionality. In this paper we propose a novel method of unsupervised feature extraction and selection with binary space partitioning, followed by a computation of information gain that is interpreted as intrinsic reward, then applied as immediate-reward signal for the reinforcement-learning. The space partitioning is executed by tiny codelets running on a simulated Turing Machine. The features are represented by concept nodes arranged in a hierarchy, in which those of a lower level become the input vectors of a higher level. |
| Tasks | |
| Published | 2018-04-10 |
| URL | http://arxiv.org/abs/1804.03611v1 |
| http://arxiv.org/pdf/1804.03611v1.pdf | |
| PWC | https://paperswithcode.com/paper/binary-space-partitioning-as-intrinsic-reward |
| Repo | |
| Framework | |
Direction Finding Based on Multi-Step Knowledge-Aided Iterative Conjugate Gradient Algorithms
| Title | Direction Finding Based on Multi-Step Knowledge-Aided Iterative Conjugate Gradient Algorithms |
| Authors | S. Pinto, R. C. de Lamare |
| Abstract | In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwanted terms found in the estimated covariance matrix of the sensor data. Furthermore, we develop a version of MS-KAI-CG equipped with forward-backward averaging, called MS-KAI-CG-FB, which is appropriate for scenarios with correlated signals. Unlike current knowledge-aided methods, which take advantage of known DoAs to enhance the estimation of the covariance matrix of the input data, the MS-KAI-CG algorithms take advantage of the knowledge of the structure of the forward-backward smoothed covariance matrix and its disturbance terms. Simulations with both uncorrelated and correlated signals show that the MS-KAI-CG algorithms outperform existing techniques. |
| Tasks | |
| Published | 2018-12-16 |
| URL | http://arxiv.org/abs/1812.07505v1 |
| http://arxiv.org/pdf/1812.07505v1.pdf | |
| PWC | https://paperswithcode.com/paper/direction-finding-based-on-multi-step |
| Repo | |
| Framework | |
Retraining-Based Iterative Weight Quantization for Deep Neural Networks
| Title | Retraining-Based Iterative Weight Quantization for Deep Neural Networks |
| Authors | Dongsoo Lee, Byeongwook Kim |
| Abstract | Model compression has gained a lot of attention due to its ability to reduce hardware resource requirements significantly while maintaining accuracy of DNNs. Model compression is especially useful for memory-intensive recurrent neural networks because smaller memory footprint is crucial not only for reducing storage requirement but also for fast inference operations. Quantization is known to be an effective model compression method and researchers are interested in minimizing the number of bits to represent parameters. In this work, we introduce an iterative technique to apply quantization, presenting high compression ratio without any modifications to the training algorithm. In the proposed technique, weight quantization is followed by retraining the model with full precision weights. We show that iterative retraining generates new sets of weights which can be quantized with decreasing quantization loss at each iteration. We also show that quantization is efficiently able to leverage pruning, another effective model compression method. Implementation issues on combining the two methods are also addressed. Our experimental results demonstrate that an LSTM model using 1-bit quantized weights is sufficient for PTB dataset without any accuracy degradation while previous methods demand at least 2-4 bits for quantized weights. |
| Tasks | Model Compression, Quantization |
| Published | 2018-05-29 |
| URL | http://arxiv.org/abs/1805.11233v1 |
| http://arxiv.org/pdf/1805.11233v1.pdf | |
| PWC | https://paperswithcode.com/paper/retraining-based-iterative-weight |
| Repo | |
| Framework | |
Learning Existing Social Conventions via Observationally Augmented Self-Play
| Title | Learning Existing Social Conventions via Observationally Augmented Self-Play |
| Authors | Adam Lerer, Alexander Peysakhovich |
| Abstract | In order for artificial agents to coordinate effectively with people, they must act consistently with existing conventions (e.g. how to navigate in traffic, which language to speak, or how to coordinate with teammates). A group’s conventions can be viewed as a choice of equilibrium in a coordination game. We consider the problem of an agent learning a policy for a coordination game in a simulated environment and then using this policy when it enters an existing group. When there are multiple possible conventions we show that learning a policy via multi-agent reinforcement learning (MARL) is likely to find policies which achieve high payoffs at training time but fail to coordinate with the real group into which the agent enters. We assume access to a small number of samples of behavior from the true convention and show that we can augment the MARL objective to help it find policies consistent with the real group’s convention. In three environments from the literature - traffic, communication, and team coordination - we observe that augmenting MARL with a small amount of imitation learning greatly increases the probability that the strategy found by MARL fits well with the existing social convention. We show that this works even in an environment where standard training methods very rarely find the true convention of the agent’s partners. |
| Tasks | Imitation Learning, Multi-agent Reinforcement Learning |
| Published | 2018-06-26 |
| URL | http://arxiv.org/abs/1806.10071v3 |
| http://arxiv.org/pdf/1806.10071v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-existing-social-conventions-in |
| Repo | |
| Framework | |
New Convergence Aspects of Stochastic Gradient Algorithms
| Title | New Convergence Aspects of Stochastic Gradient Algorithms |
| Authors | Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik, Katya Scheinberg, Martin Takáč, Marten van Dijk |
| Abstract | The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss functions, it is violated for cases where the objective function is strongly convex. In Bottou et al. (2018), a new analysis of convergence of SGD is performed under the assumption that stochastic gradients are bounded with respect to the true gradient norm. We show that for stochastic problems arising in machine learning such bound always holds; and we also propose an alternative convergence analysis of SGD with diminishing learning rate regime. We then move on to the asynchronous parallel setting, and prove convergence of Hogwild! algorithm in the same regime in the case of diminished learning rate. It is well-known that SGD converges if a sequence of learning rates ${\eta_t}$ satisfies $\sum_{t=0}^\infty \eta_t \rightarrow \infty$ and $\sum_{t=0}^\infty \eta^2_t < \infty$. We show the convergence of SGD for strongly convex objective function without using bounded gradient assumption when ${\eta_t}$ is a diminishing sequence and $\sum_{t=0}^\infty \eta_t \rightarrow \infty$. In other words, we extend the current state-of-the-art class of learning rates satisfying the convergence of SGD. |
| Tasks | |
| Published | 2018-11-10 |
| URL | https://arxiv.org/abs/1811.12403v2 |
| https://arxiv.org/pdf/1811.12403v2.pdf | |
| PWC | https://paperswithcode.com/paper/new-convergence-aspects-of-stochastic |
| Repo | |
| Framework | |
Deep Learning Towards Mobile Applications
| Title | Deep Learning Towards Mobile Applications |
| Authors | Ji Wang, Bokai Cao, Philip S. Yu, Lichao Sun, Weidong Bao, Xiaomin Zhu |
| Abstract | Recent years have witnessed an explosive growth of mobile devices. Mobile devices are permeating every aspect of our daily lives. With the increasing usage of mobile devices and intelligent applications, there is a soaring demand for mobile applications with machine learning services. Inspired by the tremendous success achieved by deep learning in many machine learning tasks, it becomes a natural trend to push deep learning towards mobile applications. However, there exist many challenges to realize deep learning in mobile applications, including the contradiction between the miniature nature of mobile devices and the resource requirement of deep neural networks, the privacy and security concerns about individuals’ data, and so on. To resolve these challenges, during the past few years, great leaps have been made in this area. In this paper, we provide an overview of the current challenges and representative achievements about pushing deep learning on mobile devices from three aspects: training with mobile data, efficient inference on mobile devices, and applications of mobile deep learning. The former two aspects cover the primary tasks of deep learning. Then, we go through our two recent applications that apply the data collected by mobile devices to inferring mood disturbance and user identification. Finally, we conclude this paper with the discussion of the future of this area. |
| Tasks | |
| Published | 2018-09-10 |
| URL | http://arxiv.org/abs/1809.03559v1 |
| http://arxiv.org/pdf/1809.03559v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-towards-mobile-applications |
| Repo | |
| Framework | |
Learning the Localization Function: Machine Learning Approach to Fingerprinting Localization
| Title | Learning the Localization Function: Machine Learning Approach to Fingerprinting Localization |
| Authors | Linchen Xiao, Arash Behboodi, Rudolf Mathar |
| Abstract | Considered as a data-driven approach, Fingerprinting Localization Solutions (FPSs) enjoy huge popularity due to their good performance and minimal environment information requirement. This papers addresses applications of artificial intelligence to solve two problems in Received Signal Strength Indicator (RSSI) based FPS, first the cumbersome training database construction and second the extrapolation of fingerprinting algorithm for similar buildings with slight environmental changes. After a concise overview of deep learning design techniques, two main techniques widely used in deep learning are exploited for the above mentioned issues namely data augmentation and transfer learning. We train a multi-layer neural network that learns the mapping from the observations to the locations. A data augmentation method is proposed to increase the training database size based on the structure of RSSI measurements and hence reducing effectively the amount of training data. Then it is shown experimentally how a model trained for a particular building can be transferred to a similar one by fine tuning with significantly smaller training numbers. The paper implicitly discusses the new guidelines to consider about deep learning designs when they are employed in a new application context. |
| Tasks | Data Augmentation, Transfer Learning |
| Published | 2018-03-21 |
| URL | http://arxiv.org/abs/1803.08153v1 |
| http://arxiv.org/pdf/1803.08153v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-the-localization-function-machine |
| Repo | |
| Framework | |
