Paper Group ANR 1386
A Convolutional Neural Network model based on Neutrosophy for Noisy Speech Recognition. Dynamical Systems as Temporal Feature Spaces. Measuring the Similarity between Materials with an Emphasis on the Materials Distinctiveness. Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting. SCNN: A General Distribution based Statistical Conv …
A Convolutional Neural Network model based on Neutrosophy for Noisy Speech Recognition
| Title | A Convolutional Neural Network model based on Neutrosophy for Noisy Speech Recognition |
| Authors | Elyas Rashno, Ahmad Akbari, Babak Nasersharif |
| Abstract | Convolutional neural networks are sensitive to unknown noisy condition in the test phase and so their performance degrades for the noisy data classification task including noisy speech recognition. In this research, a new convolutional neural network (CNN) model with data uncertainty handling; referred as NCNN (Neutrosophic Convolutional Neural Network); is proposed for classification task. Here, speech signals are used as input data and their noise is modeled as uncertainty. In this task, using speech spectrogram, a definition of uncertainty is proposed in neutrosophic (NS) domain. Uncertainty is computed for each Time-frequency point of speech spectrogram as like a pixel. Therefore, uncertainty matrix with the same size of spectrogram is created in NS domain. In the next step, a two parallel paths CNN classification model is proposed. Speech spectrogram is used as input of the first path and uncertainty matrix for the second path. The outputs of two paths are combined to compute the final output of the classifier. To show the effectiveness of the proposed method, it has been compared with conventional CNN on the isolated words of Aurora2 dataset. The proposed method achieves the average accuracy of 85.96 in noisy train data. It is more robust against Car, Airport and Subway noises with accuracies 90, 88 and 81 in test sets A, B and C, respectively. Results show that the proposed method outperforms conventional CNN with the improvement of 6, 5 and 2 percentage in test set A, test set B and test sets C, respectively. It means that the proposed method is more robust against noisy data and handle these data effectively. |
| Tasks | Noisy Speech Recognition, Speech Recognition |
| Published | 2019-01-27 |
| URL | http://arxiv.org/abs/1901.10629v2 |
| http://arxiv.org/pdf/1901.10629v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-convolutional-neural-network-model-based-on |
| Repo | |
| Framework | |
Dynamical Systems as Temporal Feature Spaces
| Title | Dynamical Systems as Temporal Feature Spaces |
| Authors | Peter Tino |
| Abstract | Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the state space. We propose a framework for rigorous analysis of such state representations in vanishing memory state space models such as echo state networks (ESN). In particular, we consider the state space a temporal feature space and the readout mapping from the state space a kernel machine operating in that feature space. We show that: (1) The usual ESN strategy of randomly generating input-to-state, as well as state coupling leads to shallow memory time series representations, corresponding to cross-correlation operator with fast exponentially decaying coefficients; (2) Imposing symmetry on dynamic coupling yields a constrained dynamic kernel matching the input time series with straightforward exponentially decaying motifs or exponentially decaying motifs of the highest frequency; (3) Simple cycle high-dimensional reservoir topology specified only through two free parameters can implement deep memory dynamic kernels with a rich variety of matching motifs. We quantify richness of feature representations imposed by dynamic kernels and demonstrate that for dynamic kernel associated with cycle reservoir topology, the kernel richness undergoes a phase transition close to the edge of stability. |
| Tasks | Time Series |
| Published | 2019-07-15 |
| URL | https://arxiv.org/abs/1907.06382v3 |
| https://arxiv.org/pdf/1907.06382v3.pdf | |
| PWC | https://paperswithcode.com/paper/dynamical-systems-as-temporal-feature-spaces |
| Repo | |
| Framework | |
Measuring the Similarity between Materials with an Emphasis on the Materials Distinctiveness
| Title | Measuring the Similarity between Materials with an Emphasis on the Materials Distinctiveness |
| Authors | Tran-Thai Dang, Tien-Lam Pham, Hiori Kino, Takashi Miyake, Hieu-Chi Dam |
| Abstract | In this study, we establish a basis for selecting similarity measures when applying machine learning techniques to solve materials science problems. This selection is considered with an emphasis on the distinctiveness between materials that reflect their nature well. We perform a case study with a dataset of rare-earth transition metal crystalline compounds represented using the Orbital Field Matrix descriptor and the Coulomb Matrix descriptor. We perform predictions of the formation energies using k-nearest neighbors regression, ridge regression, and kernel ridge regression. Through detailed analyses of the yield prediction accuracy, we examine the relationship between the characteristics of the material representation and similarity measures, and the complexity of the energy function they can capture. Empirical experiments and theoretical analysis reveal that similarity measures and kernels that minimize the loss of materials distinctiveness improve the prediction performance. |
| Tasks | |
| Published | 2019-03-23 |
| URL | http://arxiv.org/abs/1903.10867v1 |
| http://arxiv.org/pdf/1903.10867v1.pdf | |
| PWC | https://paperswithcode.com/paper/measuring-the-similarity-between-materials |
| Repo | |
| Framework | |
Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting
| Title | Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting |
| Authors | Jean Mercat, Thomas Gilles, Nicole El Zoghby, Guillaume Sandou, Dominique Beauvois, Guillermo Pita Gil |
| Abstract | This paper presents a novel vehicle motion forecasting method based on multi-head attention. It produces joint forecasts for all vehicles on a road scene as sequences of multi-modal probability density functions of their positions. Its architecture uses multi-head attention to account for complete interactions between all vehicles, and long short-term memory layers for encoding and forecasting. It relies solely on vehicle position tracks, does not need maneuver definitions, and does not represent the scene with a spatial grid. This allows it to be more versatile than similar model while combining any forecasting capabilities, namely joint forecast with interactions, uncertainty estimation, and multi-modality. The resulting prediction likelihood outperforms state-of-the-art models on the same dataset. |
| Tasks | Motion Forecasting |
| Published | 2019-10-08 |
| URL | https://arxiv.org/abs/1910.03650v3 |
| https://arxiv.org/pdf/1910.03650v3.pdf | |
| PWC | https://paperswithcode.com/paper/multi-modal-simultaneous-forecasting-of |
| Repo | |
| Framework | |
SCNN: A General Distribution based Statistical Convolutional Neural Network with Application to Video Object Detection
| Title | SCNN: A General Distribution based Statistical Convolutional Neural Network with Application to Video Object Detection |
| Authors | Tianchen Wang, Jinjun Xiong, Xiaowei Xu, Yiyu Shi |
| Abstract | Various convolutional neural networks (CNNs) were developed recently that achieved accuracy comparable with that of human beings in computer vision tasks such as image recognition, object detection and tracking, etc. Most of these networks, however, process one single frame of image at a time, and may not fully utilize the temporal and contextual correlation typically present in multiple channels of the same image or adjacent frames from a video, thus limiting the achievable throughput. This limitation stems from the fact that existing CNNs operate on deterministic numbers. In this paper, we propose a novel statistical convolutional neural network (SCNN), which extends existing CNN architectures but operates directly on correlated distributions rather than deterministic numbers. By introducing a parameterized canonical model to model correlated data and defining corresponding operations as required for CNN training and inference, we show that SCNN can process multiple frames of correlated images effectively, hence achieving significant speedup over existing CNN models. We use a CNN based video object detection as an example to illustrate the usefulness of the proposed SCNN as a general network model. Experimental results show that even a non-optimized implementation of SCNN can still achieve 178% speedup over existing CNNs with slight accuracy degradation. |
| Tasks | Object Detection, Video Object Detection |
| Published | 2019-03-15 |
| URL | http://arxiv.org/abs/1903.07663v1 |
| http://arxiv.org/pdf/1903.07663v1.pdf | |
| PWC | https://paperswithcode.com/paper/scnn-a-general-distribution-based-statistical |
| Repo | |
| Framework | |
AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling
| Title | AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling |
| Authors | Ting-Wu Chin, Ruizhou Ding, Diana Marculescu |
| Abstract | In vision-enabled autonomous systems such as robots and autonomous cars, video object detection plays a crucial role, and both its speed and accuracy are important factors to provide reliable operation. The key insight we show in this paper is that speed and accuracy are not necessarily a trade-off when it comes to image scaling. Our results show that re-scaling the image to a lower resolution will sometimes produce better accuracy. Based on this observation, we propose a novel approach, dubbed AdaScale, which adaptively selects the input image scale that improves both accuracy and speed for video object detection. To this end, our results on ImageNet VID and mini YouTube-BoundingBoxes datasets demonstrate 1.3 points and 2.7 points mAP improvement with 1.6x and 1.8x speedup, respectively. Additionally, we improve state-of-the-art video acceleration work by an extra 1.25x speedup with slightly better mAP on ImageNet VID dataset. |
| Tasks | Object Detection, Video Object Detection |
| Published | 2019-02-08 |
| URL | http://arxiv.org/abs/1902.02910v1 |
| http://arxiv.org/pdf/1902.02910v1.pdf | |
| PWC | https://paperswithcode.com/paper/adascale-towards-real-time-video-object |
| Repo | |
| Framework | |
Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG
| Title | Hierarchical Deep Feature Learning For Decoding Imagined Speech From EEG |
| Authors | Pramit Saha, Sidney Fels |
| Abstract | We propose a mixed deep neural network strategy, incorporating parallel combination of Convolutional (CNN) and Recurrent Neural Networks (RNN), cascaded with deep autoencoders and fully connected layers towards automatic identification of imagined speech from EEG. Instead of utilizing raw EEG channel data, we compute the joint variability of the channels in the form of a covariance matrix that provide spatio-temporal representations of EEG. The networks are trained hierarchically and the extracted features are passed onto the next network hierarchy until the final classification. Using a publicly available EEG based speech imagery database we demonstrate around 23.45% improvement of accuracy over the baseline method. Our approach demonstrates the promise of a mixed DNN approach for complex spatial-temporal classification problems. |
| Tasks | EEG |
| Published | 2019-04-08 |
| URL | http://arxiv.org/abs/1904.04352v1 |
| http://arxiv.org/pdf/1904.04352v1.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-deep-feature-learning-for |
| Repo | |
| Framework | |
Surrogate Optimization of Deep Neural Networks for Groundwater Predictions
| Title | Surrogate Optimization of Deep Neural Networks for Groundwater Predictions |
| Authors | Juliane Mueller, Jangho Park, Reetik Sahu, Charuleka Varadharajan, Bhavna Arora, Boris Faybishenko, Deborah Agarwal |
| Abstract | Sustainable management of groundwater resources under changing climatic conditions require an application of reliable and accurate predictions of groundwater levels. Mechanistic multi-scale, multi-physics simulation models are often too hard to use for this purpose, especially for groundwater managers who do not have access to the complex compute resources and data. Therefore, we analyzed the applicability and performance of four modern deep learning computational models for predictions of groundwater levels. We compare three methods for optimizing the models’ hyperparameters, including two surrogate model-based algorithms and a random sampling method. The models were tested using predictions of the groundwater level in Butte County, California, USA, taking into account the temporal variability of streamflow, precipitation, and ambient temperature. Our numerical study shows that the optimization of the hyperparameters can lead to reasonably accurate performance of all models (root mean squared errors of groundwater predictions of 2 meters or less), but the ‘‘simplest’’ network, namely a multilayer perceptron (MLP) performs overall better for learning and predicting groundwater data than the more advanced long short-term memory or convolutional neural networks in terms of prediction accuracy and time-to-solution, making the MLP a suitable candidate for groundwater prediction. |
| Tasks | |
| Published | 2019-08-28 |
| URL | https://arxiv.org/abs/1908.10947v3 |
| https://arxiv.org/pdf/1908.10947v3.pdf | |
| PWC | https://paperswithcode.com/paper/surrogate-optimization-of-deep-neural |
| Repo | |
| Framework | |
Generalized Off-Policy Actor-Critic
| Title | Generalized Off-Policy Actor-Critic |
| Authors | Shangtong Zhang, Wendelin Boehmer, Shimon Whiteson |
| Abstract | We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our new objective better predicts such performance. We prove the Generalized Off-Policy Policy Gradient Theorem to compute the policy gradient of the counterfactual objective and use an emphatic approach to get an unbiased sample from this policy gradient, yielding the Generalized Off-Policy Actor-Critic (Geoff-PAC) algorithm. We demonstrate the merits of Geoff-PAC over existing algorithms in Mujoco robot simulation tasks, the first empirical success of emphatic algorithms in prevailing deep RL benchmarks. |
| Tasks | |
| Published | 2019-03-27 |
| URL | https://arxiv.org/abs/1903.11329v8 |
| https://arxiv.org/pdf/1903.11329v8.pdf | |
| PWC | https://paperswithcode.com/paper/generalized-off-policy-actor-critic |
| Repo | |
| Framework | |
Controlling Model Complexity in Probabilistic Model-Based Dynamic Optimization of Neural Network Structures
| Title | Controlling Model Complexity in Probabilistic Model-Based Dynamic Optimization of Neural Network Structures |
| Authors | Shota Saito, Shinichi Shirakawa |
| Abstract | A method of simultaneously optimizing both the structure of neural networks and the connection weights in a single training loop can reduce the enormous computational cost of neural architecture search. We focus on the probabilistic model-based dynamic neural network structure optimization that considers the probability distribution of structure parameters and simultaneously optimizes both the distribution parameters and connection weights based on gradient methods. Since the existing algorithm searches for the structures that only minimize the training loss, this method might find overly complicated structures. In this paper, we propose the introduction of a penalty term to control the model complexity of obtained structures. We formulate a penalty term using the number of weights or units and derive its analytical natural gradient. The proposed method minimizes the objective function injected the penalty term based on the stochastic gradient descent. We apply the proposed method in the unit selection of a fully-connected neural network and the connection selection of a convolutional neural network. The experimental results show that the proposed method can control model complexity while maintaining performance. |
| Tasks | Neural Architecture Search |
| Published | 2019-07-15 |
| URL | https://arxiv.org/abs/1907.06341v1 |
| https://arxiv.org/pdf/1907.06341v1.pdf | |
| PWC | https://paperswithcode.com/paper/controlling-model-complexity-in-probabilistic |
| Repo | |
| Framework | |
Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently
| Title | Manifold Gradient Descent Solves Multi-Channel Sparse Blind Deconvolution Provably and Efficiently |
| Authors | Laixi Shi, Yuejie Chi |
| Abstract | Multi-channel sparse blind deconvolution, or convolutional sparse coding, refers to the problem of learning an unknown filter by observing its circulant convolutions with multiple input signals that are sparse. This problem finds numerous applications in signal processing, computer vision, and inverse problems. However, it is challenging to learn the filter efficiently due to the bilinear structure of the observations with respect to the unknown filter and inputs, leading to global ambiguities of identification. In this paper, we propose a novel approach based on nonconvex optimization over the sphere manifold by minimizing a smooth surrogate of the sparsity-promoting loss function. It is demonstrated that the manifold gradient descent with random initializations will provably recover the filter, up to scaling and shift ambiguity, as soon as the number of observations is sufficiently large under an appropriate random data model. Numerical experiments are provided to illustrate the performance of the proposed method with comparisons to existing methods. |
| Tasks | |
| Published | 2019-11-25 |
| URL | https://arxiv.org/abs/1911.11167v1 |
| https://arxiv.org/pdf/1911.11167v1.pdf | |
| PWC | https://paperswithcode.com/paper/manifold-gradient-descent-solves-multi |
| Repo | |
| Framework | |
On the Efficacy of Knowledge Distillation
| Title | On the Efficacy of Knowledge Distillation |
| Authors | Jang Hyun Cho, Bharath Hariharan |
| Abstract | In this paper, we present a thorough evaluation of the efficacy of knowledge distillation and its dependence on student and teacher architectures. Starting with the observation that more accurate teachers often don’t make good teachers, we attempt to tease apart the factors that affect knowledge distillation performance. We find crucially that larger models do not often make better teachers. We show that this is a consequence of mismatched capacity, and that small students are unable to mimic large teachers. We find typical ways of circumventing this (such as performing a sequence of knowledge distillation steps) to be ineffective. Finally, we show that this effect can be mitigated by stopping the teacher’s training early. Our results generalize across datasets and models. |
| Tasks | |
| Published | 2019-10-03 |
| URL | https://arxiv.org/abs/1910.01348v1 |
| https://arxiv.org/pdf/1910.01348v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-efficacy-of-knowledge-distillation |
| Repo | |
| Framework | |
Development details and computational benchmarking of DEPAM
| Title | Development details and computational benchmarking of DEPAM |
| Authors | Paul Nguyen Hong Duc, Dorian Cazau |
| Abstract | In the big data era of observational oceanography, passive acoustics datasets are becoming too high volume to be processed on local computers due to their processor and memory limitations. As a result there is a current need for our community to turn to cloud-based distributed computing. We present a scalable computing system for FFT (Fast Fourier Transform)-based features (e.g., Power Spectral Density) based on the Apache distributed frameworks Hadoop and Spark. These features are at the core of many different types of acoustic analysis where the need of processing data at scale with speed is evident, e.g. serving as long-term averaged learning representations of soundscapes to identify periods of acoustic interest. In addition to provide a complete description of our system implementation, we also performed a computational benchmark comparing our system to three other Scala-only, Matlab and Python based systems in standalone executions, and evaluated its scalability using the speed up metric. Our current results are very promising in terms of computational performance, as we show that our proposed Hadoop/Spark system performs reasonably well on a single node setup comparatively to state-of-the-art processing tools used by the PAM community, and that it could also fully leverage more intensive cluster resources with a almost-linear scalability behaviour above a certain dataset volume. |
| Tasks | |
| Published | 2019-03-03 |
| URL | https://arxiv.org/abs/1903.06695v2 |
| https://arxiv.org/pdf/1903.06695v2.pdf | |
| PWC | https://paperswithcode.com/paper/development-details-and-computational |
| Repo | |
| Framework | |
Multi-Objective Generalized Linear Bandits
| Title | Multi-Objective Generalized Linear Bandits |
| Authors | Shiyin Lu, Guanghui Wang, Yao Hu, Lijun Zhang |
| Abstract | In this paper, we study the multi-objective bandits (MOB) problem, where a learner repeatedly selects one arm to play and then receives a reward vector consisting of multiple objectives. MOB has found many real-world applications as varied as online recommendation and network routing. On the other hand, these applications typically contain contextual information that can guide the learning process which, however, is ignored by most of existing work. To utilize this information, we associate each arm with a context vector and assume the reward follows the generalized linear model (GLM). We adopt the notion of Pareto regret to evaluate the learner’s performance and develop a novel algorithm for minimizing it. The essential idea is to apply a variant of the online Newton step to estimate model parameters, based on which we utilize the upper confidence bound (UCB) policy to construct an approximation of the Pareto front, and then uniformly at random choose one arm from the approximate Pareto front. Theoretical analysis shows that the proposed algorithm achieves an $\tilde O(d\sqrt{T})$ Pareto regret, where $T$ is the time horizon and $d$ is the dimension of contexts, which matches the optimal result for single objective contextual bandits problem. Numerical experiments demonstrate the effectiveness of our method. |
| Tasks | Multi-Armed Bandits |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.12879v1 |
| https://arxiv.org/pdf/1905.12879v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-objective-generalized-linear-bandits |
| Repo | |
| Framework | |
Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
| Title | Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement |
| Authors | André Barreto, Diana Borsa, John Quan, Tom Schaul, David Silver, Matteo Hessel, Daniel Mankowitz, Augustin Žídek, Rémi Munos |
| Abstract | The ability to transfer skills across tasks has the potential to scale up reinforcement learning (RL) agents to environments currently out of reach. Recently, a framework based on two ideas, successor features (SFs) and generalised policy improvement (GPI), has been introduced as a principled way of transferring skills. In this paper we extend the SFs & GPI framework in two ways. One of the basic assumptions underlying the original formulation of SFs & GPI is that rewards for all tasks of interest can be computed as linear combinations of a fixed set of features. We relax this constraint and show that the theoretical guarantees supporting the framework can be extended to any set of tasks that only differ in the reward function. Our second contribution is to show that one can use the reward functions themselves as features for future tasks, without any loss of expressiveness, thus removing the need to specify a set of features beforehand. This makes it possible to combine SFs & GPI with deep learning in a more stable way. We empirically verify this claim on a complex 3D environment where observations are images from a first-person perspective. We show that the transfer promoted by SFs & GPI leads to very good policies on unseen tasks almost instantaneously. We also describe how to learn policies specialised to the new tasks in a way that allows them to be added to the agent’s set of skills, and thus be reused in the future. |
| Tasks | |
| Published | 2019-01-30 |
| URL | http://arxiv.org/abs/1901.10964v1 |
| http://arxiv.org/pdf/1901.10964v1.pdf | |
| PWC | https://paperswithcode.com/paper/transfer-in-deep-reinforcement-learning-using |
| Repo | |
| Framework | |
