Paper Group ANR 1021
Functional Gradient Boosting based on Residual Network Perception. An Improved Deep Belief Network Model for Road Safety Analyses. A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition. Large-Scale Sparse Inverse Covariance Estimation via Thresholding and Max-Det Matrix Completion. Improving the Privacy and …
Functional Gradient Boosting based on Residual Network Perception
| Title | Functional Gradient Boosting based on Residual Network Perception |
| Authors | Atsushi Nitanda, Taiji Suzuki |
| Abstract | Residual Networks (ResNets) have become state-of-the-art models in deep learning and several theoretical studies have been devoted to understanding why ResNet works so well. One attractive viewpoint on ResNet is that it is optimizing the risk in a functional space by combining an ensemble of effective features. In this paper, we adopt this viewpoint to construct a new gradient boosting method, which is known to be very powerful in data analysis. To do so, we formalize the gradient boosting perspective of ResNet mathematically using the notion of functional gradients and propose a new method called ResFGB for classification tasks by leveraging ResNet perception. Two types of generalization guarantees are provided from the optimization perspective: one is the margin bound and the other is the expected risk bound by the sample-splitting technique. Experimental results show superior performance of the proposed method over state-of-the-art methods such as LightGBM. |
| Tasks | |
| Published | 2018-02-25 |
| URL | http://arxiv.org/abs/1802.09031v2 |
| http://arxiv.org/pdf/1802.09031v2.pdf | |
| PWC | https://paperswithcode.com/paper/functional-gradient-boosting-based-on |
| Repo | |
| Framework | |
An Improved Deep Belief Network Model for Road Safety Analyses
| Title | An Improved Deep Belief Network Model for Road Safety Analyses |
| Authors | Guangyuan Pan, Liping Fu, Lalita Thakali, Matthew Muresan, Ming Yu |
| Abstract | Crash prediction is a critical component of road safety analyses. A widely adopted approach to crash prediction is application of regression based techniques. The underlying calibration process is often time-consuming, requiring significant domain knowledge and expertise and cannot be easily automated. This paper introduces a new machine learning (ML) based approach as an alternative to the traditional techniques. The proposed ML model is called regularized deep belief network, which is a deep neural network with two training steps: it is first trained using an unsupervised learning algorithm and then fine-tuned by initializing a Bayesian neural network with the trained weights from the first step. The resulting model is expected to have improved prediction power and reduced need for the time-consuming human intervention. In this paper, we attempt to demonstrate the potential of this new model for crash prediction through two case studies including a collision data set from 800 km stretch of Highway 401 and other highways in Ontario, Canada. Our intention is to show the performance of this ML approach in comparison to various traditional models including negative binomial (NB) model, kernel regression (KR), and Bayesian neural network (Bayesian NN). We also attempt to address other related issues such as effect of training data size and training parameters. |
| Tasks | Calibration |
| Published | 2018-12-17 |
| URL | http://arxiv.org/abs/1812.07410v1 |
| http://arxiv.org/pdf/1812.07410v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-improved-deep-belief-network-model-for |
| Repo | |
| Framework | |
A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition
| Title | A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition |
| Authors | Ahmadreza Ahmadi, Jun Tani |
| Abstract | This study introduces PV-RNN, a novel variational RNN inspired by the predictive-coding ideas. The model learns to extract the probabilistic structures hidden in fluctuating temporal patterns by dynamically changing the stochasticity of its latent states. Its architecture attempts to address two major concerns of variational Bayes RNNs: how can latent variables learn meaningful representations and how can the inference model transfer future observations to the latent variables. PV-RNN does both by introducing adaptive vectors mirroring the training data, whose values can then be adapted differently during evaluation. Moreover, prediction errors during backpropagation, rather than external inputs during the forward computation, are used to convey information to the network about the external data. For testing, we introduce error regression for predicting unseen sequences as inspired by predictive coding that leverages those mechanisms. The model introduces a weighting parameter, the meta-prior, to balance the optimization pressure placed on two terms of a lower bound on the marginal likelihood of the sequential data. We test the model on two datasets with probabilistic structures and show that with high values of the meta-prior the network develops deterministic chaos through which the data’s randomness is imitated. For low values, the model behaves as a random process. The network performs best on intermediate values, and is able to capture the latent probabilistic structure with good generalization. Analyzing the meta-prior’s impact on the network allows to precisely study the theoretical value and practical benefits of incorporating stochastic dynamics in our model. We demonstrate better prediction performance on a robot imitation task with our model using error regression compared to a standard variational Bayes model lacking such a procedure. |
| Tasks | |
| Published | 2018-11-04 |
| URL | https://arxiv.org/abs/1811.01339v3 |
| https://arxiv.org/pdf/1811.01339v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-embed-probabilistic-structures |
| Repo | |
| Framework | |
Large-Scale Sparse Inverse Covariance Estimation via Thresholding and Max-Det Matrix Completion
| Title | Large-Scale Sparse Inverse Covariance Estimation via Thresholding and Max-Det Matrix Completion |
| Authors | Richard Y. Zhang, Salar Fattahi, Somayeh Sojoudi |
| Abstract | The sparse inverse covariance estimation problem is commonly solved using an $\ell_{1}$-regularized Gaussian maximum likelihood estimator known as “graphical lasso”, but its computational cost becomes prohibitive for large data sets. A recent line of results showed–under mild assumptions–that the graphical lasso estimator can be retrieved by soft-thresholding the sample covariance matrix and solving a maximum determinant matrix completion (MDMC) problem. This paper proves an extension of this result, and describes a Newton-CG algorithm to efficiently solve the MDMC problem. Assuming that the thresholded sample covariance matrix is sparse with a sparse Cholesky factorization, we prove that the algorithm converges to an $\epsilon$-accurate solution in $O(n\log(1/\epsilon))$ time and $O(n)$ memory. The algorithm is highly efficient in practice: we solve the associated MDMC problems with as many as 200,000 variables to 7-9 digits of accuracy in less than an hour on a standard laptop computer running MATLAB. |
| Tasks | Matrix Completion |
| Published | 2018-02-14 |
| URL | http://arxiv.org/abs/1802.04911v3 |
| http://arxiv.org/pdf/1802.04911v3.pdf | |
| PWC | https://paperswithcode.com/paper/large-scale-sparse-inverse-covariance |
| Repo | |
| Framework | |
Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms
| Title | Improving the Privacy and Accuracy of ADMM-Based Distributed Algorithms |
| Authors | Xueru Zhang, Mohammad Mahdi Khalili, Mingyan Liu |
| Abstract | Alternating direction method of multiplier (ADMM) is a popular method used to design distributed versions of a machine learning algorithm, whereby local computations are performed on local data with the output exchanged among neighbors in an iterative fashion. During this iterative process the leakage of data privacy arises. A differentially private ADMM was proposed in prior work (Zhang & Zhu, 2017) where only the privacy loss of a single node during one iteration was bounded, a method that makes it difficult to balance the tradeoff between the utility attained through distributed computation and privacy guarantees when considering the total privacy loss of all nodes over the entire iterative process. We propose a perturbation method for ADMM where the perturbed term is correlated with the penalty parameters; this is shown to improve the utility and privacy simultaneously. The method is based on a modified ADMM where each node independently determines its own penalty parameter in every iteration and decouples it from the dual updating step size. The condition for convergence of the modified ADMM and the lower bound on the convergence rate are also derived. |
| Tasks | |
| Published | 2018-06-06 |
| URL | http://arxiv.org/abs/1806.02246v1 |
| http://arxiv.org/pdf/1806.02246v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-the-privacy-and-accuracy-of-admm |
| Repo | |
| Framework | |
Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer
| Title | Unbiasing Semantic Segmentation For Robot Perception using Synthetic Data Feature Transfer |
| Authors | Jonathan C Balloch, Varun Agrawal, Irfan Essa, Sonia Chernova |
| Abstract | Robot perception systems need to perform reliable image segmentation in real-time on noisy, raw perception data. State-of-the-art segmentation approaches use large CNN models and carefully constructed datasets; however, these models focus on accuracy at the cost of real-time inference. Furthermore, the standard semantic segmentation datasets are not large enough for training CNNs without augmentation and are not representative of noisy, uncurated robot perception data. We propose improving the performance of real-time segmentation frameworks on robot perception data by transferring features learned from synthetic segmentation data. We show that pretraining real-time segmentation architectures with synthetic segmentation data instead of ImageNet improves fine-tuning performance by reducing the bias learned in pretraining and closing the \textit{transfer gap} as a result. Our experiments show that our real-time robot perception models pretrained on synthetic data outperform those pretrained on ImageNet for every scale of fine-tuning data examined. Moreover, the degree to which synthetic pretraining outperforms ImageNet pretraining increases as the availability of robot data decreases, making our approach attractive for robotics domains where dataset collection is hard and/or expensive. |
| Tasks | Semantic Segmentation |
| Published | 2018-09-11 |
| URL | http://arxiv.org/abs/1809.03676v1 |
| http://arxiv.org/pdf/1809.03676v1.pdf | |
| PWC | https://paperswithcode.com/paper/unbiasing-semantic-segmentation-for-robot |
| Repo | |
| Framework | |
Discourse-Wizard: Discovering Deep Discourse Structure in your Conversation with RNNs
| Title | Discourse-Wizard: Discovering Deep Discourse Structure in your Conversation with RNNs |
| Authors | Chandrakant Bothe, Sven Magg, Cornelius Weber, Stefan Wermter |
| Abstract | Spoken language understanding is one of the key factors in a dialogue system, and a context in a conversation plays an important role to understand the current utterance. In this work, we demonstrate the importance of context within the dialogue for neural network models through an online web interface live demo. We developed two different neural models: a model that does not use context and a context-based model. The no-context model classifies dialogue acts at an utterance-level whereas the context-based model takes some preceding utterances into account. We make these trained neural models available as a live demo called Discourse-Wizard using a modular server architecture. The live demo provides an easy to use interface for conversational analysis and for discovering deep discourse structures in a conversation. |
| Tasks | Spoken Language Understanding |
| Published | 2018-06-29 |
| URL | http://arxiv.org/abs/1806.11420v1 |
| http://arxiv.org/pdf/1806.11420v1.pdf | |
| PWC | https://paperswithcode.com/paper/discourse-wizard-discovering-deep-discourse |
| Repo | |
| Framework | |
Combining Word Feature Vector Method with the Convolutional Neural Network for Slot Filling in Spoken Language Understanding
| Title | Combining Word Feature Vector Method with the Convolutional Neural Network for Slot Filling in Spoken Language Understanding |
| Authors | Ruixi Lin |
| Abstract | Slot filling is an important problem in Spoken Language Understanding (SLU) and Natural Language Processing (NLP), which involves identifying a user’s intent and assigning a semantic concept to each word in a sentence. This paper presents a word feature vector method and combines it into the convolutional neural network (CNN). We consider 18 word features and each word feature is constructed by merging similar word labels. By introducing the concept of external library, we propose a feature set approach that is beneficial for building the relationship between a word from the training dataset and the feature. Computational results are reported using the ATIS dataset and comparisons with traditional CNN as well as bi-directional sequential CNN are also presented. |
| Tasks | Slot Filling, Spoken Language Understanding |
| Published | 2018-06-18 |
| URL | http://arxiv.org/abs/1806.06874v1 |
| http://arxiv.org/pdf/1806.06874v1.pdf | |
| PWC | https://paperswithcode.com/paper/combining-word-feature-vector-method-with-the |
| Repo | |
| Framework | |
Constructing a Word Similarity Graph from Vector based Word Representation for Named Entity Recognition
| Title | Constructing a Word Similarity Graph from Vector based Word Representation for Named Entity Recognition |
| Authors | Miguel Feria, Juan Paolo Balbin, Francis Michael Bautista |
| Abstract | In this paper, we discuss a method for identifying a seed word that would best represent a class of named entities in a graphical representation of words and their similarities. Word networks, or word graphs, are representations of vectorized text where nodes are the words encountered in a corpus, and the weighted edges incident on the nodes represent how similar the words are to each other. We intend to build a bilingual word graph and identify seed words through community analysis that would be best used to segment a graph according to its named entities, therefore providing an unsupervised way of tagging named entities for a bilingual language base. |
| Tasks | Named Entity Recognition |
| Published | 2018-07-09 |
| URL | http://arxiv.org/abs/1807.03012v1 |
| http://arxiv.org/pdf/1807.03012v1.pdf | |
| PWC | https://paperswithcode.com/paper/constructing-a-word-similarity-graph-from |
| Repo | |
| Framework | |
Structured Evolution with Compact Architectures for Scalable Policy Optimization
| Title | Structured Evolution with Compact Architectures for Scalable Policy Optimization |
| Authors | Krzysztof Choromanski, Mark Rowland, Vikas Sindhwani, Richard E. Turner, Adrian Weller |
| Abstract | We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees. We show that this algorithm can be successfully applied to learn better quality compact policies than those using standard gradient estimation techniques. The compact policies we learn have several advantages over unstructured ones, including faster training algorithms and faster inference. These benefits are important when the policy is deployed on real hardware with limited resources. Further, compact policies provide more scalable architectures for derivative-free optimization (DFO) in high-dimensional spaces. We show that most robotics tasks from the OpenAI Gym can be solved using neural networks with less than 300 parameters, with almost linear time complexity of the inference phase, with up to 13x fewer parameters relative to the Evolution Strategies (ES) algorithm introduced by Salimans et al. (2017). We do not need heuristics such as fitness shaping to learn good quality policies, resulting in a simple and theoretically motivated training mechanism. |
| Tasks | Text-to-Image Generation |
| Published | 2018-04-06 |
| URL | http://arxiv.org/abs/1804.02395v2 |
| http://arxiv.org/pdf/1804.02395v2.pdf | |
| PWC | https://paperswithcode.com/paper/structured-evolution-with-compact |
| Repo | |
| Framework | |
Adapting Mask-RCNN for Automatic Nucleus Segmentation
| Title | Adapting Mask-RCNN for Automatic Nucleus Segmentation |
| Authors | Jeremiah W. Johnson |
| Abstract | Automatic segmentation of microscopy images is an important task in medical image processing and analysis. Nucleus detection is an important example of this task. Mask-RCNN is a recently proposed state-of-the-art algorithm for object detection, object localization, and object instance segmentation of natural images. In this paper we demonstrate that Mask-RCNN can be used to perform highly effective and efficient automatic segmentations of a wide range of microscopy images of cell nuclei, for a variety of cells acquired under a variety of conditions. |
| Tasks | Instance Segmentation, Object Detection, Object Localization, Semantic Segmentation |
| Published | 2018-05-01 |
| URL | http://arxiv.org/abs/1805.00500v1 |
| http://arxiv.org/pdf/1805.00500v1.pdf | |
| PWC | https://paperswithcode.com/paper/adapting-mask-rcnn-for-automatic-nucleus |
| Repo | |
| Framework | |
Application of Faster R-CNN model on Human Running Pattern Recognition
| Title | Application of Faster R-CNN model on Human Running Pattern Recognition |
| Authors | Kairan Yang, Feng Geng |
| Abstract | The advance algorithms like Faster Regional Convolutional Neural Network (Faster R-CNN) models are suitable to identify classified moving objects, due to the efficiency in learning the training dataset superior than ordinary CNN algorithms and the higher accuracy of labeling correct classes in the validation and testing dataset. This research examined and compared the three R-CNN type algorithms in object recognition to show the superior efficiency and accuracy of Faster R-CNN model on classifying human running patterns. Then it described the effect of Faster R-CNN in detecting different types of running patterns exhibited by a single individual or multiple individuals by conducting a dataset fitting experiment. In this study, the Faster R-CNN algorithm is implemented directly from the version released by Ross Girshick. |
| Tasks | Object Recognition |
| Published | 2018-11-13 |
| URL | http://arxiv.org/abs/1811.05147v1 |
| http://arxiv.org/pdf/1811.05147v1.pdf | |
| PWC | https://paperswithcode.com/paper/application-of-faster-r-cnn-model-on-human |
| Repo | |
| Framework | |
Robust RGB-D Face Recognition Using Attribute-Aware Loss
| Title | Robust RGB-D Face Recognition Using Attribute-Aware Loss |
| Authors | Luo Jiang, Juyong Zhang, Bailin Deng |
| Abstract | Existing convolutional neural network (CNN) based face recognition algorithms typically learn a discriminative feature mapping, using a loss function that enforces separation of features from different classes and/or aggregation of features within the same class. However, they may suffer from bias in the training data such as uneven sampling density, because they optimize the adjacency relationship of the learned features without considering the proximity of the underlying faces. Moreover, since they only use facial images for training, the learned feature mapping may not correctly indicate the relationship of other attributes such as gender and ethnicity, which can be important for some face recognition applications. In this paper, we propose a new CNN-based face recognition approach that incorporates such attributes into the training process. Using an attribute-aware loss function that regularizes the feature mapping using attribute proximity, our approach learns more discriminative features that are correlated with the attributes. We train our face recognition model on a large-scale RGB-D data set with over 100K identities captured under real application conditions. By comparing our approach with other methods on a variety of experiments, we demonstrate that depth channel and attribute-aware loss greatly improve the accuracy and robustness of face recognition. |
| Tasks | Face Recognition |
| Published | 2018-11-24 |
| URL | https://arxiv.org/abs/1811.09847v2 |
| https://arxiv.org/pdf/1811.09847v2.pdf | |
| PWC | https://paperswithcode.com/paper/robust-rgb-d-face-recognition-using-attribute |
| Repo | |
| Framework | |
Autonomous Driving System Design for Formula Student Driverless Racecar
| Title | Autonomous Driving System Design for Formula Student Driverless Racecar |
| Authors | Hanqing Tian, Jun Ni, Jibin Hu |
| Abstract | This paper summarizes the work of building the autonomous system including detection system and path tracking controller for a formula student autonomous racecar. A LIDAR-vision cooperating method of detecting traffic cone which is used as track mark is proposed. Detection algorithm of the racecar also implements a precise and high rate localization method which combines the GPS-INS data and LIDAR odometry. Besides, a track map including the location and color information of the cones is built simultaneously. Finally, the system and vehicle performance on a closed loop track is tested. This paper also briefly introduces the Formula Student Autonomous Competition (FSAC) in 2017. |
| Tasks | Autonomous Driving |
| Published | 2018-09-19 |
| URL | http://arxiv.org/abs/1809.07636v1 |
| http://arxiv.org/pdf/1809.07636v1.pdf | |
| PWC | https://paperswithcode.com/paper/autonomous-driving-system-design-for-formula |
| Repo | |
| Framework | |
Feature Selective Small Object Detection via Knowledge-based Recurrent Attentive Neural Network
| Title | Feature Selective Small Object Detection via Knowledge-based Recurrent Attentive Neural Network |
| Authors | Kai Yi, Zhiqiang Jian, Shitao Chen, Nanning Zheng |
| Abstract | At present, the performance of deep neural network in general object detection is comparable to or even surpasses that of human beings. However, due to the limitations of deep learning itself, the small proportion of feature pixels, and the occurence of blur and occlusion, the detection of small objects in complex scenes is still an open question. But we can not deny that real-time and accurate object detection is fundamental to automatic perception and subsequent perception-based decision-making and planning tasks of autonomous driving. Considering the characteristics of small objects in autonomous driving scene, we proposed a novel method named KB-RANN, which based on domain knowledge, intuitive experience and feature attentive selection. It can focus on particular parts of image features, and then it tries to stress the importance of these features and strengthenes the learning parameters of them. Our comparative experiments on KITTI and COCO datasets show that our proposed method can achieve considerable results both in speed and accuracy, and can improve the effect of small object detection through self-selection of important features and continuous enhancement of proposed method, and deployed it in our self-developed autonomous driving car. |
| Tasks | Autonomous Driving, Decision Making, Object Detection, Small Object Detection |
| Published | 2018-03-13 |
| URL | http://arxiv.org/abs/1803.05263v4 |
| http://arxiv.org/pdf/1803.05263v4.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-based-recurrent-attentive-neural |
| Repo | |
| Framework | |
