July 28, 2019

3010 words 15 mins read

Paper Group ANR 403

Stacking-based Deep Neural Network: Deep Analytic Network on Convolutional Spectral Histogram Features. Language Models of Spoken Dutch. Discovering Latent Covariance Structures for Multiple Time Series. Cooperative Automated Vehicles: a Review of Opportunities and Challenges in Socially Intelligent Vehicles Beyond Networking. Customized Routing Op …

Stacking-based Deep Neural Network: Deep Analytic Network on Convolutional Spectral Histogram Features


Title	Stacking-based Deep Neural Network: Deep Analytic Network on Convolutional Spectral Histogram Features
Authors	Cheng-Yaw Low, Andrew Beng-Jin Teoh
Abstract	Stacking-based deep neural network (S-DNN), in general, denotes a deep neural network (DNN) resemblance in terms of its very deep, feedforward network architecture. The typical S-DNN aggregates a variable number of individually learnable modules in series to assemble a DNN-alike alternative to the targeted object recognition tasks. This work likewise devises an S-DNN instantiation, dubbed deep analytic network (DAN), on top of the spectral histogram (SH) features. The DAN learning principle relies on ridge regression, and some key DNN constituents, specifically, rectified linear unit, fine-tuning, and normalization. The DAN aptitude is scrutinized on three repositories of varying domains, including FERET (faces), MNIST (handwritten digits), and CIFAR10 (natural objects). The empirical results unveil that DAN escalates the SH baseline performance over a sufficiently deep layer.
Tasks	Object Recognition
Published	2017-03-04
URL	http://arxiv.org/abs/1703.01396v2
PDF	http://arxiv.org/pdf/1703.01396v2.pdf
PWC	https://paperswithcode.com/paper/stacking-based-deep-neural-network-deep-1
Repo
Framework

Language Models of Spoken Dutch


Title	Language Models of Spoken Dutch
Authors	Lyan Verwimp, Joris Pelemans, Marieke Lycke, Hugo Van hamme, Patrick Wambacq
Abstract	In Flanders, all TV shows are subtitled. However, the process of subtitling is a very time-consuming one and can be sped up by providing the output of a speech recognizer run on the audio of the TV show, prior to the subtitling. Naturally, this speech recognition will perform much better if the employed language model is adapted to the register and the topic of the program. We present several language models trained on subtitles of television shows provided by the Flemish public-service broadcaster VRT. This data was gathered in the context of the project STON which has as purpose to facilitate the process of subtitling TV shows. One model is trained on all available data (46M word tokens), but we also trained models on a specific type of TV show or domain/topic. Language models of spoken language are quite rare due to the lack of training data. The size of this corpus is relatively large for a corpus of spoken language (compare with e.g. CGN which has 9M words), but still rather small for a language model. Thus, in practice it is advised to interpolate these models with a large background language model trained on written language. The models can be freely downloaded on http://www.esat.kuleuven.be/psi/spraak/downloads/.
Tasks	Language Modelling, Speech Recognition
Published	2017-09-12
URL	http://arxiv.org/abs/1709.03759v1
PDF	http://arxiv.org/pdf/1709.03759v1.pdf
PWC	https://paperswithcode.com/paper/language-models-of-spoken-dutch
Repo
Framework

Discovering Latent Covariance Structures for Multiple Time Series


Title	Discovering Latent Covariance Structures for Multiple Time Series
Authors	Anh Tong, Jaesik Choi
Abstract	Analyzing multivariate time series data is important to predict future events and changes of complex systems in finance, manufacturing, and administrative decisions. The expressiveness power of Gaussian Process (GP) regression methods has been significantly improved by compositional covariance structures. In this paper, we present a new GP model which naturally handles multiple time series by placing an Indian Buffet Process (IBP) prior on the presence of shared kernels. Our selective covariance structure decomposition allows exploiting shared parameters over a set of multiple, selected time series. We also investigate the well-definedness of the models when infinite latent components are introduced. We present a pragmatic search algorithm which explores a larger structure space efficiently. Experiments conducted on five real-world data sets demonstrate that our new model outperforms existing methods in term of structure discoveries and predictive performances.
Tasks	Time Series, Time Series Analysis
Published	2017-03-28
URL	https://arxiv.org/abs/1703.09528v4
PDF	https://arxiv.org/pdf/1703.09528v4.pdf
PWC	https://paperswithcode.com/paper/discovering-relational-covariance-structures
Repo
Framework

Cooperative Automated Vehicles: a Review of Opportunities and Challenges in Socially Intelligent Vehicles Beyond Networking


Title	Cooperative Automated Vehicles: a Review of Opportunities and Challenges in Socially Intelligent Vehicles Beyond Networking
Authors	Seng W. Loke
Abstract	The connected automated vehicle has been often touted as a technology that will become pervasive in society in the near future. One can view an automated vehicle as having Artificial Intelligence (AI) capabilities, being able to self-drive, sense its surroundings, recognise objects in its vicinity, and perform reasoning and decision-making. Rather than being stand alone, we examine the need for automated vehicles to cooperate and interact within their socio-cyber-physical environments, including the problems cooperation will solve, but also the issues and challenges. We review current work in cooperation for automated vehicles, based on selected examples from the literature. We conclude noting the need for the ability to behave cooperatively as a form of social-AI capability for automated vehicles, beyond sensing the immediate environment and beyond the underlying networking technology.
Tasks	Autonomous Vehicles, Decision Making
Published	2017-10-02
URL	https://arxiv.org/abs/1710.00461v2
PDF	https://arxiv.org/pdf/1710.00461v2.pdf
PWC	https://paperswithcode.com/paper/creating-a-social-brain-for-cooperative
Repo
Framework

Customized Routing Optimization Based on Gradient Boost Regressor Model


Title	Customized Routing Optimization Based on Gradient Boost Regressor Model
Authors	Chen Zheng, Clara Grzegorz Kasprowicz, Carol Saunders
Abstract	In this paper, we discussed limitation of current electronic-design-automoation (EDA) tool and proposed a machine learning framework to overcome the limitations and achieve better design quality. We explored how to efficiently extract relevant features and leverage gradient boost regressor (GBR) model to predict underestimated risky net (URN). Customized routing optimizations are applied to the URNs and results show clear timing improvement and trend to converge toward timing closure.
Tasks
Published	2017-10-28
URL	http://arxiv.org/abs/1710.11118v1
PDF	http://arxiv.org/pdf/1710.11118v1.pdf
PWC	https://paperswithcode.com/paper/customized-routing-optimization-based-on
Repo
Framework

Evaluation of Alzheimer’s Disease by Analysis of MR Images using Multilayer Perceptrons and Kohonen SOM Classifiers as an Alternative to the ADC Maps


Title	Evaluation of Alzheimer’s Disease by Analysis of MR Images using Multilayer Perceptrons and Kohonen SOM Classifiers as an Alternative to the ADC Maps
Authors	Wellington Pinheiro dos Santos, Ricardo Emmanuel de Souza, Plínio B. dos Santos Filho
Abstract	Alzheimer’s disease is the most common cause of dementia, yet hard to diagnose precisely without invasive techniques, particularly at the onset of the disease. This work approaches image analysis and classification of synthetic multispectral images composed by diffusion-weighted magnetic resonance (MR) cerebral images for the evaluation of cerebrospinal fluid area and measuring the advance of Alzheimer’s disease. A clinical 1.5 T MR imaging system was used to acquire all images presented. The classification methods are based on multilayer perceptrons and Kohonen Self-Organized Map classifiers. We assume the classes of interest can be separated by hyperquadrics. Therefore, a 2-degree polynomial network is used to classify the original image, generating the ground truth image. The classification results are used to improve the usual analysis of the apparent diffusion coefficient map.
Tasks
Published	2017-12-03
URL	http://arxiv.org/abs/1712.00712v1
PDF	http://arxiv.org/pdf/1712.00712v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-alzheimers-disease-by-analysis
Repo
Framework

A multi-task convolutional neural network for mega-city analysis using very high resolution satellite imagery and geospatial data


Title	A multi-task convolutional neural network for mega-city analysis using very high resolution satellite imagery and geospatial data
Authors	Fan Zhang, Bo Du, Liangpei Zhang
Abstract	Mega-city analysis with very high resolution (VHR) satellite images has been drawing increasing interest in the fields of city planning and social investigation. It is known that accurate land-use, urban density, and population distribution information is the key to mega-city monitoring and environmental studies. Therefore, how to generate land-use, urban density, and population distribution maps at a fine scale using VHR satellite images has become a hot topic. Previous studies have focused solely on individual tasks with elaborate hand-crafted features and have ignored the relationship between different tasks. In this study, we aim to propose a universal framework which can: 1) automatically learn the internal feature representation from the raw image data; and 2) simultaneously produce fine-scale land-use, urban density, and population distribution maps. For the first target, a deep convolutional neural network (CNN) is applied to learn the hierarchical feature representation from the raw image data. For the second target, a novel CNN-based universal framework is proposed to process the VHR satellite images and generate the land-use, urban density, and population distribution maps. To the best of our knowledge, this is the first CNN-based mega-city analysis method which can process a VHR remote sensing image with such a large data volume. A VHR satellite image (1.2 m spatial resolution) of the center of Wuhan covering an area of 2606 km2 was used to evaluate the proposed method. The experimental results confirm that the proposed method can achieve a promising accuracy for land-use, urban density, and population distribution maps.
Tasks
Published	2017-02-26
URL	http://arxiv.org/abs/1702.07985v1
PDF	http://arxiv.org/pdf/1702.07985v1.pdf
PWC	https://paperswithcode.com/paper/a-multi-task-convolutional-neural-network-for
Repo
Framework

A parameterized activation function for learning fuzzy logic operations in deep neural networks


Title	A parameterized activation function for learning fuzzy logic operations in deep neural networks
Authors	Luke B. Godfrey, Michael S. Gashler
Abstract	We present a deep learning architecture for learning fuzzy logic expressions. Our model uses an innovative, parameterized, differentiable activation function that can learn a number of logical operations by gradient descent. This activation function allows a neural network to determine the relationships between its input variables and provides insight into the logical significance of learned network parameters. We provide a theoretical basis for this parameterization and demonstrate its effectiveness and utility by successfully applying our model to five classification problems from the UCI Machine Learning Repository.
Tasks
Published	2017-08-28
URL	http://arxiv.org/abs/1708.08557v2
PDF	http://arxiv.org/pdf/1708.08557v2.pdf
PWC	https://paperswithcode.com/paper/a-parameterized-activation-function-for
Repo
Framework

Computer activity learning from system call time series


Title	Computer activity learning from system call time series
Authors	Curt Hastings, Ronnie Mainieri
Abstract	Using a previously introduced similarity function for the stream of system calls generated by a computer, we engineer a program-in-execution classifier using deep learning methods. Tested on malware classification, it significantly outperforms current state of the art. We provide a series of performance measures and tests to demonstrate the capabilities, including measurements from production use. We show how the system scales linearly with the number of endpoints. With the system we estimate the total number of malware families created over the last 10 years as 3450, in line with reasonable economic constraints. The more limited rate for new malware families than previously acknowledged implies that machine learning malware classifiers risk being tested on their training set; we achieve F1 = 0.995 in a test carefully designed to mitigate this risk.
Tasks	Malware Classification, Time Series
Published	2017-11-06
URL	http://arxiv.org/abs/1711.02088v1
PDF	http://arxiv.org/pdf/1711.02088v1.pdf
PWC	https://paperswithcode.com/paper/computer-activity-learning-from-system-call
Repo
Framework

Visual Reference Resolution using Attention Memory for Visual Dialog


Title	Visual Reference Resolution using Attention Memory for Visual Dialog
Authors	Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, Leonid Sigal
Abstract	Visual dialog is a task of answering a series of inter-dependent questions given an input image, and often requires to resolve visual references among the questions. This problem is different from visual question answering (VQA), which relies on spatial attention (a.k.a. visual grounding) estimated from an image and question pair. We propose a novel attention mechanism that exploits visual attentions in the past to resolve the current reference in the visual dialog scenario. The proposed model is equipped with an associative attention memory storing a sequence of previous (attention, key) pairs. From this memory, the model retrieves the previous attention, taking into account recency, which is most relevant for the current question, in order to resolve potentially ambiguous references. The model then merges the retrieved attention with a tentative one to obtain the final attention for the current question; specifically, we use dynamic parameter prediction to combine the two attentions conditioned on the question. Through extensive experiments on a new synthetic visual dialog dataset, we show that our model significantly outperforms the state-of-the-art (by ~16 % points) in situations, where visual reference resolution plays an important role. Moreover, the proposed model achieves superior performance (~ 2 % points improvement) in the Visual Dialog dataset, despite having significantly fewer parameters than the baselines.
Tasks	Question Answering, Visual Dialog, Visual Question Answering
Published	2017-09-23
URL	http://arxiv.org/abs/1709.07992v3
PDF	http://arxiv.org/pdf/1709.07992v3.pdf
PWC	https://paperswithcode.com/paper/visual-reference-resolution-using-attention
Repo
Framework

PDD Graph: Bridging Electronic Medical Records and Biomedical Knowledge Graphs via Entity Linking


Title	PDD Graph: Bridging Electronic Medical Records and Biomedical Knowledge Graphs via Entity Linking
Authors	Meng Wang, Jiaheng Zhang, Jun Liu, Wei Hu, Sen Wang, Xue Li, Wenqiang Liu
Abstract	Electronic medical records contain multi-format electronic medical data that consist of an abundance of medical knowledge. Facing with patient’s symptoms, experienced caregivers make right medical decisions based on their professional knowledge that accurately grasps relationships between symptoms, diagnosis and corresponding treatments. In this paper, we aim to capture these relationships by constructing a large and high-quality heterogenous graph linking patients, diseases, and drugs (PDD) in EMRs. Specifically, we propose a novel framework to extract important medical entities from MIMIC-III (Medical Information Mart for Intensive Care III) and automatically link them with the existing biomedical knowledge graphs, including ICD-9 ontology and DrugBank. The PDD graph presented in this paper is accessible on the Web via the SPARQL endpoint, and provides a pathway for medical discovery and applications, such as effective treatment recommendations.
Tasks	Entity Linking, Knowledge Graphs
Published	2017-07-17
URL	http://arxiv.org/abs/1707.05340v2
PDF	http://arxiv.org/pdf/1707.05340v2.pdf
PWC	https://paperswithcode.com/paper/pdd-graph-bridging-electronic-medical-records
Repo
Framework

Preserving Differential Privacy Between Features in Distributed Estimation


Title	Preserving Differential Privacy Between Features in Distributed Estimation
Authors	Christina Heinze-Deml, Brian McWilliams, Nicolai Meinshausen
Abstract	Privacy is crucial in many applications of machine learning. Legal, ethical and societal issues restrict the sharing of sensitive data making it difficult to learn from datasets that are partitioned between many parties. One important instance of such a distributed setting arises when information about each record in the dataset is held by different data owners (the design matrix is “vertically-partitioned”). In this setting few approaches exist for private data sharing for the purposes of statistical estimation and the classical setup of differential privacy with a “trusted curator” preparing the data does not apply. We work with the notion of $(\epsilon,\delta)$-distributed differential privacy which extends single-party differential privacy to the distributed, vertically-partitioned case. We propose PriDE, a scalable framework for distributed estimation where each party communicates perturbed random projections of their locally held features ensuring $(\epsilon,\delta)$-distributed differential privacy is preserved. For $\ell_2$-penalized supervised learning problems PriDE has bounded estimation error compared with the optimal estimates obtained without privacy constraints in the non-distributed setting. We confirm this empirically on real world and synthetic datasets.
Tasks
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00403v2
PDF	http://arxiv.org/pdf/1703.00403v2.pdf
PWC	https://paperswithcode.com/paper/preserving-differential-privacy-between
Repo
Framework

SideEye: A Generative Neural Network Based Simulator of Human Peripheral Vision


Title	SideEye: A Generative Neural Network Based Simulator of Human Peripheral Vision
Authors	Lex Fridman, Benedikt Jenik, Shaiyan Keshvari, Bryan Reimer, Christoph Zetzsche, Ruth Rosenholtz
Abstract	Foveal vision makes up less than 1% of the visual field. The other 99% is peripheral vision. Precisely what human beings see in the periphery is both obvious and mysterious in that we see it with our own eyes but can’t visualize what we see, except in controlled lab experiments. Degradation of information in the periphery is far more complex than what might be mimicked with a radial blur. Rather, behaviorally-validated models hypothesize that peripheral vision measures a large number of local texture statistics in pooling regions that overlap and grow with eccentricity. In this work, we develop a new method for peripheral vision simulation by training a generative neural network on a behaviorally-validated full-field synthesis model. By achieving a 21,000 fold reduction in running time, our approach is the first to combine realism and speed of peripheral vision simulation to a degree that provides a whole new way to approach visual design: through peripheral visualization.
Tasks
Published	2017-06-14
URL	http://arxiv.org/abs/1706.04568v2
PDF	http://arxiv.org/pdf/1706.04568v2.pdf
PWC	https://paperswithcode.com/paper/sideeye-a-generative-neural-network-based
Repo
Framework

Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image


Title	Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image
Authors	Rui Zhu, Hamed Kiani Galoogahi, Chaoyang Wang, Simon Lucey
Abstract	An emerging problem in computer vision is the reconstruction of 3D shape and pose of an object from a single image. Hitherto, the problem has been addressed through the application of canonical deep learning methods to regress from the image directly to the 3D shape and pose labels. These approaches, however, are problematic from two perspectives. First, they are minimizing the error between 3D shapes and pose labels - with little thought about the nature of this label error when reprojecting the shape back onto the image. Second, they rely on the onerous and ill-posed task of hand labeling natural images with respect to 3D shape and pose. In this paper we define the new task of pose-aware shape reconstruction from a single image, and we advocate that cheaper 2D annotations of objects silhouettes in natural images can be utilized. We design architectures of pose-aware shape reconstruction which re-project the predicted shape back on to the image using the predicted pose. Our evaluation on several object categories demonstrates the superiority of our method for predicting pose-aware 3D shapes from natural images.
Tasks
Published	2017-07-15
URL	http://arxiv.org/abs/1707.04682v2
PDF	http://arxiv.org/pdf/1707.04682v2.pdf
PWC	https://paperswithcode.com/paper/rethinking-reprojection-closing-the-loop-for
Repo
Framework

A General Neural Network Hardware Architecture on FPGA


Title	A General Neural Network Hardware Architecture on FPGA
Authors	Yufeng Hao
Abstract	Field Programmable Gate Arrays (FPGAs) plays an increasingly important role in data sampling and processing industries due to its highly parallel architecture, low power consumption, and flexibility in custom algorithms. Especially, in the artificial intelligence field, for training and implement the neural networks and machine learning algorithms, high energy efficiency hardware implement and massively parallel computing capacity are heavily demanded. Therefore, many global companies have applied FPGAs into AI and Machine learning fields such as autonomous driving and Automatic Spoken Language Recognition (Baidu) [1] [2] and Bing search (Microsoft) [3]. Considering the FPGAs great potential in these fields, we tend to implement a general neural network hardware architecture on XILINX ZU9CG System On Chip (SOC) platform [4], which contains abundant hardware resource and powerful processing capacity. The general neural network architecture on the FPGA SOC platform can perform forward and backward algorithms in deep neural networks (DNN) with high performance and easily be adjusted according to the type and scale of the neural networks.
Tasks	Autonomous Driving
Published	2017-11-06
URL	http://arxiv.org/abs/1711.05860v1
PDF	http://arxiv.org/pdf/1711.05860v1.pdf
PWC	https://paperswithcode.com/paper/a-general-neural-network-hardware
Repo
Framework