Paper Group ANR 734
SyntaxNet Models for the CoNLL 2017 Shared Task. Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model. Evolution in Virtual Worlds. Deep Over-sampling Framework for Classifying Imbalanced Data. A Machine Learning Based Intrusion Detection System for Software Defined 5G N …
SyntaxNet Models for the CoNLL 2017 Shared Task
| Title | SyntaxNet Models for the CoNLL 2017 Shared Task |
| Authors | Chris Alberti, Daniel Andor, Ivan Bogatyy, Michael Collins, Dan Gillick, Lingpeng Kong, Terry Koo, Ji Ma, Mark Omernick, Slav Petrov, Chayut Thanapirom, Zora Tung, David Weiss |
| Abstract | We describe a baseline dependency parsing system for the CoNLL2017 Shared Task. This system, which we call “ParseySaurus,” uses the DRAGNN framework [Kong et al, 2017] to combine transition-based recurrent parsing and tagging with character-based word representations. On the v1.3 Universal Dependencies Treebanks, the new system outpeforms the publicly available, state-of-the-art “Parsey’s Cousins” models by 3.47% absolute Labeled Accuracy Score (LAS) across 52 treebanks. |
| Tasks | Dependency Parsing |
| Published | 2017-03-15 |
| URL | http://arxiv.org/abs/1703.04929v1 |
| http://arxiv.org/pdf/1703.04929v1.pdf | |
| PWC | https://paperswithcode.com/paper/syntaxnet-models-for-the-conll-2017-shared |
| Repo | |
| Framework | |
Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model
| Title | Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model |
| Authors | Minkyu Choi, Jun Tani |
| Abstract | The current paper proposes a novel predictive coding type neural network model, the predictive multiple spatio-temporal scales recurrent neural network (P-MSTRNN). The P-MSTRNN learns to predict visually perceived human whole-body cyclic movement patterns by exploiting multiscale spatio-temporal constraints imposed on network dynamics by using differently sized receptive fields as well as different time constant values for each layer. After learning, the network becomes able to proactively imitate target movement patterns by inferring or recognizing corresponding intentions by means of the regression of prediction error. Results show that the network can develop a functional hierarchy by developing a different type of dynamic structure at each layer. The paper examines how model performance during pattern generation as well as predictive imitation varies depending on the stage of learning. The number of limit cycle attractors corresponding to target movement patterns increases as learning proceeds. And, transient dynamics developing early in the learning process successfully perform pattern generation and predictive imitation tasks. The paper concludes that exploitation of transient dynamics facilitates successful task performance during early learning periods. |
| Tasks | |
| Published | 2017-08-02 |
| URL | http://arxiv.org/abs/1708.00812v2 |
| http://arxiv.org/pdf/1708.00812v2.pdf | |
| PWC | https://paperswithcode.com/paper/predictive-coding-for-dynamic-visual |
| Repo | |
| Framework | |
Evolution in Virtual Worlds
| Title | Evolution in Virtual Worlds |
| Authors | Tim Taylor |
| Abstract | This chapter discusses the possibility of instilling a virtual world with mechanisms for evolution and natural selection in order to generate rich ecosystems of complex organisms in a process akin to biological evolution. Some previous work in the area is described, and successes and failures are discussed. The components of a more comprehensive framework for designing such worlds are mapped out, including the design of the individual organisms, the properties and dynamics of the environmental medium in which they are evolving, and the representational relationship between organism and environment. Some of the key issues discussed include how to allow organisms to evolve new structures and functions with few restrictions, and how to create an interconnectedness between organisms in order to generate drives for continuing evolutionary activity. |
| Tasks | |
| Published | 2017-10-17 |
| URL | http://arxiv.org/abs/1710.06055v1 |
| http://arxiv.org/pdf/1710.06055v1.pdf | |
| PWC | https://paperswithcode.com/paper/evolution-in-virtual-worlds |
| Repo | |
| Framework | |
Deep Over-sampling Framework for Classifying Imbalanced Data
| Title | Deep Over-sampling Framework for Classifying Imbalanced Data |
| Authors | Shin Ando, Chun-Yuan Huang |
| Abstract | Class imbalance is a challenging issue in practical classification problems for deep learning models as well as traditional models. Traditionally successful countermeasures such as synthetic over-sampling have had limited success with complex, structured data handled by deep learning models. In this paper, we propose Deep Over-sampling (DOS), a framework for extending the synthetic over-sampling method to exploit the deep feature space acquired by a convolutional neural network (CNN). Its key feature is an explicit, supervised representation learning, for which the training data presents each raw input sample with a synthetic embedding target in the deep feature space, which is sampled from the linear subspace of in-class neighbors. We implement an iterative process of training the CNN and updating the targets, which induces smaller in-class variance among the embeddings, to increase the discriminative power of the deep representation. We present an empirical study using public benchmarks, which shows that the DOS framework not only counteracts class imbalance better than the existing method, but also improves the performance of the CNN in the standard, balanced settings. |
| Tasks | Representation Learning |
| Published | 2017-04-25 |
| URL | http://arxiv.org/abs/1704.07515v3 |
| http://arxiv.org/pdf/1704.07515v3.pdf | |
| PWC | https://paperswithcode.com/paper/deep-over-sampling-framework-for-classifying |
| Repo | |
| Framework | |
A Machine Learning Based Intrusion Detection System for Software Defined 5G Network
| Title | A Machine Learning Based Intrusion Detection System for Software Defined 5G Network |
| Authors | Jiaqi Li, Zhifeng Zhao, Rongpeng Li |
| Abstract | As an inevitable trend of future 5G networks, Software Defined architecture has many advantages in providing central- ized control and flexible resource management. But it is also confronted with various security challenges and potential threats with emerging services and technologies. As the focus of network security, Intrusion Detection Systems (IDS) are usually deployed separately without collaboration. They are also unable to detect novel attacks with limited intelligent abilities, which are hard to meet the needs of software defined 5G. In this paper, we propose an intelligent intrusion system taking the advances of software defined technology and artificial intelligence based on Software Defined 5G architecture. It flexibly combines security function mod- ules which are adaptively invoked under centralized management and control with a globle view. It can also deal with unknown intrusions by using machine learning algorithms. Evaluation results prove that the intelligent intrusion detection system achieves a better performance. |
| Tasks | Intrusion Detection |
| Published | 2017-07-10 |
| URL | http://arxiv.org/abs/1708.04571v1 |
| http://arxiv.org/pdf/1708.04571v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-machine-learning-based-intrusion-detection |
| Repo | |
| Framework | |
Exploiting Convolutional Neural Network for Risk Prediction with Medical Feature Embedding
| Title | Exploiting Convolutional Neural Network for Risk Prediction with Medical Feature Embedding |
| Authors | Zhengping Che, Yu Cheng, Zhaonan Sun, Yan Liu |
| Abstract | The widespread availability of electronic health records (EHRs) promises to usher in the era of personalized medicine. However, the problem of extracting useful clinical representations from longitudinal EHR data remains challenging. In this paper, we explore deep neural network models with learned medical feature embedding to deal with the problems of high dimensionality and temporality. Specifically, we use a multi-layer convolutional neural network (CNN) to parameterize the model and is thus able to capture complex non-linear longitudinal evolution of EHRs. Our model can effectively capture local/short temporal dependency in EHRs, which is beneficial for risk prediction. To account for high dimensionality, we use the embedding medical features in the CNN model which hold the natural medical concepts. Our initial experiments produce promising results and demonstrate the effectiveness of both the medical feature embedding and the proposed convolutional neural network in risk prediction on cohorts of congestive heart failure and diabetes patients compared with several strong baselines. |
| Tasks | |
| Published | 2017-01-25 |
| URL | http://arxiv.org/abs/1701.07474v1 |
| http://arxiv.org/pdf/1701.07474v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploiting-convolutional-neural-network-for |
| Repo | |
| Framework | |
Learning Combinations of Sigmoids Through Gradient Estimation
| Title | Learning Combinations of Sigmoids Through Gradient Estimation |
| Authors | Stratis Ioannidis, Andrea Montanari |
| Abstract | We develop a new approach to learn the parameters of regression models with hidden variables. In a nutshell, we estimate the gradient of the regression function at a set of random points, and cluster the estimated gradients. The centers of the clusters are used as estimates for the parameters of hidden units. We justify this approach by studying a toy model, whereby the regression function is a linear combination of sigmoids. We prove that indeed the estimated gradients concentrate around the parameter vectors of the hidden units, and provide non-asymptotic bounds on the number of required samples. To the best of our knowledge, no comparable guarantees have been proven for linear combinations of sigmoids. |
| Tasks | |
| Published | 2017-08-22 |
| URL | http://arxiv.org/abs/1708.06678v2 |
| http://arxiv.org/pdf/1708.06678v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-combinations-of-sigmoids-through |
| Repo | |
| Framework | |
Fast Image Processing with Fully-Convolutional Networks
| Title | Fast Image Processing with Fully-Convolutional Networks |
| Authors | Qifeng Chen, Jia Xu, Vladlen Koltun |
| Abstract | We present an approach to accelerating a wide variety of image processing operators. Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator’s action. After training, the original operator need not be run at all. The trained network operates at full resolution and runs in constant time. We investigate the effect of network architecture on approximation accuracy, runtime, and memory footprint, and identify a specific architecture that balances these considerations. We evaluate the presented approach on ten advanced image processing operators, including multiple variational models, multiscale tone and detail manipulation, photographic style transfer, nonlocal dehazing, and nonphotorealistic stylization. All operators are approximated by the same model. Experiments demonstrate that the presented approach is significantly more accurate than prior approximation schemes. It increases approximation accuracy as measured by PSNR across the evaluated operators by 8.5 dB on the MIT-Adobe dataset (from 27.5 to 36 dB) and reduces DSSIM by a multiplicative factor of 3 compared to the most accurate prior approximation scheme, while being the fastest. We show that our models generalize across datasets and across resolutions, and investigate a number of extensions of the presented approach. The results are shown in the supplementary video at https://youtu.be/eQyfHgLx8Dc |
| Tasks | Style Transfer |
| Published | 2017-09-02 |
| URL | http://arxiv.org/abs/1709.00643v1 |
| http://arxiv.org/pdf/1709.00643v1.pdf | |
| PWC | https://paperswithcode.com/paper/fast-image-processing-with-fully |
| Repo | |
| Framework | |
Dynamic Boltzmann Machines for Second Order Moments and Generalized Gaussian Distributions
| Title | Dynamic Boltzmann Machines for Second Order Moments and Generalized Gaussian Distributions |
| Authors | Rudy Raymond, Takayuki Osogami, Sakyasingha Dasgupta |
| Abstract | Dynamic Boltzmann Machine (DyBM) has been shown highly efficient to predict time-series data. Gaussian DyBM is a DyBM that assumes the predicted data is generated by a Gaussian distribution whose first-order moment (mean) dynamically changes over time but its second-order moment (variance) is fixed. However, in many financial applications, the assumption is quite limiting in two aspects. First, even when the data follows a Gaussian distribution, its variance may change over time. Such variance is also related to important temporal economic indicators such as the market volatility. Second, financial time-series data often requires learning datasets generated by the generalized Gaussian distribution with an additional shape parameter that is important to approximate heavy-tailed distributions. Addressing those aspects, we show how to extend DyBM that results in significant performance improvement in predicting financial time-series data. |
| Tasks | Time Series |
| Published | 2017-12-17 |
| URL | http://arxiv.org/abs/1712.06132v1 |
| http://arxiv.org/pdf/1712.06132v1.pdf | |
| PWC | https://paperswithcode.com/paper/dynamic-boltzmann-machines-for-second-order |
| Repo | |
| Framework | |
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
| Title | TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering |
| Authors | Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, Gunhee Kim |
| Abstract | Vision and language understanding has emerged as a subject undergoing intense study in Artificial Intelligence. Among many tasks in this line of research, visual question answering (VQA) has been one of the most successful ones, where the goal is to learn a model that understands visual content at region-level details and finds their associations with pairs of questions and answers in the natural language form. Despite the rapid progress in the past few years, most existing work in VQA have focused primarily on images. In this paper, we focus on extending VQA to the video domain and contribute to the literature in three important ways. First, we propose three new tasks designed specifically for video VQA, which require spatio-temporal reasoning from videos to answer questions correctly. Next, we introduce a new large-scale dataset for video VQA named TGIF-QA that extends existing VQA work with our new tasks. Finally, we propose a dual-LSTM based approach with both spatial and temporal attention, and show its effectiveness over conventional VQA techniques through empirical evaluations. |
| Tasks | Question Answering, Visual Question Answering |
| Published | 2017-04-14 |
| URL | http://arxiv.org/abs/1704.04497v3 |
| http://arxiv.org/pdf/1704.04497v3.pdf | |
| PWC | https://paperswithcode.com/paper/tgif-qa-toward-spatio-temporal-reasoning-in |
| Repo | |
| Framework | |
Easy High-Dimensional Likelihood-Free Inference
| Title | Easy High-Dimensional Likelihood-Free Inference |
| Authors | Vinay Jethava, Devdatt Dubhashi |
| Abstract | We introduce a framework using Generative Adversarial Networks (GANs) for likelihood–free inference (LFI) and Approximate Bayesian Computation (ABC) where we replace the black-box simulator model with an approximator network and generate a rich set of summary features in a data driven fashion. On benchmark data sets, our approach improves on others with respect to scalability, ability to handle high dimensional data and complex probability distributions. |
| Tasks | |
| Published | 2017-11-29 |
| URL | http://arxiv.org/abs/1711.11139v2 |
| http://arxiv.org/pdf/1711.11139v2.pdf | |
| PWC | https://paperswithcode.com/paper/easy-high-dimensional-likelihood-free |
| Repo | |
| Framework | |
ConvNet-Based Localization of Anatomical Structures in 3D Medical Images
| Title | ConvNet-Based Localization of Anatomical Structures in 3D Medical Images |
| Authors | Bob D. de Vos, Jelmer M. Wolterink, Pim A. de Jong, Tim Leiner, Max A. Viergever, Ivana Išgum |
| Abstract | Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3D bounding boxes are created by combining the output of the ConvNet in all slices. In the experiments 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls. The best results were achieved in localization of structures with clearly defined boundaries (e.g. aortic arch) and the worst when the structure boundary was not clearly visible (e.g. liver). The method was more robust and accurate in localization multiple structures. |
| Tasks | |
| Published | 2017-04-19 |
| URL | http://arxiv.org/abs/1704.05629v1 |
| http://arxiv.org/pdf/1704.05629v1.pdf | |
| PWC | https://paperswithcode.com/paper/convnet-based-localization-of-anatomical |
| Repo | |
| Framework | |
Denoising Hyperspectral Image with Non-i.i.d. Noise Structure
| Title | Denoising Hyperspectral Image with Non-i.i.d. Noise Structure |
| Authors | Yang Chen, Xiangyong Cao, Qian Zhao, Deyu Meng, Zongben Xu |
| Abstract | Hyperspectral image (HSI) denoising has been attracting much research attention in remote sensing area due to its importance in improving the HSI qualities. The existing HSI denoising methods mainly focus on specific spectral and spatial prior knowledge in HSIs, and share a common underlying assumption that the embedded noise in HSI is independent and identically distributed (i.i.d.). In real scenarios, however, the noise existed in a natural HSI is always with much more complicated non-i.i.d. statistical structures and the under-estimation to this noise complexity often tends to evidently degenerate the robustness of current methods. To alleviate this issue, this paper attempts the first effort to model the HSI noise using a non-i.i.d. mixture of Gaussians (NMoG) noise assumption, which is finely in accordance with the noise characteristics possessed by a natural HSI and thus is capable of adapting various noise shapes encountered in real applications. Then we integrate such noise modeling strategy into the low-rank matrix factorization (LRMF) model and propose a NMoG-LRMF model in the Bayesian framework. A variational Bayes algorithm is designed to infer the posterior of the proposed model. All involved parameters can be recursively updated in closed-form. Compared with the current techniques, the proposed method performs more robust beyond the state-of-the-arts, as substantiated by our experiments implemented on synthetic and real noisy HSIs. |
| Tasks | Denoising |
| Published | 2017-02-01 |
| URL | http://arxiv.org/abs/1702.00098v1 |
| http://arxiv.org/pdf/1702.00098v1.pdf | |
| PWC | https://paperswithcode.com/paper/denoising-hyperspectral-image-with-non-iid |
| Repo | |
| Framework | |
Language as a matrix product state
| Title | Language as a matrix product state |
| Authors | Vasily Pestun, John Terilla, Yiannis Vlassopoulos |
| Abstract | We propose a statistical model for natural language that begins by considering language as a monoid, then representing it in complex matrices with a compatible translation invariant probability measure. We interpret the probability measure as arising via the Born rule from a translation invariant matrix product state. |
| Tasks | |
| Published | 2017-11-04 |
| URL | http://arxiv.org/abs/1711.01416v1 |
| http://arxiv.org/pdf/1711.01416v1.pdf | |
| PWC | https://paperswithcode.com/paper/language-as-a-matrix-product-state |
| Repo | |
| Framework | |
Enterprise to Computer: Star Trek chatbot
| Title | Enterprise to Computer: Star Trek chatbot |
| Authors | Grishma Jena, Mansi Vashisht, Abheek Basu, Lyle Ungar, João Sedoc |
| Abstract | Human interactions and human-computer interactions are strongly influenced by style as well as content. Adding a persona to a chatbot makes it more human-like and contributes to a better and more engaging user experience. In this work, we propose a design for a chatbot that captures the “style” of Star Trek by incorporating references from the show along with peculiar tones of the fictional characters therein. Our Enterprise to Computer bot (E2Cbot) treats Star Trek dialog style and general dialog style differently, using two recurrent neural network Encoder-Decoder models. The Star Trek dialog style uses sequence to sequence (SEQ2SEQ) models (Sutskever et al., 2014; Bahdanau et al., 2014) trained on Star Trek dialogs. The general dialog style uses Word Graph to shift the response of the SEQ2SEQ model into the Star Trek domain. We evaluate the bot both in terms of perplexity and word overlap with Star Trek vocabulary and subjectively using human evaluators. |
| Tasks | Chatbot |
| Published | 2017-08-02 |
| URL | http://arxiv.org/abs/1708.00818v1 |
| http://arxiv.org/pdf/1708.00818v1.pdf | |
| PWC | https://paperswithcode.com/paper/enterprise-to-computer-star-trek-chatbot |
| Repo | |
| Framework | |
