July 27, 2019

3087 words 15 mins read

Paper Group ANR 688

Building a Syllable Database to Solve the Problem of Khmer Word Segmentation. DeepTrend: A Deep Hierarchical Neural Network for Traffic Flow Prediction. Dual Skipping Networks. Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC. Deep De-Aliasing for Fast Compressive Sensing MRI. A World of Difference: Div …

Building a Syllable Database to Solve the Problem of Khmer Word Segmentation


Title	Building a Syllable Database to Solve the Problem of Khmer Word Segmentation
Authors	Nam Tran Van
Abstract	Word segmentation is a basic problem in natural language processing. With the languages having the complex writing system like the Khmer language in Southern of Vietnam, this problem really very intractable, posing the significant challenges. Although there are some experts in Vietnam as well as international having deeply researched this problem, there are still no reasonable results meeting the demand, in particular, no treated thoroughly the ambiguous phenomenon, in the process of Khmer language processing so far. This paper present a solution based on the syllable division into component clusters using two syllable models proposed, thereby building a Khmer syllable database, is still not actually available. This method using a lexical database updated from the online Khmer dictionaries and some supported dictionaries serving role of training data and complementary linguistic characteristics. Each component cluster is labelled and located by the first and last letter to identify entirety a syllable. This approach is workable and the test results achieve high accuracy, eliminate the ambiguity, contribute to solving the problem of word segmentation and applying efficiency in Khmer language processing.
Tasks
Published	2017-03-07
URL	http://arxiv.org/abs/1703.02166v1
PDF	http://arxiv.org/pdf/1703.02166v1.pdf
PWC	https://paperswithcode.com/paper/building-a-syllable-database-to-solve-the
Repo
Framework

DeepTrend: A Deep Hierarchical Neural Network for Traffic Flow Prediction


Title	DeepTrend: A Deep Hierarchical Neural Network for Traffic Flow Prediction
Authors	Xingyuan Dai, Rui Fu, Yilun Lin, Li Li, Fei-Yue Wang
Abstract	In this paper, we consider the temporal pattern in traffic flow time series, and implement a deep learning model for traffic flow prediction. Detrending based methods decompose original flow series into trend and residual series, in which trend describes the fixed temporal pattern in traffic flow and residual series is used for prediction. Inspired by the detrending method, we propose DeepTrend, a deep hierarchical neural network used for traffic flow prediction which considers and extracts the time-variant trend. DeepTrend has two stacked layers: extraction layer and prediction layer. Extraction layer, a fully connected layer, is used to extract the time-variant trend in traffic flow by feeding the original flow series concatenated with corresponding simple average trend series. Prediction layer, an LSTM layer, is used to make flow prediction by feeding the obtained trend from the output of extraction layer and calculated residual series. To make the model more effective, DeepTrend needs first pre-trained layer-by-layer and then fine-tuned in the entire network. Experiments show that DeepTrend can noticeably boost the prediction performance compared with some traditional prediction models and LSTM with detrending based methods.
Tasks	Time Series
Published	2017-07-11
URL	http://arxiv.org/abs/1707.03213v1
PDF	http://arxiv.org/pdf/1707.03213v1.pdf
PWC	https://paperswithcode.com/paper/deeptrend-a-deep-hierarchical-neural-network
Repo
Framework

Dual Skipping Networks


Title	Dual Skipping Networks
Authors	Changmao Cheng, Yanwei Fu, Yu-Gang Jiang, Wei Liu, Wenlian Lu, Jianfeng Feng, Xiangyang Xue
Abstract	Inspired by the recent neuroscience studies on the left-right asymmetry of the human brain in processing low and high spatial frequency information, this paper introduces a dual skipping network which carries out coarse-to-fine object categorization. Such a network has two branches to simultaneously deal with both coarse and fine-grained classification tasks. Specifically, we propose a layer-skipping mechanism that learns a gating network to predict which layers to skip in the testing stage. This layer-skipping mechanism endows the network with good flexibility and capability in practice. Evaluations are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed network model.
Tasks
Published	2017-10-28
URL	http://arxiv.org/abs/1710.10386v3
PDF	http://arxiv.org/pdf/1710.10386v3.pdf
PWC	https://paperswithcode.com/paper/dual-skipping-networks
Repo
Framework

Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC


Title	Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC
Authors	Yulai Cong, Bo Chen, Hongwei Liu, Mingyuan Zhou
Abstract	It is challenging to develop stochastic gradient based scalable inference for deep discrete latent variable models (LVMs), due to the difficulties in not only computing the gradients, but also adapting the step sizes to different latent factors and hidden layers. For the Poisson gamma belief network (PGBN), a recently proposed deep discrete LVM, we derive an alternative representation that is referred to as deep latent Dirichlet allocation (DLDA). Exploiting data augmentation and marginalization techniques, we derive a block-diagonal Fisher information matrix and its inverse for the simplex-constrained global model parameters of DLDA. Exploiting that Fisher information matrix with stochastic gradient MCMC, we present topic-layer-adaptive stochastic gradient Riemannian (TLASGR) MCMC that jointly learns simplex-constrained global parameters across all layers and topics, with topic and layer specific learning rates. State-of-the-art results are demonstrated on big data sets.
Tasks	Data Augmentation, Latent Variable Models
Published	2017-06-06
URL	http://arxiv.org/abs/1706.01724v1
PDF	http://arxiv.org/pdf/1706.01724v1.pdf
PWC	https://paperswithcode.com/paper/deep-latent-dirichlet-allocation-with-topic
Repo
Framework

Deep De-Aliasing for Fast Compressive Sensing MRI


Title	Deep De-Aliasing for Fast Compressive Sensing MRI
Authors	Simiao Yu, Hao Dong, Guang Yang, Greg Slabaugh, Pier Luigi Dragotti, Xujiong Ye, Fangde Liu, Simon Arridge, Jennifer Keegan, David Firmin, Yike Guo
Abstract	Fast Magnetic Resonance Imaging (MRI) is highly in demand for many clinical applications in order to reduce the scanning cost and improve the patient experience. This can also potentially increase the image quality by reducing the motion artefacts and contrast washout. However, once an image field of view and the desired resolution are chosen, the minimum scanning time is normally determined by the requirement of acquiring sufficient raw data to meet the Nyquist-Shannon sampling criteria. Compressive Sensing (CS) theory has been perfectly matched to the MRI scanning sequence design with much less required raw data for the image reconstruction. Inspired by recent advances in deep learning for solving various inverse problems, we propose a conditional Generative Adversarial Networks-based deep learning framework for de-aliasing and reconstructing MRI images from highly undersampled data with great promise to accelerate the data acquisition process. By coupling an innovative content loss with the adversarial loss our de-aliasing results are more realistic. Furthermore, we propose a refinement learning procedure for training the generator network, which can stabilise the training with fast convergence and less parameter tuning. We demonstrate that the proposed framework outperforms state-of-the-art CS-MRI methods, in terms of reconstruction error and perceptual image quality. In addition, our method can reconstruct each image in 0.22ms–0.37ms, which is promising for real-time applications.
Tasks	Compressive Sensing, De-aliasing, Image Reconstruction
Published	2017-05-19
URL	http://arxiv.org/abs/1705.07137v1
PDF	http://arxiv.org/pdf/1705.07137v1.pdf
PWC	https://paperswithcode.com/paper/deep-de-aliasing-for-fast-compressive-sensing
Repo
Framework

A World of Difference: Divergent Word Interpretations among People


Title	A World of Difference: Divergent Word Interpretations among People
Authors	Tianran Hu, Ruihua Song, Maya Abtahian, Philip Ding, Xing Xie, Jiebo Luo
Abstract	Divergent word usages reflect differences among people. In this paper, we present a novel angle for studying word usage divergence – word interpretations. We propose an approach that quantifies semantic differences in interpretations among different groups of people. The effectiveness of our approach is validated by quantitative evaluations. Experiment results indicate that divergences in word interpretations exist. We further apply the approach to two well studied types of differences between people – gender and region. The detected words with divergent interpretations reveal the unique features of specific groups of people. For gender, we discover that certain different interests, social attitudes, and characters between males and females are reflected in their divergent interpretations of many words. For region, we find that specific interpretations of certain words reveal the geographical and cultural features of different regions.
Tasks
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02859v2
PDF	http://arxiv.org/pdf/1703.02859v2.pdf
PWC	https://paperswithcode.com/paper/a-world-of-difference-divergent-word
Repo
Framework

Precision Learning: Reconstruction Filter Kernel Discretization


Title	Precision Learning: Reconstruction Filter Kernel Discretization
Authors	Christopher Syben, Bernhard Stimpel, Katharina Breininger, Tobias Würfl, Rebecca Fahrig, Arnd Dörfler, Andreas Maier
Abstract	In this paper, we present substantial evidence that a deep neural network will intrinsically learn the appropriate way to discretize the ideal continuous reconstruction filter. Currently, the Ram-Lak filter or heuristic filters which impose different noise assumptions are used for filtered back-projection. All of these, however, inhibit a fully data-driven reconstruction deep learning approach. In addition, the heuristic filters are not chosen in an optimal sense. To tackle this issue, we propose a formulation to directly learn the reconstruction filter. The filter is initialized with the ideal Ramp filter as a strong pre-training and learned in frequency domain. We compare the learned filter with the Ram-Lak and the Ramp filter on a numerical phantom as well as on a real CT dataset. The results show that the network properly discretizes the continuous Ramp filter and converges towards the Ram-Lak solution. In our view these observations are interesting to gain a better understanding of deep learning techniques and traditional analytic techniques such as Wiener filtering and discretization theory. Furthermore, this will allow fully trainable data-driven reconstruction deep learning approaches.
Tasks
Published	2017-10-17
URL	http://arxiv.org/abs/1710.06287v2
PDF	http://arxiv.org/pdf/1710.06287v2.pdf
PWC	https://paperswithcode.com/paper/precision-learning-reconstruction-filter
Repo
Framework

Certifying Some Distributional Robustness with Principled Adversarial Training


Title	Certifying Some Distributional Robustness with Principled Adversarial Training
Authors	Aman Sinha, Hongseok Namkoong, John Duchi
Abstract	Neural networks are vulnerable to adversarial examples and researchers have proposed many heuristic attack and defense mechanisms. We address this problem through the principled lens of distributionally robust optimization, which guarantees performance under adversarial input perturbations. By considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball, we provide a training procedure that augments model parameter updates with worst-case perturbations of training data. For smooth losses, our procedure provably achieves moderate levels of robustness with little computational or statistical cost relative to empirical risk minimization. Furthermore, our statistical guarantees allow us to efficiently certify robustness for the population loss. For imperceptible perturbations, our method matches or outperforms heuristic approaches.
Tasks
Published	2017-10-29
URL	http://arxiv.org/abs/1710.10571v4
PDF	http://arxiv.org/pdf/1710.10571v4.pdf
PWC	https://paperswithcode.com/paper/certifying-some-distributional-robustness
Repo
Framework

Feature importance scores and lossless feature pruning using Banzhaf power indices


Title	Feature importance scores and lossless feature pruning using Banzhaf power indices
Authors	Bogdan Kulynych, Carmela Troncoso
Abstract	Understanding the influence of features in machine learning is crucial to interpreting models and selecting the best features for classification. In this work we propose the use of principles from coalitional game theory to reason about importance of features. In particular, we propose the use of the Banzhaf power index as a measure of influence of features on the outcome of a classifier. We show that features having Banzhaf power index of zero can be losslessly pruned without damage to classifier accuracy. Computing the power indices does not require having access to data samples. However, if samples are available, the indices can be empirically estimated. We compute Banzhaf power indices for a neural network classifier on real-life data, and compare the results with gradient-based feature saliency, and coefficients of a logistic regression model with $L_1$ regularization.
Tasks	Feature Importance
Published	2017-11-14
URL	http://arxiv.org/abs/1711.04992v2
PDF	http://arxiv.org/pdf/1711.04992v2.pdf
PWC	https://paperswithcode.com/paper/feature-importance-scores-and-lossless
Repo
Framework

On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches


Title	On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches
Authors	Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Nicolas Papernot, Kunal Talwar, Li Zhang
Abstract	The recent, remarkable growth of machine learning has led to intense interest in the privacy of the data on which machine learning relies, and to new techniques for preserving privacy. However, older ideas about privacy may well remain valid and useful. This note reviews two recent works on privacy in the light of the wisdom of some of the early literature, in particular the principles distilled by Saltzer and Schroeder in the 1970s.
Tasks
Published	2017-08-26
URL	http://arxiv.org/abs/1708.08022v1
PDF	http://arxiv.org/pdf/1708.08022v1.pdf
PWC	https://paperswithcode.com/paper/on-the-protection-of-private-information-in
Repo
Framework

SUBIC: A Supervised Bi-Clustering Approach for Precision Medicine


Title	SUBIC: A Supervised Bi-Clustering Approach for Precision Medicine
Authors	Milad Zafar Nezhad, Dongxiao Zhu, Najibesadat Sadati, Kai Yang, Phillip Levy
Abstract	Traditional medicine typically applies one-size-fits-all treatment for the entire patient population whereas precision medicine develops tailored treatment schemes for different patient subgroups. The fact that some factors may be more significant for a specific patient subgroup motivates clinicians and medical researchers to develop new approaches to subgroup detection and analysis, which is an effective strategy to personalize treatment. In this study, we propose a novel patient subgroup detection method, called Supervised Biclustring (SUBIC) using convex optimization and apply our approach to detect patient subgroups and prioritize risk factors for hypertension (HTN) in a vulnerable demographic subgroup (African-American). Our approach not only finds patient subgroups with guidance of a clinically relevant target variable but also identifies and prioritizes risk factors by pursuing sparsity of the input variables and encouraging similarity among the input variables and between the input and target variables
Tasks
Published	2017-09-26
URL	http://arxiv.org/abs/1709.09929v1
PDF	http://arxiv.org/pdf/1709.09929v1.pdf
PWC	https://paperswithcode.com/paper/subic-a-supervised-bi-clustering-approach-for
Repo
Framework

Collect at Once, Use Effectively: Making Non-interactive Locally Private Learning Possible


Title	Collect at Once, Use Effectively: Making Non-interactive Locally Private Learning Possible
Authors	Kai Zheng, Wenlong Mou, Liwei Wang
Abstract	Non-interactive Local Differential Privacy (LDP) requires data analysts to collect data from users through noisy channel at once. In this paper, we extend the frontiers of Non-interactive LDP learning and estimation from several aspects. For learning with smooth generalized linear losses, we propose an approximate stochastic gradient oracle estimated from non-interactive LDP channel, using Chebyshev expansion. Combined with inexact gradient methods, we obtain an efficient algorithm with quasi-polynomial sample complexity bound. For the high-dimensional world, we discover that under $\ell_2$-norm assumption on data points, high-dimensional sparse linear regression and mean estimation can be achieved with logarithmic dependence on dimension, using random projection and approximate recovery. We also extend our methods to Kernel Ridge Regression. Our work is the first one that makes learning and estimation possible for a broad range of learning tasks under non-interactive LDP model.
Tasks
Published	2017-06-11
URL	http://arxiv.org/abs/1706.03316v1
PDF	http://arxiv.org/pdf/1706.03316v1.pdf
PWC	https://paperswithcode.com/paper/collect-at-once-use-effectively-making-non
Repo
Framework

Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning


Title	Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learning
Authors	Qi Wu, Peng Wang, Chunhua Shen, Ian Reid, Anton van den Hengel
Abstract	The Visual Dialogue task requires an agent to engage in a conversation about an image with a human. It represents an extension of the Visual Question Answering task in that the agent needs to answer a question about an image, but it needs to do so in light of the previous dialogue that has taken place. The key challenge in Visual Dialogue is thus maintaining a consistent, and natural dialogue while continuing to answer questions correctly. We present a novel approach that combines Reinforcement Learning and Generative Adversarial Networks (GANs) to generate more human-like responses to questions. The GAN helps overcome the relative paucity of training data, and the tendency of the typical MLE-based approach to generate overly terse answers. Critically, the GAN is tightly integrated into the attention mechanism that generates human-interpretable reasons for each answer. This means that the discriminative model of the GAN has the task of assessing whether a candidate answer is generated by a human or not, given the provided reason. This is significant because it drives the generative model to produce high quality answers that are well supported by the associated reasoning. The method also generates the state-of-the-art results on the primary benchmark.
Tasks	Question Answering, Visual Dialog, Visual Question Answering
Published	2017-11-21
URL	http://arxiv.org/abs/1711.07613v1
PDF	http://arxiv.org/pdf/1711.07613v1.pdf
PWC	https://paperswithcode.com/paper/are-you-talking-to-me-reasoned-visual-dialog
Repo
Framework

Predictive-Corrective Networks for Action Detection


Title	Predictive-Corrective Networks for Action Detection
Authors	Achal Dave, Olga Russakovsky, Deva Ramanan
Abstract	While deep feature learning has revolutionized techniques for static-image understanding, the same does not quite hold for video processing. Architectures and optimization techniques used for video are largely based off those for static images, potentially underutilizing rich video information. In this work, we rethink both the underlying network architecture and the stochastic learning paradigm for temporal data. To do so, we draw inspiration from classic theory on linear dynamic systems for modeling time series. By extending such models to include nonlinear mappings, we derive a series of novel recurrent neural networks that sequentially make top-down predictions about the future and then correct those predictions with bottom-up observations. Predictive-corrective networks have a number of desirable properties: (1) they can adaptively focus computation on “surprising” frames where predictions require large corrections, (2) they simplify learning in that only “residual-like” corrective terms need to be learned over time and (3) they naturally decorrelate an input data stream in a hierarchical fashion, producing a more reliable signal for learning at each layer of a network. We provide an extensive analysis of our lightweight and interpretable framework, and demonstrate that our model is competitive with the two-stream network on three challenging datasets without the need for computationally expensive optical flow.
Tasks	Action Detection, Optical Flow Estimation, Time Series
Published	2017-04-12
URL	http://arxiv.org/abs/1704.03615v2
PDF	http://arxiv.org/pdf/1704.03615v2.pdf
PWC	https://paperswithcode.com/paper/predictive-corrective-networks-for-action
Repo
Framework

Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial


Title	Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Authors	Mingzhe Chen, Ursula Challita, Walid Saad, Changchuan Yin, Mérouane Debbah
Abstract	Next-generation wireless networks must support ultra-reliable, low-latency communication and intelligently manage a massive number of Internet of Things (IoT) devices in real-time, within a highly dynamic environment. This need for stringent communication quality-of-service (QoS) requirements as well as mobile edge and core intelligence can only be realized by integrating fundamental notions of artificial intelligence (AI) and machine learning across the wireless infrastructure and end-user devices. In this context, this paper provides a comprehensive tutorial that introduces the main concepts of machine learning, in general, and artificial neural networks (ANNs), in particular, and their potential applications in wireless communications. For this purpose, we present a comprehensive overview on a number of key types of neural networks that include feed-forward, recurrent, spiking, and deep neural networks. For each type of neural network, we present the basic architecture and training procedure, as well as the associated challenges and opportunities. Then, we provide an in-depth overview on the variety of wireless communication problems that can be addressed using ANNs, ranging from communication using unmanned aerial vehicles to virtual reality and edge caching.For each individual application, we present the main motivation for using ANNs along with the associated challenges while also providing a detailed example for a use case scenario and outlining future works that can be addressed using ANNs. In a nutshell, this article constitutes one of the first holistic tutorials on the development of machine learning techniques tailored to the needs of future wireless networks.
Tasks
Published	2017-10-09
URL	https://arxiv.org/abs/1710.02913v2
PDF	https://arxiv.org/pdf/1710.02913v2.pdf
PWC	https://paperswithcode.com/paper/machine-learning-for-wireless-networks-with
Repo
Framework