July 30, 2019

3052 words 15 mins read

Paper Group AWR 67

Automatic Anomaly Detection in the Cloud Via Statistical Learning. Tick: a Python library for statistical learning, with a particular emphasis on time-dependent modelling. Iterative PET Image Reconstruction Using Convolutional Neural Network Representation. Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Ph …

Automatic Anomaly Detection in the Cloud Via Statistical Learning


Title	Automatic Anomaly Detection in the Cloud Via Statistical Learning
Authors	Jordan Hochenbaum, Owen S. Vallis, Arun Kejariwal
Abstract	Performance and high availability have become increasingly important drivers, amongst other drivers, for user retention in the context of web services such as social networks, and web search. Exogenic and/or endogenic factors often give rise to anomalies, making it very challenging to maintain high availability, while also delivering high performance. Given that service-oriented architectures (SOA) typically have a large number of services, with each service having a large set of metrics, automatic detection of anomalies is non-trivial. Although there exists a large body of prior research in anomaly detection, existing techniques are not applicable in the context of social network data, owing to the inherent seasonal and trend components in the time series data. To this end, we developed two novel statistical techniques for automatically detecting anomalies in cloud infrastructure data. Specifically, the techniques employ statistical learning to detect anomalies in both application, and system metrics. Seasonal decomposition is employed to filter the trend and seasonal components of the time series, followed by the use of robust statistical metrics – median and median absolute deviation (MAD) – to accurately detect anomalies, even in the presence of seasonal spikes. We demonstrate the efficacy of the proposed techniques from three different perspectives, viz., capacity planning, user behavior, and supervised learning. In particular, we used production data for evaluation, and we report Precision, Recall, and F-measure in each case.
Tasks	Anomaly Detection, Time Series
Published	2017-04-24
URL	http://arxiv.org/abs/1704.07706v1
PDF	http://arxiv.org/pdf/1704.07706v1.pdf
PWC	https://paperswithcode.com/paper/automatic-anomaly-detection-in-the-cloud-via
Repo	https://github.com/vxld014/S-H-ESD
Framework	none

Tick: a Python library for statistical learning, with a particular emphasis on time-dependent modelling


Title	Tick: a Python library for statistical learning, with a particular emphasis on time-dependent modelling
Authors	Emmanuel Bacry, Martin Bompaire, Stéphane Gaïffas, Soren Poulsen
Abstract	Tick is a statistical learning library for Python~3, with a particular emphasis on time-dependent models, such as point processes, and tools for generalized linear models and survival analysis. The core of the library is an optimization module providing model computational classes, solvers and proximal operators for regularization. tick relies on a C++ implementation and state-of-the-art optimization algorithms to provide very fast computations in a single node multi-core setting. Source code and documentation can be downloaded from https://github.com/X-DataInitiative/tick
Tasks	Point Processes, Survival Analysis
Published	2017-07-10
URL	http://arxiv.org/abs/1707.03003v2
PDF	http://arxiv.org/pdf/1707.03003v2.pdf
PWC	https://paperswithcode.com/paper/tick-a-python-library-for-statistical
Repo	https://github.com/X-DataInitiative/tick_archive
Framework	none

Iterative PET Image Reconstruction Using Convolutional Neural Network Representation


Title	Iterative PET Image Reconstruction Using Convolutional Neural Network Representation
Authors	Kuang Gong, Jiahui Guan, Kyungsang Kim, Xuezhu Zhang, Georges El Fakhri, Jinyi Qi, Quanzheng Li
Abstract	PET image reconstruction is challenging due to the ill-poseness of the inverse problem and limited number of detected photons. Recently deep neural networks have been widely and successfully used in computer vision tasks and attracted growing interests in medical imaging. In this work, we trained a deep residual convolutional neural network to improve PET image quality by using the existing inter-patient information. An innovative feature of the proposed method is that we embed the neural network in the iterative reconstruction framework for image representation, rather than using it as a post-processing tool. We formulate the objective function as a constraint optimization problem and solve it using the alternating direction method of multipliers (ADMM) algorithm. Both simulation data and hybrid real data are used to evaluate the proposed method. Quantification results show that our proposed iterative neural network method can outperform the neural network denoising and conventional penalized maximum likelihood methods.
Tasks	Denoising, Image Reconstruction
Published	2017-10-09
URL	http://arxiv.org/abs/1710.03344v1
PDF	http://arxiv.org/pdf/1710.03344v1.pdf
PWC	https://paperswithcode.com/paper/iterative-pet-image-reconstruction-using
Repo	https://github.com/zgongkuang/IterativeCNN
Framework	tf

Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena


Title	Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
Authors	Igor Shalyminov, Arash Eshghi, Oliver Lemon
Abstract	Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis; and it contains many sorts of disfluency such as mid-utterance/sentence hesitations, interruptions, and self-corrections. But training data for machine learning approaches to dialogue processing is often either cleaned-up or wholly synthetic in order to avoid such phenomena. The question then arises of how well systems trained on such clean data generalise to real spontaneous dialogue, or indeed whether they are trainable at all on naturally occurring dialogue data. To answer this question, we created a new corpus called bAbI+ by systematically adding natural spontaneous incremental dialogue phenomena such as restarts and self-corrections to the Facebook AI Research’s bAbI dialogues dataset. We then explore the performance of a state-of-the-art retrieval model, MemN2N, on this more natural dataset. Results show that the semantic accuracy of the MemN2N model drops drastically; and that although it is in principle able to learn to process the constructions in bAbI+, it needs an impractical amount of training data to do so. Finally, we go on to show that an incremental, semantic parser – DyLan – shows 100% semantic accuracy on both bAbI and bAbI+, highlighting the generalisation properties of linguistically informed dialogue models.
Tasks
Published	2017-09-22
URL	http://arxiv.org/abs/1709.07840v1
PDF	http://arxiv.org/pdf/1709.07840v1.pdf
PWC	https://paperswithcode.com/paper/challenging-neural-dialogue-models-with
Repo	https://github.com/ishalyminov/babi_tools
Framework	none

mixup: Beyond Empirical Risk Minimization


Title	mixup: Beyond Empirical Risk Minimization
Authors	Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz
Abstract	Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple linear behavior in-between training examples. Our experiments on the ImageNet-2012, CIFAR-10, CIFAR-100, Google commands and UCI datasets show that mixup improves the generalization of state-of-the-art neural network architectures. We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
Tasks	Image Classification, Semi-Supervised Image Classification
Published	2017-10-25
URL	http://arxiv.org/abs/1710.09412v2
PDF	http://arxiv.org/pdf/1710.09412v2.pdf
PWC	https://paperswithcode.com/paper/mixup-beyond-empirical-risk-minimization
Repo	https://github.com/scut-aitcm/CompetitiveSENet
Framework	tf

A Spatiotemporal Oriented Energy Network for Dynamic Texture Recognition


Title	A Spatiotemporal Oriented Energy Network for Dynamic Texture Recognition
Authors	Isma Hadji, Richard P. Wildes
Abstract	This paper presents a novel hierarchical spatiotemporal orientation representation for spacetime image analysis. It is designed to combine the benefits of the multilayer architecture of ConvNets and a more controlled approach to spacetime analysis. A distinguishing aspect of the approach is that unlike most contemporary convolutional networks no learning is involved; rather, all design decisions are specified analytically with theoretical motivations. This approach makes it possible to understand what information is being extracted at each stage and layer of processing as well as to minimize heuristic choices in design. Another key aspect of the network is its recurrent nature, whereby the output of each layer of processing feeds back to the input. To keep the network size manageable across layers, a novel cross-channel feature pooling is proposed. The multilayer architecture that results systematically reveals hierarchical image structure in terms of multiscale, multiorientation properties of visual spacetime. To illustrate its utility, the network has been applied to the task of dynamic texture recognition. Empirical evaluation on multiple standard datasets shows that it sets a new state-of-the-art.
Tasks	Dynamic Texture Recognition
Published	2017-08-22
URL	http://arxiv.org/abs/1708.06690v1
PDF	http://arxiv.org/pdf/1708.06690v1.pdf
PWC	https://paperswithcode.com/paper/a-spatiotemporal-oriented-energy-network-for
Repo	https://github.com/hadjisma/soe-net
Framework	tf

Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting


Title	Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Authors	Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, Matthias Nießner
Abstract	We introduce a novel method to obtain high-quality 3D reconstructions from consumer RGB-D sensors. Our core idea is to simultaneously optimize for geometry encoded in a signed distance field (SDF), textures from automatically-selected keyframes, and their camera poses along with material and scene lighting. To this end, we propose a joint surface reconstruction approach that is based on Shape-from-Shading (SfS) techniques and utilizes the estimation of spatially-varying spherical harmonics (SVSH) from subvolumes of the reconstructed scene. Through extensive examples and evaluations, we demonstrate that our method dramatically increases the level of detail in the reconstructed scene geometry and contributes highly to consistent surface texture recovery.
Tasks	3D Reconstruction
Published	2017-08-04
URL	http://arxiv.org/abs/1708.01670v1
PDF	http://arxiv.org/pdf/1708.01670v1.pdf
PWC	https://paperswithcode.com/paper/intrinsic3d-high-quality-3d-reconstruction-by
Repo	https://github.com/NVlabs/intrinsic3d
Framework	none

Learning Intelligent Dialogs for Bounding Box Annotation


Title	Learning Intelligent Dialogs for Bounding Box Annotation
Authors	Ksenia Konyushkova, Jasper Uijlings, Christoph Lampert, Vittorio Ferrari
Abstract	We introduce Intelligent Annotation Dialogs for bounding box annotation. We train an agent to automatically choose a sequence of actions for a human annotator to produce a bounding box in a minimal amount of time. Specifically, we consider two actions: box verification, where the annotator verifies a box generated by an object detector, and manual box drawing. We explore two kinds of agents, one based on predicting the probability that a box will be positively verified, and the other based on reinforcement learning. We demonstrate that (1) our agents are able to learn efficient annotation strategies in several scenarios, automatically adapting to the image difficulty, the desired quality of the boxes, and the detector strength; (2) in all scenarios the resulting annotation dialogs speed up annotation compared to manual box drawing alone and box verification alone, while also outperforming any fixed combination of verification and drawing in most scenarios; (3) in a realistic scenario where the detector is iteratively re-trained, our agents evolve a series of strategies that reflect the shifting trade-off between verification and drawing as the detector grows stronger.
Tasks
Published	2017-12-21
URL	http://arxiv.org/abs/1712.08087v3
PDF	http://arxiv.org/pdf/1712.08087v3.pdf
PWC	https://paperswithcode.com/paper/learning-intelligent-dialogs-for-bounding-box
Repo	https://github.com/google/intelligent_annotation_dialogs
Framework	tf

Variational Reasoning for Question Answering with Knowledge Graph


Title	Variational Reasoning for Question Answering with Knowledge Graph
Authors	Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J. Smola, Le Song
Abstract	Knowledge graph (KG) is known to be helpful for the task of question answering (QA), since it provides well-structured relational information between entities, and allows one to further infer indirect facts. However, it is challenging to build QA systems which can learn to reason over knowledge graphs based on question-answer pairs alone. First, when people ask questions, their expressions are noisy (for example, typos in texts, or variations in pronunciations), which is non-trivial for the QA system to match those mentioned entities to the knowledge graph. Second, many questions require multi-hop logic reasoning over the knowledge graph to retrieve the answers. To address these challenges, we propose a novel and unified deep learning architecture, and an end-to-end variational learning algorithm which can handle noise in questions, and learn multi-hop reasoning simultaneously. Our method achieves state-of-the-art performance on a recent benchmark dataset in the literature. We also derive a series of new benchmark datasets, including questions for multi-hop reasoning, questions paraphrased by neural translation model, and questions in human voice. Our method yields very promising results on all these challenging datasets.
Tasks	Knowledge Graphs, Question Answering
Published	2017-09-12
URL	http://arxiv.org/abs/1709.04071v5
PDF	http://arxiv.org/pdf/1709.04071v5.pdf
PWC	https://paperswithcode.com/paper/variational-reasoning-for-question-answering
Repo	https://github.com/yuyuz/MetaQA
Framework	none

Neural Factorization Machines for Sparse Predictive Analytics


Title	Neural Factorization Machines for Sparse Predictive Analytics
Authors	Xiangnan He, Tat-Seng Chua
Abstract	Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of binary features via one-hot encoding, making the resultant feature vector highly sparse. To learn from such sparse data effectively, it is crucial to account for the interactions between features. Factorization Machines (FMs) are a popular solution for efficiently using the second-order feature interactions. However, FM models feature interactions in a linear way, which can be insufficient for capturing the non-linear and complex inherent structure of real-world data. While deep neural networks have recently been applied to learn non-linear feature interactions in industry, such as the Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile makes them difficult to train. In this paper, we propose a novel model Neural Factorization Machine (NFM) for prediction under sparse settings. NFM seamlessly combines the linearity of FM in modelling second-order feature interactions and the non-linearity of neural network in modelling higher-order feature interactions. Conceptually, NFM is more expressive than FM since FM can be seen as a special case of NFM without hidden layers. Empirical results on two regression tasks show that with one hidden layer only, NFM significantly outperforms FM with a 7.3% relative improvement. Compared to the recent deep learning methods Wide&Deep and DeepCross, our NFM uses a shallower structure but offers better performance, being much easier to train and tune in practice.
Tasks
Published	2017-08-16
URL	http://arxiv.org/abs/1708.05027v1
PDF	http://arxiv.org/pdf/1708.05027v1.pdf
PWC	https://paperswithcode.com/paper/neural-factorization-machines-for-sparse
Repo	https://github.com/shenweichen/DeepCTR
Framework	tf

Tacotron: Towards End-to-End Speech Synthesis


Title	Tacotron: Towards End-to-End Speech Synthesis
Authors	Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, Rif A. Saurous
Abstract	A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module. Building these components often requires extensive domain expertise and may contain brittle design choices. In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters. Given <text, audio> pairs, the model can be trained completely from scratch with random initialization. We present several key techniques to make the sequence-to-sequence framework perform well for this challenging task. Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness. In addition, since Tacotron generates speech at the frame level, it’s substantially faster than sample-level autoregressive methods.
Tasks	Speech Synthesis, Text-To-Speech Synthesis
Published	2017-03-29
URL	http://arxiv.org/abs/1703.10135v2
PDF	http://arxiv.org/pdf/1703.10135v2.pdf
PWC	https://paperswithcode.com/paper/tacotron-towards-end-to-end-speech-synthesis
Repo	https://github.com/0fengzi0/tacotron
Framework	tf

Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D


Title	Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D
Authors	Allan Grønlund, Kasper Green Larsen, Alexander Mathiasen, Jesper Sindahl Nielsen, Stefan Schneider, Mingzhou Song
Abstract	The $k$-Means clustering problem on $n$ points is NP-Hard for any dimension $d\ge 2$, however, for the 1D case there exists exact polynomial time algorithms. Previous literature reported an $O(kn^2)$ time dynamic programming algorithm that uses $O(kn)$ space. It turns out that the problem has been considered under a different name more than twenty years ago. We present all the existing work that had been overlooked and compare the various solutions theoretically. Moreover, we show how to reduce the space usage for some of them, as well as generalize them to data structures that can quickly report an optimal $k$-Means clustering for any $k$. Finally we also generalize all the algorithms to work for the absolute distance and to work for any Bregman Divergence. We complement our theoretical contributions by experiments that compare the practical performance of the various algorithms.
Tasks
Published	2017-01-25
URL	http://arxiv.org/abs/1701.07204v4
PDF	http://arxiv.org/pdf/1701.07204v4.pdf
PWC	https://paperswithcode.com/paper/fast-exact-k-means-k-medians-and-bregman
Repo	https://github.com/dstein64/kmeans1d
Framework	none

Dilated Recurrent Neural Networks


Title	Dilated Recurrent Neural Networks
Authors	Shiyu Chang, Yang Zhang, Wei Han, Mo Yu, Xiaoxiao Guo, Wei Tan, Xiaodong Cui, Michael Witbrock, Mark Hasegawa-Johnson, Thomas S. Huang
Abstract	Learning with recurrent neural networks (RNNs) on long sequences is a notoriously difficult task. There are three major challenges: 1) complex dependencies, 2) vanishing and exploding gradients, and 3) efficient parallelization. In this paper, we introduce a simple yet effective RNN connection structure, the DilatedRNN, which simultaneously tackles all of these challenges. The proposed architecture is characterized by multi-resolution dilated recurrent skip connections and can be combined flexibly with diverse RNN cells. Moreover, the DilatedRNN reduces the number of parameters needed and enhances training efficiency significantly, while matching state-of-the-art performance (even with standard RNN cells) in tasks involving very long-term dependencies. To provide a theory-based quantification of the architecture’s advantages, we introduce a memory capacity measure, the mean recurrent length, which is more suitable for RNNs with long skip connections than existing measures. We rigorously prove the advantages of the DilatedRNN over other recurrent neural architectures. The code for our method is publicly available at https://github.com/code-terminator/DilatedRNN
Tasks	Sequential Image Classification
Published	2017-10-05
URL	http://arxiv.org/abs/1710.02224v3
PDF	http://arxiv.org/pdf/1710.02224v3.pdf
PWC	https://paperswithcode.com/paper/dilated-recurrent-neural-networks
Repo	https://github.com/code-terminator/DilatedRNN
Framework	tf

The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics


Title	The Minor Fall, the Major Lift: Inferring Emotional Valence of Musical Chords through Lyrics
Authors	Artemy Kolchinsky, Nakul Dhande, Kengjeun Park, Yong-Yeol Ahn
Abstract	We investigate the association between musical chords and lyrics by analyzing a large dataset of user-contributed guitar tablatures. Motivated by the idea that the emotional content of chords is reflected in the words used in corresponding lyrics, we analyze associations between lyrics and chord categories. We also examine the usage patterns of chords and lyrics in different musical genres, historical eras, and geographical regions. Our overall results confirms a previously known association between Major chords and positive valence. We also report a wide variation in this association across regions, genres, and eras. Our results suggest possible existence of different emotional associations for other types of chords.
Tasks
Published	2017-06-26
URL	http://arxiv.org/abs/1706.08609v2
PDF	http://arxiv.org/pdf/1706.08609v2.pdf
PWC	https://paperswithcode.com/paper/the-minor-fall-the-major-lift-inferring
Repo	https://github.com/artemyk/chordsentiment
Framework	none

Sparse quadratic classification rules via linear dimension reduction


Title	Sparse quadratic classification rules via linear dimension reduction
Authors	Irina Gaynanova, Tianying Wang
Abstract	We consider the problem of high-dimensional classification between the two groups with unequal covariance matrices. Rather than estimating the full quadratic discriminant rule, we propose to perform simultaneous variable selection and linear dimension reduction on original data, with the subsequent application of quadratic discriminant analysis on the reduced space. In contrast to quadratic discriminant analysis, the proposed framework doesn’t require estimation of precision matrices and scales linearly with the number of measurements, making it especially attractive for the use on high-dimensional datasets. We support the methodology with theoretical guarantees on variable selection consistency, and empirical comparison with competing approaches. We apply the method to gene expression data of breast cancer patients, and confirm the crucial importance of ESR1 gene in differentiating estrogen receptor status.
Tasks	Dimensionality Reduction
Published	2017-11-13
URL	http://arxiv.org/abs/1711.04817v2
PDF	http://arxiv.org/pdf/1711.04817v2.pdf
PWC	https://paperswithcode.com/paper/sparse-quadratic-classification-rules-via
Repo	https://github.com/irinagain/DAP
Framework	none