July 27, 2019

3060 words 15 mins read

Paper Group ANR 743

Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation. The cognitive roots of regularization in language. Generalized Linear Model Regression under Distance-to-set Penalties. ArbiText: Arbitrary-Oriented Text Detection in Unconstrained Scene. Semi-Supervised Generation with Cluster-aware Generative Models. U …

Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation


Title	Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation
Authors	Ivan Vulić, Nikola Mrkšić, Anna Korhonen
Abstract	Existing approaches to automatic VerbNet-style verb classification are heavily dependent on feature engineering and therefore limited to languages with mature NLP pipelines. In this work, we propose a novel cross-lingual transfer method for inducing VerbNets for multiple languages. To the best of our knowledge, this is the first study which demonstrates how the architectures for learning word embeddings can be applied to this challenging syntactic-semantic task. Our method uses cross-lingual translation pairs to tie each of the six target languages into a bilingual vector space with English, jointly specialising the representations to encode the relational information from English VerbNet. A standard clustering algorithm is then run on top of the VerbNet-specialised representations, using vector dimensions as features for learning verb classes. Our results show that the proposed cross-lingual transfer approach sets new state-of-the-art verb classification performance across all six target languages explored in this work.
Tasks	Cross-Lingual Transfer, Feature Engineering, Learning Word Embeddings, Word Embeddings
Published	2017-07-21
URL	http://arxiv.org/abs/1707.06945v1
PDF	http://arxiv.org/pdf/1707.06945v1.pdf
PWC	https://paperswithcode.com/paper/cross-lingual-induction-and-transfer-of-verb
Repo
Framework

The cognitive roots of regularization in language


Title	The cognitive roots of regularization in language
Authors	Vanessa Ferdinand, Simon Kirby, Kenny Smith
Abstract	Regularization occurs when the output a learner produces is less variable than the linguistic data they observed. In an artificial language learning experiment, we show that there exist at least two independent sources of regularization bias in cognition: a domain-general source based on cognitive load and a domain-specific source triggered by linguistic stimuli. Both of these factors modulate how frequency information is encoded and produced, but only the production-side modulations result in regularization (i.e. cause learners to eliminate variation from the observed input). We formalize the definition of regularization as the reduction of entropy and find that entropy measures are better at identifying regularization behavior than frequency-based analyses. Using our experimental data and a model of cultural transmission, we generate predictions for the amount of regularity that would develop in each experimental condition if the artificial language were transmitted over several generations of learners. Here we find that the effect of cognitive constraints can become more complex when put into the context of cultural evolution: although learning biases certainly carry information about the course of language evolution, we should not expect a one-to-one correspondence between the micro-level processes that regularize linguistic datasets and the macro-level evolution of linguistic regularity.
Tasks
Published	2017-03-09
URL	http://arxiv.org/abs/1703.03442v2
PDF	http://arxiv.org/pdf/1703.03442v2.pdf
PWC	https://paperswithcode.com/paper/the-cognitive-roots-of-regularization-in
Repo
Framework

Generalized Linear Model Regression under Distance-to-set Penalties


Title	Generalized Linear Model Regression under Distance-to-set Penalties
Authors	Jason Xu, Eric C. Chi, Kenneth Lange
Abstract	Estimation in generalized linear models (GLM) is complicated by the presence of constraints. One can handle constraints by maximizing a penalized log-likelihood. Penalties such as the lasso are effective in high dimensions, but often lead to unwanted shrinkage. This paper explores instead penalizing the squared distance to constraint sets. Distance penalties are more flexible than algebraic and regularization penalties, and avoid the drawback of shrinkage. To optimize distance penalized objectives, we make use of the majorization-minimization principle. Resulting algorithms constructed within this framework are amenable to acceleration and come with global convergence guarantees. Applications to shape constraints, sparse regression, and rank-restricted matrix regression on synthetic and real data showcase strong empirical performance, even under non-convex constraints.
Tasks
Published	2017-11-03
URL	http://arxiv.org/abs/1711.01341v1
PDF	http://arxiv.org/pdf/1711.01341v1.pdf
PWC	https://paperswithcode.com/paper/generalized-linear-model-regression-under
Repo
Framework

ArbiText: Arbitrary-Oriented Text Detection in Unconstrained Scene


Title	ArbiText: Arbitrary-Oriented Text Detection in Unconstrained Scene
Authors	Daitao Xing, Zichen Li, Xin Chen, Yi Fang
Abstract	Arbitrary-oriented text detection in the wild is a very challenging task, due to the aspect ratio, scale, orientation, and illumination variations. In this paper, we propose a novel method, namely Arbitrary-oriented Text (or ArbText for short) detector, for efficient text detection in unconstrained natural scene images. Specifically, we first adopt the circle anchors rather than the rectangular ones to represent bounding boxes, which is more robust to orientation variations. Subsequently, we incorporate a pyramid pooling module into the Single Shot MultiBox Detector framework, in order to simultaneously explore the local and global visual information, which can, therefore, generate more confidential detection results. Experiments on established scene-text datasets, such as the ICDAR 2015 and MSRA-TD500 datasets, have demonstrated the supe rior performance of the proposed method, compared to the state-of-the-art approaches.
Tasks
Published	2017-11-30
URL	http://arxiv.org/abs/1711.11249v1
PDF	http://arxiv.org/pdf/1711.11249v1.pdf
PWC	https://paperswithcode.com/paper/arbitext-arbitrary-oriented-text-detection-in
Repo
Framework

Semi-Supervised Generation with Cluster-aware Generative Models


Title	Semi-Supervised Generation with Cluster-aware Generative Models
Authors	Lars Maaløe, Marco Fraccaro, Ole Winther
Abstract	Deep generative models trained with large amounts of unlabelled data have proven to be powerful within the domain of unsupervised learning. Many real life data sets contain a small amount of labelled data points, that are typically disregarded when training generative models. We propose the Cluster-aware Generative Model, that uses unlabelled information to infer a latent representation that models the natural clustering of the data, and additional labelled data points to refine this clustering. The generative performances of the model significantly improve when labelled information is exploited, obtaining a log-likelihood of -79.38 nats on permutation invariant MNIST, while also achieving competitive semi-supervised classification accuracies. The model can also be trained fully unsupervised, and still improve the log-likelihood performance with respect to related methods.
Tasks
Published	2017-04-03
URL	http://arxiv.org/abs/1704.00637v1
PDF	http://arxiv.org/pdf/1704.00637v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-generation-with-cluster-aware
Repo
Framework

Unsupervised Video Understanding by Reconciliation of Posture Similarities


Title	Unsupervised Video Understanding by Reconciliation of Posture Similarities
Authors	Timo Milbich, Miguel Bautista, Ekaterina Sutter, Bjorn Ommer
Abstract	Understanding human activity and being able to explain it in detail surpasses mere action classification by far in both complexity and value. The challenge is thus to describe an activity on the basis of its most fundamental constituents, the individual postures and their distinctive transitions. Supervised learning of such a fine-grained representation based on elementary poses is very tedious and does not scale. Therefore, we propose a completely unsupervised deep learning procedure based solely on video sequences, which starts from scratch without requiring pre-trained networks, predefined body models, or keypoints. A combinatorial sequence matching algorithm proposes relations between frames from subsets of the training data, while a CNN is reconciling the transitivity conflicts of the different subsets to learn a single concerted pose embedding despite changes in appearance across sequences. Without any manual annotation, the model learns a structured representation of postures and their temporal development. The model not only enables retrieval of similar postures but also temporal super-resolution. Additionally, based on a recurrent formulation, next frames can be synthesized.
Tasks	Action Classification, Super-Resolution, Video Understanding
Published	2017-08-03
URL	http://arxiv.org/abs/1708.01191v1
PDF	http://arxiv.org/pdf/1708.01191v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-video-understanding-by
Repo
Framework

Anatomical labeling of brain CT scan anomalies using multi-context nearest neighbor relation networks


Title	Anatomical labeling of brain CT scan anomalies using multi-context nearest neighbor relation networks
Authors	Srikrishna Varadarajan, Muktabh Mayank Srivastava, Monika Grewal, Pulkit Kumar
Abstract	This work is an endeavor to develop a deep learning methodology for automated anatomical labeling of a given region of interest (ROI) in brain computed tomography (CT) scans. We combine both local and global context to obtain a representation of the ROI. We then use Relation Networks (RNs) to predict the corresponding anatomy of the ROI based on its relationship score for each class. Further, we propose a novel strategy employing nearest neighbors approach for training RNs. We train RNs to learn the relationship of the target ROI with the joint representation of its nearest neighbors in each class instead of all data-points in each class. The proposed strategy leads to better training of RNs along with increased performance as compared to training baseline RN network.
Tasks	Computed Tomography (CT)
Published	2017-10-25
URL	http://arxiv.org/abs/1710.09180v2
PDF	http://arxiv.org/pdf/1710.09180v2.pdf
PWC	https://paperswithcode.com/paper/anatomical-labeling-of-brain-ct-scan
Repo
Framework

Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization


Title	Dispersion for Data-Driven Algorithm Design, Online Learning, and Private Optimization
Authors	Maria-Florina Balcan, Travis Dick, Ellen Vitercik
Abstract	Data-driven algorithm design, that is, choosing the best algorithm for a specific application, is a crucial problem in modern data science. Practitioners often optimize over a parameterized algorithm family, tuning parameters based on problems from their domain. These procedures have historically come with no guarantees, though a recent line of work studies algorithm selection from a theoretical perspective. We advance the foundations of this field in several directions: we analyze online algorithm selection, where problems arrive one-by-one and the goal is to minimize regret, and private algorithm selection, where the goal is to find good parameters over a set of problems without revealing sensitive information contained therein. We study important algorithm families, including SDP-rounding schemes for problems formulated as integer quadratic programs, and greedy techniques for canonical subset selection problems. In these cases, the algorithm’s performance is a volatile and piecewise Lipschitz function of its parameters, since tweaking the parameters can completely change the algorithm’s behavior. We give a sufficient and general condition, dispersion, defining a family of piecewise Lipschitz functions that can be optimized online and privately, which includes the functions measuring the performance of the algorithms we study. Intuitively, a set of piecewise Lipschitz functions is dispersed if no small region contains many of the functions’ discontinuities. We present general techniques for online and private optimization of the sum of dispersed piecewise Lipschitz functions. We improve over the best-known regret bounds for a variety of problems, prove regret bounds for problems not previously studied, and give matching lower bounds. We also give matching upper and lower bounds on the utility loss due to privacy. Moreover, we uncover dispersion in auction design and pricing problems.
Tasks
Published	2017-11-08
URL	http://arxiv.org/abs/1711.03091v4
PDF	http://arxiv.org/pdf/1711.03091v4.pdf
PWC	https://paperswithcode.com/paper/dispersion-for-data-driven-algorithm-design
Repo
Framework

Learning Simpler Language Models with the Differential State Framework


Title	Learning Simpler Language Models with the Differential State Framework
Authors	Alexander G. Ororbia II, Tomas Mikolov, David Reitter
Abstract	Learning useful information across long time lags is a critical and difficult problem for temporal neural models in tasks such as language modeling. Existing architectures that address the issue are often complex and costly to train. The Differential State Framework (DSF) is a simple and high-performing design that unifies previously introduced gated neural models. DSF models maintain longer-term memory by learning to interpolate between a fast-changing data-driven representation and a slowly changing, implicitly stable state. This requires hardly any more parameters than a classical, simple recurrent network. Within the DSF framework, a new architecture is presented, the Delta-RNN. In language modeling at the word and character levels, the Delta-RNN outperforms popular complex architectures, such as the Long Short Term Memory (LSTM) and the Gated Recurrent Unit (GRU), and, when regularized, performs comparably to several state-of-the-art baselines. At the subword level, the Delta-RNN’s performance is comparable to that of complex gated architectures.
Tasks	Language Modelling
Published	2017-03-26
URL	http://arxiv.org/abs/1703.08864v4
PDF	http://arxiv.org/pdf/1703.08864v4.pdf
PWC	https://paperswithcode.com/paper/learning-simpler-language-models-with-the
Repo
Framework

Automatic Analysis of EEGs Using Big Data and Hybrid Deep Learning Architectures


Title	Automatic Analysis of EEGs Using Big Data and Hybrid Deep Learning Architectures
Authors	Meysam Golmohammadi, Amir Hossein Harati Nejad Torbati, Silvia Lopez de Diego, Iyad Obeid, Joseph Picone
Abstract	Objective: A clinical decision support tool that automatically interprets EEGs can reduce time to diagnosis and enhance real-time applications such as ICU monitoring. Clinicians have indicated that a sensitivity of 95% with a specificity below 5% was the minimum requirement for clinical acceptance. We propose a highperformance classification system based on principles of big data and machine learning. Methods: A hybrid machine learning system that uses hidden Markov models (HMM) for sequential decoding and deep learning networks for postprocessing is proposed. These algorithms were trained and evaluated using the TUH EEG Corpus, which is the world’s largest publicly available database of clinical EEG data. Results: Our approach delivers a sensitivity above 90% while maintaining a specificity below 5%. This system detects three events of clinical interest: (1) spike and/or sharp waves, (2) periodic lateralized epileptiform discharges, (3) generalized periodic epileptiform discharges. It also detects three events used to model background noise: (1) artifacts, (2) eye movement (3) background. Conclusions: A hybrid HMM/deep learning system can deliver a low false alarm rate on EEG event detection, making automated analysis a viable option for clinicians. Significance: The TUH EEG Corpus enables application of highly data consumptive machine learning algorithms to EEG analysis. Performance is approaching clinical acceptance for real-time applications.
Tasks	EEG
Published	2017-12-28
URL	http://arxiv.org/abs/1712.09771v1
PDF	http://arxiv.org/pdf/1712.09771v1.pdf
PWC	https://paperswithcode.com/paper/automatic-analysis-of-eegs-using-big-data-and
Repo
Framework

Learning with Feature Evolvable Streams


Title	Learning with Feature Evolvable Streams
Authors	Bo-Jian Hou, Lijun Zhang, Zhi-Hua Zhou
Abstract	Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones. In this paper, we propose a novel learning paradigm: \emph{Feature Evolvable Streaming Learning} where old features would vanish and new features would occur. Rather than relying on only the current features, we attempt to recover the vanished features and exploit it to improve performance. Specifically, we learn two models from the recovered features and the current features, respectively. To benefit from the recovered features, we develop two ensemble methods. In the first method, we combine the predictions from two models and theoretically show that with the assistance of old features, the performance on new features can be improved. In the second approach, we dynamically select the best single prediction and establish a better performance guarantee when the best model switches. Experiments on both synthetic and real data validate the effectiveness of our proposal.
Tasks
Published	2017-06-16
URL	http://arxiv.org/abs/1706.05259v2
PDF	http://arxiv.org/pdf/1706.05259v2.pdf
PWC	https://paperswithcode.com/paper/learning-with-feature-evolvable-streams
Repo
Framework

A Quantum Extension of Variational Bayes Inference


Title	A Quantum Extension of Variational Bayes Inference
Authors	Hideyuki Miyahara, Yuki Sughiyama
Abstract	Variational Bayes (VB) inference is one of the most important algorithms in machine learning and widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this Letter, we generalize VB by using quantum mechanics, and propose a new algorithm, which we call quantum annealing variational Bayes (QAVB) inference. We then show that QAVB drastically improve the performance of VB by applying them to a clustering problem described by a Gaussian mixture model. Finally, we discuss an intuitive understanding on how QAVB works well.
Tasks
Published	2017-12-13
URL	http://arxiv.org/abs/1712.04709v1
PDF	http://arxiv.org/pdf/1712.04709v1.pdf
PWC	https://paperswithcode.com/paper/a-quantum-extension-of-variational-bayes
Repo
Framework

Shape DNA: Basic Generating Functions for Geometric Moment Invariants


Title	Shape DNA: Basic Generating Functions for Geometric Moment Invariants
Authors	Erbo Li, Yazhou Huang, Dong Xu, Hua Li
Abstract	Geometric moment invariants (GMIs) have been widely used as basic tool in shape analysis and information retrieval. Their structure and characteristics determine efficiency and effectiveness. Two fundamental building blocks or generating functions (GFs) for invariants are discovered, which are dot product and vector product of point vectors in Euclidean space. The primitive invariants (PIs) can be derived by carefully selecting different products of GFs and calculating the corresponding multiple integrals, which translates polynomials of coordinates of point vectors into geometric moments. Then the invariants themselves are expressed in the form of product of moments. This procedure is just like DNA encoding proteins. All GMIs available in the literature can be decomposed into linear combinations of PIs. This paper shows that Hu’s seven well known GMIs in computer vision have a more deep structure, which can be further divided into combination of simpler PIs. In practical uses, low order independent GMIs are of particular interest. In this paper, a set of PIs for similarity transformation and affine transformation in 2D are presented, which are simpler to use, and some of which are newly reported. The discovery of the two generating functions provides a new perspective of better understanding shapes in 2D and 3D Euclidean spaces, and the method proposed can be further extended to higher dimensional spaces and different manifolds, such as curves, surfaces and so on.
Tasks	Information Retrieval
Published	2017-03-07
URL	http://arxiv.org/abs/1703.02242v3
PDF	http://arxiv.org/pdf/1703.02242v3.pdf
PWC	https://paperswithcode.com/paper/shape-dna-basic-generating-functions-for
Repo
Framework

Position Tracking for Virtual Reality Using Commodity WiFi


Title	Position Tracking for Virtual Reality Using Commodity WiFi
Authors	Manikanta Kotaru, Sachin Katti
Abstract	Today, experiencing virtual reality (VR) is a cumbersome experience which either requires dedicated infrastructure like infrared cameras to track the headset and hand-motion controllers (e.g., Oculus Rift, HTC Vive), or provides only 3-DoF (Degrees of Freedom) tracking which severely limits the user experience (e.g., Samsung Gear). To truly enable VR everywhere, we need position tracking to be available as a ubiquitous service. This paper presents WiCapture, a novel approach which leverages commodity WiFi infrastructure, which is ubiquitous today, for tracking purposes. We prototype WiCapture using off-the-shelf WiFi radios and show that it achieves an accuracy of 0.88 cm compared to sophisticated infrared based tracking systems like the Oculus, while providing much higher range, resistance to occlusion, ubiquity and ease of deployment.
Tasks
Published	2017-03-09
URL	http://arxiv.org/abs/1703.03468v2
PDF	http://arxiv.org/pdf/1703.03468v2.pdf
PWC	https://paperswithcode.com/paper/position-tracking-for-virtual-reality-using
Repo
Framework

Design, Analysis and Application of A Volumetric Convolutional Neural Network


Title	Design, Analysis and Application of A Volumetric Convolutional Neural Network
Authors	Xiaqing Pan, Yueru Chen, C. -C. Jay Kuo
Abstract	The design, analysis and application of a volumetric convolutional neural network (VCNN) are studied in this work. Although many CNNs have been proposed in the literature, their design is empirical. In the design of the VCNN, we propose a feed-forward K-means clustering algorithm to determine the filter number and size at each convolutional layer systematically. For the analysis of the VCNN, the cause of confusing classes in the output of the VCNN is explained by analyzing the relationship between the filter weights (also known as anchor vectors) from the last fully-connected layer to the output. Furthermore, a hierarchical clustering method followed by a random forest classification method is proposed to boost the classification performance among confusing classes. For the application of the VCNN, we examine the 3D shape classification problem and conduct experiments on a popular ModelNet40 dataset. The proposed VCNN offers the state-of-the-art performance among all volume-based CNN methods.
Tasks
Published	2017-02-01
URL	http://arxiv.org/abs/1702.00158v1
PDF	http://arxiv.org/pdf/1702.00158v1.pdf
PWC	https://paperswithcode.com/paper/design-analysis-and-application-of-a
Repo
Framework