Paper Group ANR 396
Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method. Intel RealSense Stereoscopic Depth Cameras. Adaptive Plant Propagation Algorithm for Solving Economic Load Dispatch Problem. GP-SUM. Gaussian Processes Filtering of non-Gaussian Beliefs. Faster Clustering via Non-Backtracking Random Walks. Corpus specificity in LSA and Wo …
Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method
| Title | Large Scale Empirical Risk Minimization via Truncated Adaptive Newton Method |
| Authors | Mark Eisen, Aryan Mokhtari, Alejandro Ribeiro |
| Abstract | We consider large scale empirical risk minimization (ERM) problems, where both the problem dimension and variable size is large. In these cases, most second order methods are infeasible due to the high cost in both computing the Hessian over all samples and computing its inverse in high dimensions. In this paper, we propose a novel adaptive sample size second-order method, which reduces the cost of computing the Hessian by solving a sequence of ERM problems corresponding to a subset of samples and lowers the cost of computing the Hessian inverse using a truncated eigenvalue decomposition. We show that while we geometrically increase the size of the training set at each stage, a single iteration of the truncated Newton method is sufficient to solve the new ERM within its statistical accuracy. Moreover, for a large number of samples we are allowed to double the size of the training set at each stage, and the proposed method subsequently reaches the statistical accuracy of the full training set approximately after two effective passes. In addition to this theoretical result, we show empirically on a number of well known data sets that the proposed truncated adaptive sample size algorithm outperforms stochastic alternatives for solving ERM problems. |
| Tasks | |
| Published | 2017-05-22 |
| URL | http://arxiv.org/abs/1705.07957v1 |
| http://arxiv.org/pdf/1705.07957v1.pdf | |
| PWC | https://paperswithcode.com/paper/large-scale-empirical-risk-minimization-via |
| Repo | |
| Framework | |
Intel RealSense Stereoscopic Depth Cameras
| Title | Intel RealSense Stereoscopic Depth Cameras |
| Authors | Leonid Keselman, John Iselin Woodfill, Anders Grunnet-Jepsen, Achintya Bhowmik |
| Abstract | We present a comprehensive overview of the stereoscopic Intel RealSense RGBD imaging systems. We discuss these systems’ mode-of-operation, functional behavior and include models of their expected performance, shortcomings, and limitations. We provide information about the systems’ optical characteristics, their correlation algorithms, and how these properties can affect different applications, including 3D reconstruction and gesture recognition. Our discussion covers the Intel RealSense R200 and the Intel RealSense D400 (formally RS400). |
| Tasks | 3D Reconstruction, Gesture Recognition |
| Published | 2017-05-16 |
| URL | http://arxiv.org/abs/1705.05548v2 |
| http://arxiv.org/pdf/1705.05548v2.pdf | |
| PWC | https://paperswithcode.com/paper/intel-realsense-stereoscopic-depth-cameras |
| Repo | |
| Framework | |
Adaptive Plant Propagation Algorithm for Solving Economic Load Dispatch Problem
| Title | Adaptive Plant Propagation Algorithm for Solving Economic Load Dispatch Problem |
| Authors | Sayan Nag |
| Abstract | Optimization problems in design engineering are complex by nature, often because of the involvement of critical objective functions accompanied by a number of rigid constraints associated with the products involved. One such problem is Economic Load Dispatch (ED) problem which focuses on the optimization of the fuel cost while satisfying some system constraints. Classical optimization algorithms are not sufficient and also inefficient for the ED problem involving highly nonlinear, and non-convex functions both in the objective and in the constraints. This led to the development of metaheuristic optimization approaches which can solve the ED problem almost efficiently. This paper presents a novel robust plant intelligence based Adaptive Plant Propagation Algorithm (APPA) which is used to solve the classical ED problem. The application of the proposed method to the 3-generator and 6-generator systems shows the efficiency and robustness of the proposed algorithm. A comparative study with another state-of-the-art algorithm (APSO) demonstrates the quality of the solution achieved by the proposed method along with the convergence characteristics of the proposed approach. |
| Tasks | |
| Published | 2017-08-04 |
| URL | http://arxiv.org/abs/1708.07040v1 |
| http://arxiv.org/pdf/1708.07040v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-plant-propagation-algorithm-for |
| Repo | |
| Framework | |
GP-SUM. Gaussian Processes Filtering of non-Gaussian Beliefs
| Title | GP-SUM. Gaussian Processes Filtering of non-Gaussian Beliefs |
| Authors | Maria Bauza, Alberto Rodriguez |
| Abstract | This work studies the problem of stochastic dynamic filtering and state propagation with complex beliefs. The main contribution is GP-SUM, a filtering algorithm tailored to dynamic systems and observation models expressed as Gaussian Processes (GP), and to states represented as a weighted sum of Gaussians. The key attribute of GP-SUM is that it does not rely on linearizations of the dynamic or observation models, or on unimodal Gaussian approximations of the belief, hence enables tracking complex state distributions. The algorithm can be seen as a combination of a sampling-based filter with a probabilistic Bayes filter. On the one hand, GP-SUM operates by sampling the state distribution and propagating each sample through the dynamic system and observation models. On the other hand, it achieves effective sampling and accurate probabilistic propagation by relying on the GP form of the system, and the sum-of-Gaussian form of the belief. We show that GP-SUM outperforms several GP-Bayes and Particle Filters on a standard benchmark. We also demonstrate its use in a pushing task, predicting with experimental accuracy the naturally occurring non-Gaussian distributions. |
| Tasks | Gaussian Processes |
| Published | 2017-09-23 |
| URL | http://arxiv.org/abs/1709.08120v3 |
| http://arxiv.org/pdf/1709.08120v3.pdf | |
| PWC | https://paperswithcode.com/paper/gp-sum-gaussian-processes-filtering-of-non |
| Repo | |
| Framework | |
Faster Clustering via Non-Backtracking Random Walks
| Title | Faster Clustering via Non-Backtracking Random Walks |
| Authors | Brian Rappaport, Anuththari Gamage, Shuchin Aeron |
| Abstract | This paper presents VEC-NBT, a variation on the unsupervised graph clustering technique VEC, which improves upon the performance of the original algorithm significantly for sparse graphs. VEC employs a novel application of the state-of-the-art word2vec model to embed a graph in Euclidean space via random walks on the nodes of the graph. In VEC-NBT, we modify the original algorithm to use a non-backtracking random walk instead of the normal backtracking random walk used in VEC. We introduce a modification to a non-backtracking random walk, which we call a begrudgingly-backtracking random walk, and show empirically that using this model of random walks for VEC-NBT requires shorter walks on the graph to obtain results with comparable or greater accuracy than VEC, especially for sparser graphs. |
| Tasks | Graph Clustering |
| Published | 2017-08-26 |
| URL | http://arxiv.org/abs/1708.07967v1 |
| http://arxiv.org/pdf/1708.07967v1.pdf | |
| PWC | https://paperswithcode.com/paper/faster-clustering-via-non-backtracking-random |
| Repo | |
| Framework | |
Corpus specificity in LSA and Word2vec: the role of out-of-domain documents
| Title | Corpus specificity in LSA and Word2vec: the role of out-of-domain documents |
| Authors | Edgar Altszyler, Mariano Sigman, Diego Fernandez Slezak |
| Abstract | Latent Semantic Analysis (LSA) and Word2vec are some of the most widely used word embeddings. Despite the popularity of these techniques, the precise mechanisms by which they acquire new semantic relations between words remain unclear. In the present article we investigate whether LSA and Word2vec capacity to identify relevant semantic dimensions increases with size of corpus. One intuitive hypothesis is that the capacity to identify relevant dimensions should increase as the amount of data increases. However, if corpus size grow in topics which are not specific to the domain of interest, signal to noise ratio may weaken. Here we set to examine and distinguish these alternative hypothesis. To investigate the effect of corpus specificity and size in word-embeddings we study two ways for progressive elimination of documents: the elimination of random documents vs. the elimination of documents unrelated to a specific task. We show that Word2vec can take advantage of all the documents, obtaining its best performance when it is trained with the whole corpus. On the contrary, the specialization (removal of out-of-domain documents) of the training corpus, accompanied by a decrease of dimensionality, can increase LSA word-representation quality while speeding up the processing time. Furthermore, we show that the specialization without the decrease in LSA dimensionality can produce a strong performance reduction in specific tasks. From a cognitive-modeling point of view, we point out that LSA’s word-knowledge acquisitions may not be efficiently exploiting higher-order co-occurrences and global relations, whereas Word2vec does. |
| Tasks | Word Embeddings |
| Published | 2017-12-28 |
| URL | http://arxiv.org/abs/1712.10054v1 |
| http://arxiv.org/pdf/1712.10054v1.pdf | |
| PWC | https://paperswithcode.com/paper/corpus-specificity-in-lsa-and-word2vec-the |
| Repo | |
| Framework | |
Scale-invariant unconstrained online learning
| Title | Scale-invariant unconstrained online learning |
| Authors | Wojciech Kotłowski |
| Abstract | We consider a variant of online convex optimization in which both the instances (input vectors) and the comparator (weight vector) are unconstrained. We exploit a natural scale invariance symmetry in our unconstrained setting: the predictions of the optimal comparator are invariant under any linear transformation of the instances. Our goal is to design online algorithms which also enjoy this property, i.e. are scale-invariant. We start with the case of coordinate-wise invariance, in which the individual coordinates (features) can be arbitrarily rescaled. We give an algorithm, which achieves essentially optimal regret bound in this setup, expressed by means of a coordinate-wise scale-invariant norm of the comparator. We then study general invariance with respect to arbitrary linear transformations. We first give a negative result, showing that no algorithm can achieve a meaningful bound in terms of scale-invariant norm of the comparator in the worst case. Next, we compliment this result with a positive one, providing an algorithm which “almost” achieves the desired bound, incurring only a logarithmic overhead in terms of the norm of the instances. |
| Tasks | |
| Published | 2017-08-23 |
| URL | http://arxiv.org/abs/1708.07042v1 |
| http://arxiv.org/pdf/1708.07042v1.pdf | |
| PWC | https://paperswithcode.com/paper/scale-invariant-unconstrained-online-learning |
| Repo | |
| Framework | |
A Historical Review of Forty Years of Research on CMAC
| Title | A Historical Review of Forty Years of Research on CMAC |
| Authors | Frank Z. Xing |
| Abstract | The Cerebellar Model Articulation Controller (CMAC) is an influential brain-inspired computing model in many relevant fields. Since its inception in the 1970s, the model has been intensively studied and many variants of the prototype, such as Kernel-CMAC, Self-Organizing Map CMAC, and Linguistic CMAC, have been proposed. This review article focus on how the CMAC model is gradually developed and refined to meet the demand of fast, adaptive, and robust control. Two perspective, CMAC as a neural network and CMAC as a table look-up technique are presented. Three aspects of the model: the architecture, learning algorithms and applications are discussed. In the end, some potential future research directions on this model are suggested. |
| Tasks | |
| Published | 2017-02-08 |
| URL | http://arxiv.org/abs/1702.02277v1 |
| http://arxiv.org/pdf/1702.02277v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-historical-review-of-forty-years-of |
| Repo | |
| Framework | |
Listen to Your Face: Inferring Facial Action Units from Audio Channel
| Title | Listen to Your Face: Inferring Facial Action Units from Audio Channel |
| Authors | Zibo Meng, Shizhong Han, Yan Tong |
| Abstract | Extensive efforts have been devoted to recognizing facial action units (AUs). However, it is still challenging to recognize AUs from spontaneous facial displays especially when they are accompanied with speech. Different from all prior work that utilized visual observations for facial AU recognition, this paper presents a novel approach that recognizes speech-related AUs exclusively from audio signals based on the fact that facial activities are highly correlated with voice during speech. Specifically, dynamic and physiological relationships between AUs and phonemes are modeled through a continuous time Bayesian network (CTBN); then AU recognition is performed by probabilistic inference via the CTBN model. A pilot audiovisual AU-coded database has been constructed to evaluate the proposed audio-based AU recognition framework. The database consists of a “clean” subset with frontal and neutral faces and a challenging subset collected with large head movements and occlusions. Experimental results on this database show that the proposed CTBN model achieves promising recognition performance for 7 speech-related AUs and outperforms the state-of-the-art visual-based methods especially for those AUs that are activated at low intensities or “hardly visible” in the visual channel. Furthermore, the CTBN model yields more impressive recognition performance on the challenging subset, where the visual-based approaches suffer significantly. |
| Tasks | |
| Published | 2017-06-23 |
| URL | http://arxiv.org/abs/1706.07536v2 |
| http://arxiv.org/pdf/1706.07536v2.pdf | |
| PWC | https://paperswithcode.com/paper/listen-to-your-face-inferring-facial-action |
| Repo | |
| Framework | |
Loss Functions for Multiset Prediction
| Title | Loss Functions for Multiset Prediction |
| Authors | Sean Welleck, Zixin Yao, Yu Gai, Jialin Mao, Zheng Zhang, Kyunghyun Cho |
| Abstract | We study the problem of multiset prediction. The goal of multiset prediction is to train a predictor that maps an input to a multiset consisting of multiple items. Unlike existing problems in supervised learning, such as classification, ranking and sequence generation, there is no known order among items in a target multiset, and each item in the multiset may appear more than once, making this problem extremely challenging. In this paper, we propose a novel multiset loss function by viewing this problem from the perspective of sequential decision making. The proposed multiset loss function is empirically evaluated on two families of datasets, one synthetic and the other real, with varying levels of difficulty, against various baseline loss functions including reinforcement learning, sequence, and aggregated distribution matching loss functions. The experiments reveal the effectiveness of the proposed loss function over the others. |
| Tasks | Decision Making |
| Published | 2017-11-14 |
| URL | http://arxiv.org/abs/1711.05246v2 |
| http://arxiv.org/pdf/1711.05246v2.pdf | |
| PWC | https://paperswithcode.com/paper/loss-functions-for-multiset-prediction |
| Repo | |
| Framework | |
A Corpus of Deep Argumentative Structures as an Explanation to Argumentative Relations
| Title | A Corpus of Deep Argumentative Structures as an Explanation to Argumentative Relations |
| Authors | Paul Reisert, Naoya Inoue, Naoaki Okazaki, Kentaro Inui |
| Abstract | In this paper, we compose a new task for deep argumentative structure analysis that goes beyond shallow discourse structure analysis. The idea is that argumentative relations can reasonably be represented with a small set of predefined patterns. For example, using value judgment and bipolar causality, we can explain a support relation between two argumentative segments as follows: Segment 1 states that something is good, and Segment 2 states that it is good because it promotes something good when it happens. We are motivated by the following questions: (i) how do we formulate the task?, (ii) can a reasonable pattern set be created?, and (iii) do the patterns work? To examine the task feasibility, we conduct a three-stage, detailed annotation study using 357 argumentative relations from the argumentative microtext corpus, a small, but highly reliable corpus. We report the coverage of explanations captured by our patterns on a test set composed of 270 relations. Our coverage result of 74.6% indicates that argumentative relations can reasonably be explained by our small pattern set. Our agreement result of 85.9% shows that a reasonable inter-annotator agreement can be achieved. To assist with future work in computational argumentation, the annotated corpus is made publicly available. |
| Tasks | |
| Published | 2017-12-07 |
| URL | http://arxiv.org/abs/1712.02480v1 |
| http://arxiv.org/pdf/1712.02480v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-corpus-of-deep-argumentative-structures-as |
| Repo | |
| Framework | |
Dictionary Learning and Sparse Coding-based Denoising for High-Resolution Task Functional Connectivity MRI Analysis
| Title | Dictionary Learning and Sparse Coding-based Denoising for High-Resolution Task Functional Connectivity MRI Analysis |
| Authors | Seongah Jeong, Xiang Li, Jiarui Yang, Quanzheng Li, Vahid Tarokh |
| Abstract | We propose a novel denoising framework for task functional Magnetic Resonance Imaging (tfMRI) data to delineate the high-resolution spatial pattern of the brain functional connectivity via dictionary learning and sparse coding (DLSC). In order to address the limitations of the unsupervised DLSC-based fMRI studies, we utilize the prior knowledge of task paradigm in the learning step to train a data-driven dictionary and to model the sparse representation. We apply the proposed DLSC-based method to Human Connectome Project (HCP) motor tfMRI dataset. Studies on the functional connectivity of cerebrocerebellar circuits in somatomotor networks show that the DLSC-based denoising framework can significantly improve the prominent connectivity patterns, in comparison to the temporal non-local means (tNLM)-based denoising method as well as the case without denoising, which is consistent and neuroscientifically meaningful within motor area. The promising results show that the proposed method can provide an important foundation for the high-resolution functional connectivity analysis, and provide a better approach for fMRI preprocessing. |
| Tasks | Denoising, Dictionary Learning |
| Published | 2017-07-21 |
| URL | http://arxiv.org/abs/1707.06962v1 |
| http://arxiv.org/pdf/1707.06962v1.pdf | |
| PWC | https://paperswithcode.com/paper/dictionary-learning-and-sparse-coding-based |
| Repo | |
| Framework | |
A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise
| Title | A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise |
| Authors | Adrien Saumard |
| Abstract | We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of linear contrast estimation was tackled in the literature with this level of generality on the model. We solve here the case of a quadratic contrast, by separating the behavior of a linearized empirical process and the empirical process driven by the squares of functions of models. |
| Tasks | |
| Published | 2017-02-16 |
| URL | http://arxiv.org/abs/1702.05063v2 |
| http://arxiv.org/pdf/1702.05063v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-concentration-inequality-for-the-excess |
| Repo | |
| Framework | |
Finer Grained Entity Typing with TypeNet
| Title | Finer Grained Entity Typing with TypeNet |
| Authors | Shikhar Murty, Patrick Verga, Luke Vilnis, Andrew McCallum |
| Abstract | We consider the challenging problem of entity typing over an extremely fine grained set of types, wherein a single mention or entity can have many simultaneous and often hierarchically-structured types. Despite the importance of the problem, there is a relative lack of resources in the form of fine-grained, deep type hierarchies aligned to existing knowledge bases. In response, we introduce TypeNet, a dataset of entity types consisting of over 1941 types organized in a hierarchy, obtained by manually annotating a mapping from 1081 Freebase types to WordNet. We also experiment with several models comparable to state-of-the-art systems and explore techniques to incorporate a structure loss on the hierarchy with the standard mention typing loss, as a first step towards future research on this dataset. |
| Tasks | Entity Typing |
| Published | 2017-11-15 |
| URL | http://arxiv.org/abs/1711.05795v1 |
| http://arxiv.org/pdf/1711.05795v1.pdf | |
| PWC | https://paperswithcode.com/paper/finer-grained-entity-typing-with-typenet |
| Repo | |
| Framework | |
Asymptotic Bias of Stochastic Gradient Search
| Title | Asymptotic Bias of Stochastic Gradient Search |
| Authors | Vladislav B. Tadic, Arnaud Doucet |
| Abstract | The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and Lojasiewicz inequality), tight bounds on the asymptotic bias of the iterates generated by such an algorithm are derived. The obtained results hold under mild conditions and cover a broad class of high-dimensional nonlinear algorithms. Using these results, the asymptotic properties of the policy-gradient (reinforcement) learning and adaptive population Monte Carlo sampling are studied. Relying on the same results, the asymptotic behavior of the recursive maximum split-likelihood estimation in hidden Markov models is analyzed, too. |
| Tasks | |
| Published | 2017-08-30 |
| URL | http://arxiv.org/abs/1709.00291v1 |
| http://arxiv.org/pdf/1709.00291v1.pdf | |
| PWC | https://paperswithcode.com/paper/asymptotic-bias-of-stochastic-gradient-search |
| Repo | |
| Framework | |
