Paper Group ANR 1251
Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery. Graph Representation Learning: A Survey. Modulated binary cliquenet. Binary Space Partitioning Forests. Differentially private sub-Gaussian location estimators. Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emoti …
Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery
| Title | Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery |
| Authors | Richard Y. Zhang, Somayeh Sojoudi, Javad Lavaei |
| Abstract | Nonconvex matrix recovery is known to contain no spurious local minima under a restricted isometry property (RIP) with a sufficiently small RIP constant $\delta$. If $\delta$ is too large, however, then counterexamples containing spurious local minima are known to exist. In this paper, we introduce a proof technique that is capable of establishing sharp thresholds on $\delta$ to guarantee the inexistence of spurious local minima. Using the technique, we prove that in the case of a rank-1 ground truth, an RIP constant of $\delta<1/2$ is both necessary and sufficient for exact recovery from any arbitrary initial point (such as a random point). We also prove a local recovery result: given an initial point $x_{0}$ satisfying $f(x_{0})\le(1-\delta)^{2}f(0)$, any descent algorithm that converges to second-order optimality guarantees exact recovery. |
| Tasks | |
| Published | 2019-01-07 |
| URL | https://arxiv.org/abs/1901.01631v3 |
| https://arxiv.org/pdf/1901.01631v3.pdf | |
| PWC | https://paperswithcode.com/paper/sharp-restricted-isometry-bounds-for-the |
| Repo | |
| Framework | |
Graph Representation Learning: A Survey
| Title | Graph Representation Learning: A Survey |
| Authors | Fenxiao Chen, Yuncheng Wang, Bin Wang, C. -C. Jay Kuo |
| Abstract | Research on graph representation learning has received a lot of attention in recent years since many data in real-world applications come in form of graphs. High-dimensional graph data are often in irregular form, which makes them more difficult to analyze than image/video/audio data defined on regular lattices. Various graph embedding techniques have been developed to convert the raw graph data into a low-dimensional vector representation while preserving the intrinsic graph properties. In this review, we first explain the graph embedding task and its challenges. Next, we review a wide range of graph embedding techniques with insights. Then, we evaluate several state-of-the-art methods against small and large datasets and compare their performance. Finally, potential applications and future directions are presented. |
| Tasks | Graph Embedding, Graph Representation Learning, Representation Learning |
| Published | 2019-09-03 |
| URL | https://arxiv.org/abs/1909.00958v1 |
| https://arxiv.org/pdf/1909.00958v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-representation-learning-a-survey |
| Repo | |
| Framework | |
Modulated binary cliquenet
| Title | Modulated binary cliquenet |
| Authors | Jinpeng Xia, Jiasong Wu, Youyong Kong, Pinzheng Zhang, Lotfi Senhadji, Huazhong Shu |
| Abstract | Although Convolutional Neural Networks (CNNs) achieve effectiveness in various computer vision tasks, the significant requirement of storage of such networks hinders the deployment on computationally limited devices. In this paper, we propose a new compact and portable deep learning network named Modulated Binary Cliquenet (MBCliqueNet) aiming to improve the portability of CNNs based on binarized filters while achieving comparable performance with the full-precision CNNs like Resnet. In MBCliqueNet, we introduce a novel modulated operation to approximate the unbinarized filters and gives an initialization method to speed up its convergence. We reduce the extra parameters caused by modulated operation with parameters sharing. As a result, the proposed MBCliqueNet can reduce the required storage space of convolutional filters by a factor of at least 32, in contrast to the full-precision model, and achieve better performance than other state-of-the-art binarized models. More importantly, our model compares even better with some full-precision models like Resnet on the dataset we used. |
| Tasks | |
| Published | 2019-02-27 |
| URL | http://arxiv.org/abs/1902.10460v1 |
| http://arxiv.org/pdf/1902.10460v1.pdf | |
| PWC | https://paperswithcode.com/paper/modulated-binary-cliquenet |
| Repo | |
| Framework | |
Binary Space Partitioning Forests
| Title | Binary Space Partitioning Forests |
| Authors | Xuhui Fan, Bin Li, Scott Anthony Sisson |
| Abstract | The Binary Space Partitioning~(BSP)-Tree process is proposed to produce flexible 2-D partition structures which are originally used as a Bayesian nonparametric prior for relational modelling. It can hardly be applied to other learning tasks such as regression trees because extending the BSP-Tree process to a higher dimensional space is nontrivial. This paper is the first attempt to extend the BSP-Tree process to a d-dimensional (d>2) space. We propose to generate a cutting hyperplane, which is assumed to be parallel to d-2 dimensions, to cut each node in the d-dimensional BSP-tree. By designing a subtle strategy to sample two free dimensions from d dimensions, the extended BSP-Tree process can inherit the essential self-consistency property from the original version. Based on the extended BSP-Tree process, an ensemble model, which is named the BSP-Forest, is further developed for regression tasks. Thanks to the retained self-consistency property, we can thus significantly reduce the geometric calculations in the inference stage. Compared to its counterpart, the Mondrian Forest, the BSP-Forest can achieve similar performance with fewer cuts due to its flexibility. The BSP-Forest also outperforms other (Bayesian) regression forests on a number of real-world data sets. |
| Tasks | |
| Published | 2019-03-22 |
| URL | http://arxiv.org/abs/1903.09348v1 |
| http://arxiv.org/pdf/1903.09348v1.pdf | |
| PWC | https://paperswithcode.com/paper/binary-space-partitioning-forests |
| Repo | |
| Framework | |
Differentially private sub-Gaussian location estimators
| Title | Differentially private sub-Gaussian location estimators |
| Authors | Marco Avella-Medina, Victor-Emmanuel Brunel |
| Abstract | We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differential privacy and show that a naive application of the Laplace mechanism can lead to sub-optimal results. We design two private algorithms for estimating the median that lead to estimators with sub-Gaussian type errors. Unlike most existing differentially private median estimators, both algorithms are well defined for unbounded random variables that are not even required to have finite moments. We then turn to the problem of sub-Gaussian mean estimation and show that under heavy tails natural differentially private alternatives lead to strictly worse deviations than their non-private sub-Gaussian counterparts. This is in sharp contrast with recent results that show that from an asymptotic perspective the cost of differential privacy is negligible. |
| Tasks | |
| Published | 2019-06-27 |
| URL | https://arxiv.org/abs/1906.11923v1 |
| https://arxiv.org/pdf/1906.11923v1.pdf | |
| PWC | https://paperswithcode.com/paper/differentially-private-sub-gaussian-location |
| Repo | |
| Framework | |
Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emotion Recognition and Event Recognition as Use Cases
| Title | Graph Neural Networks for Image Understanding Based on Multiple Cues: Group Emotion Recognition and Event Recognition as Use Cases |
| Authors | Xin Guo, Luisa F. Polania, Bin Zhu, Charles Boncelet, Kenneth E. Barner |
| Abstract | A graph neural network (GNN) for image understanding based on multiple cues is proposed in this paper. Compared to traditional feature and decision fusion approaches that neglect the fact that features can interact and exchange information, the proposed GNN is able to pass information among features extracted from different models. Two image understanding tasks, namely group-level emotion recognition (GER) and event recognition, which are highly semantic and require the interaction of several deep models to synthesize multiple cues, were selected to validate the performance of the proposed method. It is shown through experiments that the proposed method achieves state-of-the-art performance on the selected image understanding tasks. In addition, a new group-level emotion recognition database is introduced and shared in this paper. |
| Tasks | Emotion Recognition |
| Published | 2019-09-19 |
| URL | https://arxiv.org/abs/1909.12911v2 |
| https://arxiv.org/pdf/1909.12911v2.pdf | |
| PWC | https://paperswithcode.com/paper/graph-neural-networks-for-image-understanding |
| Repo | |
| Framework | |
Parallel Algorithm for Non-Monotone DR-Submodular Maximization
| Title | Parallel Algorithm for Non-Monotone DR-Submodular Maximization |
| Authors | Alina Ene, Huy L. Nguyen |
| Abstract | In this work, we give a new parallel algorithm for the problem of maximizing a non-monotone diminishing returns submodular function subject to a cardinality constraint. For any desired accuracy $\epsilon$, our algorithm achieves a $1/e - \epsilon$ approximation using $O(\log{n} \log(1/\epsilon) / \epsilon^3)$ parallel rounds of function evaluations. The approximation guarantee nearly matches the best approximation guarantee known for the problem in the sequential setting and the number of parallel rounds is nearly-optimal for any constant $\epsilon$. Previous algorithms achieve worse approximation guarantees using $\Omega(\log^2{n})$ parallel rounds. Our experimental evaluation suggests that our algorithm obtains solutions whose objective value nearly matches the value obtained by the state of the art sequential algorithms, and it outperforms previous parallel algorithms in number of parallel rounds, iterations, and solution quality. |
| Tasks | |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.13272v1 |
| https://arxiv.org/pdf/1905.13272v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-algorithm-for-non-monotone-dr |
| Repo | |
| Framework | |
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
| Title | Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech |
| Authors | David Harwath, Wei-Ning Hsu, James Glass |
| Abstract | In this paper, we present a method for learning discrete linguistic units by incorporating vector quantization layers into neural models of visually grounded speech. We show that our method is capable of capturing both word-level and sub-word units, depending on how it is configured. What differentiates this paper from prior work on speech unit learning is the choice of training objective. Rather than using a reconstruction-based loss, we use a discriminative, multimodal grounding objective which forces the learned units to be useful for semantic image retrieval. We evaluate the sub-word units on the ZeroSpeech 2019 challenge, achieving a 27.3% reduction in ABX error rate over the top-performing submission, while keeping the bitrate approximately the same. We also present experiments demonstrating the noise robustness of these units. Finally, we show that a model with multiple quantizers can simultaneously learn phone-like detectors at a lower layer and word-like detectors at a higher layer. We show that these detectors are highly accurate, discovering 279 words with an F1 score of greater than 0.5. |
| Tasks | Image Retrieval, Quantization |
| Published | 2019-11-21 |
| URL | https://arxiv.org/abs/1911.09602v2 |
| https://arxiv.org/pdf/1911.09602v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-hierarchical-discrete-linguistic |
| Repo | |
| Framework | |
Variation-aware Binarized Memristive Networks
| Title | Variation-aware Binarized Memristive Networks |
| Authors | Corey Lammie, Olga Krestinskaya, Alex James, Mostafa Rahimi Azghadi |
| Abstract | The quantization of weights to binary states in Deep Neural Networks (DNNs) can replace resource-hungry multiply accumulate operations with simple accumulations. Such Binarized Neural Networks (BNNs) exhibit greatly reduced resource and power requirements. In addition, memristors have been shown as promising synaptic weight elements in DNNs. In this paper, we propose and simulate novel Binarized Memristive Convolutional Neural Network (BMCNN) architectures employing hybrid weight and parameter representations. We train the proposed architectures offline and then map the trained parameters to our binarized memristive devices for inference. To take into account the variations in memristive devices, and to study their effect on the performance, we introduce variations in $R_{ON}$ and $R_{OFF}$. Moreover, we introduce means to mitigate the adverse effect of memristive variations in our proposed networks. Finally, we benchmark our BMCNNs and variation-aware BMCNNs using the MNIST dataset. |
| Tasks | Quantization |
| Published | 2019-10-14 |
| URL | https://arxiv.org/abs/1910.05920v1 |
| https://arxiv.org/pdf/1910.05920v1.pdf | |
| PWC | https://paperswithcode.com/paper/variation-aware-binarized-memristive-networks |
| Repo | |
| Framework | |
CMS Sematrix: A Tool to Aid the Development of Clinical Quality Measures (CQMs)
| Title | CMS Sematrix: A Tool to Aid the Development of Clinical Quality Measures (CQMs) |
| Authors | Michael A. Schwemmer, Po-Hsu Chen, Mithun Balakrishna, Amy Leibrand, Aaron Leonard, Nancy J. McMillan, Jeffrey J. Geppert |
| Abstract | As part of the effort to improve quality and to reduce national healthcare costs, the Centers for Medicare and Medicaid Services (CMS) are responsible for creating and maintaining an array of clinical quality measures (CQMs) for assessing healthcare structure, process, outcome, and patient experience across various conditions, clinical specialties, and settings. The development and maintenance of CQMs involves substantial and ongoing evaluation of the evidence on the measure’s properties: importance, reliability, validity, feasibility, and usability. As such, CMS conducts monthly environmental scans of the published clinical and health service literature. Conducting time consuming, exhaustive evaluations of the ever-changing healthcare literature presents one of the largest challenges to an evidence-based approach to healthcare quality improvement. Thus, it is imperative to leverage automated techniques to aid CMS in the identification of clinical and health services literature relevant to CQMs. Additionally, the estimated labor hours and related cost savings of using CMS Sematrix compared to a traditional literature review are roughly 818 hours and 122,000 dollars for a single monthly environmental scan. |
| Tasks | |
| Published | 2019-02-05 |
| URL | http://arxiv.org/abs/1902.01918v1 |
| http://arxiv.org/pdf/1902.01918v1.pdf | |
| PWC | https://paperswithcode.com/paper/cms-sematrix-a-tool-to-aid-the-development-of |
| Repo | |
| Framework | |
Accurate Layerwise Interpretable Competence Estimation
| Title | Accurate Layerwise Interpretable Competence Estimation |
| Authors | Vickram Rajendran, William LeVine |
| Abstract | Estimating machine learning performance ‘in the wild’ is both an important and unsolved problem. In this paper, we seek to examine, understand, and predict the pointwise competence of classification models. Our contributions are twofold: First, we establish a statistically rigorous definition of competence that generalizes the common notion of classifier confidence; second, we present the ALICE (Accurate Layerwise Interpretable Competence Estimation) Score, a pointwise competence estimator for any classifier. By considering distributional, data, and model uncertainty, ALICE empirically shows accurate competence estimation in common failure situations such as class-imbalanced datasets, out-of-distribution datasets, and poorly trained models. Our contributions allow us to accurately predict the competence of any classification model given any input and error function. We compare our score with state-of-the-art confidence estimators such as model confidence and Trust Score, and show significant improvements in competence prediction over these methods on datasets such as DIGITS, CIFAR10, and CIFAR100. |
| Tasks | |
| Published | 2019-10-24 |
| URL | https://arxiv.org/abs/1910.11363v1 |
| https://arxiv.org/pdf/1910.11363v1.pdf | |
| PWC | https://paperswithcode.com/paper/accurate-layerwise-interpretable-competence |
| Repo | |
| Framework | |
A weighted random survival forest
| Title | A weighted random survival forest |
| Authors | Lev V. Utkin, Andrei V. Konstantinov, Viacheslav S. Chukanov, Mikhail V. Kots, Mikhail A. Ryabinin, Anna A. Meldo |
| Abstract | A weighted random survival forest is presented in the paper. It can be regarded as a modification of the random forest improving its performance. The main idea underlying the proposed model is to replace the standard procedure of averaging used for estimation of the random survival forest hazard function by weighted avaraging where the weights are assigned to every tree and can be veiwed as training paremeters which are computed in an optimal way by solving a standard quadratic optimization problem maximizing Harrell’s C-index. Numerical examples with real data illustrate the outperformance of the proposed model in comparison with the original random survival forest. |
| Tasks | |
| Published | 2019-01-01 |
| URL | http://arxiv.org/abs/1901.00213v1 |
| http://arxiv.org/pdf/1901.00213v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-weighted-random-survival-forest |
| Repo | |
| Framework | |
Search on the Replay Buffer: Bridging Planning and Reinforcement Learning
| Title | Search on the Replay Buffer: Bridging Planning and Reinforcement Learning |
| Authors | Benjamin Eysenbach, Ruslan Salakhutdinov, Sergey Levine |
| Abstract | The history of learning for control has been an exciting back and forth between two broad classes of algorithms: planning and reinforcement learning. Planning algorithms effectively reason over long horizons, but assume access to a local policy and distance metric over collision-free paths. Reinforcement learning excels at learning policies and the relative values of states, but fails to plan over long horizons. Despite the successes of each method in various domains, tasks that require reasoning over long horizons with limited feedback and high-dimensional observations remain exceedingly challenging for both planning and reinforcement learning algorithms. Frustratingly, these sorts of tasks are potentially the most useful, as they are simple to design (a human only need to provide an example goal state) and avoid reward shaping, which can bias the agent towards finding a sub-optimal solution. We introduce a general control algorithm that combines the strengths of planning and reinforcement learning to effectively solve these tasks. Our aim is to decompose the task of reaching a distant goal state into a sequence of easier tasks, each of which corresponds to reaching a subgoal. Planning algorithms can automatically find these waypoints, but only if provided with suitable abstractions of the environment – namely, a graph consisting of nodes and edges. Our main insight is that this graph can be constructed via reinforcement learning, where a goal-conditioned value function provides edge weights, and nodes are taken to be previously seen observations in a replay buffer. Using graph search over our replay buffer, we can automatically generate this sequence of subgoals, even in image-based environments. Our algorithm, search on the replay buffer (SoRB), enables agents to solve sparse reward tasks over one hundred steps, and generalizes substantially better than standard RL algorithms. |
| Tasks | |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05253v1 |
| https://arxiv.org/pdf/1906.05253v1.pdf | |
| PWC | https://paperswithcode.com/paper/search-on-the-replay-buffer-bridging-planning |
| Repo | |
| Framework | |
Situational Fusion of Visual Representation for Visual Navigation
| Title | Situational Fusion of Visual Representation for Visual Navigation |
| Authors | William B. Shen, Danfei Xu, Yuke Zhu, Leonidas J. Guibas, Li Fei-Fei, Silvio Savarese |
| Abstract | A complex visual navigation task puts an agent in different situations which call for a diverse range of visual perception abilities. For example, to “go to the nearest chair’', the agent might need to identify a chair in a living room using semantics, follow along a hallway using vanishing point cues, and avoid obstacles using depth. Therefore, utilizing the appropriate visual perception abilities based on a situational understanding of the visual environment can empower these navigation models in unseen visual environments. We propose to train an agent to fuse a large set of visual representations that correspond to diverse visual perception abilities. To fully utilize each representation, we develop an action-level representation fusion scheme, which predicts an action candidate from each representation and adaptively consolidate these action candidates into the final action. Furthermore, we employ a data-driven inter-task affinity regularization to reduce redundancies and improve generalization. Our approach leads to a significantly improved performance in novel environments over ImageNet-pretrained baseline and other fusion methods. |
| Tasks | Visual Navigation |
| Published | 2019-08-24 |
| URL | https://arxiv.org/abs/1908.09073v1 |
| https://arxiv.org/pdf/1908.09073v1.pdf | |
| PWC | https://paperswithcode.com/paper/situational-fusion-of-visual-representation |
| Repo | |
| Framework | |
Efficient Robotic Task Generalization Using Deep Model Fusion Reinforcement Learning
| Title | Efficient Robotic Task Generalization Using Deep Model Fusion Reinforcement Learning |
| Authors | Tianying Wang, Hao Zhang, Wei Qi Toh, Hongyuan Zhu, Cheston Tan, Yan Wu, Yong Liu, Wei Jing |
| Abstract | Learning-based methods have been used to pro-gram robotic tasks in recent years. However, extensive training is usually required not only for the initial task learning but also for generalizing the learned model to the same task but in different environments. In this paper, we propose a novel Deep Reinforcement Learning algorithm for efficient task generalization and environment adaptation in the robotic task learning problem. The proposed method is able to efficiently generalize the previously learned task by model fusion to solve the environment adaptation problem. The proposed Deep Model Fusion (DMF) method reuses and combines the previously trained model to improve the learning efficiency and results.Besides, we also introduce a Multi-objective Guided Reward(MGR) shaping technique to further improve training efficiency.The proposed method was benchmarked with previous methods in various environments to validate its effectiveness. |
| Tasks | |
| Published | 2019-12-11 |
| URL | https://arxiv.org/abs/1912.05205v1 |
| https://arxiv.org/pdf/1912.05205v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-robotic-task-generalization-using |
| Repo | |
| Framework | |
