Paper Group ANR 1158
Learning graphs from data: A signal representation perspective. A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI. A strong baseline for question relevancy ranking. Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation. Securing Distributed Gradient Descent in High Dimensional …
Learning graphs from data: A signal representation perspective
| Title | Learning graphs from data: A signal representation perspective |
| Authors | Xiaowen Dong, Dorina Thanou, Michael Rabbat, Pascal Frossard |
| Abstract | The construction of a meaningful graph topology plays a crucial role in the effective representation, processing, analysis and visualization of structured data. When a natural choice of the graph is not readily available from the data sets, it is thus desirable to infer or learn a graph topology from the data. In this tutorial overview, we survey solutions to the problem of graph learning, including classical viewpoints from statistics and physics, and more recent approaches that adopt a graph signal processing (GSP) perspective. We further emphasize the conceptual similarities and differences between classical and GSP-based graph inference methods, and highlight the potential advantage of the latter in a number of theoretical and practical scenarios. We conclude with several open issues and challenges that are keys to the design of future signal processing and machine learning algorithms for learning graphs from data. |
| Tasks | |
| Published | 2018-06-03 |
| URL | https://arxiv.org/abs/1806.00848v3 |
| https://arxiv.org/pdf/1806.00848v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-graphs-from-data-a-signal |
| Repo | |
| Framework | |
A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI
| Title | A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI |
| Authors | Risheng Liu, Yuxi Zhang, Shichao Cheng, Xin Fan, Zhongxuan Luo |
| Abstract | Magnetic Resonance Imaging (MRI) is one of the most dynamic and safe imaging techniques available for clinical applications. However, the rather slow speed of MRI acquisitions limits the patient throughput and potential indi cations. Compressive Sensing (CS) has proven to be an efficient technique for accelerating MRI acquisition. The most widely used CS-MRI model, founded on the premise of reconstructing an image from an incompletely filled k-space, leads to an ill-posed inverse problem. In the past years, lots of efforts have been made to efficiently optimize the CS-MRI model. Inspired by deep learning techniques, some preliminary works have tried to incorporate deep architectures into CS-MRI process. Unfortunately, the convergence issues (due to the experience-based networks) and the robustness (i.e., lack real-world noise modeling) of these deeply trained optimization methods are still missing. In this work, we develop a new paradigm to integrate designed numerical solvers and the data-driven architectures for CS-MRI. By introducing an optimal condition checking mechanism, we can successfully prove the convergence of our established deep CS-MRI optimization scheme. Furthermore, we explicitly formulate the Rician noise distributions within our framework and obtain an extended CS-MRI network to handle the real-world nosies in the MRI process. Extensive experimental results verify that the proposed paradigm outperforms the existing state-of-the-art techniques both in reconstruction accuracy and efficiency as well as robustness to noises in real scene. |
| Tasks | Compressive Sensing |
| Published | 2018-11-09 |
| URL | http://arxiv.org/abs/1811.03782v3 |
| http://arxiv.org/pdf/1811.03782v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-theoretically-guaranteed-deep-optimization |
| Repo | |
| Framework | |
A strong baseline for question relevancy ranking
| Title | A strong baseline for question relevancy ranking |
| Authors | Ana V. González-Garduño, Isabelle Augenstein, Anders Søgaard |
| Abstract | The best systems at the SemEval-16 and SemEval-17 community question answering shared tasks – a task that amounts to question relevancy ranking – involve complex pipelines and manual feature engineering. Despite this, many of these still fail at beating the IR baseline, i.e., the rankings provided by Google’s search engine. We present a strong baseline for question relevancy ranking by training a simple multi-task feed forward network on a bag of 14 distance measures for the input question pair. This baseline model, which is fast to train and uses only language-independent features, outperforms the best shared task systems on the task of retrieving relevant previously asked questions. |
| Tasks | Community Question Answering, Feature Engineering, Question Answering |
| Published | 2018-08-27 |
| URL | http://arxiv.org/abs/1808.08836v1 |
| http://arxiv.org/pdf/1808.08836v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-strong-baseline-for-question-relevancy |
| Repo | |
| Framework | |
Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation
| Title | Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation |
| Authors | Alistair Shilton, Sunil Gupta, Santu Rana, Pratibha Vellanki, Cheng Li, Laurence Park, Svetha Venkatesh, Alessandra Sutti, David Rubin, Thomas Dorin, Alireza Vahid, Murray Height |
| Abstract | The paper presents a novel approach to direct covariance function learning for Bayesian optimisation, with particular emphasis on experimental design problems where an existing corpus of condensed knowledge is present. The method presented borrows techniques from reproducing kernel Banach space theory (specifically m-kernels) and leverages them to convert (or re-weight) existing covariance functions into new, problem-specific covariance functions. The key advantage of this approach is that rather than relying on the user to manually select (with some hyperparameter tuning and experimentation) an appropriate covariance function it constructs the covariance function to specifically match the problem at hand. The technique is demonstrated on two real-world problems - specifically alloy design and short-polymer fibre manufacturing - as well as a selected test function. |
| Tasks | Bayesian Optimisation |
| Published | 2018-02-15 |
| URL | http://arxiv.org/abs/1802.05370v2 |
| http://arxiv.org/pdf/1802.05370v2.pdf | |
| PWC | https://paperswithcode.com/paper/covariance-function-pre-training-with-m |
| Repo | |
| Framework | |
Securing Distributed Gradient Descent in High Dimensional Statistical Learning
| Title | Securing Distributed Gradient Descent in High Dimensional Statistical Learning |
| Authors | Lili Su, Jiaming Xu |
| Abstract | We consider unreliable distributed learning systems wherein the training data is kept confidential by external workers, and the learner has to interact closely with those workers to train a model. In particular, we assume that there exists a system adversary that can adaptively compromise some workers; the compromised workers deviate from their local designed specifications by sending out arbitrarily malicious messages. We assume in each communication round, up to $q$ out of the $m$ workers suffer Byzantine faults. Each worker keeps a local sample of size $n$ and the total sample size is $N=nm$. We propose a secured variant of the gradient descent method that can tolerate up to a constant fraction of Byzantine workers, i.e., $q/m = O(1)$. Moreover, we show the statistical estimation error of the iterates converges in $O(\log N)$ rounds to $O(\sqrt{q/N} + \sqrt{d/N})$, where $d$ is the model dimension. As long as $q=O(d)$, our proposed algorithm achieves the optimal error rate $O(\sqrt{d/N})$. Our results are obtained under some technical assumptions. Specifically, we assume strongly-convex population risk. Nevertheless, the empirical risk (sample version) is allowed to be non-convex. The core of our method is to robustly aggregate the gradients computed by the workers based on the filtering procedure proposed by Steinhardt et al. On the technical front, deviating from the existing literature on robustly estimating a finite-dimensional mean vector, we establish a {\em uniform} concentration of the sample covariance matrix of gradients, and show that the aggregated gradient, as a function of model parameter, converges uniformly to the true gradient function. To get a near-optimal uniform concentration bound, we develop a new matrix concentration inequality, which might be of independent interest. |
| Tasks | |
| Published | 2018-04-26 |
| URL | https://arxiv.org/abs/1804.10140v3 |
| https://arxiv.org/pdf/1804.10140v3.pdf | |
| PWC | https://paperswithcode.com/paper/securing-distributed-machine-learning-in-high |
| Repo | |
| Framework | |
Predicting Out-of-View Feature Points for Model-Based Camera Pose Estimation
| Title | Predicting Out-of-View Feature Points for Model-Based Camera Pose Estimation |
| Authors | Oliver Moolan-Feroze, Andrew Calway |
| Abstract | In this work we present a novel framework that uses deep learning to predict object feature points that are out-of-view in the input image. This system was developed with the application of model-based tracking in mind, particularly in the case of autonomous inspection robots, where only partial views of the object are available. Out-of-view prediction is enabled by applying scaling to the feature point labels during network training. This is combined with a recurrent neural network architecture designed to provide the final prediction layers with rich feature information from across the spatial extent of the input image. To show the versatility of these out-of-view predictions, we describe how to integrate them in both a particle filter tracker and an optimisation based tracker. To evaluate our work we compared our framework with one that predicts only points inside the image. We show that as the amount of the object in view decreases, being able to predict outside the image bounds adds robustness to the final pose estimation. |
| Tasks | Pose Estimation |
| Published | 2018-03-05 |
| URL | http://arxiv.org/abs/1803.01577v1 |
| http://arxiv.org/pdf/1803.01577v1.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-out-of-view-feature-points-for |
| Repo | |
| Framework | |
Position Bias Estimation for Unbiased Learning-to-Rank in eCommerce Search
| Title | Position Bias Estimation for Unbiased Learning-to-Rank in eCommerce Search |
| Authors | Grigor Aslanyan, Utkarsh Porwal |
| Abstract | The Unbiased Learning-to-Rank framework has been recently proposed as a general approach to systematically remove biases, such as position bias, from learning-to-rank models. The method takes two steps - estimating click propensities and using them to train unbiased models. Most common methods proposed in the literature for estimating propensities involve some degree of intervention in the live search engine. An alternative approach proposed recently uses an Expectation Maximization (EM) algorithm to estimate propensities by using ranking features for estimating relevances. In this work we propose a novel method to directly estimate propensities which does not use any intervention in live search or rely on modeling relevance. Rather, we take advantage of the fact that the same query-document pair may naturally change ranks over time. This typically occurs for eCommerce search because of change of popularity of items over time, existence of time dependent ranking features, or addition or removal of items to the index (an item getting sold or a new item being listed). However, our method is general and can be applied to any search engine for which the rank of the same document may naturally change over time for the same query. We derive a simple likelihood function that depends on propensities only, and by maximizing the likelihood we are able to get estimates of the propensities. We apply this method to eBay search data to estimate click propensities for web and mobile search and compare these with estimates using the EM method. We also use simulated data to show that the method gives reliable estimates of the “true” simulated propensities. Finally, we train an unbiased learning-to-rank model for eBay search using the estimated propensities and show that it outperforms both baselines - one without position bias correction and one with position bias correction using the EM method. |
| Tasks | Learning-To-Rank |
| Published | 2018-12-21 |
| URL | https://arxiv.org/abs/1812.09338v3 |
| https://arxiv.org/pdf/1812.09338v3.pdf | |
| PWC | https://paperswithcode.com/paper/direct-estimation-of-position-bias-for |
| Repo | |
| Framework | |
Efficient Video Understanding via Layered Multi Frame-Rate Analysis
| Title | Efficient Video Understanding via Layered Multi Frame-Rate Analysis |
| Authors | Ziyao Tang, Yongxi Lu, Tara Javidi |
| Abstract | One of the greatest challenges in the design of a real-time perception system for autonomous driving vehicles and drones is the conflicting requirement of safety (high prediction accuracy) and efficiency. Traditional approaches use a single frame rate for the entire system. Motivated by the observation that the lack of robustness against environmental factors is the major weakness of compact ConvNet architectures, we propose a dual frame-rate system that brings in the best of both worlds: A modulator stream that executes an expensive models robust to environmental factors at a low frame rate to extract slowly changing features describing the environment, and a prediction stream that executes a light-weight model at real-time to extract transient signals that describes particularities of the current frame. The advantage of our design is validated by our extensive empirical study, showing that our solution leads to consistent improvements using a variety of backbone architecture choice and input resolutions. These findings suggest multiple frame-rate systems as a promising direction in designing efficient perception for autonomous agents. |
| Tasks | Autonomous Driving, Video Understanding |
| Published | 2018-11-24 |
| URL | http://arxiv.org/abs/1811.09834v1 |
| http://arxiv.org/pdf/1811.09834v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-video-understanding-via-layered |
| Repo | |
| Framework | |
A Stochastic Large-scale Machine Learning Algorithm for Distributed Features and Observations
| Title | A Stochastic Large-scale Machine Learning Algorithm for Distributed Features and Observations |
| Authors | Biyi Fang, Diego Klabjan |
| Abstract | As the size of modern data sets exceeds the disk and memory capacities of a single computer, machine learning practitioners have resorted to parallel and distributed computing. Given that optimization is one of the pillars of machine learning and predictive modeling, distributed optimization methods have recently garnered ample attention, in particular when either observations or features are distributed, but not both. We propose a general stochastic algorithm where observations, features, and gradient components can be sampled in a double distributed setting, i.e., with both features and observations distributed. Very technical analyses establish convergence properties of the algorithm under different conditions on the learning rate (diminishing to zero or constant). Computational experiments in Spark demonstrate a superior performance of our algorithm versus a benchmark in early iterations of the algorithm, which is due to the stochastic components of the algorithm. |
| Tasks | Distributed Optimization |
| Published | 2018-03-29 |
| URL | https://arxiv.org/abs/1803.11287v2 |
| https://arxiv.org/pdf/1803.11287v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-stochastic-large-scale-machine-learning |
| Repo | |
| Framework | |
Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems
| Title | Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems |
| Authors | Junjie Liang, Jinlong Hu, Shoubin Dong, Vasant Honavar |
| Abstract | We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models for reliably recommending the N top-ranked items. The proposed models optimize a variant of the widely used discounted cumulative gain (DCG) objective function which differs from DCG in two important aspects: (i) It limits the evaluation of DCG only on the top N items in the ranked lists, thereby eliminating the impact of low-ranked items on the learned ranking function; and (ii) it incorporates weights that allow the model to leverage multiple types of implicit feedback with differing levels of reliability or trustworthiness. Because the resulting objective function is non-smooth and hence challenging to optimize, we consider two smooth approximations of the objective function, using the traditional sigmoid function and the rectified linear unit (ReLU). We propose a family of learning-to-rank algorithms (Top-N-Rank) that work with any smooth objective function. Then, a more efficient variant, Top-N-Rank.ReLU, is introduced, which effectively exploits the properties of ReLU function to reduce the computational complexity of Top-N-Rank from quadratic to linear in the average number of items rated by users. The results of our experiments using two widely used benchmarks, namely, the MovieLens data set and the Amazon Video Games data set demonstrate that: (i) The `top-N truncation’ of the objective function substantially improves the ranking quality of the top N recommendations; (ii) using the ReLU for smoothing the objective function yields significant improvement in both ranking quality as well as runtime as compared to using the sigmoid; and (iii) Top-N-Rank.ReLU substantially outperforms the well-performing list-wise ranking methods in terms of ranking quality. | |
| Tasks | Learning-To-Rank, Recommendation Systems |
| Published | 2018-12-10 |
| URL | http://arxiv.org/abs/1812.04109v2 |
| http://arxiv.org/pdf/1812.04109v2.pdf | |
| PWC | https://paperswithcode.com/paper/top-n-rank-a-scalable-list-wise-ranking |
| Repo | |
| Framework | |
On the Brain Networks of Complex Problem Solving
| Title | On the Brain Networks of Complex Problem Solving |
| Authors | Abdullah Alchihabi, Omer Ekmekci, Baran B. Kivilcim, Sharlene D. Newman, Fatos T. Yarman Vural |
| Abstract | Complex problem solving is a high level cognitive process which has been thoroughly studied over the last decade. The Tower of London (TOL) is a task that has been widely used to study problem-solving. In this study, we aim to explore the underlying cognitive network dynamics among anatomical regions of complex problem solving and its sub-phases, namely planning and execution. A new brain network construction model establishing dynamic functional brain networks using fMRI is proposed. The first step of the model is a preprocessing pipeline that manages to decrease the spatial redundancy while increasing the temporal resolution of the fMRI recordings. Then, dynamic brain networks are estimated using artificial neural networks. The network properties of the estimated brain networks are studied in order to identify regions of interest, such as hubs and subgroups of densely connected brain regions. The major similarities and dissimilarities of the network structure of planning and execution phases are highlighted. Our findings show the hubs and clusters of densely interconnected regions during both subtasks. It is observed that there are more hubs during the planning phase compared to the execution phase, and the clusters are more strongly connected during planning compared to execution. |
| Tasks | |
| Published | 2018-10-10 |
| URL | http://arxiv.org/abs/1810.05077v1 |
| http://arxiv.org/pdf/1810.05077v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-brain-networks-of-complex-problem |
| Repo | |
| Framework | |
Dynamically Context-Sensitive Time-Decay Attention for Dialogue Modeling
| Title | Dynamically Context-Sensitive Time-Decay Attention for Dialogue Modeling |
| Authors | Shang-Yu Su, Pei-Chieh Yuan, Yun-Nung Chen |
| Abstract | Spoken language understanding (SLU) is an essential component in conversational systems. Considering that contexts provide informative cues for better understanding, history can be leveraged for contextual SLU. However, most prior work only paid attention to the related content in history utterances and ignored the temporal information. In dialogues, it is intuitive that the most recent utterances are more important than the least recent ones, and time-aware attention should be in a decaying manner. Therefore, this paper allows the model to automatically learn a time-decay attention function where the attentional weights can be dynamically decided based on the content of each role’s contexts, which effectively integrates both content-aware and time-aware perspectives and demonstrates remarkable flexibility to complex dialogue contexts. The experiments on the benchmark Dialogue State Tracking Challenge (DSTC4) dataset show that the proposed dynamically context-sensitive time-decay attention mechanisms significantly improve the state-of-the-art model for contextual understanding performance. |
| Tasks | Dialogue State Tracking, Spoken Language Understanding |
| Published | 2018-09-05 |
| URL | http://arxiv.org/abs/1809.01557v2 |
| http://arxiv.org/pdf/1809.01557v2.pdf | |
| PWC | https://paperswithcode.com/paper/dynamically-context-sensitive-time-decay |
| Repo | |
| Framework | |
Intervention Harvesting for Context-Dependent Examination-Bias Estimation
| Title | Intervention Harvesting for Context-Dependent Examination-Bias Estimation |
| Authors | Zhichong Fang, Aman Agarwal, Thorsten Joachims |
| Abstract | Accurate estimates of examination bias are crucial for unbiased learning-to-rank from implicit feedback in search engines and recommender systems, since they enable the use of Inverse Propensity Score (IPS) weighting techniques to address selection biases and missing data. Unfortunately, existing examination-bias estimators are limited to the Position-Based Model (PBM), where the examination bias may only depend on the rank of the document. To overcome this limitation, we propose a Contextual Position-Based Model (CPBM) where the examination bias may also depend on a context vector describing the query and the user. Furthermore, we propose an effective estimator for the CPBM based on intervention harvesting. A key feature of the estimator is that it does not require disruptive interventions but merely exploits natural variation resulting from the use of multiple historic ranking functions. Real-world experiments on the ArXiv search engine and semi-synthetic experiments on the Yahoo Learning-To-Rank dataset demonstrate the superior effectiveness and robustness of the new approach. |
| Tasks | Learning-To-Rank, Recommendation Systems |
| Published | 2018-11-05 |
| URL | https://arxiv.org/abs/1811.01802v3 |
| https://arxiv.org/pdf/1811.01802v3.pdf | |
| PWC | https://paperswithcode.com/paper/intervention-harvesting-for-context-dependent |
| Repo | |
| Framework | |
Consistent polynomial-time unseeded graph matching for Lipschitz graphons
| Title | Consistent polynomial-time unseeded graph matching for Lipschitz graphons |
| Authors | Yuan Zhang |
| Abstract | We propose a consistent polynomial-time method for the unseeded node matching problem for networks with smooth underlying structures. Despite widely conjectured by the research community that the structured graph matching problem to be significantly easier than its worst case counterpart, well-known to be NP-hard, the statistical version of the problem has stood a challenge that resisted any solution both provable and polynomial-time. The closest existing work requires quasi-polynomial time. Our method is based on the latest advances in graphon estimation techniques and analysis on the concentration of empirical Wasserstein distances. Its core is a simple yet unconventional sampling-and-matching scheme that reduces the problem from unseeded to seeded. Our method allows flexible efficiencies, is convenient to analyze and potentially can be extended to more general settings. Our work enables a rich variety of subsequent estimations and inferences. |
| Tasks | Graph Matching, Graphon Estimation |
| Published | 2018-07-29 |
| URL | http://arxiv.org/abs/1807.11027v1 |
| http://arxiv.org/pdf/1807.11027v1.pdf | |
| PWC | https://paperswithcode.com/paper/consistent-polynomial-time-unseeded-graph |
| Repo | |
| Framework | |
Learning to Rank Query Graphs for Complex Question Answering over Knowledge Graphs
| Title | Learning to Rank Query Graphs for Complex Question Answering over Knowledge Graphs |
| Authors | Gaurav Maheshwari, Priyansh Trivedi, Denis Lukovnikov, Nilesh Chakraborty, Asja Fischer, Jens Lehmann |
| Abstract | In this paper, we conduct an empirical investigation of neural query graph ranking approaches for the task of complex question answering over knowledge graphs. We experiment with six different ranking models and propose a novel self-attention based slot matching model which exploits the inherent structure of query graphs, our logical form of choice. Our proposed model generally outperforms the other models on two QA datasets over the DBpedia knowledge graph, evaluated in different settings. In addition, we show that transfer learning from the larger of those QA datasets to the smaller dataset yields substantial improvements, effectively offsetting the general lack of training data. |
| Tasks | Graph Ranking, Knowledge Graphs, Learning-To-Rank, Question Answering, Transfer Learning |
| Published | 2018-11-02 |
| URL | http://arxiv.org/abs/1811.01118v1 |
| http://arxiv.org/pdf/1811.01118v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-rank-query-graphs-for-complex |
| Repo | |
| Framework | |
