Paper Group ANR 279
Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems. Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes. Task Specific Adversarial Cost Function. Global Continuous Optimization with Error Bound and Fast Convergence. Swipe Mosaics from Video. Observing and Recom …
Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems
| Title | Funneled Bayesian Optimization for Design, Tuning and Control of Autonomous Systems |
| Authors | Ruben Martinez-Cantin |
| Abstract | Bayesian optimization has become a fundamental global optimization algorithm in many problems where sample efficiency is of paramount importance. Recently, there has been proposed a large number of new applications in fields such as robotics, machine learning, experimental design, simulation, etc. In this paper, we focus on several problems that appear in robotics and autonomous systems: algorithm tuning, automatic control and intelligent design. All those problems can be mapped to global optimization problems. However, they become hard optimization problems. Bayesian optimization internally uses a probabilistic surrogate model (e.g.: Gaussian process) to learn from the process and reduce the number of samples required. In order to generalize to unknown functions in a black-box fashion, the common assumption is that the underlying function can be modeled with a stationary process. Nonstationary Gaussian process regression cannot generalize easily and it typically requires prior knowledge of the function. Some works have designed techniques to generalize Bayesian optimization to nonstationary functions in an indirect way, but using techniques originally designed for regression, where the objective is to improve the quality of the surrogate model everywhere. Instead optimization should focus on improving the surrogate model near the optimum. In this paper, we present a novel kernel function specially designed for Bayesian optimization, that allows nonstationary behavior of the surrogate model in an adaptive local region. In our experiments, we found that this new kernel results in an improved local search (exploitation), without penalizing the global search (exploration). We provide results in well-known benchmarks and real applications. The new method outperforms the state of the art in Bayesian optimization both in stationary and nonstationary problems. |
| Tasks | |
| Published | 2016-10-02 |
| URL | http://arxiv.org/abs/1610.00366v2 |
| http://arxiv.org/pdf/1610.00366v2.pdf | |
| PWC | https://paperswithcode.com/paper/funneled-bayesian-optimization-for-design |
| Repo | |
| Framework | |
Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes
| Title | Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes |
| Authors | Mohammad Gheshlaghi Azar, Eva Dyer, Konrad Kording |
| Abstract | Finding efficient and provable methods to solve non-convex optimization problems is an outstanding challenge in machine learning and optimization theory. A popular approach used to tackle non-convex problems is to use convex relaxation techniques to find a convex surrogate for the problem. Unfortunately, convex relaxations typically must be found on a problem-by-problem basis. Thus, providing a general-purpose strategy to estimate a convex relaxation would have a wide reaching impact. Here, we introduce Convex Relaxation Regression (CoRR), an approach for learning convex relaxations for a class of smooth functions. The main idea behind our approach is to estimate the convex envelope of a function $f$ by evaluating $f$ at a set of $T$ random points and then fitting a convex function to these function evaluations. We prove that with probability greater than $1-\delta$, the solution of our algorithm converges to the global optimizer of $f$ with error $\mathcal{O} \Big( \big(\frac{\log(1/\delta) }{T} \big)^{\alpha} \Big)$ for some $\alpha> 0$. Our approach enables the use of convex optimization tools to solve a class of non-convex optimization problems. |
| Tasks | |
| Published | 2016-02-05 |
| URL | http://arxiv.org/abs/1602.02191v3 |
| http://arxiv.org/pdf/1602.02191v3.pdf | |
| PWC | https://paperswithcode.com/paper/convex-relaxation-regression-black-box |
| Repo | |
| Framework | |
Task Specific Adversarial Cost Function
| Title | Task Specific Adversarial Cost Function |
| Authors | Antonia Creswell, Anil A. Bharath |
| Abstract | The cost function used to train a generative model should fit the purpose of the model. If the model is intended for tasks such as generating perceptually correct samples, it is beneficial to maximise the likelihood of a sample drawn from the model, Q, coming from the same distribution as the training data, P. This is equivalent to minimising the Kullback-Leibler (KL) distance, KL[QP]. However, if the model is intended for tasks such as retrieval or classification it is beneficial to maximise the likelihood that a sample drawn from the training data is captured by the model, equivalent to minimising KL[PQ]. The cost function used in adversarial training optimises the Jensen-Shannon entropy which can be seen as an even interpolation between KL[QP] and KL[PQ]. Here, we propose an alternative adversarial cost function which allows easy tuning of the model for either task. Our task specific cost function is evaluated on a dataset of hand-written characters in the following tasks: Generation, retrieval and one-shot learning. |
| Tasks | One-Shot Learning |
| Published | 2016-09-27 |
| URL | http://arxiv.org/abs/1609.08661v1 |
| http://arxiv.org/pdf/1609.08661v1.pdf | |
| PWC | https://paperswithcode.com/paper/task-specific-adversarial-cost-function |
| Repo | |
| Framework | |
Global Continuous Optimization with Error Bound and Fast Convergence
| Title | Global Continuous Optimization with Error Bound and Fast Convergence |
| Authors | Kenji Kawaguchi, Yu Maruyama, Xiaoyu Zheng |
| Abstract | This paper considers global optimization with a black-box unknown objective function that can be non-convex and non-differentiable. Such a difficult optimization problem arises in many real-world applications, such as parameter tuning in machine learning, engineering design problem, and planning with a complex physics simulator. This paper proposes a new global optimization algorithm, called Locally Oriented Global Optimization (LOGO), to aim for both fast convergence in practice and finite-time error bound in theory. The advantage and usage of the new algorithm are illustrated via theoretical analysis and an experiment conducted with 11 benchmark test functions. Further, we modify the LOGO algorithm to specifically solve a planning problem via policy search with continuous state/action space and long time horizon while maintaining its finite-time error bound. We apply the proposed planning method to accident management of a nuclear power plant. The result of the application study demonstrates the practical utility of our method. |
| Tasks | |
| Published | 2016-07-17 |
| URL | http://arxiv.org/abs/1607.04817v1 |
| http://arxiv.org/pdf/1607.04817v1.pdf | |
| PWC | https://paperswithcode.com/paper/global-continuous-optimization-with-error |
| Repo | |
| Framework | |
Swipe Mosaics from Video
| Title | Swipe Mosaics from Video |
| Authors | Malcolm Reynolds, Tom S. F. Haines, Gabriel J. Brostow |
| Abstract | A panoramic image mosaic is an attractive visualization for viewing many overlapping photos, but its images must be both captured and processed correctly to produce an acceptable composite. We propose Swipe Mosaics, an interactive visualization that places the individual video frames on a 2D planar map that represents the layout of the physical scene. Compared to traditional panoramic mosaics, our capture is easier because the user can both translate the camera center and film moving subjects. Processing and display degrade gracefully if the footage lacks distinct, overlapping, non-repeating texture. Our proposed visual odometry algorithm produces a distribution over (x,y) translations for image pairs. Inferring a distribution of possible camera motions allows us to better cope with parallax, lack of texture, dynamic scenes, and other phenomena that hurt deterministic reconstruction techniques. Robustness is obtained by training on synthetic scenes with known camera motions. We show that Swipe Mosaics are easy to generate, support a wide range of difficult scenes, and are useful for documenting a scene for closer inspection. |
| Tasks | Visual Odometry |
| Published | 2016-09-26 |
| URL | http://arxiv.org/abs/1609.08080v1 |
| http://arxiv.org/pdf/1609.08080v1.pdf | |
| PWC | https://paperswithcode.com/paper/swipe-mosaics-from-video |
| Repo | |
| Framework | |
Observing and Recommending from a Social Web with Biases
| Title | Observing and Recommending from a Social Web with Biases |
| Authors | Steffen Staab, Sophie Stalla-Bourdillon, Laura Carmichael |
| Abstract | The research question this report addresses is: how, and to what extent, those directly involved with the design, development and employment of a specific black box algorithm can be certain that it is not unlawfully discriminating (directly and/or indirectly) against particular persons with protected characteristics (e.g. gender, race and ethnicity)? |
| Tasks | |
| Published | 2016-04-25 |
| URL | http://arxiv.org/abs/1604.07180v1 |
| http://arxiv.org/pdf/1604.07180v1.pdf | |
| PWC | https://paperswithcode.com/paper/observing-and-recommending-from-a-social-web |
| Repo | |
| Framework | |
Learning Opposites Using Neural Networks
| Title | Learning Opposites Using Neural Networks |
| Authors | Shivam Kalra, Aditya Sriram, Shahryar Rahnamayan, H. R. Tizhoosh |
| Abstract | Many research works have successfully extended algorithms such as evolutionary algorithms, reinforcement agents and neural networks using “opposition-based learning” (OBL). Two types of the “opposites” have been defined in the literature, namely \textit{type-I} and \textit{type-II}. The former are linear in nature and applicable to the variable space, hence easy to calculate. On the other hand, type-II opposites capture the “oppositeness” in the output space. In fact, type-I opposites are considered a special case of type-II opposites where inputs and outputs have a linear relationship. However, in many real-world problems, inputs and outputs do in fact exhibit a nonlinear relationship. Therefore, type-II opposites are expected to be better in capturing the sense of “opposition” in terms of the input-output relation. In the absence of any knowledge about the problem at hand, there seems to be no intuitive way to calculate the type-II opposites. In this paper, we introduce an approach to learn type-II opposites from the given inputs and their outputs using the artificial neural networks (ANNs). We first perform \emph{opposition mining} on the sample data, and then use the mined data to learn the relationship between input $x$ and its opposite $\breve{x}$. We have validated our algorithm using various benchmark functions to compare it against an evolving fuzzy inference approach that has been recently introduced. The results show the better performance of a neural approach to learn the opposites. This will create new possibilities for integrating oppositional schemes within existing algorithms promising a potential increase in convergence speed and/or accuracy. |
| Tasks | |
| Published | 2016-09-16 |
| URL | http://arxiv.org/abs/1609.05123v1 |
| http://arxiv.org/pdf/1609.05123v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-opposites-using-neural-networks |
| Repo | |
| Framework | |
Summary Transfer: Exemplar-based Subset Selection for Video Summarization
| Title | Summary Transfer: Exemplar-based Subset Selection for Video Summarization |
| Authors | Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman |
| Abstract | Video summarization has unprecedented importance to help us digest, browse, and search today’s ever-growing video collections. We propose a novel subset selection technique that leverages supervision in the form of human-created summaries to perform automatic keyframe-based video summarization. The main idea is to nonparametrically transfer summary structures from annotated videos to unseen test videos. We show how to extend our method to exploit semantic side information about the video’s category/genre to guide the transfer process by those training videos semantically consistent with the test input. We also show how to generalize our method to subshot-based summarization, which not only reduces computational costs but also provides more flexible ways of defining visual similarity across subshots spanning several frames. We conduct extensive evaluation on several benchmarks and demonstrate promising results, outperforming existing methods in several settings. |
| Tasks | Video Summarization |
| Published | 2016-03-10 |
| URL | http://arxiv.org/abs/1603.03369v3 |
| http://arxiv.org/pdf/1603.03369v3.pdf | |
| PWC | https://paperswithcode.com/paper/summary-transfer-exemplar-based-subset |
| Repo | |
| Framework | |
Formula of Volume of Revolution with Integration by Parts and Extension
| Title | Formula of Volume of Revolution with Integration by Parts and Extension |
| Authors | Yi Liu, Jingwei Liu |
| Abstract | A calculation formula of volume of revolution with integration by parts of definite integral is derived based on monotone function, and extended to a general case that curved trapezoids is determined by continuous, piecewise strictly monotone and differential function. And, two examples are given, ones curvilinear trapezoids is determined by Kepler equation, and the other curvilinear trapezoids is a function transmuted from Kepler equation. |
| Tasks | |
| Published | 2016-09-04 |
| URL | http://arxiv.org/abs/1609.04771v1 |
| http://arxiv.org/pdf/1609.04771v1.pdf | |
| PWC | https://paperswithcode.com/paper/formula-of-volume-of-revolution-with |
| Repo | |
| Framework | |
A Discrete Firefly Algorithm to Solve a Rich Vehicle Routing Problem Modelling a Newspaper Distribution System with Recycling Policy
| Title | A Discrete Firefly Algorithm to Solve a Rich Vehicle Routing Problem Modelling a Newspaper Distribution System with Recycling Policy |
| Authors | E. Osaba, Xin-She Yang, F. Diaz, E. Onieva, A. D. Masegosa, A. Perallos |
| Abstract | A real-world newspaper distribution problem with recycling policy is tackled in this work. In order to meet all the complex restrictions contained in such a problem, it has been modeled as a rich vehicle routing problem, which can be more specifically considered as an asymmetric and clustered vehicle routing problem with simultaneous pickup and deliveries, variable costs and forbidden paths (AC-VRP-SPDVCFP). This is the first study of such a problem in the literature. For this reason, a benchmark composed by 15 instances has been also proposed. In the design of this benchmark, real geographical positions have been used, located in the province of Bizkaia, Spain. For the proper treatment of this AC-VRP-SPDVCFP, a discrete firefly algorithm (DFA) has been developed. This application is the first application of the firefly algorithm to any rich vehicle routing problem. To prove that the proposed DFA is a promising technique, its performance has been compared with two other well-known techniques: an evolutionary algorithm and an evolutionary simulated annealing. Our results have shown that the DFA has outperformed these two classic meta-heuristics. |
| Tasks | |
| Published | 2016-04-14 |
| URL | http://arxiv.org/abs/1604.04146v1 |
| http://arxiv.org/pdf/1604.04146v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-discrete-firefly-algorithm-to-solve-a-rich |
| Repo | |
| Framework | |
Scalable Modeling of Multivariate Longitudinal Data for Prediction of Chronic Kidney Disease Progression
| Title | Scalable Modeling of Multivariate Longitudinal Data for Prediction of Chronic Kidney Disease Progression |
| Authors | Joseph Futoma, Mark Sendak, C. Blake Cameron, Katherine Heller |
| Abstract | Prediction of the future trajectory of a disease is an important challenge for personalized medicine and population health management. However, many complex chronic diseases exhibit large degrees of heterogeneity, and furthermore there is not always a single readily available biomarker to quantify disease severity. Even when such a clinical variable exists, there are often additional related biomarkers routinely measured for patients that may better inform the predictions of their future disease state. To this end, we propose a novel probabilistic generative model for multivariate longitudinal data that captures dependencies between multivariate trajectories. We use a Gaussian process based regression model for each individual trajectory, and build off ideas from latent class models to induce dependence between their mean functions. We fit our method using a scalable variational inference algorithm to a large dataset of longitudinal electronic patient health records, and find that it improves dynamic predictions compared to a recent state of the art method. Our local accountable care organization then uses the model predictions during chart reviews of high risk patients with chronic kidney disease. |
| Tasks | |
| Published | 2016-08-16 |
| URL | http://arxiv.org/abs/1608.04615v1 |
| http://arxiv.org/pdf/1608.04615v1.pdf | |
| PWC | https://paperswithcode.com/paper/scalable-modeling-of-multivariate |
| Repo | |
| Framework | |
Temporally coherent 4D reconstruction of complex dynamic scenes
| Title | Temporally coherent 4D reconstruction of complex dynamic scenes |
| Authors | Armin Mustafa, Hansung Kim, Jean-Yves Guillemaut, Adrian Hilton |
| Abstract | This paper presents an approach for reconstruction of 4D temporally coherent models of complex dynamic scenes. No prior knowledge is required of scene structure or camera calibration allowing reconstruction from multiple moving cameras. Sparse-to-dense temporal correspondence is integrated with joint multi-view segmentation and reconstruction to obtain a complete 4D representation of static and dynamic objects. Temporal coherence is exploited to overcome visual ambiguities resulting in improved reconstruction of complex scenes. Robust joint segmentation and reconstruction of dynamic objects is achieved by introducing a geodesic star convexity constraint. Comparative evaluation is performed on a variety of unstructured indoor and outdoor dynamic scenes with hand-held cameras and multiple people. This demonstrates reconstruction of complete temporally coherent 4D scene models with improved nonrigid object segmentation and shape reconstruction. |
| Tasks | Calibration, Scene Segmentation, Semantic Segmentation |
| Published | 2016-03-10 |
| URL | http://arxiv.org/abs/1603.03381v2 |
| http://arxiv.org/pdf/1603.03381v2.pdf | |
| PWC | https://paperswithcode.com/paper/temporally-coherent-4d-reconstruction-of |
| Repo | |
| Framework | |
Underwater Fish Tracking for Moving Cameras based on Deformable Multiple Kernels
| Title | Underwater Fish Tracking for Moving Cameras based on Deformable Multiple Kernels |
| Authors | Meng-Che Chuang, Jenq-Neng Hwang, Jian-Hui Ye, Shih-Chia Huang, Kresimir Williams |
| Abstract | Fishery surveys that call for the use of single or multiple underwater cameras have been an emerging technology as a non-extractive mean to estimate the abundance of fish stocks. Tracking live fish in an open aquatic environment posts challenges that are different from general pedestrian or vehicle tracking in surveillance applications. In many rough habitats fish are monitored by cameras installed on moving platforms, where tracking is even more challenging due to inapplicability of background models. In this paper, a novel tracking algorithm based on the deformable multiple kernels (DMK) is proposed to address these challenges. Inspired by the deformable part model (DPM) technique, a set of kernels is defined to represent the holistic object and several parts that are arranged in a deformable configuration. Color histogram, texture histogram and the histogram of oriented gradients (HOG) are extracted and serve as object features. Kernel motion is efficiently estimated by the mean-shift algorithm on color and texture features to realize tracking. Furthermore, the HOG-feature deformation costs are adopted as soft constraints on kernel positions to maintain the part configuration. Experimental results on practical video set from underwater moving cameras show the reliable performance of the proposed method with much less computational cost comparing with state-of-the-art techniques. |
| Tasks | |
| Published | 2016-03-05 |
| URL | http://arxiv.org/abs/1603.01695v1 |
| http://arxiv.org/pdf/1603.01695v1.pdf | |
| PWC | https://paperswithcode.com/paper/underwater-fish-tracking-for-moving-cameras |
| Repo | |
| Framework | |
Comparison of feature extraction and dimensionality reduction methods for single channel extracellular spike sorting
| Title | Comparison of feature extraction and dimensionality reduction methods for single channel extracellular spike sorting |
| Authors | Anupam Mitra, Anagh Pathak, Kaushik Majumdar |
| Abstract | Spikes in the membrane electrical potentials of neurons play a major role in the functioning of nervous systems of animals. Obtaining the spikes from different neurons has been a challenging problem for decades. Several schemes have been proposed for spike sorting to isolate the spikes of individual neurons from electrical recordings in extracellular media. However, there is much scope for improvement in the accuracies obtained using the prevailing methods of spike sorting. To determine more effective spike sorting strategies using well known methods, we compared different types of signal features and techniques for dimensionality reduction in feature space. We tried to determine an optimum or near optimum feature extraction and dimensionality reduction methods and an optimum or near optimum number of features for spike sorting. We assessed relative performance of well known methods on simulated recordings specially designed for development and benchmarking of spike sorting schemes, with varying number of spike classes and the well established method of $k$-means clustering of selected features. We found that almost all well known methods performed quite well. Nevertheless, from spike waveforms of 64 samples, sampled at 24 kHz, using principal component analysis (PCA) to select around 46 to 55 features led to the better spike sorting performance than most other methods (Wilcoxon signed rank sum test, $p < 0.001$). |
| Tasks | Dimensionality Reduction |
| Published | 2016-02-10 |
| URL | http://arxiv.org/abs/1602.03379v1 |
| http://arxiv.org/pdf/1602.03379v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparison-of-feature-extraction-and |
| Repo | |
| Framework | |
How Well Do Local Algorithms Solve Semidefinite Programs?
| Title | How Well Do Local Algorithms Solve Semidefinite Programs? |
| Authors | Zhou Fan, Andrea Montanari |
| Abstract | Several probabilistic models from high-dimensional statistics and machine learning reveal an intriguing –and yet poorly understood– dichotomy. Either simple local algorithms succeed in estimating the object of interest, or even sophisticated semi-definite programming (SDP) relaxations fail. In order to explore this phenomenon, we study a classical SDP relaxation of the minimum graph bisection problem, when applied to Erd\H{o}s-Renyi random graphs with bounded average degree $d>1$, and obtain several types of results. First, we use a dual witness construction (using the so-called non-backtracking matrix of the graph) to upper bound the SDP value. Second, we prove that a simple local algorithm approximately solves the SDP to within a factor $2d^2/(2d^2+d-1)$ of the upper bound. In particular, the local algorithm is at most $8/9$ suboptimal, and $1+O(1/d)$ suboptimal for large degree. We then analyze a more sophisticated local algorithm, which aggregates information according to the harmonic measure on the limiting Galton-Watson (GW) tree. The resulting lower bound is expressed in terms of the conductance of the GW tree and matches surprisingly well the empirically determined SDP values on large-scale Erd\H{o}s-Renyi graphs. We finally consider the planted partition model. In this case, purely local algorithms are known to fail, but they do succeed if a small amount of side information is available. Our results imply quantitative bounds on the threshold for partial recovery using SDP in this model. |
| Tasks | |
| Published | 2016-10-17 |
| URL | http://arxiv.org/abs/1610.05350v1 |
| http://arxiv.org/pdf/1610.05350v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-well-do-local-algorithms-solve |
| Repo | |
| Framework | |
