Paper Group ANR 920
Combinatorial Bayesian Optimization using the Graph Cartesian Product. Deep Algorithm Unrolling for Blind Image Deblurring. Adversarial Robustness May Be at Odds With Simplicity. On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems. On the approximation of the solution of partial differential equations by …
Combinatorial Bayesian Optimization using the Graph Cartesian Product
| Title | Combinatorial Bayesian Optimization using the Graph Cartesian Product |
| Authors | Changyong Oh, Jakub M. Tomczak, Efstratios Gavves, Max Welling |
| Abstract | This paper focuses on Bayesian Optimization (BO) for objectives on combinatorial search spaces, including ordinal and categorical variables. Despite the abundance of potential applications of Combinatorial BO, including chipset configuration search and neural architecture search, only a handful of methods have been proposed. We introduce COMBO, a new Gaussian Process (GP) BO. COMBO quantifies “smoothness” of functions on combinatorial search spaces by utilizing a combinatorial graph. The vertex set of the combinatorial graph consists of all possible joint assignments of the variables, while edges are constructed using the graph Cartesian product of the sub-graphs that represent the individual variables. On this combinatorial graph, we propose an ARD diffusion kernel with which the GP is able to model high-order interactions between variables leading to better performance. Moreover, using the Horseshoe prior for the scale parameter in the ARD diffusion kernel results in an effective variable selection procedure, making COMBO suitable for high dimensional problems. Computationally, in COMBO the graph Cartesian product allows the Graph Fourier Transform calculation to scale linearly instead of exponentially. We validate COMBO in a wide array of realistic benchmarks, including weighted maximum satisfiability problems and neural architecture search. COMBO outperforms consistently the latest state-of-the-art while maintaining computational and statistical efficiency. |
| Tasks | Neural Architecture Search |
| Published | 2019-02-01 |
| URL | https://arxiv.org/abs/1902.00448v2 |
| https://arxiv.org/pdf/1902.00448v2.pdf | |
| PWC | https://paperswithcode.com/paper/combinatorial-bayesian-optimization-using |
| Repo | |
| Framework | |
Deep Algorithm Unrolling for Blind Image Deblurring
| Title | Deep Algorithm Unrolling for Blind Image Deblurring |
| Authors | Yuelong Li, Mohammad Tofighi, Junyi Geng, Vishal Monga, Yonina C. Eldar |
| Abstract | Blind image deblurring remains a topic of enduring interest. Learning based approaches, especially those that employ neural networks have emerged to complement traditional model based methods and in many cases achieve vastly enhanced performance. That said, neural network approaches are generally empirically designed and the underlying structures are difficult to interpret. In recent years, a promising technique called algorithm unrolling has been developed that has helped connect iterative algorithms such as those for sparse coding to neural network architectures. However, such connections have not been made yet for blind image deblurring. In this paper, we propose a neural network architecture based on this idea. We first present an iterative algorithm that may be considered as a generalization of the traditional total-variation regularization method in the gradient domain. We then unroll the algorithm to construct a neural network for image deblurring which we refer to as Deep Unrolling for Blind Deblurring (DUBLID). Key algorithm parameters are learned with the help of training images. Our proposed deep network DUBLID achieves significant practical performance gains while enjoying interpretability at the same time. Extensive experimental results show that DUBLID outperforms many state-of-the-art methods and in addition is computationally faster. |
| Tasks | Blind Image Deblurring, Deblurring |
| Published | 2019-02-09 |
| URL | https://arxiv.org/abs/1902.03493v3 |
| https://arxiv.org/pdf/1902.03493v3.pdf | |
| PWC | https://paperswithcode.com/paper/an-algorithm-unrolling-approach-to-deep-blind |
| Repo | |
| Framework | |
Adversarial Robustness May Be at Odds With Simplicity
| Title | Adversarial Robustness May Be at Odds With Simplicity |
| Authors | Preetum Nakkiran |
| Abstract | Current techniques in machine learning are so far are unable to learn classifiers that are robust to adversarial perturbations. However, they are able to learn non-robust classifiers with very high accuracy, even in the presence of random perturbations. Towards explaining this gap, we highlight the hypothesis that $\textit{robust classification may require more complex classifiers (i.e. more capacity) than standard classification.}$ In this note, we show that this hypothesis is indeed possible, by giving several theoretical examples of classification tasks and sets of “simple” classifiers for which: (1) There exists a simple classifier with high standard accuracy, and also high accuracy under random $\ell_\infty$ noise. (2) Any simple classifier is not robust: it must have high adversarial loss with $\ell_\infty$ perturbations. (3) Robust classification is possible, but only with more complex classifiers (exponentially more complex, in some examples). Moreover, $\textit{there is a quantitative trade-off between robustness and standard accuracy among simple classifiers.}$ This suggests an alternate explanation of this phenomenon, which appears in practice: the tradeoff may occur not because the classification task inherently requires such a tradeoff (as in [Tsipras-Santurkar-Engstrom-Turner-Madry `18]), but because the structure of our current classifiers imposes such a tradeoff. | |
| Tasks | |
| Published | 2019-01-02 |
| URL | http://arxiv.org/abs/1901.00532v1 |
| http://arxiv.org/pdf/1901.00532v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-robustness-may-be-at-odds-with |
| Repo | |
| Framework | |
On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems
| Title | On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems |
| Authors | Stefano V. Albrecht, Subramanian Ramamoorthy |
| Abstract | While many multiagent algorithms are designed for homogeneous systems (i.e. all agents are identical), there are important applications which require an agent to coordinate its actions without knowing a priori how the other agents behave. One method to make this problem feasible is to assume that the other agents draw their latent policy (or type) from a specific set, and that a domain expert could provide a specification of this set, albeit only a partially correct one. Algorithms have been proposed by several researchers to compute posterior beliefs over such policy libraries, which can then be used to determine optimal actions. In this paper, we provide theoretical guidance on two central design parameters of this method: Firstly, it is important that the user choose a posterior which can learn the true distribution of latent types, as otherwise suboptimal actions may be chosen. We analyse convergence properties of two existing posterior formulations and propose a new posterior which can learn correlated distributions. Secondly, since the types are provided by an expert, they may be inaccurate in the sense that they do not predict the agents’ observed actions. We provide a novel characterisation of optimality which allows experts to use efficient model checking algorithms to verify optimality of types. |
| Tasks | |
| Published | 2019-07-15 |
| URL | https://arxiv.org/abs/1907.06995v1 |
| https://arxiv.org/pdf/1907.06995v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-convergence-and-optimality-of-best |
| Repo | |
| Framework | |
On the approximation of the solution of partial differential equations by artificial neural networks trained by a multilevel Levenberg-Marquardt method
| Title | On the approximation of the solution of partial differential equations by artificial neural networks trained by a multilevel Levenberg-Marquardt method |
| Authors | Henri Calandra, Serge Gratton, Elisa Riccietti, Xavier Vasseur |
| Abstract | This paper is concerned with the approximation of the solution of partial differential equations by means of artificial neural networks. Here a feedforward neural network is used to approximate the solution of the partial differential equation. The learning problem is formulated as a least squares problem, choosing the residual of the partial differential equation as a loss function, whereas a multilevel Levenberg-Marquardt method is employed as a training method. This setting allows us to get further insight into the potential of multilevel methods. Indeed, when the least squares problem arises from the training of artificial neural networks, the variables subject to optimization are not related by any geometrical constraints and the standard interpolation and restriction operators cannot be employed any longer. A heuristic, inspired by algebraic multigrid methods, is then proposed to construct the multilevel transfer operators. Numerical experiments show encouraging results related to the efficiency of the new multilevel optimization method for the training of artificial neural networks, compared to the standard corresponding one-level procedure. |
| Tasks | |
| Published | 2019-04-09 |
| URL | http://arxiv.org/abs/1904.04685v1 |
| http://arxiv.org/pdf/1904.04685v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-approximation-of-the-solution-of |
| Repo | |
| Framework | |
Human acceptability judgements for extractive sentence compression
| Title | Human acceptability judgements for extractive sentence compression |
| Authors | Abram Handler, Brian Dillon, Brendan O’Connor |
| Abstract | Recent approaches to English-language sentence compression rely on parallel corpora consisting of sentence-compression pairs. However, a sentence may be shortened in many different ways, which each might be suited to the needs of a particular application. Therefore, in this work, we collect and model crowdsourced judgements of the acceptability of many possible sentence shortenings. We then show how a model of such judgements can be used to support a flexible approach to the compression task. We release our model and dataset for future work. |
| Tasks | Sentence Compression |
| Published | 2019-02-01 |
| URL | http://arxiv.org/abs/1902.00489v1 |
| http://arxiv.org/pdf/1902.00489v1.pdf | |
| PWC | https://paperswithcode.com/paper/human-acceptability-judgements-for-extractive |
| Repo | |
| Framework | |
Accelerating Optimization Algorithms With Dynamic Parameter Selections Using Convolutional Neural Networks For Inverse Problems In Image Processing
| Title | Accelerating Optimization Algorithms With Dynamic Parameter Selections Using Convolutional Neural Networks For Inverse Problems In Image Processing |
| Authors | Byung Hyun Lee, Se Young Chun |
| Abstract | Recent advances using deep neural networks (DNNs) for solving inverse problems in image processing have significantly outperformed conventional optimization algorithm based methods. Most works train DNNs to learn 1) forward models and image priors implicitly for direct mappings from given measurements to solutions, 2) data-driven priors as proximal operators in conventional iterative algorithms, or 3) forward models, priors and/or static stepsizes in unfolded structures of optimization iterations. Here we investigate another way of utilizing convolutional neural network (CNN) for empirically accelerating conventional optimization for solving inverse problems in image processing. We propose a CNN to yield parameters in optimization algorithms that have been chosen heuristically, but have shown to be crucial for good empirical performance. Our CNN-incorporated scaled gradient projection methods, without compromising theoretical properties, significantly improve empirical convergence rate over conventional optimization based methods in large-scale inverse problems such as image inpainting, compressive image recovery with partial Fourier samples, deblurring and sparse view CT. During testing, our proposed methods dynamically select parameters every iterations to speed up convergence robustly for different degradation levels, noise, or regularization parameters as compared to direct mapping approach. |
| Tasks | Deblurring, Image Inpainting, Image Reconstruction |
| Published | 2019-02-07 |
| URL | https://arxiv.org/abs/1902.02449v2 |
| https://arxiv.org/pdf/1902.02449v2.pdf | |
| PWC | https://paperswithcode.com/paper/speeding-up-scaled-gradient-projection |
| Repo | |
| Framework | |
X-LineNet: Detecting Aircraft in Remote Sensing Images by a pair of Intersecting Line Segments
| Title | X-LineNet: Detecting Aircraft in Remote Sensing Images by a pair of Intersecting Line Segments |
| Authors | Haoran Wei, Yue Zhang, Bing Wang, Yang Yang, Hao Li, Hongqi Wang |
| Abstract | Motivated by the development of deep convolution neural networks (DCNNs), tremendous progress has been gained in the field of aircraft detection. These DCNNs based detectors mainly belong to top-down approaches, which first enumerate massive potential locations of objects with the form of rectangular regions, and then identify whether they are objects or not. Compared with these top-down approaches, this paper shows that aircraft detection via bottom-up approach still performs competitively in the era of deep learning. We present a novel one-stage and anchor-free aircraft detection model in a bottom-up manner, which formulates the task as detection of two intersecting line segments inside each target and grouping of them without any rectangular region classification. This model is named as X-LineNet. With simple post-processing, X-LineNet can simultaneously provide multiple representation forms of the detection result: the horizontal bounding box, the rotating bounding box, and the pentagonal mask. The pentagonal mask is a more accurate representation form which has less redundancy and can better represent aircraft than that of rectangular box. Experiments show that X-LineNet outperforms state-of-the-art one-stage object detectors and is competitive compared with advanced two-stage detectors on both UCAS-AOD and NWPU VHR-10 open dataset in the field of aircraft detection. |
| Tasks | |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12474v3 |
| https://arxiv.org/pdf/1907.12474v3.pdf | |
| PWC | https://paperswithcode.com/paper/x-linenet-detecting-aircraft-in-remote |
| Repo | |
| Framework | |
Generalization Error Bounds Via Rényi-, $f$-Divergences and Maximal Leakage
| Title | Generalization Error Bounds Via Rényi-, $f$-Divergences and Maximal Leakage |
| Authors | Amedeo Roberto Esposito, Michael Gastpar, Ibrahim Issa |
| Abstract | In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables. These results find applications in adaptive data analysis, where multiple dependencies are introduced and in learning theory, where they can be employed to bound the generalization error of a learning algorithm. Bounds are given in terms of $\alpha-$Divergence, Sibson’s Mutual Information and $f-$Divergence. A case of particular interest is the Maximal Leakage (or Sibson’s Mutual Information of order infinity) since this measure is robust to post-processing and composes adaptively. This bound can also be seen as a generalization of classical bounds, such as Hoeffding’s and McDiarmid’s inequalities, to the case of dependent random variables. |
| Tasks | |
| Published | 2019-12-01 |
| URL | https://arxiv.org/abs/1912.01439v2 |
| https://arxiv.org/pdf/1912.01439v2.pdf | |
| PWC | https://paperswithcode.com/paper/generalization-error-bounds-via-renyi-f |
| Repo | |
| Framework | |
SpeechYOLO: Detection and Localization of Speech Objects
| Title | SpeechYOLO: Detection and Localization of Speech Objects |
| Authors | Yael Segal, Tzeviya Sylvia Fuchs, Joseph Keshet |
| Abstract | In this paper, we propose to apply object detection methods from the vision domain on the speech recognition domain, by treating audio fragments as objects. More specifically, we present SpeechYOLO, which is inspired by the YOLO algorithm for object detection in images. The goal of SpeechYOLO is to localize boundaries of utterances within the input signal, and to correctly classify them. Our system is composed of a convolutional neural network, with a simple least-mean-squares loss function. We evaluated the system on several keyword spotting tasks, that include corpora of read speech and spontaneous speech. Our system compares favorably with other algorithms trained for both localization and classification. |
| Tasks | Keyword Spotting, Object Detection, Speech Recognition |
| Published | 2019-04-14 |
| URL | https://arxiv.org/abs/1904.07704v2 |
| https://arxiv.org/pdf/1904.07704v2.pdf | |
| PWC | https://paperswithcode.com/paper/speechyolo-detection-and-localization-of |
| Repo | |
| Framework | |
An In-Vehicle KWS System with Multi-Source Fusion for Vehicle Applications
| Title | An In-Vehicle KWS System with Multi-Source Fusion for Vehicle Applications |
| Authors | Yue Tan, Kan Zheng, Lei Lei |
| Abstract | In order to maximize detection precision rate as well as the recall rate, this paper proposes an in-vehicle multi-source fusion scheme in Keyword Spotting (KWS) System for vehicle applications. Vehicle information, as a new source for the original system, is collected by an in-vehicle data acquisition platform while the user is driving. A Deep Neural Network (DNN) is trained to extract acoustic features and make a speech classification. Based on the posterior probabilities obtained from DNN, the vehicle information including the speed and direction of vehicle is applied to choose the suitable parameter from a pair of sensitivity values for the KWS system. The experimental results show that the KWS system with the proposed multi-source fusion scheme can achieve better performances in term of precision rate, recall rate, and mean square error compared to the system without it. |
| Tasks | Keyword Spotting |
| Published | 2019-02-12 |
| URL | http://arxiv.org/abs/1902.04326v2 |
| http://arxiv.org/pdf/1902.04326v2.pdf | |
| PWC | https://paperswithcode.com/paper/an-in-vehicle-kws-system-with-multi-source |
| Repo | |
| Framework | |
Empowering swarm-based optimizers by multi-scale search to enhance Gradient Descent initialization performance
| Title | Empowering swarm-based optimizers by multi-scale search to enhance Gradient Descent initialization performance |
| Authors | Mojtaba Moattari, Mohammad Hassan Moradi, Reza Boostani |
| Abstract | Swarm-based optimizers like Particle Swarm Optimization or Imperialistic Competitive Algorithm that act under influences of cooperation or competition among groups, are unable to search in multiple volumes of locality or globality and do not have nested localities. As hybrid optimizers, they may not give satisfactory results as initializers in Gradient Descent approximators used in plenty of multimodal problems like nonlinear subspace learning and neural network training, which have hierarchies of convex spaces due to nonlinearity and multi-layer nature of these models. To search in various levels of scale in a homogenous way, a framework is proposed to equip PSO and ICA a multi-scale search capability. Then, the resulted optimizers are evaluated in single and GD-hybridized mode. Hybrid evaluation as GD randomizer is implemented with the help of a nonlinear subspace filtering objective function over EEG data and optimization loss and validation data accuracy is compared with other hybrids containing GD. A single evaluation is also taken place between the proposed ones, PSO, ICA, CLPSO, and CICA, which are used more in hybrid learning-based approaches. Evaluations were with respect to solution error. Before concluding the paper, it is shown and analyzed that proposed optimizers outperform algorithms of related context both in single and hybrid-GD mode. |
| Tasks | EEG |
| Published | 2019-06-13 |
| URL | https://arxiv.org/abs/1907.08220v1 |
| https://arxiv.org/pdf/1907.08220v1.pdf | |
| PWC | https://paperswithcode.com/paper/empowering-swarm-based-optimizers-by-multi |
| Repo | |
| Framework | |
Image Enhancement Network Trained by Using HDR images
| Title | Image Enhancement Network Trained by Using HDR images |
| Authors | Yuma Kinoshita, Hitoshi Kiya |
| Abstract | In this paper, a novel image enhancement network is proposed, where HDR images are used for generating training data for our network. Most of conventional image enhancement methods, including Retinex based methods, do not take into account restoring lost pixel values caused by clipping and quantizing. In addition, recently proposed CNN based methods still have a limited scope of application or a limited performance, due to network architectures. In contrast, the proposed method have a higher performance and a simpler network architecture than existing CNN based methods. Moreover, the proposed method enables us to restore lost pixel values. Experimental results show that the proposed method can provides higher-quality images than conventional image enhancement methods including a CNN based method, in terms of TMQI and NIQE. |
| Tasks | Image Enhancement |
| Published | 2019-01-17 |
| URL | http://arxiv.org/abs/1901.05686v2 |
| http://arxiv.org/pdf/1901.05686v2.pdf | |
| PWC | https://paperswithcode.com/paper/image-enhancement-network-trained-by-using |
| Repo | |
| Framework | |
Very Long Term Field of View Prediction for 360-degree Video Streaming
| Title | Very Long Term Field of View Prediction for 360-degree Video Streaming |
| Authors | Chenge Li, Weixi Zhang, Yong Liu, Yao Wang |
| Abstract | 360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user’s future FoV not only based on the user’s own past FoV center trajectory but also other users’ future FoV locations. We propose multiple prediction models based on two different FoV representations: one using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users’ FoVs are very helpful for improving long-term predictions. |
| Tasks | |
| Published | 2019-02-04 |
| URL | http://arxiv.org/abs/1902.01439v1 |
| http://arxiv.org/pdf/1902.01439v1.pdf | |
| PWC | https://paperswithcode.com/paper/very-long-term-field-of-view-prediction-for |
| Repo | |
| Framework | |
Nonlinear input design as optimal control of a Hamiltonian system
| Title | Nonlinear input design as optimal control of a Hamiltonian system |
| Authors | Jack Umenberger, Thomas B. Schön |
| Abstract | We propose an input design method for a general class of parametric probabilistic models, including nonlinear dynamical systems with process noise. The goal of the procedure is to select inputs such that the parameter posterior distribution concentrates about the true value of the parameters; however, exact computation of the posterior is intractable. By representing (samples from) the posterior as trajectories from a certain Hamiltonian system, we transform the input design task into an optimal control problem. The method is illustrated via numerical examples, including MRI pulse sequence design. |
| Tasks | |
| Published | 2019-03-06 |
| URL | http://arxiv.org/abs/1903.02250v1 |
| http://arxiv.org/pdf/1903.02250v1.pdf | |
| PWC | https://paperswithcode.com/paper/nonlinear-input-design-as-optimal-control-of |
| Repo | |
| Framework | |
