Paper Group ANR 816
Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning. Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator. Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks. SAAGs: Biased Stochastic Variance Reduction Methods for Large-scale Learni …
Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning
| Title | Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning |
| Authors | Thomas G. Dietterich, George Trimponias, Zhitang Chen |
| Abstract | Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and identify conditions under which an MDP with exogenous state can be decomposed into an exogenous Markov Reward Process involving only the exogenous state+reward and an endogenous Markov Decision Process defined with respect to only the endogenous rewards. We also derive a variance-covariance condition under which Monte Carlo policy evaluation on the endogenous MDP is accelerated compared to using the full MDP. Similar speedups are likely to carry over to all RL algorithms. We develop two algorithms for discovering the exogenous variables and test them on several MDPs. Results show that the algorithms are practical and can significantly speed up reinforcement learning. |
| Tasks | |
| Published | 2018-06-05 |
| URL | http://arxiv.org/abs/1806.01584v1 |
| http://arxiv.org/pdf/1806.01584v1.pdf | |
| PWC | https://paperswithcode.com/paper/discovering-and-removing-exogenous-state |
| Repo | |
| Framework | |
Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator
| Title | Distributionally Robust Inverse Covariance Estimation: The Wasserstein Shrinkage Estimator |
| Authors | Viet Anh Nguyen, Daniel Kuhn, Peyman Mohajerin Esfahani |
| Abstract | We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a $p$-dimensional Gaussian random vector from $n$ independent samples. The proposed model minimizes the worst case (maximum) of Stein’s loss across all normal reference distributions within a prescribed Wasserstein distance from the normal distribution characterized by the sample mean and the sample covariance matrix. We prove that this estimation problem is equivalent to a semidefinite program that is tractable in theory but beyond the reach of general purpose solvers for practically relevant problem dimensions $p$. In the absence of any prior structural information, the estimation problem has an analytical solution that is naturally interpreted as a nonlinear shrinkage estimator. Besides being invertible and well-conditioned even for $p>n$, the new shrinkage estimator is rotation-equivariant and preserves the order of the eigenvalues of the sample covariance matrix. These desirable properties are not imposed ad hoc but emerge naturally from the underlying distributionally robust optimization model. Finally, we develop a sequential quadratic approximation algorithm for efficiently solving the general estimation problem subject to conditional independence constraints typically encountered in Gaussian graphical models. |
| Tasks | |
| Published | 2018-05-18 |
| URL | http://arxiv.org/abs/1805.07194v1 |
| http://arxiv.org/pdf/1805.07194v1.pdf | |
| PWC | https://paperswithcode.com/paper/distributionally-robust-inverse-covariance |
| Repo | |
| Framework | |
Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks
| Title | Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks |
| Authors | Anton Ragni, Qiujia Li, Mark Gales, Yu Wang |
| Abstract | The standard approach to assess reliability of automatic speech transcriptions is through the use of confidence scores. If accurate, these scores provide a flexible mechanism to flag transcription errors for upstream and downstream applications. One challenging type of errors that recognisers make are deletions. These errors are not accounted for by the standard confidence estimation schemes and are hard to rectify in the upstream and downstream processing. High deletion rates are prominent in limited resource and highly mismatched training/testing conditions studied under IARPA Babel and Material programs. This paper looks at the use of bidirectional recurrent neural networks to yield confidence estimates in predicted as well as deleted words. Several simple schemes are examined for combination. To assess usefulness of this approach, the combined confidence score is examined for untranscribed data selection that favours transcriptions with lower deletion errors. Experiments are conducted using IARPA Babel/Material program languages. |
| Tasks | |
| Published | 2018-10-30 |
| URL | http://arxiv.org/abs/1810.13025v1 |
| http://arxiv.org/pdf/1810.13025v1.pdf | |
| PWC | https://paperswithcode.com/paper/confidence-estimation-and-deletion-prediction |
| Repo | |
| Framework | |
SAAGs: Biased Stochastic Variance Reduction Methods for Large-scale Learning
| Title | SAAGs: Biased Stochastic Variance Reduction Methods for Large-scale Learning |
| Authors | Vinod Kumar Chauhan, Anuj Sharma, Kalpana Dahiya |
| Abstract | Stochastic approximation is one of the effective approach to deal with the large-scale machine learning problems and the recent research has focused on reduction of variance, caused by the noisy approximations of the gradients. In this paper, we have proposed novel variants of SAAG-I and II (Stochastic Average Adjusted Gradient) (Chauhan et al. 2017), called SAAG-III and IV, respectively. Unlike SAAG-I, starting point is set to average of previous epoch in SAAG-III, and unlike SAAG-II, the snap point and starting point are set to average and last iterate of previous epoch in SAAG-IV, respectively. To determine the step size, we have used Stochastic Backtracking-Armijo line Search (SBAS) which performs line search only on selected mini-batch of data points. Since backtracking line search is not suitable for large-scale problems and the constants used to find the step size, like Lipschitz constant, are not always available so SBAS could be very effective in such cases. We have extended SAAGs (I, II, III and IV) to solve non-smooth problems and designed two update rules for smooth and non-smooth problems. Moreover, our theoretical results have proved linear convergence of SAAG-IV for all the four combinations of smoothness and strong-convexity, in expectation. Finally, our experimental studies have proved the efficacy of proposed methods against the state-of-art techniques. |
| Tasks | |
| Published | 2018-07-24 |
| URL | http://arxiv.org/abs/1807.08934v3 |
| http://arxiv.org/pdf/1807.08934v3.pdf | |
| PWC | https://paperswithcode.com/paper/saags-biased-stochastic-variance-reduction |
| Repo | |
| Framework | |
Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data
| Title | Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data |
| Authors | Charilaos I. Kanatsoulis, Xiao Fu, Nicholas D. Sidiropoulos, Mingyi Hong |
| Abstract | The sum-of-correlations (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view CCA to the multiview case, and thus has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information – which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers which are frequently used in data analytics. The updates of the proposed algorithm are lightweight and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm. |
| Tasks | |
| Published | 2018-04-24 |
| URL | http://arxiv.org/abs/1804.08806v1 |
| http://arxiv.org/pdf/1804.08806v1.pdf | |
| PWC | https://paperswithcode.com/paper/structured-sumcor-multiview-canonical |
| Repo | |
| Framework | |
Finite LTL Synthesis with Environment Assumptions and Quality Measures
| Title | Finite LTL Synthesis with Environment Assumptions and Quality Measures |
| Authors | Alberto Camacho, Meghyn Bienvenu, Sheila A. McIlraith |
| Abstract | In this paper, we investigate the problem of synthesizing strategies for linear temporal logic (LTL) specifications that are interpreted over finite traces – a problem that is central to the automated construction of controllers, robot programs, and business processes. We study a natural variant of the finite LTL synthesis problem in which strategy guarantees are predicated on specified environment behavior. We further explore a quantitative extension of LTL that supports specification of quality measures, utilizing it to synthesize high-quality strategies. We propose new notions of optimality and associated algorithms that yield strategies that best satisfy specified quality measures. Our algorithms utilize an automata-game approach, positioning them well for future implementation via existing state-of-the-art techniques. |
| Tasks | |
| Published | 2018-08-31 |
| URL | http://arxiv.org/abs/1808.10831v1 |
| http://arxiv.org/pdf/1808.10831v1.pdf | |
| PWC | https://paperswithcode.com/paper/finite-ltl-synthesis-with-environment |
| Repo | |
| Framework | |
Semantic Image Inpainting Through Improved Wasserstein Generative Adversarial Networks
| Title | Semantic Image Inpainting Through Improved Wasserstein Generative Adversarial Networks |
| Authors | Patricia Vitoria, Joan Sintes, Coloma Ballester |
| Abstract | Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works. |
| Tasks | Image Inpainting |
| Published | 2018-12-03 |
| URL | http://arxiv.org/abs/1812.01071v1 |
| http://arxiv.org/pdf/1812.01071v1.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-image-inpainting-through-improved |
| Repo | |
| Framework | |
Improving the Segmentation of Anatomical Structures in Chest Radiographs using U-Net with an ImageNet Pre-trained Encoder
| Title | Improving the Segmentation of Anatomical Structures in Chest Radiographs using U-Net with an ImageNet Pre-trained Encoder |
| Authors | Maayan Frid-Adar, Avi Ben-Cohen, Rula Amer, Hayit Greenspan |
| Abstract | Accurate segmentation of anatomical structures in chest radiographs is essential for many computer-aided diagnosis tasks. In this paper we investigate the latest fully-convolutional architectures for the task of multi-class segmentation of the lungs field, heart and clavicles in a chest radiograph. In addition, we explore the influence of using different loss functions in the training process of a neural network for semantic segmentation. We evaluate all models on a common benchmark of 247 X-ray images from the JSRT database and ground-truth segmentation masks from the SCR dataset. Our best performing architecture, is a modified U-Net that benefits from pre-trained encoder weights. This model outperformed the current state-of-the-art methods tested on the same benchmark, with Jaccard overlap scores of 96.1% for lung fields, 90.6% for heart and 85.5% for clavicles. |
| Tasks | Semantic Segmentation |
| Published | 2018-10-04 |
| URL | http://arxiv.org/abs/1810.02113v1 |
| http://arxiv.org/pdf/1810.02113v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-the-segmentation-of-anatomical |
| Repo | |
| Framework | |
Fashionable Modelling with Flux
| Title | Fashionable Modelling with Flux |
| Authors | Michael Innes, Elliot Saba, Keno Fischer, Dhairya Gandhi, Marco Concetto Rudilosso, Neethu Mariya Joy, Tejan Karmali, Avik Pal, Viral Shah |
| Abstract | Machine learning as a discipline has seen an incredible surge of interest in recent years due in large part to a perfect storm of new theory, superior tooling, renewed interest in its capabilities. We present in this paper a framework named Flux that shows how further refinement of the core ideas of machine learning, built upon the foundation of the Julia programming language, can yield an environment that is simple, easily modifiable, and performant. We detail the fundamental principles of Flux as a framework for differentiable programming, give examples of models that are implemented within Flux to display many of the language and framework-level features that contribute to its ease of use and high productivity, display internal compiler techniques used to enable the acceleration and performance that lies at the heart of Flux, and finally give an overview of the larger ecosystem that Flux fits inside of. |
| Tasks | |
| Published | 2018-11-01 |
| URL | http://arxiv.org/abs/1811.01457v3 |
| http://arxiv.org/pdf/1811.01457v3.pdf | |
| PWC | https://paperswithcode.com/paper/fashionable-modelling-with-flux |
| Repo | |
| Framework | |
Deep Multiple Instance Learning for Zero-shot Image Tagging
| Title | Deep Multiple Instance Learning for Zero-shot Image Tagging |
| Authors | Shafin Rahman, Salman Khan |
| Abstract | In-line with the success of deep learning on traditional recognition problem, several end-to-end deep models for zero-shot recognition have been proposed in the literature. These models are successful to predict a single unseen label given an input image, but does not scale to cases where multiple unseen objects are present. In this paper, we model this problem within the framework of Multiple Instance Learning (MIL). To the best of our knowledge, we propose the first end-to-end trainable deep MIL framework for the multi-label zero-shot tagging problem. Due to its novel design, the proposed framework has several interesting features: (1) Unlike previous deep MIL models, it does not use any off-line procedure (e.g., Selective Search or EdgeBoxes) for bag generation. (2) During test time, it can process any number of unseen labels given their semantic embedding vectors. (3) Using only seen labels per image as weak annotation, it can produce a bounding box for each predicted labels. We experiment with the NUS-WIDE dataset and achieve superior performance across conventional, zero-shot and generalized zero-shot tagging tasks. |
| Tasks | Multiple Instance Learning, Zero-Shot Learning |
| Published | 2018-03-16 |
| URL | http://arxiv.org/abs/1803.06051v1 |
| http://arxiv.org/pdf/1803.06051v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-multiple-instance-learning-for-zero-shot |
| Repo | |
| Framework | |
Generative Adversarial Networks for Unsupervised Object Co-localization
| Title | Generative Adversarial Networks for Unsupervised Object Co-localization |
| Authors | Junsuk Choe, Joo Hyun Park, Hyunjung Shim |
| Abstract | This paper introduces a novel approach for unsupervised object co-localization using Generative Adversarial Networks (GANs). GAN is a powerful tool that can implicitly learn unknown data distributions in an unsupervised manner. From the observation that GAN discriminator is highly influenced by pixels where objects appear, we analyze the internal layers of discriminator and visualize the activated pixels. Our important finding is that high image diversity of GAN, which is a main goal in GAN research, is ironically disadvantageous for object localization, because such discriminators focus not only on the target object, but also on the various objects, such as background objects. Based on extensive evaluations and experimental studies, we show the image diversity and localization performance have a negative correlation. In addition, our approach achieves meaningful accuracy for unsupervised object co-localization using publicly available benchmark datasets, even comparable to state-of-the-art weakly-supervised approach. |
| Tasks | Object Localization |
| Published | 2018-06-01 |
| URL | http://arxiv.org/abs/1806.00236v2 |
| http://arxiv.org/pdf/1806.00236v2.pdf | |
| PWC | https://paperswithcode.com/paper/generative-adversarial-networks-for-1 |
| Repo | |
| Framework | |
Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization
| Title | Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization |
| Authors | Zhe Wang, Yi Zhou, Yingbin Liang, Guanghui Lan |
| Abstract | Cubic regularization (CR) is an optimization method with emerging popularity due to its capability to escape saddle points and converge to second-order stationary solutions for nonconvex optimization. However, CR encounters a high sample complexity issue for finite-sum problems with a large data size. %Various inexact variants of CR have been proposed to improve the sample complexity. In this paper, we propose a stochastic variance-reduced cubic-regularization (SVRC) method under random sampling, and study its convergence guarantee as well as sample complexity. We show that the iteration complexity of SVRC for achieving a second-order stationary solution within $\epsilon$ accuracy is $O(\epsilon^{-3/2})$, which matches the state-of-art result on CR types of methods. Moreover, our proposed variance reduction scheme significantly reduces the per-iteration sample complexity. The resulting total Hessian sample complexity of our SVRC is ${\Oc}(N^{2/3} \epsilon^{-3/2})$, which outperforms the state-of-art result by a factor of $O(N^{2/15})$. We also study our SVRC under random sampling without replacement scheme, which yields a lower per-iteration sample complexity, and hence justifies its practical applicability. |
| Tasks | |
| Published | 2018-02-20 |
| URL | http://arxiv.org/abs/1802.07372v2 |
| http://arxiv.org/pdf/1802.07372v2.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-variance-reduced-cubic |
| Repo | |
| Framework | |
Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
| Title | Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks |
| Authors | Yarin Gal, Lewis Smith |
| Abstract | We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised Bayesian neural networks (BNNs) satisfy these. We continue by studying near-idealised BNNs using HMC inference, demonstrating the theoretical ideas in practice. We experiment with HMC on synthetic data derived from MNIST for which we know the ground-truth image density, showing that near-perfect epistemic uncertainty correlates to density under image manifold, and that adversarial images lie off the manifold in our setting. This suggests why MC dropout, which can be seen as performing approximate inference, has been observed to be an effective defence against adversarial examples in practice; We highlight failure-cases of non-idealised BNNs relying on dropout, suggesting a new attack for dropout models and a new defence as well. Lastly, we demonstrate the defence on a cats-vs-dogs image classification task with a VGG13 variant. |
| Tasks | Image Classification |
| Published | 2018-06-02 |
| URL | http://arxiv.org/abs/1806.00667v3 |
| http://arxiv.org/pdf/1806.00667v3.pdf | |
| PWC | https://paperswithcode.com/paper/sufficient-conditions-for-idealised-models-to |
| Repo | |
| Framework | |
Teaching Meaningful Explanations
| Title | Teaching Meaningful Explanations |
| Authors | Noel C. F. Codella, Michael Hind, Karthikeyan Natesan Ramamurthy, Murray Campbell, Amit Dhurandhar, Kush R. Varshney, Dennis Wei, Aleksandra Mojsilovic |
| Abstract | The adoption of machine learning in high-stakes applications such as healthcare and law has lagged in part because predictions are not accompanied by explanations comprehensible to the domain user, who often holds the ultimate responsibility for decisions and outcomes. In this paper, we propose an approach to generate such explanations in which training data is augmented to include, in addition to features and labels, explanations elicited from domain users. A joint model is then learned to produce both labels and explanations from the input features. This simple idea ensures that explanations are tailored to the complexity expectations and domain knowledge of the consumer. Evaluation spans multiple modeling techniques on a game dataset, a (visual) aesthetics dataset, a chemical odor dataset and a Melanoma dataset showing that our approach is generalizable across domains and algorithms. Results demonstrate that meaningful explanations can be reliably taught to machine learning algorithms, and in some cases, also improve modeling accuracy. |
| Tasks | |
| Published | 2018-05-29 |
| URL | http://arxiv.org/abs/1805.11648v2 |
| http://arxiv.org/pdf/1805.11648v2.pdf | |
| PWC | https://paperswithcode.com/paper/teaching-meaningful-explanations |
| Repo | |
| Framework | |
Learning End-to-End Goal-Oriented Dialog with Multiple Answers
| Title | Learning End-to-End Goal-Oriented Dialog with Multiple Answers |
| Authors | Janarthanan Rajendran, Jatin Ganhotra, Satinder Singh, Lazaros Polymenakos |
| Abstract | In a dialog, there can be multiple valid next utterances at any point. The present end-to-end neural methods for dialog do not take this into account. They learn with the assumption that at any time there is only one correct next utterance. In this work, we focus on this problem in the goal-oriented dialog setting where there are different paths to reach a goal. We propose a new method, that uses a combination of supervised learning and reinforcement learning approaches to address this issue. We also propose a new and more effective testbed, permuted-bAbI dialog tasks, by introducing multiple valid next utterances to the original-bAbI dialog tasks, which allows evaluation of goal-oriented dialog systems in a more realistic setting. We show that there is a significant drop in performance of existing end-to-end neural methods from 81.5% per-dialog accuracy on original-bAbI dialog tasks to 30.3% on permuted-bAbI dialog tasks. We also show that our proposed method improves the performance and achieves 47.3% per-dialog accuracy on permuted-bAbI dialog tasks. |
| Tasks | Goal-Oriented Dialog |
| Published | 2018-08-24 |
| URL | http://arxiv.org/abs/1808.09996v1 |
| http://arxiv.org/pdf/1808.09996v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-end-to-end-goal-oriented-dialog-with |
| Repo | |
| Framework | |
