Paper Group ANR 258
PAC-Reasoning in Relational Domains. A Framework for the construction of upper bounds on the number of affine linear regions of ReLU feed-forward neural networks. Engineering Cooperative Smart Things based on Embodied Cognition. Dealing with sequences in the RGBDT space. Marginal Weighted Maximum Log-likelihood for Efficient Learning of Perturb-and …
PAC-Reasoning in Relational Domains
| Title | PAC-Reasoning in Relational Domains |
| Authors | Ondrej Kuzelka, Yuyi Wang, Jesse Davis, Steven Schockaert |
| Abstract | We consider the problem of predicting plausible missing facts in relational data, given a set of imperfect logical rules. In particular, our aim is to provide bounds on the (expected) number of incorrect inferences that are made in this way. Since for classical inference it is in general impossible to bound this number in a non-trivial way, we consider two inference relations that weaken, but remain close in spirit to classical inference. |
| Tasks | |
| Published | 2018-03-15 |
| URL | http://arxiv.org/abs/1803.05768v3 |
| http://arxiv.org/pdf/1803.05768v3.pdf | |
| PWC | https://paperswithcode.com/paper/pac-reasoning-in-relational-domains |
| Repo | |
| Framework | |
A Framework for the construction of upper bounds on the number of affine linear regions of ReLU feed-forward neural networks
| Title | A Framework for the construction of upper bounds on the number of affine linear regions of ReLU feed-forward neural networks |
| Authors | Peter Hinz, Sara van de Geer |
| Abstract | We present a framework to derive upper bounds on the number of regions that feed-forward neural networks with ReLU activation functions are affine linear on. It is based on an inductive analysis that keeps track of the number of such regions per dimensionality of their images within the layers. More precisely, the information about the number regions per dimensionality is pushed through the layers starting with one region of the input dimension of the neural network and using a recursion based on an analysis of how many regions per output dimensionality a subsequent layer with a certain width can induce on an input region with a given dimensionality. The final bound on the number of regions depends on the number and widths of the layers of the neural network and on some additional parameters that were used for the recursion. It is stated in terms of the $L1$-norm of the last column of a product of matrices and provides a unifying treatment of several previously known bounds: Depending on the choice of the recursion parameters that determine these matrices, it is possible to obtain the bounds from Mont'{u}far (2014), (2017) and Serra et. al. (2017) as special cases. For the latter, which is the strongest of these bounds, the formulation in terms of matrices provides new insight. In particular, by using explicit formulas for a Jordan-like decomposition of the involved matrices, we achieve new tighter results for the asymptotic setting, where the number of layers of the same fixed width tends to infinity. |
| Tasks | |
| Published | 2018-06-05 |
| URL | https://arxiv.org/abs/1806.01918v3 |
| https://arxiv.org/pdf/1806.01918v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-framework-for-the-construction-of-upper |
| Repo | |
| Framework | |
Engineering Cooperative Smart Things based on Embodied Cognition
| Title | Engineering Cooperative Smart Things based on Embodied Cognition |
| Authors | Nathalia Moraes do Nascimento, Carlos Jose Pereira de Lucena |
| Abstract | The goal of the Internet of Things (IoT) is to transform any thing around us, such as a trash can or a street light, into a smart thing. A smart thing has the ability of sensing, processing, communicating and/or actuating. In order to achieve the goal of a smart IoT application, such as minimizing waste transportation costs or reducing energy consumption, the smart things in the application scenario must cooperate with each other without a centralized control. Inspired by known approaches to design swarm of cooperative and autonomous robots, we modeled our smart things based on the embodied cognition concept. Each smart thing is a physical agent with a body composed of a microcontroller, sensors and actuators, and a brain that is represented by an artificial neural network. This type of agent is commonly called an embodied agent. The behavior of these embodied agents is autonomously configured through an evolutionary algorithm that is triggered according to the application performance. To illustrate, we have designed three homogeneous prototypes for smart street lights based on an evolved network. This application has shown that the proposed approach results in a feasible way of modeling decentralized smart things with self-developed and cooperative capabilities. |
| Tasks | |
| Published | 2018-01-12 |
| URL | http://arxiv.org/abs/1801.04345v1 |
| http://arxiv.org/pdf/1801.04345v1.pdf | |
| PWC | https://paperswithcode.com/paper/engineering-cooperative-smart-things-based-on |
| Repo | |
| Framework | |
Dealing with sequences in the RGBDT space
| Title | Dealing with sequences in the RGBDT space |
| Authors | Gabriel Moyà, Antoni Jaume-i-Capó, Javier Varona |
| Abstract | Most of the current research in computer vision is focused on working with single images without taking in account temporal information. We present a probabilistic non-parametric model that mixes multiple information cues from devices to segment regions that contain moving objects in image sequences. We prepared an experimental setup to show the importance of using previous information for obtaining an accurate segmentation result, using a novel dataset that provides sequences in the RGBDT space. We label the detected regions ts with a state-of-the-art human detector. Each one of the detected regions is at least marked as human once. |
| Tasks | |
| Published | 2018-05-10 |
| URL | http://arxiv.org/abs/1805.03897v1 |
| http://arxiv.org/pdf/1805.03897v1.pdf | |
| PWC | https://paperswithcode.com/paper/dealing-with-sequences-in-the-rgbdt-space |
| Repo | |
| Framework | |
Marginal Weighted Maximum Log-likelihood for Efficient Learning of Perturb-and-Map models
| Title | Marginal Weighted Maximum Log-likelihood for Efficient Learning of Perturb-and-Map models |
| Authors | Tatiana Shpakova, Francis Bach, Anton Osokin |
| Abstract | We consider the structured-output prediction problem through probabilistic approaches and generalize the “perturb-and-MAP” framework to more challenging weighted Hamming losses, which are crucial in applications. While in principle our approach is a straightforward marginalization, it requires solving many related MAP inference problems. We show that for log-supermodular pairwise models these operations can be performed efficiently using the machinery of dynamic graph cuts. We also propose to use double stochastic gradient descent, both on the data and on the perturbations, for efficient learning. Our framework can naturally take weak supervision (e.g., partial labels) into account. We conduct a set of experiments on medium-scale character recognition and image segmentation, showing the benefits of our algorithms. |
| Tasks | Semantic Segmentation |
| Published | 2018-11-21 |
| URL | http://arxiv.org/abs/1811.08725v1 |
| http://arxiv.org/pdf/1811.08725v1.pdf | |
| PWC | https://paperswithcode.com/paper/marginal-weighted-maximum-log-likelihood-for |
| Repo | |
| Framework | |
Certifying Global Optimality of Graph Cuts via Semidefinite Relaxation: A Performance Guarantee for Spectral Clustering
| Title | Certifying Global Optimality of Graph Cuts via Semidefinite Relaxation: A Performance Guarantee for Spectral Clustering |
| Authors | Shuyang Ling, Thomas Strohmer |
| Abstract | Spectral clustering has become one of the most widely used clustering techniques when the structure of the individual clusters is non-convex or highly anisotropic. Yet, despite its immense popularity, there exists fairly little theory about performance guarantees for spectral clustering. This issue is partly due to the fact that spectral clustering typically involves two steps which complicated its theoretical analysis: first, the eigenvectors of the associated graph Laplacian are used to embed the dataset, and second, k-means clustering algorithm is applied to the embedded dataset to get the labels. This paper is devoted to the theoretical foundations of spectral clustering and graph cuts. We consider a convex relaxation of graph cuts, namely ratio cuts and normalized cuts, that makes the usual two-step approach of spectral clustering obsolete and at the same time gives rise to a rigorous theoretical analysis of graph cuts and spectral clustering. We derive deterministic bounds for successful spectral clustering via a spectral proximity condition that naturally depends on the algebraic connectivity of each cluster and the inter-cluster connectivity. Moreover, we demonstrate by means of some popular examples that our bounds can achieve near-optimality. Our findings are also fundamental for the theoretical understanding of kernel k-means. Numerical simulations confirm and complement our analysis. |
| Tasks | |
| Published | 2018-06-29 |
| URL | http://arxiv.org/abs/1806.11429v3 |
| http://arxiv.org/pdf/1806.11429v3.pdf | |
| PWC | https://paperswithcode.com/paper/certifying-global-optimality-of-graph-cuts |
| Repo | |
| Framework | |
On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes
| Title | On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes |
| Authors | Xiaoyu Li, Francesco Orabona |
| Abstract | Stochastic gradient descent is the method of choice for large scale optimization of machine learning objective functions. Yet, its performance is greatly variable and heavily depends on the choice of the stepsizes. This has motivated a large body of research on adaptive stepsizes. However, there is currently a gap in our theoretical understanding of these methods, especially in the non-convex setting. In this paper, we start closing this gap: we theoretically analyze in the convex and non-convex settings a generalized version of the AdaGrad stepsizes. We show sufficient conditions for these stepsizes to achieve almost sure asymptotic convergence of the gradients to zero, proving the first guarantee for generalized AdaGrad stepsizes in the non-convex setting. Moreover, we show that these stepsizes allow to automatically adapt to the level of noise of the stochastic gradients in both the convex and non-convex settings, interpolating between $O(1/T)$ and $O(1/\sqrt{T})$, up to logarithmic terms. |
| Tasks | |
| Published | 2018-05-21 |
| URL | http://arxiv.org/abs/1805.08114v3 |
| http://arxiv.org/pdf/1805.08114v3.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-convergence-of-stochastic-gradient-1 |
| Repo | |
| Framework | |
Artificial Intelligence for the Public Sector: Opportunities and challenges of cross-sector collaboration
| Title | Artificial Intelligence for the Public Sector: Opportunities and challenges of cross-sector collaboration |
| Authors | Slava Jankin Mikhaylov, Marc Esteve, Averill Campion |
| Abstract | Public sector organisations are increasingly interested in using data science and artificial intelligence capabilities to deliver policy and generate efficiencies in high uncertainty environments. The long-term success of data science and AI in the public sector relies on effectively embedding it into delivery solutions for policy implementation. However, governments cannot do this integration of AI into public service delivery on their own. The UK Government Industrial Strategy is clear that delivering on the AI grand challenge requires collaboration between universities and public and private sectors. This cross-sectoral collaborative approach is the norm in applied AI centres of excellence around the world. Despite their popularity, cross-sector collaborations entail serious management challenges that hinder their success. In this article we discuss the opportunities and challenges from AI for public sector. Finally, we propose a series of strategies to successfully manage these cross-sectoral collaborations. |
| Tasks | |
| Published | 2018-09-12 |
| URL | http://arxiv.org/abs/1809.04399v1 |
| http://arxiv.org/pdf/1809.04399v1.pdf | |
| PWC | https://paperswithcode.com/paper/artificial-intelligence-for-the-public-sector |
| Repo | |
| Framework | |
Unifying Decision-Making: a Review on Evolutionary Theories on Rationality and Cognitive Biases
| Title | Unifying Decision-Making: a Review on Evolutionary Theories on Rationality and Cognitive Biases |
| Authors | Catarina Moreira |
| Abstract | In this paper, we make a review on the concepts of rationality across several different fields, namely in economics, psychology and evolutionary biology and behavioural ecology. We review how processes like natural selection can help us understand the evolution of cognition and how cognitive biases might be a consequence of this natural selection. In the end we argue that humans are not irrational, but rather rationally bounded and we complement the discussion on how quantum cognitive models can contribute for the modelling and prediction of human paradoxical decisions. |
| Tasks | Decision Making |
| Published | 2018-11-29 |
| URL | http://arxiv.org/abs/1811.12455v1 |
| http://arxiv.org/pdf/1811.12455v1.pdf | |
| PWC | https://paperswithcode.com/paper/unifying-decision-making-a-review-on |
| Repo | |
| Framework | |
Quantum enhanced cross-validation for near-optimal neural networks architecture selection
| Title | Quantum enhanced cross-validation for near-optimal neural networks architecture selection |
| Authors | Priscila G. M. dos Santos, Rodrigo S. Sousa, Ismael C. S. Araujo, Adenilton J. da Silva |
| Abstract | This paper proposes a quantum-classical algorithm to evaluate and select classical artificial neural networks architectures. The proposed algorithm is based on a probabilistic quantum memory and the possibility to train artificial neural networks in superposition. We obtain an exponential quantum speedup in the evaluation of neural networks. We also verify experimentally through a reduced experimental analysis that the proposed algorithm can be used to select near-optimal neural networks. |
| Tasks | |
| Published | 2018-08-27 |
| URL | http://arxiv.org/abs/1808.09058v1 |
| http://arxiv.org/pdf/1808.09058v1.pdf | |
| PWC | https://paperswithcode.com/paper/quantum-enhanced-cross-validation-for-near |
| Repo | |
| Framework | |
Automatic Feature Weight Determination using Indexing and Pseudo-Relevance Feedback for Multi-feature Content-Based Image Retrieval
| Title | Automatic Feature Weight Determination using Indexing and Pseudo-Relevance Feedback for Multi-feature Content-Based Image Retrieval |
| Authors | Asheet Kumar, Shivam Choudhary, Vaibhav Singh Khokhar, Vikas Meena, Chiranjoy Chattopadhyay |
| Abstract | Content-based image retrieval (CBIR) is one of the most active research areas in multimedia information retrieval. Given a query image, the task is to search relevant images in a repository. Low level features like color, texture, and shape feature vectors of an image are always considered to be an important attribute in CBIR system. Thus the performance of the CBIR system can be enhanced by combining these feature vectors. In this paper, we propose a novel CBIR framework by applying to index using multiclass SVM and finding the appropriate weights of the individual features automatically using the relevance ratio and mean difference. We have taken four feature descriptors to represent color, texture and shape features. During retrieval, feature vectors of query image are combined, weighted and compared with feature vectors of images in the database to rank order the results. Experiments were performed on four benchmark datasets and performance is compared with existing techniques to validate the superiority of our proposed framework. |
| Tasks | Content-Based Image Retrieval, Image Retrieval, Information Retrieval |
| Published | 2018-12-11 |
| URL | http://arxiv.org/abs/1812.04215v1 |
| http://arxiv.org/pdf/1812.04215v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-feature-weight-determination-using |
| Repo | |
| Framework | |
Born Again Neural Networks
| Title | Born Again Neural Networks |
| Authors | Tommaso Furlanello, Zachary C. Lipton, Michael Tschannen, Laurent Itti, Anima Anandkumar |
| Abstract | Knowledge distillation (KD) consists of transferring knowledge from one machine learning model (the teacher}) to another (the student). Commonly, the teacher is a high-capacity model with formidable performance, while the student is more compact. By transferring knowledge, one hopes to benefit from the student’s compactness. %we desire a compact model with performance close to the teacher’s. We study KD from a new perspective: rather than compressing models, we train students parameterized identically to their teachers. Surprisingly, these {Born-Again Networks (BANs), outperform their teachers significantly, both on computer vision and language modeling tasks. Our experiments with BANs based on DenseNets demonstrate state-of-the-art performance on the CIFAR-10 (3.5%) and CIFAR-100 (15.5%) datasets, by validation error. Additional experiments explore two distillation objectives: (i) Confidence-Weighted by Teacher Max (CWTM) and (ii) Dark Knowledge with Permuted Predictions (DKPP). Both methods elucidate the essential components of KD, demonstrating a role of the teacher outputs on both predicted and non-predicted classes. We present experiments with students of various capacities, focusing on the under-explored case where students overpower teachers. Our experiments show significant advantages from transferring knowledge between DenseNets and ResNets in either direction. |
| Tasks | Language Modelling |
| Published | 2018-05-12 |
| URL | http://arxiv.org/abs/1805.04770v2 |
| http://arxiv.org/pdf/1805.04770v2.pdf | |
| PWC | https://paperswithcode.com/paper/born-again-neural-networks |
| Repo | |
| Framework | |
Security and Privacy Issues in Deep Learning
| Title | Security and Privacy Issues in Deep Learning |
| Authors | Ho Bae, Jaehee Jang, Dahuin Jung, Hyemi Jang, Heonseok Ha, Sungroh Yoon |
| Abstract | With the development of machine learning (ML), expectations for artificial intelligence (AI) technology have been increasing daily. In particular, deep neural networks have shown outstanding performance results in many fields. Many applications are deeply involved in our daily life, such as making significant decisions in application areas based on predictions or classifications, in which a DL model could be relevant. Hence, if a DL model causes mispredictions or misclassifications due to malicious external influences, then it can cause very large difficulties in real life. Moreover, training DL models involve an enormous amount of data and the training data often include sensitive information. Therefore, DL models should not expose the privacy of such data. In this paper, we review the vulnerabilities and the developed defense methods on the security of the models and data privacy under the notion of secure and private AI (SPAI). We also discuss current challenges and open issues. |
| Tasks | |
| Published | 2018-07-31 |
| URL | https://arxiv.org/abs/1807.11655v3 |
| https://arxiv.org/pdf/1807.11655v3.pdf | |
| PWC | https://paperswithcode.com/paper/security-and-privacy-issues-in-deep-learning |
| Repo | |
| Framework | |
Representing pictures with emotions
| Title | Representing pictures with emotions |
| Authors | António Filipe Fonseca |
| Abstract | Modern research in content-based image retrieval systems (CIBR) has become progressively more focused on the richness of human semantics. Several approaches may be used to reduced the ‘semantic gap’ between the high-level human experience and the low level visual features of pictures. Object ontology, among others, is one of the methods. In this paper we investigate the use of a codified emotion ontology over global color features of images to annotate the images at a high semantic level. In order to speed up the annotation process the images are sampled so that each digital image is represented by a random subset of its content. We test within controlled conditions how this random subset may represent the adequate high level emotional concept presented in the image. We monitor this information reducing process with entropy measures, showing that controlled random sampling can capture with significant relevance high level concepts for picture representation. |
| Tasks | Content-Based Image Retrieval, Image Retrieval |
| Published | 2018-12-06 |
| URL | http://arxiv.org/abs/1812.02523v2 |
| http://arxiv.org/pdf/1812.02523v2.pdf | |
| PWC | https://paperswithcode.com/paper/representing-pictures-with-emotions |
| Repo | |
| Framework | |
Learning Embeddings for Product Visual Search with Triplet Loss and Online Sampling
| Title | Learning Embeddings for Product Visual Search with Triplet Loss and Online Sampling |
| Authors | Eric Dodds, Huy Nguyen, Simao Herdade, Jack Culpepper, Andrew Kae, Pierre Garrigues |
| Abstract | In this paper, we propose learning an embedding function for content-based image retrieval within the e-commerce domain using the triplet loss and an online sampling method that constructs triplets from within a minibatch. We compare our method to several strong baselines as well as recent works on the DeepFashion and Stanford Online Product datasets. Our approach significantly outperforms the state-of-the-art on the DeepFashion dataset. With a modification to favor sampling minibatches from a single product category, the same approach demonstrates competitive results when compared to the state-of-the-art for the Stanford Online Products dataset. |
| Tasks | Content-Based Image Retrieval, Image Retrieval |
| Published | 2018-10-10 |
| URL | http://arxiv.org/abs/1810.04652v1 |
| http://arxiv.org/pdf/1810.04652v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-embeddings-for-product-visual-search |
| Repo | |
| Framework | |
