Paper Group ANR 49
High- and Low-level image component decomposition using VAEs for improved reconstruction and anomaly detection. Information Losses in Neural Classifiers from Sampling. Off-Policy Policy Gradient Algorithms by Constraining the State Distribution Shift. Online Replanning in Belief Space for Partially Observable Task and Motion Problems. Point Clouds …
High- and Low-level image component decomposition using VAEs for improved reconstruction and anomaly detection
| Title | High- and Low-level image component decomposition using VAEs for improved reconstruction and anomaly detection |
| Authors | David Zimmerer, Jens Petersen, Klaus Maier-Hein |
| Abstract | Variational Auto-Encoders have often been used for unsupervised pretraining, feature extraction and out-of-distribution and anomaly detection in the medical field. However, VAEs often lack the ability to produce sharp images and learn high-level features. We propose to alleviate these issues by adding a new branch to conditional hierarchical VAEs. This enforces a division between higher-level and lower-level features. Despite the additional computational overhead compared to a normal VAE it results in sharper and better reconstructions and can capture the data distribution similarly well (indicated by a similar or slightly better OoD detection performance). |
| Tasks | Anomaly Detection |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.12161v1 |
| https://arxiv.org/pdf/1911.12161v1.pdf | |
| PWC | https://paperswithcode.com/paper/high-and-low-level-image-component |
| Repo | |
| Framework | |
Information Losses in Neural Classifiers from Sampling
| Title | Information Losses in Neural Classifiers from Sampling |
| Authors | Brandon Foggo, Nanpeng Yu, Jie Shi, Yuanqi Gao |
| Abstract | This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidden representation of that model. It then bounds this expected total variation as a function of the size of randomly sampled datasets in a fairly general setting, and without bringing in any additional dependence on model complexity. It ultimately obtains bounds on information losses that are less sensitive to input compression and in general much smaller than existing bounds. The paper then uses these bounds to explain some recent experimental findings of information compression in neural networks which cannot be explained by previous work. Finally, the paper shows that not only are these bounds much smaller than existing ones, but that they also correspond well with experiments. |
| Tasks | Active Learning |
| Published | 2019-02-15 |
| URL | https://arxiv.org/abs/1902.05991v3 |
| https://arxiv.org/pdf/1902.05991v3.pdf | |
| PWC | https://paperswithcode.com/paper/asymptotic-finite-sample-information-losses |
| Repo | |
| Framework | |
Off-Policy Policy Gradient Algorithms by Constraining the State Distribution Shift
| Title | Off-Policy Policy Gradient Algorithms by Constraining the State Distribution Shift |
| Authors | Riashat Islam, Komal K. Teru, Deepak Sharma, Joelle Pineau |
| Abstract | Off-policy deep reinforcement learning (RL) algorithms are incapable of learning solely from batch offline data without online interactions with the environment, due to the phenomenon known as \textit{extrapolation error}. This is often due to past data available in the replay buffer that may be quite different from the data distribution under the current policy. We argue that most off-policy learning methods fundamentally suffer from a \textit{state distribution shift} due to the mismatch between the state visitation distribution of the data collected by the behavior and target policies. This data distribution shift between current and past samples can significantly impact the performance of most modern off-policy based policy optimization algorithms. In this work, we first do a systematic analysis of state distribution mismatch in off-policy learning, and then develop a novel off-policy policy optimization method to constraint the state distribution shift. To do this, we first estimate the state distribution based on features of the state, using a density estimator and then develop a novel constrained off-policy gradient objective that minimizes the state distribution shift. Our experimental results on continuous control tasks show that minimizing this distribution mismatch can significantly improve performance in most popular practical off-policy policy gradient algorithms. |
| Tasks | Continuous Control |
| Published | 2019-11-16 |
| URL | https://arxiv.org/abs/1911.06970v2 |
| https://arxiv.org/pdf/1911.06970v2.pdf | |
| PWC | https://paperswithcode.com/paper/off-policy-policy-gradient-algorithms-by |
| Repo | |
| Framework | |
Online Replanning in Belief Space for Partially Observable Task and Motion Problems
| Title | Online Replanning in Belief Space for Partially Observable Task and Motion Problems |
| Authors | Caelan Reed Garrett, Chris Paxton, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox |
| Abstract | To solve multi-step manipulation tasks in the real world, an autonomous robot must take actions to observe its environment and react to unexpected observations. This may require opening a drawer to observe its contents or moving an object out of the way to examine the space behind it. Upon receiving a new observation, the robot must update its belief about the world and compute a new plan of action. In this work, we present an online planning and execution system for robots faced with these challenges. We perform deterministic cost-sensitive planning in the space of hybrid belief states to select likely-to-succeed observation actions and continuous control actions. After execution and observation, we replan using our new state estimate. We initially enforce that planner reuses the structure of the unexecuted tail of the last plan. This both improves planning efficiency and ensures that the overall policy does not undo its progress towards achieving the goal. Our approach is able to efficiently solve partially observable problems both in simulation and in a real-world kitchen. |
| Tasks | Continuous Control |
| Published | 2019-11-11 |
| URL | https://arxiv.org/abs/1911.04577v2 |
| https://arxiv.org/pdf/1911.04577v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-replanning-in-belief-space-for |
| Repo | |
| Framework | |
Point Clouds Learning with Attention-based Graph Convolution Networks
| Title | Point Clouds Learning with Attention-based Graph Convolution Networks |
| Authors | Zhuyang Xie, Junzhou Chen, Bo Peng |
| Abstract | Point clouds data, as one kind of representation of 3D objects, are the most primitive output obtained by 3D sensors. Unlike 2D images, point clouds are disordered and unstructured. Hence it is not straightforward to apply classification techniques such as the convolution neural network to point clouds analysis directly. To solve this problem, we propose a novel network structure, named Attention-based Graph Convolution Networks (AGCN), to extract point clouds features. Taking the learning process as a message propagation between adjacent points, we introduce an attention mechanism to AGCN for analyzing the relationships between local features of the points. In addition, we introduce an additional global graph structure network to compensate for the relative information of the individual points in the graph structure network. The proposed network is also extended to an encoder-decoder structure for segmentation tasks. Experimental results show that the proposed network can achieve state-of-the-art performance in both classification and segmentation tasks. |
| Tasks | |
| Published | 2019-05-31 |
| URL | https://arxiv.org/abs/1905.13445v1 |
| https://arxiv.org/pdf/1905.13445v1.pdf | |
| PWC | https://paperswithcode.com/paper/point-clouds-learning-with-attention-based |
| Repo | |
| Framework | |
Graded Relevance Assessments and Graded Relevance Measures of NTCIR: A Survey of the First Twenty Years
| Title | Graded Relevance Assessments and Graded Relevance Measures of NTCIR: A Survey of the First Twenty Years |
| Authors | Tetsuya Sakai |
| Abstract | NTCIR was the first large-scale IR evaluation conference to construct test collections with graded relevance assessments: the NTCIR-1 test collections from 1998 already featured relevant and partially relevant documents. In this paper, I first describe a few graded-relevance measures that originated from NTCIR (and a few variants) which are used across different NTCIR tasks. I then provide a survey on the use of graded relevance assessments and of graded relevance measures in the past NTCIR tasks which primarily tackled ranked retrieval. My survey shows that the majority of the past tasks fully utilised graded relevance by means of graded evaluation measures, but not all of them; interestingly, even a few relatively recent tasks chose to adhere to binary relevance measures. I conclude this paper by a summary of my survey in table form, and a brief discussion on what may lie beyond graded relevance. |
| Tasks | |
| Published | 2019-03-27 |
| URL | http://arxiv.org/abs/1903.11272v1 |
| http://arxiv.org/pdf/1903.11272v1.pdf | |
| PWC | https://paperswithcode.com/paper/graded-relevance-assessments-and-graded |
| Repo | |
| Framework | |
Ludii and XCSP: Playing and Solving Logic Puzzles
| Title | Ludii and XCSP: Playing and Solving Logic Puzzles |
| Authors | Cédric Piette, Éric Piette, Matthew Stephenson, Dennis J. N. J. Soemers, Cameron Browne |
| Abstract | Many of the famous single-player games, commonly called puzzles, can be shown to be NP-Complete. Indeed, this class of complexity contains hundreds of puzzles, since people particularly appreciate completing an intractable puzzle, such as Sudoku, but also enjoy the ability to check their solution easily once it’s done. For this reason, using constraint programming is naturally suited to solve them. In this paper, we focus on logic puzzles described in the Ludii general game system and we propose using the XCSP formalism in order to solve them with any CSP solver. |
| Tasks | |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.00245v1 |
| https://arxiv.org/pdf/1907.00245v1.pdf | |
| PWC | https://paperswithcode.com/paper/ludii-and-xcsp-playing-and-solving-logic |
| Repo | |
| Framework | |
Analytic expressions for the output evolution of a deep neural network
| Title | Analytic expressions for the output evolution of a deep neural network |
| Authors | Anastasia Borovykh |
| Abstract | We present a novel methodology based on a Taylor expansion of the network output for obtaining analytical expressions for the expected value of the network weights and output under stochastic training. Using these analytical expressions the effects of the hyperparameters and the noise variance of the optimization algorithm on the performance of the deep neural network are studied. In the early phases of training with a small noise coefficient, the output is equivalent to a linear model. In this case the network can generalize better due to the noise preventing the output from fully converging on the train data, however the noise does not result in any explicit regularization. In the later training stages, when higher order approximations are required, the impact of the noise becomes more significant, i.e. in a model which is non-linear in the weights noise can regularize the output function resulting in better generalization as witnessed by its influence on the weight Hessian, a commonly used metric for generalization capabilities. |
| Tasks | |
| Published | 2019-12-18 |
| URL | https://arxiv.org/abs/1912.08526v1 |
| https://arxiv.org/pdf/1912.08526v1.pdf | |
| PWC | https://paperswithcode.com/paper/analytic-expressions-for-the-output-evolution |
| Repo | |
| Framework | |
The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents
| Title | The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents |
| Authors | Shangmin Guo, Yi Ren, Serhii Havrylov, Stella Frank, Ivan Titov, Kenny Smith |
| Abstract | Since first introduced, computer simulation has been an increasingly important tool in evolutionary linguistics. Recently, with the development of deep learning techniques, research in grounded language learning has also started to focus on facilitating the emergence of compositional languages without pre-defined elementary linguistic knowledge. In this work, we explore the emergence of compositional languages for numeric concepts in multi-agent communication systems. We demonstrate that compositional language for encoding numeric concepts can emerge through iterated learning in populations of deep neural network agents. However, language properties greatly depend on the input representations given to agents. We found that compositional languages only emerge if they require less iterations to be fully learnt than other non-degenerate languages for agents on a given input representation. |
| Tasks | |
| Published | 2019-10-11 |
| URL | https://arxiv.org/abs/1910.05291v1 |
| https://arxiv.org/pdf/1910.05291v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-emergence-of-compositional-languages-for |
| Repo | |
| Framework | |
Smartphone Multi-modal Biometric Authentication: Database and Evaluation
| Title | Smartphone Multi-modal Biometric Authentication: Database and Evaluation |
| Authors | Raghavendra Ramachandra, Martin Stokkenes, Amir Mohammadi, Sushma Venkatesh, Kiran Raja, Pankaj Wasnik, Eric Poiret, Sébastien Marcel, Christoph Busch |
| Abstract | Biometric-based verification is widely employed on the smartphones for various applications, including financial transactions. In this work, we present a new multimodal biometric dataset (face, voice, and periocular) acquired using a smartphone. The new dataset is comprised of 150 subjects that are captured in six different sessions reflecting real-life scenarios of smartphone assisted authentication. One of the unique features of this dataset is that it is collected in four different geographic locations representing a diverse population and ethnicity. Additionally, we also present a multimodal Presentation Attack (PA) or spoofing dataset using a low-cost Presentation Attack Instrument (PAI) such as print and electronic display attacks. The novel acquisition protocols and the diversity of the data subjects collected from different geographic locations will allow developing a novel algorithm for either unimodal or multimodal biometrics. Further, we also report the performance evaluation of the baseline biometric verification and Presentation Attack Detection (PAD) on the newly collected dataset. |
| Tasks | |
| Published | 2019-12-05 |
| URL | https://arxiv.org/abs/1912.02487v1 |
| https://arxiv.org/pdf/1912.02487v1.pdf | |
| PWC | https://paperswithcode.com/paper/smartphone-multi-modal-biometric |
| Repo | |
| Framework | |
Strong Baselines for Complex Word Identification across Multiple Languages
| Title | Strong Baselines for Complex Word Identification across Multiple Languages |
| Authors | Pierre Finnimore, Elisabeth Fritzsch, Daniel King, Alison Sneyd, Aneeq Ur Rehman, Fernando Alva-Manchego, Andreas Vlachos |
| Abstract | Complex Word Identification (CWI) is the task of identifying which words or phrases in a sentence are difficult to understand by a target audience. The latest CWI Shared Task released data for two settings: monolingual (i.e. train and test in the same language) and cross-lingual (i.e. test in a language not seen during training). The best monolingual models relied on language-dependent features, which do not generalise in the cross-lingual setting, while the best cross-lingual model used neural networks with multi-task learning. In this paper, we present monolingual and cross-lingual CWI models that perform as well as (or better than) most models submitted to the latest CWI Shared Task. We show that carefully selected features and simple learning models can achieve state-of-the-art performance, and result in strong baselines for future development in this area. Finally, we discuss how inconsistencies in the annotation of the data can explain some of the results obtained. |
| Tasks | Complex Word Identification, Multi-Task Learning |
| Published | 2019-04-11 |
| URL | http://arxiv.org/abs/1904.05953v1 |
| http://arxiv.org/pdf/1904.05953v1.pdf | |
| PWC | https://paperswithcode.com/paper/strong-baselines-for-complex-word |
| Repo | |
| Framework | |
Improving Neural Network Classifier using Gradient-based Floating Centroid Method
| Title | Improving Neural Network Classifier using Gradient-based Floating Centroid Method |
| Authors | Mazharul Islam, Shuangrong Liu, Lin Wang, Xiaojing Zhang |
| Abstract | Floating centroid method (FCM) offers an efficient way to solve a fixed-centroid problem for the neural network classifiers. However, evolutionary computation as its optimization method restrains the FCM to achieve satisfactory performance for different neural network structures, because of the high computational complexity and inefficiency. Traditional gradient-based methods have been extensively adopted to optimize the neural network classifiers. In this study, a gradient-based floating centroid (GDFC) method is introduced to address the fixed centroid problem for the neural network classifiers optimized by gradient-based methods. Furthermore, a new loss function for optimizing GDFC is introduced. The experimental results display that GDFC obtains promising classification performance than the comparison methods on the benchmark datasets. |
| Tasks | |
| Published | 2019-07-21 |
| URL | https://arxiv.org/abs/1907.08996v1 |
| https://arxiv.org/pdf/1907.08996v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-neural-network-classifier-using |
| Repo | |
| Framework | |
(q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs
| Title | (q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs |
| Authors | Anton Mallasto, Jes Frellsen, Wouter Boomsma, Aasa Feragen |
| Abstract | Generative Adversial Networks (GANs) have made a major impact in computer vision and machine learning as generative models. Wasserstein GANs (WGANs) brought Optimal Transport (OT) theory into GANs, by minimizing the $1$-Wasserstein distance between model and data distributions as their objective function. Since then, WGANs have gained considerable interest due to their stability and theoretical framework. We contribute to the WGAN literature by introducing the family of $(q,p)$-Wasserstein GANs, which allow the use of more general $p$-Wasserstein metrics for $p\geq 1$ in the GAN learning procedure. While the method is able to incorporate any cost function as the ground metric, we focus on studying the $l^q$ metrics for $q\geq 1$. This is a notable generalization as in the WGAN literature the OT distances are commonly based on the $l^2$ ground metric. We demonstrate the effect of different $p$-Wasserstein distances in two toy examples. Furthermore, we show that the ground metric does make a difference, by comparing different $(q,p)$ pairs on the MNIST and CIFAR-10 datasets. Our experiments demonstrate that changing the ground metric and $p$ can notably improve on the common $(q,p) = (2,1)$ case. |
| Tasks | |
| Published | 2019-02-10 |
| URL | http://arxiv.org/abs/1902.03642v1 |
| http://arxiv.org/pdf/1902.03642v1.pdf | |
| PWC | https://paperswithcode.com/paper/qp-wasserstein-gans-comparing-ground-metrics |
| Repo | |
| Framework | |
OpBerg: Discovering causal sentences using optimal alignments
| Title | OpBerg: Discovering causal sentences using optimal alignments |
| Authors | Justin Wood, Nicholas J. Matiasz, Alcino J. Silva, William Hsu, Alexej Abyzov, Wei Wang |
| Abstract | The biological literature is rich with sentences that describe causal relations. Methods that automatically extract such sentences can help biologists to synthesize the literature and even discover latent relations that had not been articulated explicitly. Current methods for extracting causal sentences are based on either machine learning or a predefined database of causal terms. Machine learning approaches require a large set of labeled training data and can be susceptible to noise. Methods based on predefined databases are limited by the quality of their curation and are unable to capture new concepts or mistakes in the input. We address these challenges by adapting and improving a method designed for a seemingly unrelated problem: finding alignments between genomic sequences. This paper presents a novel and outperforming method for extracting causal relations from text by aligning the part-of-speech representations of an input set with that of known causal sentences. Our experiments show that when applied to the task of finding causal sentences in biological literature, our method improves on the accuracy of other methods in a computationally efficient manner. |
| Tasks | |
| Published | 2019-04-03 |
| URL | http://arxiv.org/abs/1904.02032v1 |
| http://arxiv.org/pdf/1904.02032v1.pdf | |
| PWC | https://paperswithcode.com/paper/opberg-discovering-causal-sentences-using |
| Repo | |
| Framework | |
NNStreamer: Stream Processing Paradigm for Neural Networks, Toward Efficient Development and Execution of On-Device AI Applications
| Title | NNStreamer: Stream Processing Paradigm for Neural Networks, Toward Efficient Development and Execution of On-Device AI Applications |
| Authors | MyungJoo Ham, Ji Joong Moon, Geunsik Lim, Wook Song, Jaeyun Jung, Hyoungjoo Ahn, Sangjung Woo, Youngchul Cho, Jinhyuck Park, Sewon Oh, Hong-Seok Kim |
| Abstract | We propose nnstreamer, a software system that handles neural networks as filters of stream pipelines, applying the stream processing paradigm to neural network applications. A new trend with the wide-spread of deep neural network applications is on-device AI; i.e., processing neural networks directly on mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy issues, data transmission costs, and operational costs signifies the need for on-device AI especially when a huge number of devices with real-time data processing are deployed. Nnstreamer efficiently handles neural networks with complex data stream pipelines on devices, improving the overall performance significantly with minimal efforts. Besides, nnstreamer simplifies the neural network pipeline implementations and allows reusing off-shelf multimedia stream filters directly; thus it reduces the developmental costs significantly. Nnstreamer is already being deployed with a product releasing soon and is open source software applicable to a wide range of hardware architectures and software platforms. |
| Tasks | |
| Published | 2019-01-12 |
| URL | http://arxiv.org/abs/1901.04985v1 |
| http://arxiv.org/pdf/1901.04985v1.pdf | |
| PWC | https://paperswithcode.com/paper/nnstreamer-stream-processing-paradigm-for |
| Repo | |
| Framework | |
