Paper Group NANR 28
CRAP: Semi-supervised Learning via Conditional Rotation Angle Prediction. The Surprising Behavior Of Graph Neural Networks. INTERNAL-CONSISTENCY CONSTRAINTS FOR EMERGENT COMMUNICATION. Improving Dirichlet Prior Network for Out-of-Distribution Example Detection. Negative Sampling in Variational Autoencoders. Global reasoning network for image super- …
CRAP: Semi-supervised Learning via Conditional Rotation Angle Prediction
| Title | CRAP: Semi-supervised Learning via Conditional Rotation Angle Prediction |
| Authors | Anonymous |
| Abstract | Self-supervised learning (SlfSL), aiming at learning feature representations through ingeniously designed pretext tasks without human annotation, has achieved compelling progress in the past few years. Very recently, SlfSL has also been identified as a promising solution for semi-supervised learning (SemSL) since it offers a new paradigm to utilize unlabeled data. This work further explores this direction by proposing a new framework to seamlessly couple SlfSL with SemSL. Our insight is that the prediction target in SemSL can be modeled as the latent factor in the predictor for the SlfSL target. Marginalizing over the latent factor naturally derives a new formulation which marries the prediction targets of these two learning processes. By implementing this framework through a simple-but-effective SlfSL approach – rotation angle prediction, we create a new SemSL approach called Conditional Rotation Angle Prediction (CRAP). Specifically, CRAP is featured by adopting a module which predicts the image rotation angle \textbf{conditioned on the candidate image class}. Through experimental evaluation, we show that CRAP achieves superior performance over the other existing ways of combining SlfSL and SemSL. Moreover, the proposed SemSL framework is highly extendable. By augmenting CRAP with a simple SemSL technique and a modification of the rotation angle prediction task, our method has already achieved the state-of-the-art SemSL performance. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BJxoz1rKwr |
| https://openreview.net/pdf?id=BJxoz1rKwr | |
| PWC | https://paperswithcode.com/paper/crap-semi-supervised-learning-via-conditional |
| Repo | |
| Framework | |
The Surprising Behavior Of Graph Neural Networks
| Title | The Surprising Behavior Of Graph Neural Networks |
| Authors | Anonymous |
| Abstract | We highlight a lack of understanding of the behaviour of Graph Neural Networks (GNNs) in various topological contexts. We present 4 experimental studies which counter-intuitively demonstrate that the performance of GNNs is weakly dependent on the topology, sensitive to structural noise and the modality (attributes or edges) of information, and degraded by strong coupling between nodal attributes and structure. We draw on the empirical results to recommend reporting of topological context in GNN evaluation and propose a simple (attribute-structure) decoupling method to improve GNN performance. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BkgOM1rKvr |
| https://openreview.net/pdf?id=BkgOM1rKvr | |
| PWC | https://paperswithcode.com/paper/the-surprising-behavior-of-graph-neural |
| Repo | |
| Framework | |
INTERNAL-CONSISTENCY CONSTRAINTS FOR EMERGENT COMMUNICATION
| Title | INTERNAL-CONSISTENCY CONSTRAINTS FOR EMERGENT COMMUNICATION |
| Authors | Anonymous |
| Abstract | When communicating, humans rely on internally-consistent language representations. That is, as speakers, we expect listeners to behave the same way we do when we listen. This work proposes several methods for encouraging such internal consistency in dialog agents in an emergent communication setting. We consider two hypotheses about the effect of internal-consistency constraints: 1) that they improve agents’ ability to refer to unseen referents, and 2) that they improve agents’ ability to generalize across communicative roles (e.g. performing as a speaker de- spite only being trained as a listener). While we do not find evidence in favor of the former, our results show significant support for the latter. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SkgJOAEtvr |
| https://openreview.net/pdf?id=SkgJOAEtvr | |
| PWC | https://paperswithcode.com/paper/internal-consistency-constraints-for-emergent |
| Repo | |
| Framework | |
Improving Dirichlet Prior Network for Out-of-Distribution Example Detection
| Title | Improving Dirichlet Prior Network for Out-of-Distribution Example Detection |
| Authors | Anonymous |
| Abstract | Determining the source of uncertainties in the predictions of AI systems are important. It allows the users to act in an informative manner to improve the safety of such systems, applied to the real-world sensitive applications. Predictive uncertainties can originate from the uncertainty in model parameters, data uncertainty or due to distributional mismatch between training and test examples. While recently, significant progress has been made to improve the predictive uncertainty estimation of deep learning models, most of these approaches either conflate the distributional uncertainty with model uncertainty or data uncertainty. In contrast, the Dirichlet Prior Network (DPN) can model distributional uncertainty distinctly by parameterizing a prior Dirichlet over the predictive categorical distributions. However, their complex loss function by explicitly incorporating KL divergence between Dirichlet distributions often makes the error surface ill-suited to optimize for challenging datasets with multiple classes. In this paper, we present an improved DPN framework by proposing a novel loss function using the standard cross-entropy loss along with a regularization term to control the sharpness of the output Dirichlet distributions from the network. Our proposed loss function aims to improve the training efficiency of the DPN framework for challenging classification tasks with large number of classes. In our experiments using synthetic and real datasets, we demonstrate that our DPN models can distinguish the distributional uncertainty from other uncertainty types. Our proposed approach significantly improves DPN frameworks and outperform the existing OOD detectors on CIFAR-10 and CIFAR-100 dataset while also being able to recognize distributional uncertainty distinctly. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=Bye4iaEFwr |
| https://openreview.net/pdf?id=Bye4iaEFwr | |
| PWC | https://paperswithcode.com/paper/improving-dirichlet-prior-network-for-out-of |
| Repo | |
| Framework | |
Negative Sampling in Variational Autoencoders
| Title | Negative Sampling in Variational Autoencoders |
| Authors | Anonymous |
| Abstract | We propose negative sampling as an approach to improve the notoriously bad out-of-distribution likelihood estimates of Variational Autoencoder models. Our model pushes latent images of negative samples away from the prior. When the source of negative samples is an auxiliary dataset, such a model can vastly improve on baselines when evaluated on OOD detection tasks. Perhaps more surprisingly, we present a fully unsupervised variant that can also significantly improve detection performance: using the output of the generator as a source of negative samples results in a fully unsupervised model that can be interpreted as adversarially trained. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=BJlqYlrtPB |
| https://openreview.net/pdf?id=BJlqYlrtPB | |
| PWC | https://paperswithcode.com/paper/negative-sampling-in-variational-autoencoders |
| Repo | |
| Framework | |
Global reasoning network for image super-resolution
| Title | Global reasoning network for image super-resolution |
| Authors | Anonymous |
| Abstract | Recent image super-resolution(SR) studies leverage very deep convolutional neural networks and the rich hierarchical features they offered, which leads to better reconstruction performance than conventional methods. However, the small receptive fields in the up-sampling and reconstruction process of those models stop them to take full advantage of global contextual information. This causes problems for further performance improvement. In this paper, inspired by image reconstruction principles of human visual system, we propose an image super-resolution global reasoning network (SRGRN) to effectively learn the correlations between different regions of an image, through global reasoning. Specifically, we propose global reasoning up-sampling module (GRUM) and global reasoning reconstruction block (GRRB). They construct a graph model to perform relation reasoning on regions of low resolution (LR) images.They aim to reason the interactions between different regions in the up-sampling and reconstruction process and thus leverage more contextual information to generate accurate details. Our proposed SRGRN are more robust and can handle low resolution images that are corrupted by multiple types of degradation. Extensive experiments on different benchmark data-sets show that our model outperforms other state-of-the-art methods. Also our model is lightweight and consumes less computing power, which makes it very suitable for real life deployment. |
| Tasks | Image Reconstruction, Image Super-Resolution, Super-Resolution |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=S1gE6TEYDB |
| https://openreview.net/pdf?id=S1gE6TEYDB | |
| PWC | https://paperswithcode.com/paper/global-reasoning-network-for-image-super |
| Repo | |
| Framework | |
LOSSLESS SINGLE IMAGE SUPER RESOLUTION FROM LOW-QUALITY JPG IMAGES
| Title | LOSSLESS SINGLE IMAGE SUPER RESOLUTION FROM LOW-QUALITY JPG IMAGES |
| Authors | Yong Shi, Biao Li, Bo Wang, Zhiquan Qi, Jiabin Liu, Fan Meng |
| Abstract | Super Resolution (SR) is a fundamental and important low-level computer vision (CV) task. Different from traditional SR models, this study concentrates on a specific but realistic SR issue: How can we obtain satisfied SR results from compressed JPG (C-JPG) image, which widely exists on the Internet. In general, C-JPG can release storage space while keeping considerable quality in visual. However, further image processing operations, e.g., SR, will suffer from enlarging inner artificial details and result in unacceptable outputs. To address this problem, we propose a novel SR structure with two specifically designed components, as well as a cycle loss. In short, there are mainly three contributions to this paper. First, our research can generate high-qualified SR images for prevalent C-JPG images. Second, we propose a functional sub-model to recover information for C-JPG images, instead of the perspective of noise elimination in traditional SR approaches. Third, we further integrate cycle loss into SR solver to build a hybrid loss function for better SR generation. Experiments show that our approach achieves outstanding performance among state-of-the-art methods. |
| Tasks | Image Super-Resolution, Super-Resolution |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=r1l0VCNKwB |
| https://openreview.net/pdf?id=r1l0VCNKwB | |
| PWC | https://paperswithcode.com/paper/lossless-single-image-super-resolution-from |
| Repo | |
| Framework | |
Multiagent Reinforcement Learning in Games with an Iterated Dominance Solution
| Title | Multiagent Reinforcement Learning in Games with an Iterated Dominance Solution |
| Authors | Anonymous |
| Abstract | Multiagent reinforcement learning (MARL) attempts to optimize policies of intelligent agents interacting in the same environment. However, it may fail to converge to a Nash equilibrium in some games. We study independent MARL under the more demanding solution concept of iterated elimination of strictly dominated strategies. In dominance solvable games, if players iteratively eliminate strictly dominated strategies until no further strategies can be eliminated, we obtain a single strategy profile. We show that convergence to the iterated dominance solution is guaranteed for several reinforcement learning algorithms (for multiple independent learners). We illustrate an application of our results by studying mechanism design for principal-agent problems, where a principal wishes to incentivize agents to exert costly effort in a joint project when it can only observe whether the project succeeded, but not whether agents actually exerted effort. We show that MARL converges to the desired outcome if the rewards are designed so that exerting effort is the iterated dominance solution, but fails if it is merely a Nash equilibrium. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=ryl1r1BYDS |
| https://openreview.net/pdf?id=ryl1r1BYDS | |
| PWC | https://paperswithcode.com/paper/multiagent-reinforcement-learning-in-games |
| Repo | |
| Framework | |
On Symmetry and Initialization for Neural Networks
| Title | On Symmetry and Initialization for Neural Networks |
| Authors | Anonymous |
| Abstract | This work provides an additional step in the theoretical understanding of neural networks. We consider neural networks with one hidden layer and show that when learning symmetric functions, one can choose initial conditions so that standard SGD training efficiently produces generalization guarantees. We empirically verify this and show that this does not hold when the initial conditions are chosen at random. The proof of convergence investigates the interaction between the two layers of the network. Our results highlight the importance of using symmetry in the design of neural networks. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=Skeh-xBYDH |
| https://openreview.net/pdf?id=Skeh-xBYDH | |
| PWC | https://paperswithcode.com/paper/on-symmetry-and-initialization-for-neural-1 |
| Repo | |
| Framework | |
Molecule Property Prediction and Classification with Graph Hypernetworks
| Title | Molecule Property Prediction and Classification with Graph Hypernetworks |
| Authors | Anonymous |
| Abstract | Graph neural networks are currently leading the performance charts in learning-based molecule property prediction and classification. Computational chemistry has, therefore, become the a prominent testbed for generic graph neural networks, as well as for specialized message passing methods. In this work, we demonstrate that the replacement of the underlying networks with hypernetworks leads to a boost in performance, obtaining state of the art results in various benchmarks. A major difficulty in the application of hypernetworks is their lack of stability. We tackle this by combining the current message and the first message. A recent work has tackled the training instability of hypernetworks in the context of error correcting codes, by replacing the activation function of the message passing network with a low-order Taylor approximation of it. We demonstrate that our generic solution can replace this domain-specific solution. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=SJgg_yHKvH |
| https://openreview.net/pdf?id=SJgg_yHKvH | |
| PWC | https://paperswithcode.com/paper/molecule-property-prediction-and |
| Repo | |
| Framework | |
Learning Latent Representations for Inverse Dynamics using Generalized Experiences
| Title | Learning Latent Representations for Inverse Dynamics using Generalized Experiences |
| Authors | Anonymous |
| Abstract | Many practical robot locomotion tasks require agents to use control policies that can be parameterized by goals. Popular deep reinforcement learning approaches in this direction involve learning goal-conditioned policies or value functions, or Inverse Dynamics Models (IDMs). IDMs map an agent’s current state and desired goal to the required actions. We show that the key to achieving good performance with IDMs lies in learning the information shared between equivalent experiences, so that they can be generalized to unseen scenarios. We design a training process that guides the learning of latent representations to encode this shared information. Using a limited number of environment interactions, our agent is able to efficiently navigate to arbitrary points in the goal space. We demonstrate the effectiveness of our approach in high-dimensional locomotion environments such as the Mujoco Ant, PyBullet Humanoid, and PyBullet Minitaur. We provide quantitative and qualitative results to show that our method clearly outperforms competing baseline approaches. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HylloR4YDr |
| https://openreview.net/pdf?id=HylloR4YDr | |
| PWC | https://paperswithcode.com/paper/learning-latent-representations-for-inverse |
| Repo | |
| Framework | |
Word embedding re-examined: is the symmetrical factorization optimal?
| Title | Word embedding re-examined: is the symmetrical factorization optimal? |
| Authors | Anonymous |
| Abstract | As observed in previous works, many word embedding methods exhibit two interesting properties: (1) words having similar semantic meanings are embedded closely; (2) analogy structure exists in the embedding space, such that ‘‘emph{Paris} is to \emph{France} as \emph{Berlin} is to \emph{Germany}''. We theoretically analyze the inner mechanism leading to these nice properties. Specifically, the embedding can be viewed as a linear transformation from the word-context co-occurrence space to the embedding space. We reveal how the relative distances between nodes change during this transforming process. Such linear transformation will result in these good properties. Based on the analysis, we also provide the answer to a question whether the symmetrical factorization (e.g., \texttt{word2vec}) is better than traditional SVD method. We propose a method to improve the embedding further. The experiments on real datasets verify our analysis. |
| Tasks | |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HklCk1BtwS |
| https://openreview.net/pdf?id=HklCk1BtwS | |
| PWC | https://paperswithcode.com/paper/word-embedding-re-examined-is-the-symmetrical |
| Repo | |
| Framework | |
FoveaBox: Beyound Anchor-based Object Detection
| Title | FoveaBox: Beyound Anchor-based Object Detection |
| Authors | Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei Li, Jianbo Shi |
| Abstract | We present FoveaBox, an accurate, flexible, and completely anchor-free framework for object detection. While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors. Instead, FoveaBox directly learns the object existing possibility and the bounding box coordinates without anchor reference. This is achieved by: (a) predicting category-sensitive semantic maps for the object existing possibility, and (b) producing category-agnostic bounding box for each position that potentially contains an object. The scales of target boxes are naturally associated with feature pyramid representations. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis. Without bells and whistles, FoveaBox achieves state-of-the-art single model performance on the standard COCO detection benchmark. More importantly, FoveaBox avoids all computation and hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. We believe the simple and effective approach will serve as a solid baseline and help ease future research for object detection. |
| Tasks | Object Detection |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=HyxFF34FPr |
| https://openreview.net/pdf?id=HyxFF34FPr | |
| PWC | https://paperswithcode.com/paper/foveabox-beyound-anchor-based-object |
| Repo | |
| Framework | |
Unsupervised Representation Learning by Predicting Random Distances
| Title | Unsupervised Representation Learning by Predicting Random Distances |
| Authors | Anonymous |
| Abstract | Deep neural networks have gained tremendous success in a broad range of machine learning tasks due to its remarkable capability to learn semantic-rich features from high-dimensional data. However, they often require large-scale labelled data to successfully learn such features, which significantly hinders their adaption into unsupervised learning tasks, such as anomaly detection and clustering, and limits their applications into critical domains where obtaining massive labelled data is prohibitively expensive. To enable downstream unsupervised learning on those domains, in this work we propose to learn features without using any labelled data by training neural networks to predict data distances in a randomly projected space. Random mapping is a highly efficient yet theoretical proven approach to obtain approximately preserved distances. To well predict these random distances, the representation learner is optimised to learn class structures that are implicitly embedded in the randomly projected space. Experimental results on 19 real-world datasets show our learned representations substantially outperform state-of-the-art competing methods in both anomaly detection and clustering tasks. |
| Tasks | Anomaly Detection, Representation Learning, Unsupervised Representation Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rkgIW1HKPB |
| https://openreview.net/pdf?id=rkgIW1HKPB | |
| PWC | https://paperswithcode.com/paper/unsupervised-representation-learning-by-4 |
| Repo | |
| Framework | |
Keep Doing What Worked: Behavior Modelling Priors for Offline Reinforcement Learning
| Title | Keep Doing What Worked: Behavior Modelling Priors for Offline Reinforcement Learning |
| Authors | Anonymous |
| Abstract | Off-policy reinforcement learning algorithms promise to be applicable in settings where only a fixed data-set (batch) of environment interactions is available and no new experience can be acquired. This property makes these algorithms appealing for real world problems such as robot control. In practice, however, standard off-policy algorithms fail in the batch setting for continuous control. In this paper, we propose a simple solution to this problem. It admits the use of data generated by arbitrary behavior policies and uses a learned prior – the advantage-weighted behavior model (ABM) – to bias the RL policy towards actions that have previously been executed and are likely to be successful on the new task. Our method can be seen as an extension of recent work on batch-RL that enables stable learning from conflicting data-sources. We find improvements on competitive baselines in a variety of RL tasks – including standard continuous control benchmarks and multi-task learning for simulated and real-world robots. |
| Tasks | Continuous Control, Multi-Task Learning |
| Published | 2020-01-01 |
| URL | https://openreview.net/forum?id=rke7geHtwH |
| https://openreview.net/pdf?id=rke7geHtwH | |
| PWC | https://paperswithcode.com/paper/keep-doing-what-worked-behavior-modelling |
| Repo | |
| Framework | |
