Paper Group AWR 198
BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication). Neural Network Training with Approximate Logarithmic Computations. Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension. Predicting Research Trends From Arxiv. A Conceptual Framework for Evaluating Fairness in Search. DDGK: Lea …
BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
| Title | BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication) |
| Authors | Marek Rosa, Olga Afanasjeva, Simon Andersson, Joseph Davidson, Nicholas Guttenberg, Petr Hlubuček, Martin Poliak, Jaroslav Vítku, Jan Feyereisl |
| Abstract | In this work, we propose a novel memory-based multi-agent meta-learning architecture and learning procedure that allows for learning of a shared communication policy that enables the emergence of rapid adaptation to new and unseen environments by learning to learn learning algorithms through communication. Behavior, adaptation and learning to adapt emerges from the interactions of homogeneous experts inside a single agent. The proposed architecture should allow for generalization beyond the level seen in existing methods, in part due to the use of a single policy shared by all experts within the agent as well as the inherent modularity of ‘Badger’. |
| Tasks | Meta-Learning |
| Published | 2019-12-03 |
| URL | https://arxiv.org/abs/1912.01513v1 |
| https://arxiv.org/pdf/1912.01513v1.pdf | |
| PWC | https://paperswithcode.com/paper/badger-learning-to-learn-learning-algorithms |
| Repo | https://github.com/GoodAI/badger-2019 |
| Framework | pytorch |
Neural Network Training with Approximate Logarithmic Computations
| Title | Neural Network Training with Approximate Logarithmic Computations |
| Authors | Arnab Sanyal, Peter A. Beerel, Keith M. Chugg |
| Abstract | The high computational complexity associated with training deep neural networks limits online and real-time training on edge devices. This paper proposed an end-to-end training and inference scheme that eliminates multiplications by approximate operations in the log-domain which has the potential to significantly reduce implementation complexity. We implement the entire training procedure in the log-domain, with fixed-point data representations. This training procedure is inspired by hardware-friendly approximations of log-domain addition which are based on look-up tables and bit-shifts. We show that our 16-bit log-based training can achieve classification accuracy within approximately 1% of the equivalent floating-point baselines for a number of commonly used datasets. |
| Tasks | |
| Published | 2019-10-22 |
| URL | https://arxiv.org/abs/1910.09876v1 |
| https://arxiv.org/pdf/1910.09876v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-network-training-with-approximate |
| Repo | https://github.com/arnabsanyal/lnsdnn |
| Framework | none |
Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension
| Title | Explore, Propose, and Assemble: An Interpretable Model for Multi-Hop Reading Comprehension |
| Authors | Yichen Jiang, Nitish Joshi, Yen-Chun Chen, Mohit Bansal |
| Abstract | Multi-hop reading comprehension requires the model to explore and connect relevant information from multiple sentences/documents in order to answer the question about the context. To achieve this, we propose an interpretable 3-module system called Explore-Propose-Assemble reader (EPAr). First, the Document Explorer iteratively selects relevant documents and represents divergent reasoning chains in a tree structure so as to allow assimilating information from all chains. The Answer Proposer then proposes an answer from every root-to-leaf path in the reasoning tree. Finally, the Evidence Assembler extracts a key sentence containing the proposed answer from every path and combines them to predict the final answer. Intuitively, EPAr approximates the coarse-to-fine-grained comprehension behavior of human readers when facing multiple long documents. We jointly optimize our 3 modules by minimizing the sum of losses from each stage conditioned on the previous stage’s output. On two multi-hop reading comprehension datasets WikiHop and MedHop, our EPAr model achieves significant improvements over the baseline and competitive results compared to the state-of-the-art model. We also present multiple reasoning-chain-recovery tests and ablation studies to demonstrate our system’s ability to perform interpretable and accurate reasoning. |
| Tasks | Multi-Hop Reading Comprehension, Reading Comprehension |
| Published | 2019-06-12 |
| URL | https://arxiv.org/abs/1906.05210v1 |
| https://arxiv.org/pdf/1906.05210v1.pdf | |
| PWC | https://paperswithcode.com/paper/explore-propose-and-assemble-an-interpretable |
| Repo | https://github.com/jiangycTarheel/EPAr |
| Framework | tf |
Predicting Research Trends From Arxiv
| Title | Predicting Research Trends From Arxiv |
| Authors | Steffen Eger, Chao Li, Florian Netzer, Iryna Gurevych |
| Abstract | We perform trend detection on two datasets of Arxiv papers, derived from its machine learning (cs.LG) and natural language processing (cs.CL) categories. Our approach is bottom-up: we first rank papers by their normalized citation counts, then group top-ranked papers into different categories based on the tasks that they pursue and the methods they use. We then analyze these resulting topics. We find that the dominating paradigm in cs.CL revolves around natural language generation problems and those in cs.LG revolve around reinforcement learning and adversarial principles. By extrapolation, we predict that these topics will remain lead problems/approaches in their fields in the short- and mid-term. |
| Tasks | Text Generation |
| Published | 2019-03-07 |
| URL | http://arxiv.org/abs/1903.02831v1 |
| http://arxiv.org/pdf/1903.02831v1.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-research-trends-from-arxiv |
| Repo | https://github.com/UKPLab/refresh2018-predicting-trends-from-arxiv |
| Framework | none |
A Conceptual Framework for Evaluating Fairness in Search
| Title | A Conceptual Framework for Evaluating Fairness in Search |
| Authors | Anubrata Das, Matthew Lease |
| Abstract | While search efficacy has been evaluated traditionally on the basis of result relevance, fairness of search has attracted recent attention. In this work, we define a notion of distributional fairness and provide a conceptual framework for evaluating search results based on it. As part of this, we formulate a set of axioms which an ideal evaluation framework should satisfy for distributional fairness. We show how existing TREC test collections can be repurposed to study fairness, and we measure potential data bias to inform test collection design for fair search. A set of analyses show metric divergence between relevance and fairness, and we describe a simple but flexible interpolation strategy for integrating relevance and fairness into a single metric for optimization and evaluation. |
| Tasks | |
| Published | 2019-07-22 |
| URL | https://arxiv.org/abs/1907.09328v1 |
| https://arxiv.org/pdf/1907.09328v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-conceptual-framework-for-evaluating |
| Repo | https://github.com/anubrata/anubrata.github.io |
| Framework | none |
DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
| Title | DDGK: Learning Graph Representations for Deep Divergence Graph Kernels |
| Authors | Rami Al-Rfou, Dustin Zelle, Bryan Perozzi |
| Abstract | Can neural networks learn to compare graphs without feature engineering? In this paper, we show that it is possible to learn representations for graph similarity with neither domain knowledge nor supervision (i.e.\ feature engineering or labeled graphs). We propose Deep Divergence Graph Kernels, an unsupervised method for learning representations over graphs that encodes a relaxed notion of graph isomorphism. Our method consists of three parts. First, we learn an encoder for each anchor graph to capture its structure. Second, for each pair of graphs, we train a cross-graph attention network which uses the node representations of an anchor graph to reconstruct another graph. This approach, which we call isomorphism attention, captures how well the representations of one graph can encode another. We use the attention-augmented encoder’s predictions to define a divergence score for each pair of graphs. Finally, we construct an embedding space for all graphs using these pair-wise divergence scores. Unlike previous work, much of which relies on 1) supervision, 2) domain specific knowledge (e.g. a reliance on Weisfeiler-Lehman kernels), and 3) known node alignment, our unsupervised method jointly learns node representations, graph representations, and an attention-based alignment between graphs. Our experimental results show that Deep Divergence Graph Kernels can learn an unsupervised alignment between graphs, and that the learned representations achieve competitive results when used as features on a number of challenging graph classification tasks. Furthermore, we illustrate how the learned attention allows insight into the the alignment of sub-structures across graphs. |
| Tasks | Feature Engineering, Graph Classification, Graph Similarity |
| Published | 2019-04-21 |
| URL | http://arxiv.org/abs/1904.09671v1 |
| http://arxiv.org/pdf/1904.09671v1.pdf | |
| PWC | https://paperswithcode.com/paper/ddgk-learning-graph-representations-for-deep |
| Repo | https://github.com/google-research/google-research/tree/master/graph_embedding/ddgk |
| Framework | tf |
Seeing the Wind: Visual Wind Speed Prediction with a Coupled Convolutional and Recurrent Neural Network
| Title | Seeing the Wind: Visual Wind Speed Prediction with a Coupled Convolutional and Recurrent Neural Network |
| Authors | Jennifer L Cardona, Michael F Howland, John O Dabiri |
| Abstract | Wind energy resource quantification, air pollution monitoring, and weather forecasting all rely on rapid, accurate measurement of local wind conditions. Visual observations of the effects of wind—the swaying of trees and flapping of flags, for example—encode information regarding local wind conditions that can potentially be leveraged for visual anemometry that is inexpensive and ubiquitous. Here, we demonstrate a coupled convolutional neural network and recurrent neural network architecture that extracts the wind speed encoded in visually recorded flow-structure interactions of a flag and tree in naturally occurring wind. Predictions for wind speeds ranging from 0.75-11 m/s showed agreement with measurements from a cup anemometer on site, with a root-mean-squared error approaching the natural wind speed variability due to atmospheric turbulence. Generalizability of the network was demonstrated by successful prediction of wind speed based on recordings of other flags in the field and in a controlled wind tunnel test. Furthermore, physics-based scaling of the flapping dynamics accurately predicts the dependence of the network performance on the video frame rate and duration. |
| Tasks | Weather Forecasting |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1905.13290v3 |
| https://arxiv.org/pdf/1905.13290v3.pdf | |
| PWC | https://paperswithcode.com/paper/seeing-the-wind-visual-wind-speed-prediction |
| Repo | https://github.com/luoye2333/ResNetLSTM |
| Framework | pytorch |
Safe squeezing for antisparse coding
| Title | Safe squeezing for antisparse coding |
| Authors | Clément Elvira, Cédric Herzet |
| Abstract | Spreading the information over all coefficients of a representation is a desirable property in many applications such as digital communication or machine learning. This so-called antisparse representation can be obtained by solving a convex program involving an $\ell_\infty$-norm penalty combined with a quadratic discrepancy. In this paper, we propose a new methodology, dubbed safe squeezing, to accelerate the computation of antisparse representation. We describe a test that allows to detect saturated entries in the solution of the optimization problem. The contribution of these entries is compacted into a single vector, thus operating a form of dimensionality reduction. We propose two algorithms to solve the resulting lower dimensional problem. Numerical experiments show the effectiveness of the proposed method to detect the saturated components of the solution and illustrates the induced computational gains in the resolution of the antisparse problem. |
| Tasks | Dimensionality Reduction |
| Published | 2019-11-18 |
| URL | https://arxiv.org/abs/1911.07508v2 |
| https://arxiv.org/pdf/1911.07508v2.pdf | |
| PWC | https://paperswithcode.com/paper/safe-squeezing-for-antisparse-coding |
| Repo | https://github.com/c-elvira/safe-squeezing |
| Framework | none |
Networks for Joint Affine and Non-parametric Image Registration
| Title | Networks for Joint Affine and Non-parametric Image Registration |
| Authors | Zhengyang Shen, Xu Han, Zhenlin Xu, Marc Niethammer |
| Abstract | We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a Unet-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF component to provide a non-parametric refinement based on the current estimate of the transformation map. Once the model is trained, a registration is completed in one forward pass. To evaluate the performance, we conducted longitudinal and cross-subject experiments on 3D magnetic resonance images (MRI) of the knee of the Osteoarthritis Initiative (OAI) dataset. Results show that our framework achieves comparable performance to state-of-the-art medical image registration approaches, but it is much faster, with a better control of transformation regularity including the ability to produce approximately symmetric transformations, and combining affine and non-parametric registration. |
| Tasks | Image Registration, Medical Image Registration |
| Published | 2019-03-21 |
| URL | http://arxiv.org/abs/1903.08811v1 |
| http://arxiv.org/pdf/1903.08811v1.pdf | |
| PWC | https://paperswithcode.com/paper/networks-for-joint-affine-and-non-parametric |
| Repo | https://github.com/uncbiag/registration |
| Framework | pytorch |
On Reducing Negative Jacobian Determinant of the Deformation Predicted by Deep Registration Networks
| Title | On Reducing Negative Jacobian Determinant of the Deformation Predicted by Deep Registration Networks |
| Authors | Dongyang Kuang |
| Abstract | Image registration is a fundamental step in medical image analysis. Ideally, the transformation that registers one image to another should be a diffeomorphism that is both invertible and smooth. Traditional methods like geodesic shooting approach the problem via differential geometry, with theoretical guarantees that the resulting transformation will be smooth and invertible. Most previous research using unsupervised deep neural networks for registration have used a local smoothness constraint (typically, a spatial variation loss) to address the smoothness issue. These networks usually produce non-invertible transformations with ``folding’’ in multiple voxel locations, indicated by a negative determinant of the Jacobian matrix of the transformation. While using a loss function that specifically penalizes the folding is a straightforward solution, this usually requires carefully tuning the regularization strength, especially when there are also other losses. In this paper we address this problem from a different angle, by investigating possible training mechanisms that will help the network avoid negative Jacobians and produce smoother deformations. We contribute two independent ideas in this direction. Both ideas greatly reduce the number of folding locations in the predicted deformation, without making changes to the hyperparameters or the architecture used in the existing baseline registration network. | |
| Tasks | Image Registration |
| Published | 2019-06-28 |
| URL | https://arxiv.org/abs/1907.00068v1 |
| https://arxiv.org/pdf/1907.00068v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-reducing-negative-jacobian-determinant-of |
| Repo | https://github.com/dykuang/Medical-image-registration |
| Framework | tf |
Curls & Whey: Boosting Black-Box Adversarial Attacks
| Title | Curls & Whey: Boosting Black-Box Adversarial Attacks |
| Authors | Yucheng Shi, Siyu Wang, Yahong Han |
| Abstract | Image classifiers based on deep neural networks suffer from harassment caused by adversarial examples. Two defects exist in black-box iterative attacks that generate adversarial examples by incrementally adjusting the noise-adding direction for each step. On the one hand, existing iterative attacks add noises monotonically along the direction of gradient ascent, resulting in a lack of diversity and adaptability of the generated iterative trajectories. On the other hand, it is trivial to perform adversarial attack by adding excessive noises, but currently there is no refinement mechanism to squeeze redundant noises. In this work, we propose Curls & Whey black-box attack to fix the above two defects. During Curls iteration, by combining gradient ascent and descent, we curl' up iterative trajectories to integrate more diversity and transferability into adversarial examples. Curls iteration also alleviates the diminishing marginal effect in existing iterative attacks. The Whey optimization further squeezes the whey’ of noises by exploiting the robustness of adversarial perturbation. Extensive experiments on Imagenet and Tiny-Imagenet demonstrate that our approach achieves impressive decrease on noise magnitude in l2 norm. Curls & Whey attack also shows promising transferability against ensemble models as well as adversarially trained models. In addition, we extend our attack to the targeted misclassification, effectively reducing the difficulty of targeted attacks under black-box condition. |
| Tasks | Adversarial Attack |
| Published | 2019-04-02 |
| URL | http://arxiv.org/abs/1904.01160v1 |
| http://arxiv.org/pdf/1904.01160v1.pdf | |
| PWC | https://paperswithcode.com/paper/curls-whey-boosting-black-box-adversarial |
| Repo | https://github.com/walegahaha/Curls-Whey |
| Framework | pytorch |
Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data
| Title | Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data |
| Authors | Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Duc Thanh Nguyen, Sai-Kit Yeung |
| Abstract | Deep learning techniques for point cloud data have demonstrated great potentials in solving classical problems in 3D computer vision such as 3D object classification and segmentation. Several recent 3D object classification methods have reported state-of-the-art performance on CAD model datasets such as ModelNet40 with high accuracy (~92%). Despite such impressive results, in this paper, we argue that object classification is still a challenging task when objects are framed with real-world settings. To prove this, we introduce ScanObjectNN, a new real-world point cloud object dataset based on scanned indoor scene data. From our comprehensive benchmark, we show that our dataset poses great challenges to existing point cloud classification techniques as objects from real-world scans are often cluttered with background and/or are partial due to occlusions. We identify three key open problems for point cloud object classification, and propose new point cloud classification neural networks that achieve state-of-the-art performance on classifying objects with cluttered background. Our dataset and code are publicly available in our project page https://hkust-vgd.github.io/scanobjectnn/. |
| Tasks | 3D Object Classification, Object Classification |
| Published | 2019-08-13 |
| URL | https://arxiv.org/abs/1908.04616v2 |
| https://arxiv.org/pdf/1908.04616v2.pdf | |
| PWC | https://paperswithcode.com/paper/revisiting-point-cloud-classification-a-new |
| Repo | https://github.com/hkust-vgd/scanobjectnn |
| Framework | tf |
Survey of Artificial Intelligence for Card Games and Its Application to the Swiss Game Jass
| Title | Survey of Artificial Intelligence for Card Games and Its Application to the Swiss Game Jass |
| Authors | Joel Niklaus, Michele Alberti, Vinaychandran Pondenkandath, Rolf Ingold, Marcus Liwicki |
| Abstract | In the last decades we have witnessed the success of applications of Artificial Intelligence to playing games. In this work we address the challenging field of games with hidden information and card games in particular. Jass is a very popular card game in Switzerland and is closely connected with Swiss culture. To the best of our knowledge, performances of Artificial Intelligence agents in the game of Jass do not outperform top players yet. Our contribution to the community is two-fold. First, we provide an overview of the current state-of-the-art of Artificial Intelligence methods for card games in general. Second, we discuss their application to the use-case of the Swiss card game Jass. This paper aims to be an entry point for both seasoned researchers and new practitioners who want to join in the Jass challenge. |
| Tasks | Card Games |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.04439v1 |
| https://arxiv.org/pdf/1906.04439v1.pdf | |
| PWC | https://paperswithcode.com/paper/survey-of-artificial-intelligence-for-card |
| Repo | https://github.com/tobiasemrich/SchafkopfRL |
| Framework | pytorch |
Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation
| Title | Regularizing Proxies with Multi-Adversarial Training for Unsupervised Domain-Adaptive Semantic Segmentation |
| Authors | Tong Shen, Dong Gong, Wei Zhang, Chunhua Shen, Tao Mei |
| Abstract | Training a semantic segmentation model requires a large amount of pixel-level annotation, hampering its application at scale. With computer graphics, we can generate almost unlimited training data with precise annotation. However,a deep model trained with synthetic data usually cannot directly generalize well to realistic images due to domain shift. It has been observed that highly confident labels for the unlabeled real images may be predicted relying on the labeled synthetic data. To tackle the unsupervised domain adaptation problem, we explore the possibilities to generate high-quality labels as proxy labels to supervise the training on target data. Specifically, we propose a novel proxy-based method using multi-adversarial training. We first train the model using synthetic data (source domain). Multiple discriminators are used to align the features be-tween the source and target domain (real images) at different levels. Then we focus on obtaining and selecting high-quality proxy labels by incorporating both the confidence of the class predictor and that from the adversarial discriminators. Our discriminators not only work as a regularizer to encourage feature alignment but also provide an alternative confidence measure for generating proxy labels. Relying on the generated high-quality proxies, our model can be trained in a “supervised manner” on the target do-main. On two major tasks, GTA5->Cityscapes and SYNTHIA->Cityscapes, our method achieves state-of-the-art results, outperforming the previous by a large margin. |
| Tasks | Domain Adaptation, Semantic Segmentation, Unsupervised Domain Adaptation |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12282v1 |
| https://arxiv.org/pdf/1907.12282v1.pdf | |
| PWC | https://paperswithcode.com/paper/regularizing-proxies-with-multi-adversarial |
| Repo | https://github.com/ascust/MADA |
| Framework | mxnet |
Extreme Points Derived Confidence Map as a Cue For Class-Agnostic Segmentation Using Deep Neural Network
| Title | Extreme Points Derived Confidence Map as a Cue For Class-Agnostic Segmentation Using Deep Neural Network |
| Authors | Shadab Khan, Ahmed H. Shahin, Javier Villafruela, Jianbing Shen, Ling Shao |
| Abstract | To automate the process of segmenting an anatomy of interest, we can learn a model from previously annotated data. The learning-based approach uses annotations to train a model that tries to emulate the expert labeling on a new data set. While tremendous progress has been made using such approaches, labeling of medical images remains a time-consuming and expensive task. In this paper, we evaluate the utility of extreme points in learning to segment. Specifically, we propose a novel approach to compute a confidence map from extreme points that quantitatively encodes the priors derived from extreme points. We use the confidence map as a cue to train a deep neural network based on ResNet-101 and PSP module to develop a class-agnostic segmentation model that outperforms state-of-the-art method that employs extreme points as a cue. Further, we evaluate a realistic use-case by using our model to generate training data for supervised learning (U-Net) and observed that U-Net performs comparably when trained with either the generated data or the ground truth data. These findings suggest that models trained using cues can be used to generate reliable training data. |
| Tasks | |
| Published | 2019-06-06 |
| URL | https://arxiv.org/abs/1906.02421v1 |
| https://arxiv.org/pdf/1906.02421v1.pdf | |
| PWC | https://paperswithcode.com/paper/extreme-points-derived-confidence-map-as-a |
| Repo | https://github.com/ahmedshahin9/AssistedAnnotator |
| Framework | pytorch |
