February 2, 2020

3389 words 16 mins read

Paper Group AWR 72

Topic Sensitive Attention on Generic Corpora Corrects Sense Bias in Pretrained Embeddings. RandAugment: Practical automated data augmentation with a reduced search space. TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors. Big Transfer (BiT): General Visual Representation Learning. Semantic Segmentation of …

Topic Sensitive Attention on Generic Corpora Corrects Sense Bias in Pretrained Embeddings


Title	Topic Sensitive Attention on Generic Corpora Corrects Sense Bias in Pretrained Embeddings
Authors	Vihari Piratla, Sunita Sarawagi, Soumen Chakrabarti
Abstract	Given a small corpus $\mathcal D_T$ pertaining to a limited set of focused topics, our goal is to train embeddings that accurately capture the sense of words in the topic in spite of the limited size of $\mathcal D_T$. These embeddings may be used in various tasks involving $\mathcal D_T$. A popular strategy in limited data settings is to adapt pre-trained embeddings $\mathcal E$ trained on a large corpus. To correct for sense drift, fine-tuning, regularization, projection, and pivoting have been proposed recently. Among these, regularization informed by a word’s corpus frequency performed well, but we improve upon it using a new regularizer based on the stability of its cooccurrence with other words. However, a thorough comparison across ten topics, spanning three tasks, with standardized settings of hyper-parameters, reveals that even the best embedding adaptation strategies provide small gains beyond well-tuned baselines, which many earlier comparisons ignored. In a bold departure from adapting pretrained embeddings, we propose using $\mathcal D_T$ to probe, attend to, and borrow fragments from any large, topic-rich source corpus (such as Wikipedia), which need not be the corpus used to pretrain embeddings. This step is made scalable and practical by suitable indexing. We reach the surprising conclusion that even limited corpus augmentation is more useful than adapting embeddings, which suggests that non-dominant sense information may be irrevocably obliterated from pretrained embeddings and cannot be salvaged by adaptation.
Tasks
Published	2019-06-05
URL	https://arxiv.org/abs/1906.02688v2
PDF	https://arxiv.org/pdf/1906.02688v2.pdf
PWC	https://paperswithcode.com/paper/topic-sensitive-attention-on-generic-corpora
Repo	https://github.com/vihari/focussed_embs
Framework	tf

RandAugment: Practical automated data augmentation with a reduced search space


Title	RandAugment: Practical automated data augmentation with a reduced search space
Authors	Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, Quoc V. Le
Abstract	Recent work has shown that data augmentation has the potential to significantly improve the generalization of deep learning models. Recently, automated augmentation strategies have led to state-of-the-art results in image classification and object detection. While these strategies were optimized for improving validation accuracy, they also led to state-of-the-art results in semi-supervised learning and improved robustness to common corruptions of images. An obstacle to a large-scale adoption of these methods is a separate search phase which increases the training complexity and may substantially increase the computational cost. Additionally, due to the separate search phase, these approaches are unable to adjust the regularization strength based on model or dataset size. Automated augmentation policies are often found by training small models on small datasets and subsequently applied to train larger models. In this work, we remove both of these obstacles. RandAugment has a significantly reduced search space which allows it to be trained on the target task with no need for a separate proxy task. Furthermore, due to the parameterization, the regularization strength may be tailored to different model and dataset sizes. RandAugment can be used uniformly across different tasks and datasets and works out of the box, matching or surpassing all previous automated augmentation approaches on CIFAR-10/100, SVHN, and ImageNet. On the ImageNet dataset we achieve 85.0% accuracy, a 0.6% increase over the previous state-of-the-art and 1.0% increase over baseline augmentation. On object detection, RandAugment leads to 1.0-1.3% improvement over baseline augmentation, and is within 0.3% mAP of AutoAugment on COCO. Finally, due to its interpretable hyperparameter, RandAugment may be used to investigate the role of data augmentation with varying model and dataset size. Code is available online.
Tasks	Data Augmentation, Image Classification, Object Detection
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13719v2
PDF	https://arxiv.org/pdf/1909.13719v2.pdf
PWC	https://paperswithcode.com/paper/randaugment-practical-data-augmentation-with
Repo	https://github.com/etetteh/sota-data-augmentation-and-optimizers
Framework	pytorch

TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors


Title	TactileGCN: A Graph Convolutional Network for Predicting Grasp Stability with Tactile Sensors
Authors	Alberto Garcia-Garcia, Brayan Stiven Zapata-Impata, Sergio Orts-Escolano, Pablo Gil, Jose Garcia-Rodriguez
Abstract	Tactile sensors provide useful contact data during the interaction with an object which can be used to accurately learn to determine the stability of a grasp. Most of the works in the literature represented tactile readings as plain feature vectors or matrix-like tactile images, using them to train machine learning models. In this work, we explore an alternative way of exploiting tactile information to predict grasp stability by leveraging graph-like representations of tactile data, which preserve the actual spatial arrangement of the sensor’s taxels and their locality. In experimentation, we trained a Graph Neural Network to binary classify grasps as stable or slippery ones. To train such network and prove its predictive capabilities for the problem at hand, we captured a novel dataset of approximately 5000 three-fingered grasps across 41 objects for training and 1000 grasps with 10 unknown objects for testing. Our experiments prove that this novel approach can be effectively used to predict grasp stability.
Tasks
Published	2019-01-18
URL	http://arxiv.org/abs/1901.06181v1
PDF	http://arxiv.org/pdf/1901.06181v1.pdf
PWC	https://paperswithcode.com/paper/tactilegcn-a-graph-convolutional-network-for
Repo	https://github.com/3dperceptionlab/biotacsp-stability-set-v2
Framework	none

Big Transfer (BiT): General Visual Representation Learning


Title	Big Transfer (BiT): General Visual Representation Learning
Authors	Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, Neil Houlsby
Abstract	Transfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes – from 1 example per class to 1M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.
Tasks	Fine-Grained Image Classification, Image Classification, Representation Learning
Published	2019-12-24
URL	https://arxiv.org/abs/1912.11370v2
PDF	https://arxiv.org/pdf/1912.11370v2.pdf
PWC	https://paperswithcode.com/paper/large-scale-learning-of-general-visual
Repo	https://github.com/SoojungYang/supervised_pretraining_GN_WS
Framework	tf

Semantic Segmentation of Panoramic Images Using a Synthetic Dataset


Title	Semantic Segmentation of Panoramic Images Using a Synthetic Dataset
Authors	Yuanyou Xu, Kaiwei Wang, Kailun Yang, Dongming Sun, Jia Fu
Abstract	Panoramic images have advantages in information capacity and scene stability due to their large field of view (FoV). In this paper, we propose a method to synthesize a new dataset of panoramic image. We managed to stitch the images taken from different directions into panoramic images, together with their labeled images, to yield the panoramic semantic segmentation dataset denominated as SYNTHIA-PANO. For the purpose of finding out the effect of using panoramic images as training dataset, we designed and performed a comprehensive set of experiments. Experimental results show that using panoramic images as training data is beneficial to the segmentation result. In addition, it has been shown that by using panoramic images with a 180 degree FoV as training data the model has better performance. Furthermore, the model trained with panoramic images also has a better capacity to resist the image distortion.
Tasks	Semantic Segmentation
Published	2019-09-02
URL	https://arxiv.org/abs/1909.00532v1
PDF	https://arxiv.org/pdf/1909.00532v1.pdf
PWC	https://paperswithcode.com/paper/semantic-segmentation-of-panoramic-images
Repo	https://github.com/Francis515/SYNTHIA-PANO
Framework	none

Spatial and Colour Opponency in Anatomically Constrained Deep Networks


Title	Spatial and Colour Opponency in Anatomically Constrained Deep Networks
Authors	Ethan Harris, Daniela Mihai, Jonathon Hare
Abstract	Colour vision has long fascinated scientists, who have sought to understand both the physiology of the mechanics of colour vision and the psychophysics of colour perception. We consider representations of colour in anatomically constrained convolutional deep neural networks. Following ideas from neuroscience, we classify cells in early layers into groups relating to their spectral and spatial functionality. We show the emergence of single and double opponent cells in our networks and characterise how the distribution of these cells changes under the constraint of a retinal bottleneck. Our experiments not only open up a new understanding of how deep networks process spatial and colour information, but also provide new tools to help understand the black box of deep learning. The code for all experiments is avaialable at \url{https://github.com/ecs-vlc/opponency}.
Tasks
Published	2019-10-14
URL	https://arxiv.org/abs/1910.11086v1
PDF	https://arxiv.org/pdf/1910.11086v1.pdf
PWC	https://paperswithcode.com/paper/spatial-and-colour-opponency-in-anatomically
Repo	https://github.com/ecs-vlc/opponency
Framework	pytorch

AtomNAS: Fine-Grained End-to-End Neural Architecture Search


Title	AtomNAS: Fine-Grained End-to-End Neural Architecture Search
Authors	Jieru Mei, Yingwei Li, Xiaochen Lian, Xiaojie Jin, Linjie Yang, Alan Yuille, Jianchao Yang
Abstract	Search space design is very critical to neural architecture search (NAS) algorithms. We propose a fine-grained search space comprised of atomic blocks, a minimal search unit that is much smaller than the ones used in recent NAS algorithms. This search space allows a mix of operations by composing different types of atomic blocks, while the search space in previous methods only allows homogeneous operations. Based on this search space, we propose a resource-aware architecture search framework which automatically assigns the computational resources (e.g., output channel numbers) for each operation by jointly considering the performance and the computational cost. In addition, to accelerate the search process, we propose a dynamic network shrinkage technique which prunes the atomic blocks with negligible influence on outputs on the fly. Instead of a search-and-retrain two-stage paradigm, our method simultaneously searches and trains the target architecture. Our method achieves state-of-the-art performance under several FLOPs configurations on ImageNet with a small searching cost. We open our entire codebase at: https://github.com/meijieru/AtomNAS.
Tasks	Neural Architecture Search
Published	2019-12-20
URL	https://arxiv.org/abs/1912.09640v2
PDF	https://arxiv.org/pdf/1912.09640v2.pdf
PWC	https://paperswithcode.com/paper/atomnas-fine-grained-end-to-end-neural-1
Repo	https://github.com/meijieru/AtomNAS
Framework	pytorch

An α-Matte Boundary Defocus Model Based Cascaded Network for Multi-focus Image Fusion


Title	An α-Matte Boundary Defocus Model Based Cascaded Network for Multi-focus Image Fusion
Authors	Haoyu Ma, Qingmin Liao, Juncheng Zhang, Shaojun Liu, Jing-Hao Xue
Abstract	Capturing an all-in-focus image with a single camera is difficult since the depth of field of the camera is usually limited. An alternative method to obtain the all-in-focus image is to fuse several images focusing at different depths. However, existing multi-focus image fusion methods cannot obtain clear results for areas near the focused/defocused boundary (FDB). In this paper, a novel {\alpha}-matte boundary defocus model is proposed to generate realistic training data with the defocus spread effect precisely modeled, especially for areas near the FDB. Based on this {\alpha}-matte defocus model and the generated data, a cascaded boundary aware convolutional network termed MMF-Net is proposed and trained, aiming to achieve clearer fusion results around the FDB. More specifically, the MMF-Net consists of two cascaded sub-nets for initial fusion and boundary fusion, respectively; these two sub-nets are designed to first obtain a guidance map of FDB and then refine the fusion near the FDB. Experiments demonstrate that with the help of the new {\alpha}-matte boundary defocus model, the proposed MMF-Net outperforms the state-of-the-art methods both qualitatively and quantitatively.
Tasks
Published	2019-10-29
URL	https://arxiv.org/abs/1910.13136v2
PDF	https://arxiv.org/pdf/1910.13136v2.pdf
PWC	https://paperswithcode.com/paper/an-matte-boundary-defocus-model-based
Repo	https://github.com/xytmhy/MMF-Net-alpha-Matte-Boundary-Defocus-Model-Fusion
Framework	pytorch

Ablation Studies in Artificial Neural Networks


Title	Ablation Studies in Artificial Neural Networks
Authors	Richard Meyes, Melanie Lu, Constantin Waubert de Puiseau, Tobias Meisen
Abstract	Ablation studies have been widely used in the field of neuroscience to tackle complex biological systems such as the extensively studied Drosophila central nervous system, the vertebrate brain and more interestingly and most delicately, the human brain. In the past, these kinds of studies were utilized to uncover structure and organization in the brain, i.e. a mapping of features inherent to external stimuli onto different areas of the neocortex. considering the growth in size and complexity of state-of-the-art artificial neural networks (ANNs) and the corresponding growth in complexity of the tasks that are tackled by these networks, the question arises whether ablation studies may be used to investigate these networks for a similar organization of their inner representations. In this paper, we address this question and performed two ablation studies in two fundamentally different ANNs to investigate their inner representations of two well-known benchmark datasets from the computer vision domain. We found that features distinct to the local and global structure of the data are selectively represented in specific parts of the network. Furthermore, some of these representations are redundant, awarding the network a certain robustness to structural damages. We further determined the importance of specific parts of the network for the classification task solely based on the weight structure of single units. Finally, we examined the ability of damaged networks to recover from the consequences of ablations by means of recovery training. We argue that ablations studies are a feasible method to investigate knowledge representations in ANNs and are especially helpful to examine a networks robustness to structural damages, a feature of ANNs that will become increasingly important for future safety-critical applications.
Tasks
Published	2019-01-24
URL	http://arxiv.org/abs/1901.08644v2
PDF	http://arxiv.org/pdf/1901.08644v2.pdf
PWC	https://paperswithcode.com/paper/ablation-studies-in-artificial-neural
Repo	https://github.com/RichardMeyes/AblationStudies
Framework	pytorch

Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference


Title	Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference
Authors	Niccolò Dalmasso, Taylor Pospisil, Ann B. Lee, Rafael Izbicki, Peter E. Freeman, Alex I. Malz
Abstract	It is well known in astronomy that propagating non-Gaussian prediction uncertainty in photometric redshift estimates is key to reducing bias in downstream cosmological analyses. Similarly, likelihood-free inference approaches, which are beginning to emerge as a tool for cosmological analysis, require a characterization of the full uncertainty landscape of the parameters of interest given observed data. However, most machine learning (ML) or training-based methods with open-source software target point prediction or classification, and hence fall short in quantifying uncertainty in complex regression and parameter inference settings. As an alternative to methods that focus on predicting the response (or parameters) $\mathbf{y}$ from features $\mathbf{x}$, we provide nonparametric conditional density estimation (CDE) tools for approximating and validating the entire probability density function (PDF) $\mathrm{p}(\mathbf{y}\mathbf{x})$ of $\mathbf{y}$ given (i.e., conditional on) $\mathbf{x}$. As there is no one-size-fits-all CDE method, the goal of this work is to provide a comprehensive range of statistical tools and open-source software for nonparametric CDE and method assessment which can accommodate different types of settings and be easily fit to the problem at hand. Specifically, we introduce four CDE software packages in $\texttt{Python}$ and $\texttt{R}$ based on ML prediction methods adapted and optimized for CDE: $\texttt{NNKCDE}$, $\texttt{RFCDE}$, $\texttt{FlexCode}$, and $\texttt{DeepCDE}$. Furthermore, we present the $\texttt{cdetools}$ package, which includes functions for computing a CDE loss function for tuning and assessing the quality of individual PDFs, along with diagnostic functions. We provide sample code in $\texttt{Python}$ and $\texttt{R}$ as well as examples of applications to photometric redshift estimation and likelihood-free cosmological inference via CDE.
Tasks	Density Estimation, Model Selection, Photometric Redshift Estimation
Published	2019-08-30
URL	https://arxiv.org/abs/1908.11523v2
PDF	https://arxiv.org/pdf/1908.11523v2.pdf
PWC	https://paperswithcode.com/paper/conditional-density-estimation-tools-in
Repo	https://github.com/Mr8ND/cdetools_applications
Framework	none

Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models


Title	Purifying Interaction Effects with the Functional ANOVA: An Efficient Algorithm for Recovering Identifiable Additive Models
Authors	Benjamin Lengerich, Sarah Tan, Chun-Hao Chang, Giles Hooker, Rich Caruana
Abstract	Recent methods for training generalized additive models (GAMs) with pairwise interactions achieve state-of-the-art accuracy on a variety of datasets. Adding interactions to GAMs, however, introduces an identifiability problem: effects can be freely moved between main effects and interaction effects without changing the model predictions. In some cases, this can lead to contradictory interpretations of the same underlying function. This is a critical problem because a central motivation of GAMs is model interpretability. In this paper, we use the Functional ANOVA decomposition to uniquely define interaction effects and thus produce identifiable additive models with purified interactions. To compute this decomposition, we present a fast, exact, mass-moving algorithm that transforms any piecewise-constant function (such as a tree-based model) into a purified, canonical representation. We apply this algorithm to several datasets and show large disparity, including contradictions, between the apparent and the purified effects.
Tasks
Published	2019-11-12
URL	https://arxiv.org/abs/1911.04974v1
PDF	https://arxiv.org/pdf/1911.04974v1.pdf
PWC	https://paperswithcode.com/paper/purifying-interaction-effects-with-the
Repo	https://github.com/microsoft/interpret
Framework	none

Hierarchical Gating Networks for Sequential Recommendation


Title	Hierarchical Gating Networks for Sequential Recommendation
Authors	Chen Ma, Peng Kang, Xue Liu
Abstract	The chronological order of user-item interactions is a key feature in many recommender systems, where the items that users will interact may largely depend on those items that users just accessed recently. However, with the tremendous increase of users and items, sequential recommender systems still face several challenging problems: (1) the hardness of modeling the long-term user interests from sparse implicit feedback; (2) the difficulty of capturing the short-term user interests given several items the user just accessed. To cope with these challenges, we propose a hierarchical gating network (HGN), integrated with the Bayesian Personalized Ranking (BPR) to capture both the long-term and short-term user interests. Our HGN consists of a feature gating module, an instance gating module, and an item-item product module. In particular, our feature gating and instance gating modules select what item features can be passed to the downstream layers from the feature and instance levels, respectively. Our item-item product module explicitly captures the item relations between the items that users accessed in the past and those items users will access in the future. We extensively evaluate our model with several state-of-the-art methods and different validation metrics on five real-world datasets. The experimental results demonstrate the effectiveness of our model on Top-N sequential recommendation.
Tasks	Recommendation Systems
Published	2019-06-21
URL	https://arxiv.org/abs/1906.09217v1
PDF	https://arxiv.org/pdf/1906.09217v1.pdf
PWC	https://paperswithcode.com/paper/hierarchical-gating-networks-for-sequential
Repo	https://github.com/wuliwei9278/HGN_baseline
Framework	pytorch

Partial Scanning Transmission Electron Microscopy with Deep Learning


Title	Partial Scanning Transmission Electron Microscopy with Deep Learning
Authors	Jeffrey M. Ede, Richard Beanland
Abstract	Compressed sensing algorithms are used to decrease electron microscope scan time and electron beam exposure with minimal information loss. Following successful applications of deep learning to compressed sensing, we have developed a two-stage multiscale generative adversarial neural network to complete realistic 512$\times$512 scanning transmission electron micrographs from spiral, jittered gridlike, and other partial scans. For spiral scans and mean squared error based pre-training, this enables electron beam coverage to be decreased by 17.9$\times$ with a 3.8% test set root mean squared intensity error, and by 87.0$\times$ with a 6.2% error. Our generator networks are trained on partial scans created from a new dataset of 16227 scanning transmission electron micrographs. High performance is achieved with adaptive learning rate clipping of loss spikes and an auxiliary trainer network. Our source code, new dataset, and pre-trained models have been made publicly available at https://github.com/Jeffrey-Ede/partial-STEM
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1905.13667v2
PDF	https://arxiv.org/pdf/1905.13667v2.pdf
PWC	https://paperswithcode.com/paper/partial-scan-electron-microscopy-with-deep
Repo	https://github.com/Jeffrey-Ede/ALRC
Framework	tf

Alzheimer’s Disease Brain MRI Classification: Challenges and Insights


Title	Alzheimer’s Disease Brain MRI Classification: Challenges and Insights
Authors	Yi Ren Fung, Ziqiang Guan, Ritesh Kumar, Joie Yeahuay Wu, Madalina Fiterau
Abstract	In recent years, many papers have reported state-of-the-art performance on Alzheimer’s Disease classification with MRI scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset using convolutional neural networks. However, we discover that when we split that data into training and testing sets at the subject level, we are not able to obtain similar performance, bringing the validity of many of the previous studies into question. Furthermore, we point out that previous works use different subsets of the ADNI data, making comparison across similar works tricky. In this study, we present the results of three splitting methods, discuss the motivations behind their validity, and report our results using all of the available subjects.
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04231v1
PDF	https://arxiv.org/pdf/1906.04231v1.pdf
PWC	https://paperswithcode.com/paper/alzheimers-disease-brain-mri-classification
Repo	https://github.com/Information-Fusion-Lab-Umass/alzheimers-cnn-study
Framework	pytorch

Boundary-Aware Network for Fast and High-Accuracy Portrait Segmentation


Title	Boundary-Aware Network for Fast and High-Accuracy Portrait Segmentation
Authors	Xi Chen, Donglian Qi, Jianxin Shen
Abstract	Compared with other semantic segmentation tasks, portrait segmentation requires both higher precision and faster inference speed. However, this problem has not been well studied in previous works. In this paper, we propose a lightweight network architecture, called Boundary-Aware Network (BANet) which selectively extracts detail information in boundary area to make high-quality segmentation output with real-time( >25FPS) speed. In addition, we design a new loss function called refine loss which supervises the network with image level gradient information. Our model is able to produce finer segmentation results which has richer details than annotations.
Tasks	Semantic Segmentation
Published	2019-01-12
URL	http://arxiv.org/abs/1901.03814v1
PDF	http://arxiv.org/pdf/1901.03814v1.pdf
PWC	https://paperswithcode.com/paper/boundary-aware-network-for-fast-and-high
Repo	https://github.com/lewisluk/BoundaryAwareNetwork
Framework	tf