July 30, 2019

3160 words 15 mins read

Paper Group AWR 34

Towards Reverse-Engineering Black-Box Neural Networks. Multiple Source Domain Adaptation with Adversarial Training of Neural Networks. A deep generative model for gene expression profiles from single-cell RNA sequencing. MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach. PACRR: A Position-Aware Neural IR Model f …

Towards Reverse-Engineering Black-Box Neural Networks


Title	Towards Reverse-Engineering Black-Box Neural Networks
Authors	Seong Joon Oh, Max Augustin, Bernt Schiele, Mario Fritz
Abstract	Many deployed learned models are black boxes: given input, returns output. Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable. This work shows that such attributes of neural networks can be exposed from a sequence of queries. This has multiple implications. On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks – we show that the revealed internal information helps generate more effective adversarial examples against the black box model. On the other hand, this technique can be used for better protection of private content from automatic recognition models using adversarial examples. Our paper suggests that it is actually hard to draw a line between white box and black box models.
Tasks
Published	2017-11-06
URL	http://arxiv.org/abs/1711.01768v3
PDF	http://arxiv.org/pdf/1711.01768v3.pdf
PWC	https://paperswithcode.com/paper/towards-reverse-engineering-black-box-neural
Repo	https://github.com/markliou/model_distillation
Framework	tf

Multiple Source Domain Adaptation with Adversarial Training of Neural Networks


Title	Multiple Source Domain Adaptation with Adversarial Training of Neural Networks
Authors	Han Zhao, Shanghang Zhang, Guanhang Wu, João P. Costeira, José M. F. Moura, Geoffrey J. Gordon
Abstract	While domain adaptation has been actively researched in recent years, most theoretical results and algorithms focus on the single-source-single-target adaptation setting. Naive application of such algorithms on multiple source domain adaptation problem may lead to suboptimal solutions. As a step toward bridging the gap, we propose a new generalization bound for domain adaptation when there are multiple source domains with labeled instances and one target domain with unlabeled instances. Compared with existing bounds, the new bound does not require expert knowledge about the target distribution, nor the optimal combination rule for multisource domains. Interestingly, our theory also leads to an efficient learning strategy using adversarial neural networks: we show how to interpret it as learning feature representations that are invariant to the multiple domain shifts while still being discriminative for the learning task. To this end, we propose two models, both of which we call multisource domain adversarial networks (MDANs): the first model optimizes directly our bound, while the second model is a smoothed approximation of the first one, leading to a more data-efficient and task-adaptive model. The optimization tasks of both models are minimax saddle point problems that can be optimized by adversarial training. To demonstrate the effectiveness of MDANs, we conduct extensive experiments showing superior adaptation performance on three real-world datasets: sentiment analysis, digit classification, and vehicle counting.
Tasks	Domain Adaptation, Sentiment Analysis
Published	2017-05-26
URL	http://arxiv.org/abs/1705.09684v2
PDF	http://arxiv.org/pdf/1705.09684v2.pdf
PWC	https://paperswithcode.com/paper/multiple-source-domain-adaptation-with
Repo	https://github.com/daoyuan98/MSDA
Framework	tf

A deep generative model for gene expression profiles from single-cell RNA sequencing


Title	A deep generative model for gene expression profiles from single-cell RNA sequencing
Authors	Romain Lopez, Jeffrey Regier, Michael Cole, Michael Jordan, Nir Yosef
Abstract	We propose a probabilistic model for interpreting gene expression levels that are observed through single-cell RNA sequencing. In the model, each cell has a low-dimensional latent representation. Additional latent variables account for technical effects that may erroneously set some observations of gene expression levels to zero. Conditional distributions are specified by neural networks, giving the proposed model enough flexibility to fit the data well. We use variational inference and stochastic optimization to approximate the posterior distribution. The inference procedure scales to over one million cells, whereas competing algorithms do not. Even for smaller datasets, for several tasks, the proposed procedure outperforms state-of-the-art methods like ZIFA and ZINB-WaVE. We also extend our framework to account for batch effects and other confounding factors, and propose a Bayesian hypothesis test for differential expression that outperforms DESeq2.
Tasks	Stochastic Optimization
Published	2017-09-07
URL	http://arxiv.org/abs/1709.02082v4
PDF	http://arxiv.org/pdf/1709.02082v4.pdf
PWC	https://paperswithcode.com/paper/a-deep-generative-model-for-gene-expression
Repo	https://github.com/romain-lopez/scVI-reproducibility
Framework	tf

MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach


Title	MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach
Authors	Diego Moussallem, Ricardo Usbeck, Michael Röder, Axel-Cyrille Ngonga Ngomo
Abstract	Entity linking has recently been the subject of a significant body of research. Currently, the best performing approaches rely on trained mono-lingual models. Porting these approaches to other languages is consequently a difficult endeavor as it requires corresponding training data and retraining of the models. We address this drawback by presenting a novel multilingual, knowledge-based agnostic and deterministic approach to entity linking, dubbed MAG. MAG is based on a combination of context-based retrieval on structured knowledge bases and graph algorithms. We evaluate MAG on 23 data sets and in 7 languages. Our results show that the best approach trained on English datasets (PBOH) achieves a micro F-measure that is up to 4 times worse on datasets in other languages. MAG, on the other hand, achieves state-of-the-art performance on English datasets and reaches a micro F-measure that is up to 0.6 higher than that of PBOH on non-English languages.
Tasks	Entity Linking
Published	2017-07-17
URL	http://arxiv.org/abs/1707.05288v3
PDF	http://arxiv.org/pdf/1707.05288v3.pdf
PWC	https://paperswithcode.com/paper/mag-a-multilingual-knowledge-base-agnostic
Repo	https://github.com/AKSW/AGDISTIS
Framework	none

PACRR: A Position-Aware Neural IR Model for Relevance Matching


Title	PACRR: A Position-Aware Neural IR Model for Relevance Matching
Authors	Kai Hui, Andrew Yates, Klaus Berberich, Gerard de Melo
Abstract	In order to adopt deep learning for information retrieval, models are needed that can capture all relevant information required to assess the relevance of a document to a given user query. While previous works have successfully captured unigram term matches, how to fully employ position-dependent information such as proximity and term dependencies has been insufficiently explored. In this work, we propose a novel neural IR model named PACRR aiming at better modeling position-dependent interactions between a query and a document. Extensive experiments on six years’ TREC Web Track data confirm that the proposed model yields better results under multiple benchmarks.
Tasks	Ad-Hoc Information Retrieval, Information Retrieval
Published	2017-04-12
URL	http://arxiv.org/abs/1704.03940v3
PDF	http://arxiv.org/pdf/1704.03940v3.pdf
PWC	https://paperswithcode.com/paper/pacrr-a-position-aware-neural-ir-model-for
Repo	https://github.com/MatanRad/Neural-IR-Project
Framework	none

Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition


Title	Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Authors	Kensho Hara, Hirokatsu Kataoka, Yutaka Satoh
Abstract	Convolutional neural networks with spatio-temporal 3D kernels (3D CNNs) have an ability to directly extract spatio-temporal features from videos for action recognition. Although the 3D kernels tend to overfit because of a large number of their parameters, the 3D CNNs are greatly improved by using recent huge video databases. However, the architecture of 3D CNNs is relatively shallow against to the success of very deep neural networks in 2D-based CNNs, such as residual networks (ResNets). In this paper, we propose a 3D CNNs based on ResNets toward a better action representation. We describe the training procedure of our 3D ResNets in details. We experimentally evaluate the 3D ResNets on the ActivityNet and Kinetics datasets. The 3D ResNets trained on the Kinetics did not suffer from overfitting despite the large number of parameters of the model, and achieved better performance than relatively shallow networks, such as C3D. Our code and pretrained models (e.g. Kinetics and ActivityNet) are publicly available at https://github.com/kenshohara/3D-ResNets.
Tasks	Action Recognition In Videos, Hand-Gesture Recognition, Temporal Action Localization
Published	2017-08-25
URL	http://arxiv.org/abs/1708.07632v1
PDF	http://arxiv.org/pdf/1708.07632v1.pdf
PWC	https://paperswithcode.com/paper/learning-spatio-temporal-features-with-3d
Repo	https://github.com/kenshohara/3D-ResNets
Framework	torch

Learning and Visualizing Localized Geometric Features Using 3D-CNN: An Application to Manufacturability Analysis of Drilled Holes


Title	Learning and Visualizing Localized Geometric Features Using 3D-CNN: An Application to Manufacturability Analysis of Drilled Holes
Authors	Sambit Ghadai, Aditya Balu, Adarsh Krishnamurthy, Soumik Sarkar
Abstract	3D Convolutional Neural Networks (3D-CNN) have been used for object recognition based on the voxelized shape of an object. However, interpreting the decision making process of these 3D-CNNs is still an infeasible task. In this paper, we present a unique 3D-CNN based Gradient-weighted Class Activation Mapping method (3D-GradCAM) for visual explanations of the distinct local geometric features of interest within an object. To enable efficient learning of 3D geometries, we augment the voxel data with surface normals of the object boundary. We then train a 3D-CNN with this augmented data and identify the local features critical for decision-making using 3D GradCAM. An application of this feature identification framework is to recognize difficult-to-manufacture drilled hole features in a complex CAD geometry. The framework can be extended to identify difficult-to-manufacture features at multiple spatial scales leading to a real-time design for manufacturability decision support system.
Tasks	3D Object Recognition, Decision Making, Object Recognition
Published	2017-11-13
URL	http://arxiv.org/abs/1711.04851v3
PDF	http://arxiv.org/pdf/1711.04851v3.pdf
PWC	https://paperswithcode.com/paper/learning-and-visualizing-localized-geometric
Repo	https://github.com/idealab-isu/GPView
Framework	none

Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning


Title	Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
Authors	NhatHai Phan, Xintao Wu, Han Hu, Dejing Dou
Abstract	In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks. To achieve this, we figure out a way to perturb affine transformations of neurons, and loss functions used in deep neural networks. In addition, our mechanism intentionally adds “more noise” into features which are “less relevant” to the model output, and vice-versa. Our theoretical analysis further derives the sensitivities and error bounds of our mechanism. Rigorous experiments conducted on MNIST and CIFAR-10 datasets show that our mechanism is highly effective and outperforms existing solutions.
Tasks
Published	2017-09-18
URL	http://arxiv.org/abs/1709.05750v2
PDF	http://arxiv.org/pdf/1709.05750v2.pdf
PWC	https://paperswithcode.com/paper/adaptive-laplace-mechanism-differential
Repo	https://github.com/haiphanNJIT/PrivateDeepLearning
Framework	tf

Improving Hypernymy Extraction with Distributional Semantic Classes


Title	Improving Hypernymy Extraction with Distributional Semantic Classes
Authors	Alexander Panchenko, Dmitry Ustalov, Stefano Faralli, Simone P. Ponzetto, Chris Biemann
Abstract	In this paper, we show how distributionally-induced semantic classes can be helpful for extracting hypernyms. We present methods for inducing sense-aware semantic classes using distributional semantics and using these induced semantic classes for filtering noisy hypernymy relations. Denoising of hypernyms is performed by labeling each semantic class with its hypernyms. On the one hand, this allows us to filter out wrong extractions using the global structure of distributionally similar senses. On the other hand, we infer missing hypernyms via label propagation to cluster terms. We conduct a large-scale crowdsourcing study showing that processing of automatically extracted hypernyms using our approach improves the quality of the hypernymy extraction in terms of both precision and recall. Furthermore, we show the utility of our method in the domain taxonomy induction task, achieving the state-of-the-art results on a SemEval’16 task on taxonomy induction.
Tasks	Denoising
Published	2017-11-08
URL	http://arxiv.org/abs/1711.02918v2
PDF	http://arxiv.org/pdf/1711.02918v2.pdf
PWC	https://paperswithcode.com/paper/improving-hypernymy-extraction-with
Repo	https://github.com/uhh-lt/mangosteen
Framework	none

Explaining Anomalies in Groups with Characterizing Subspace Rules


Title	Explaining Anomalies in Groups with Characterizing Subspace Rules
Authors	Meghanath Macha, Leman Akoglu
Abstract	Anomaly detection has numerous applications and has been studied vastly. We consider a complementary problem that has a much sparser literature: anomaly description. Interpretation of anomalies is crucial for practitioners for sense-making, troubleshooting, and planning actions. To this end, we present a new approach called x-PACS (for eXplaining Patterns of Anomalies with Characterizing Subspaces), which “reverse-engineers” the known anomalies by identifying (1) the groups (or patterns) that they form, and (2) the characterizing subspace and feature rules that separate each anomalous pattern from normal instances. Explaining anomalies in groups not only saves analyst time and gives insight into various types of anomalies, but also draws attention to potentially critical, repeating anomalies. In developing x-PACS, we first construct a desiderata for the anomaly description problem. From a descriptive data mining perspective, our method exhibits five desired properties in our desiderata. Namely, it can unearth anomalous patterns (i) of multiple different types, (ii) hidden in arbitrary subspaces of a high dimensional space, (iii) interpretable by the analysts, (iv) different from normal patterns of the data, and finally (v) succinct, providing the shortest data description. Furthermore, x-PACS is highly parallelizable and scales linearly in terms of data size. No existing work on anomaly description satisfies all of these properties simultaneously. While not our primary goal, the anomalous patterns we find serve as interpretable “signatures” and can be used for detection. We show the effectiveness of x-PACS in explanation as well as detection on real-world datasets as compared to state-of-the-art.
Tasks	Anomaly Detection
Published	2017-08-20
URL	http://arxiv.org/abs/1708.05929v4
PDF	http://arxiv.org/pdf/1708.05929v4.pdf
PWC	https://paperswithcode.com/paper/explaining-anomalies-in-groups-with
Repo	https://github.com/meghanathmacha/xPACS
Framework	none


Title	A Fast and Scalable Joint Estimator for Learning Multiple Related Sparse Gaussian Graphical Models
Authors	Beilun Wang, Ji Gao, Yanjun Qi
Abstract	Estimating multiple sparse Gaussian Graphical Models (sGGMs) jointly for many related tasks (large $K$) under a high-dimensional (large $p$) situation is an important task. Most previous studies for the joint estimation of multiple sGGMs rely on penalized log-likelihood estimators that involve expensive and difficult non-smooth optimizations. We propose a novel approach, FASJEM for \underline{fa}st and \underline{s}calable \underline{j}oint structure-\underline{e}stimation of \underline{m}ultiple sGGMs at a large scale. As the first study of joint sGGM using the Elementary Estimator framework, our work has three major contributions: (1) We solve FASJEM through an entry-wise manner which is parallelizable. (2) We choose a proximal algorithm to optimize FASJEM. This improves the computational efficiency from $O(Kp^3)$ to $O(Kp^2)$ and reduces the memory requirement from $O(Kp^2)$ to $O(K)$. (3) We theoretically prove that FASJEM achieves a consistent estimation with a convergence rate of $O(\log(Kp)/n_{tot})$. On several synthetic and four real-world datasets, FASJEM shows significant improvements over baselines on accuracy, computational complexity, and memory costs.
Tasks
Published	2017-02-09
URL	http://arxiv.org/abs/1702.02715v3
PDF	http://arxiv.org/pdf/1702.02715v3.pdf
PWC	https://paperswithcode.com/paper/a-fast-and-scalable-joint-estimator-for
Repo	https://github.com/QData/FASJEM
Framework	torch

Hierarchical Surface Prediction for 3D Object Reconstruction


Title	Hierarchical Surface Prediction for 3D Object Reconstruction
Authors	Christian Häne, Shubham Tulsiani, Jitendra Malik
Abstract	Recently, Convolutional Neural Networks have shown promising results for 3D geometry prediction. They can make predictions from very little input data such as a single color image. A major limitation of such approaches is that they only predict a coarse resolution voxel grid, which does not capture the surface of the objects well. We propose a general framework, called hierarchical surface prediction (HSP), which facilitates prediction of high resolution voxel grids. The main insight is that it is sufficient to predict high resolution voxels around the predicted surfaces. The exterior and interior of the objects can be represented with coarse resolution voxels. Our approach is not dependent on a specific input type. We show results for geometry prediction from color images, depth images and shape completion from partial voxel grids. Our analysis shows that our high resolution predictions are more accurate than low resolution predictions.
Tasks	3D Object Reconstruction, Object Reconstruction
Published	2017-04-03
URL	http://arxiv.org/abs/1704.00710v2
PDF	http://arxiv.org/pdf/1704.00710v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-surface-prediction-for-3d-object
Repo	https://github.com/vnoves/aectech2019-sketchto3d-backend
Framework	none

Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients


Title	Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
Authors	Andrew Slavin Ross, Finale Doshi-Velez
Abstract	Deep neural networks have proven remarkably effective at solving many classification problems, but have been criticized recently for two major weaknesses: the reasons behind their predictions are uninterpretable, and the predictions themselves can often be fooled by small adversarial perturbations. These problems pose major obstacles for the adoption of neural networks in domains that require security or transparency. In this work, we evaluate the effectiveness of defenses that differentiably penalize the degree to which small changes in inputs can alter model predictions. Across multiple attacks, architectures, defenses, and datasets, we find that neural networks trained with this input gradient regularization exhibit robustness to transferred adversarial examples generated to fool all of the other models. We also find that adversarial examples generated to fool gradient-regularized models fool all other models equally well, and actually lead to more “legitimate,” interpretable misclassifications as rated by people (which we confirm in a human subject experiment). Finally, we demonstrate that regularizing input gradients makes them more naturally interpretable as rationales for model predictions. We conclude by discussing this relationship between interpretability and robustness in deep neural networks.
Tasks
Published	2017-11-26
URL	http://arxiv.org/abs/1711.09404v1
PDF	http://arxiv.org/pdf/1711.09404v1.pdf
PWC	https://paperswithcode.com/paper/improving-the-adversarial-robustness-and
Repo	https://github.com/dtak/adversarial-robustness-public
Framework	tf

STWalk: Learning Trajectory Representations in Temporal Graphs


Title	STWalk: Learning Trajectory Representations in Temporal Graphs
Authors	Supriya Pandhre, Himangi Mittal, Manish Gupta, Vineeth N Balasubramanian
Abstract	Analyzing the temporal behavior of nodes in time-varying graphs is useful for many applications such as targeted advertising, community evolution and outlier detection. In this paper, we present a novel approach, STWalk, for learning trajectory representations of nodes in temporal graphs. The proposed framework makes use of structural properties of graphs at current and previous time-steps to learn effective node trajectory representations. STWalk performs random walks on a graph at a given time step (called space-walk) as well as on graphs from past time-steps (called time-walk) to capture the spatio-temporal behavior of nodes. We propose two variants of STWalk to learn trajectory representations. In one algorithm, we perform space-walk and time-walk as part of a single step. In the other variant, we perform space-walk and time-walk separately and combine the learned representations to get the final trajectory embedding. Extensive experiments on three real-world temporal graph datasets validate the effectiveness of the learned representations when compared to three baseline methods. We also show the goodness of the learned trajectory embeddings for change point detection, as well as demonstrate that arithmetic operations on these trajectory representations yield interesting and interpretable results.
Tasks	Change Point Detection, Outlier Detection
Published	2017-11-11
URL	http://arxiv.org/abs/1711.04150v1
PDF	http://arxiv.org/pdf/1711.04150v1.pdf
PWC	https://paperswithcode.com/paper/stwalk-learning-trajectory-representations-in
Repo	https://github.com/supriya-pandhre/STWalk
Framework	none

11K Hands: Gender recognition and biometric identification using a large dataset of hand images


Title	11K Hands: Gender recognition and biometric identification using a large dataset of hand images
Authors	Mahmoud Afifi
Abstract	The human hand possesses distinctive features which can reveal gender information. In addition, the hand is considered one of the primary biometric traits used to identify a person. In this work, we propose a large dataset of human hand images (dorsal and palmar sides) with detailed ground-truth information for gender recognition and biometric identification. Using this dataset, a convolutional neural network (CNN) can be trained effectively for the gender recognition task. Based on this, we design a two-stream CNN to tackle the gender recognition problem. This trained model is then used as a feature extractor to feed a set of support vector machine classifiers for the biometric identification task. We show that the dorsal side of hand images, captured by a regular digital camera, convey effective distinctive features similar to, if not better, those available in the palmar hand images. To facilitate access to the proposed dataset and replication of our experiments, the dataset, trained CNN models, and Matlab source code are available at (https://goo.gl/rQJndd).
Tasks
Published	2017-11-12
URL	http://arxiv.org/abs/1711.04322v9
PDF	http://arxiv.org/pdf/1711.04322v9.pdf
PWC	https://paperswithcode.com/paper/11k-hands-gender-recognition-and-biometric
Repo	https://github.com/mahmoudnafifi/11K-Hands
Framework	tf