July 30, 2019

3263 words 16 mins read

Paper Group AWR 56

Fully Automated Fact Checking Using External Sources. Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning. Superhuman Accuracy on the SNEMI3D Connectomics Challenge. A General Framework for Adversarial Examples with Objectives. Rocket Launching: A Universal and Efficient Framework for Training Well-performing Lig …

Fully Automated Fact Checking Using External Sources


Title	Fully Automated Fact Checking Using External Sources
Authors	Georgi Karadzhov, Preslav Nakov, Lluis Marquez, Alberto Barron-Cedeno, Ivan Koychev
Abstract	Given the constantly growing proliferation of false claims online in recent years, there has been also a growing research interest in automatically distinguishing false rumors from factually true claims. Here, we propose a general-purpose framework for fully-automatic fact checking using external sources, tapping the potential of the entire Web as a knowledge source to confirm or reject a claim. Our framework uses a deep neural network with LSTM text encoding to combine semantic kernels with task-specific embeddings that encode a claim together with pieces of potentially-relevant text fragments from the Web, taking the source reliability into account. The evaluation results show good performance on two different tasks and datasets: (i) rumor detection and (ii) fact checking of the answers to a question in community question answering forums.
Tasks	Community Question Answering, Question Answering
Published	2017-10-01
URL	http://arxiv.org/abs/1710.00341v1
PDF	http://arxiv.org/pdf/1710.00341v1.pdf
PWC	https://paperswithcode.com/paper/fully-automated-fact-checking-using-external
Repo	https://github.com/gkaradzhov/FactcheckingRANLP
Framework	none

Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning


Title	Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
Authors	Rui Luo, Jianhong Wang, Yaodong Yang, Zhanxing Zhu, Jun Wang
Abstract	We propose a new sampling method, the thermostat-assisted continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large datasets and multimodal distributions. It simulates the Nos'e-Hoover dynamics of a continuously-tempered Hamiltonian system built on the distribution of interest. A significant advantage of this method is that it is not only able to efficiently draw representative i.i.d. samples when the distribution contains multiple isolated modes, but capable of adaptively neutralising the noise arising from mini-batches and maintaining accurate sampling. While the properties of this method have been studied using synthetic distributions, experiments on three real datasets also demonstrated the gain of performance over several strong baselines with various types of neural networks plunged in.
Tasks
Published	2017-11-30
URL	http://arxiv.org/abs/1711.11511v5
PDF	http://arxiv.org/pdf/1711.11511v5.pdf
PWC	https://paperswithcode.com/paper/thermostat-assisted-continuously-tempered
Repo	https://github.com/hsvgbkhgbv/TACTHMC
Framework	pytorch

Superhuman Accuracy on the SNEMI3D Connectomics Challenge


Title	Superhuman Accuracy on the SNEMI3D Connectomics Challenge
Authors	Kisuk Lee, Jonathan Zung, Peter Li, Viren Jain, H. Sebastian Seung
Abstract	For the past decade, convolutional networks have been used for 3D reconstruction of neurons from electron microscopic (EM) brain images. Recent years have seen great improvements in accuracy, as evidenced by submissions to the SNEMI3D benchmark challenge. Here we report the first submission to surpass the estimate of human accuracy provided by the SNEMI3D leaderboard. A variant of 3D U-Net is trained on a primary task of predicting affinities between nearest neighbor voxels, and an auxiliary task of predicting long-range affinities. The training data is augmented by simulated image defects. The nearest neighbor affinities are used to create an oversegmentation, and then supervoxels are greedily agglomerated based on mean affinity. The resulting SNEMI3D score exceeds the estimate of human accuracy by a large margin. While one should be cautious about extrapolating from the SNEMI3D benchmark to real-world accuracy of large-scale neural circuit reconstruction, our result inspires optimism that the goal of full automation may be realizable in the future.
Tasks	3D Reconstruction, Electron Microscopy Image Segmentation
Published	2017-05-31
URL	http://arxiv.org/abs/1706.00120v1
PDF	http://arxiv.org/pdf/1706.00120v1.pdf
PWC	https://paperswithcode.com/paper/superhuman-accuracy-on-the-snemi3d
Repo	https://github.com/seung-lab/DeepEM
Framework	none

A General Framework for Adversarial Examples with Objectives


Title	A General Framework for Adversarial Examples with Objectives
Authors	Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter
Abstract	Images perturbed subtly to be misclassified by neural networks, called adversarial examples, have emerged as a technically deep challenge and an important concern for several application domains. Most research on adversarial examples takes as its only constraint that the perturbed images are similar to the originals. However, real-world application of these ideas often requires the examples to satisfy additional objectives, which are typically enforced through custom modifications of the perturbation process. In this paper, we propose adversarial generative nets (AGNs), a general methodology to train a generator neural network to emit adversarial examples satisfying desired objectives. We demonstrate the ability of AGNs to accommodate a wide range of objectives, including imprecise ones difficult to model, in two application domains. In particular, we demonstrate physical adversarial examples—eyeglass frames designed to fool face recognition—with better robustness, inconspicuousness, and scalability than previous approaches, as well as a new attack to fool a handwritten-digit classifier.
Tasks	Face Recognition
Published	2017-12-31
URL	http://arxiv.org/abs/1801.00349v2
PDF	http://arxiv.org/pdf/1801.00349v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-generative-nets-neural-network
Repo	https://github.com/drewbarot/Un-CNN
Framework	tf

Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net


Title	Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net
Authors	Guorui Zhou, Ying Fan, Runpeng Cui, Weijie Bian, Xiaoqiang Zhu, Kun Gai
Abstract	Models applied on real time response task, like click-through rate (CTR) prediction model, require high accuracy and rigorous response time. Therefore, top-performing deep models of high depth and complexity are not well suited for these applications with the limitations on the inference time. In order to further improve the neural networks’ performance given the time and computational limitations, we propose an approach that exploits a cumbersome net to help train the lightweight net for prediction. We dub the whole process rocket launching, where the cumbersome booster net is used to guide the learning of the target light net throughout the whole training process. We analyze different loss functions aiming at pushing the light net to behave similarly to the booster net, and adopt the loss with best performance in our experiments. We use one technique called gradient block to improve the performance of the light net and booster net further. Experiments on benchmark datasets and real-life industrial advertisement data present that our light model can get performance only previously achievable with more complex models.
Tasks	Click-Through Rate Prediction
Published	2017-08-14
URL	http://arxiv.org/abs/1708.04106v3
PDF	http://arxiv.org/pdf/1708.04106v3.pdf
PWC	https://paperswithcode.com/paper/rocket-launching-a-universal-and-efficient
Repo	https://github.com/zhougr1993/Rocket-Launching
Framework	pytorch

Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights


Title	Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
Authors	Aojun Zhou, Anbang Yao, Yiwen Guo, Lin Xu, Yurong Chen
Abstract	This paper presents incremental network quantization (INQ), a novel method, targeting to efficiently convert any pre-trained full-precision convolutional neural network (CNN) model into a low-precision version whose weights are constrained to be either powers of two or zero. Unlike existing methods which are struggled in noticeable accuracy loss, our INQ has the potential to resolve this issue, as benefiting from two innovations. On one hand, we introduce three interdependent operations, namely weight partition, group-wise quantization and re-training. A well-proven measure is employed to divide the weights in each layer of a pre-trained CNN model into two disjoint groups. The weights in the first group are responsible to form a low-precision base, thus they are quantized by a variable-length encoding method. The weights in the other group are responsible to compensate for the accuracy loss from the quantization, thus they are the ones to be re-trained. On the other hand, these three operations are repeated on the latest re-trained group in an iterative manner until all the weights are converted into low-precision ones, acting as an incremental network quantization and accuracy enhancement procedure. Extensive experiments on the ImageNet classification task using almost all known deep CNN architectures including AlexNet, VGG-16, GoogleNet and ResNets well testify the efficacy of the proposed method. Specifically, at 5-bit quantization, our models have improved accuracy than the 32-bit floating-point references. Taking ResNet-18 as an example, we further show that our quantized models with 4-bit, 3-bit and 2-bit ternary weights have improved or very similar accuracy against its 32-bit floating-point baseline. Besides, impressive results with the combination of network pruning and INQ are also reported. The code is available at https://github.com/Zhouaojun/Incremental-Network-Quantization.
Tasks	Quantization
Published	2017-02-10
URL	http://arxiv.org/abs/1702.03044v2
PDF	http://arxiv.org/pdf/1702.03044v2.pdf
PWC	https://paperswithcode.com/paper/incremental-network-quantization-towards
Repo	https://github.com/Zhouaojun/Incremental-Network-Quantization
Framework	none

SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation


Title	SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation
Authors	Weiyue Wang, Ronald Yu, Qiangui Huang, Ulrich Neumann
Abstract	We introduce Similarity Group Proposal Network (SGPN), a simple and intuitive deep learning framework for 3D object instance segmentation on point clouds. SGPN uses a single network to predict point grouping proposals and a corresponding semantic class for each proposal, from which we can directly extract instance segmentation results. Important to the effectiveness of SGPN is its novel representation of 3D instance segmentation results in the form of a similarity matrix that indicates the similarity between each pair of points in embedded feature space, thus producing an accurate grouping proposal for each point. To the best of our knowledge, SGPN is the first framework to learn 3D instance-aware semantic segmentation on point clouds. Experimental results on various 3D scenes show the effectiveness of our method on 3D instance segmentation, and we also evaluate the capability of SGPN to improve 3D object detection and semantic segmentation results. We also demonstrate its flexibility by seamlessly incorporating 2D CNN features into the framework to boost performance.
Tasks	3D Instance Segmentation, 3D Object Detection, 3D Semantic Instance Segmentation, Instance Segmentation, Object Detection, Scene Segmentation, Semantic Segmentation
Published	2017-11-23
URL	https://arxiv.org/abs/1711.08588v2
PDF	https://arxiv.org/pdf/1711.08588v2.pdf
PWC	https://paperswithcode.com/paper/sgpn-similarity-group-proposal-network-for-3d
Repo	https://github.com/laughtervv/SGPN
Framework	tf

6-DoF Object Pose from Semantic Keypoints


Title	6-DoF Object Pose from Semantic Keypoints
Authors	Georgios Pavlakos, Xiaowei Zhou, Aaron Chan, Konstantinos G. Derpanis, Kostas Daniilidis
Abstract	This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.
Tasks	Keypoint Detection, Pose Estimation
Published	2017-03-14
URL	http://arxiv.org/abs/1703.04670v1
PDF	http://arxiv.org/pdf/1703.04670v1.pdf
PWC	https://paperswithcode.com/paper/6-dof-object-pose-from-semantic-keypoints
Repo	https://github.com/geopavlakos/object3d
Framework	torch

Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks


Title	Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Authors	Nils Reimers, Iryna Gurevych
Abstract	Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance. However, little is published which parameters and design choices should be evaluated or selected making the correct hyperparameter optimization often a “black art that requires expert experiences” (Snoek et al., 2012). In this paper, we evaluate the importance of different network design choices and hyperparameters for five common linguistic sequence tagging tasks (POS, Chunking, NER, Entity Recognition, and Event Detection). We evaluated over 50.000 different setups and found, that some parameters, like the pre-trained word embeddings or the last layer of the network, have a large impact on the performance, while other parameters, for example the number of LSTM layers or the number of recurrent units, are of minor importance. We give a recommendation on a configuration that performs well among different tasks.
Tasks	Chunking, Hyperparameter Optimization, Word Embeddings
Published	2017-07-21
URL	http://arxiv.org/abs/1707.06799v2
PDF	http://arxiv.org/pdf/1707.06799v2.pdf
PWC	https://paperswithcode.com/paper/optimal-hyperparameters-for-deep-lstm
Repo	https://github.com/jiesutd/NCRFpp
Framework	pytorch

RelNN: A Deep Neural Model for Relational Learning


Title	RelNN: A Deep Neural Model for Relational Learning
Authors	Seyed Mehran Kazemi, David Poole
Abstract	Statistical relational AI (StarAI) aims at reasoning and learning in noisy domains described in terms of objects and relationships by combining probability with first-order logic. With huge advances in deep learning in the current years, combining deep networks with first-order logic has been the focus of several recent studies. Many of the existing attempts, however, only focus on relations and ignore object properties. The attempts that do consider object properties are limited in terms of modelling power or scalability. In this paper, we develop relational neural networks (RelNNs) by adding hidden layers to relational logistic regression (the relational counterpart of logistic regression). We learn latent properties for objects both directly and through general rules. Back-propagation is used for training these models. A modular, layer-wise architecture facilitates utilizing the techniques developed within deep learning community to our architecture. Initial experiments on eight tasks over three real-world datasets show that RelNNs are promising models for relational learning.
Tasks	Relational Reasoning
Published	2017-12-07
URL	http://arxiv.org/abs/1712.02831v1
PDF	http://arxiv.org/pdf/1712.02831v1.pdf
PWC	https://paperswithcode.com/paper/relnn-a-deep-neural-model-for-relational
Repo	https://github.com/Mehran-k/RelNN
Framework	none

Block-Simultaneous Direction Method of Multipliers: A proximal primal-dual splitting algorithm for nonconvex problems with multiple constraints


Title	Block-Simultaneous Direction Method of Multipliers: A proximal primal-dual splitting algorithm for nonconvex problems with multiple constraints
Authors	Fred Moolekamp, Peter Melchior
Abstract	We introduce a generalization of the linearized Alternating Direction Method of Multipliers to optimize a real-valued function $f$ of multiple arguments with potentially multiple constraints $g_\circ$ on each of them. The function $f$ may be nonconvex as long as it is convex in every argument, while the constraints $g_\circ$ need to be convex but not smooth. If $f$ is smooth, the proposed Block-Simultaneous Direction Method of Multipliers (bSDMM) can be interpreted as a proximal analog to inexact coordinate descent methods under constraints. Unlike alternative approaches for joint solvers of multiple-constraint problems, we do not require linear operators $L$ of a constraint function $g(L\ \cdot)$ to be invertible or linked between each other. bSDMM is well-suited for a range of optimization problems, in particular for data analysis, where $f$ is the likelihood function of a model and $L$ could be a transformation matrix describing e.g. finite differences or basis transforms. We apply bSDMM to the Non-negative Matrix Factorization task of a hyperspectral unmixing problem and demonstrate convergence and effectiveness of multiple constraints on both matrix factors. The algorithms are implemented in python and released as an open-source package.
Tasks	Hyperspectral Unmixing
Published	2017-08-30
URL	http://arxiv.org/abs/1708.09066v1
PDF	http://arxiv.org/pdf/1708.09066v1.pdf
PWC	https://paperswithcode.com/paper/block-simultaneous-direction-method-of
Repo	https://github.com/pmelchior/proxmin
Framework	none

A Scalable Deep Neural Network Architecture for Multi-Building and Multi-Floor Indoor Localization Based on Wi-Fi Fingerprinting


Title	A Scalable Deep Neural Network Architecture for Multi-Building and Multi-Floor Indoor Localization Based on Wi-Fi Fingerprinting
Authors	Kyeong Soo Kim, Sanghyuk Lee, Kaizhu Huang
Abstract	One of the key technologies for future large-scale location-aware services covering a complex of multi-story buildings — e.g., a big shopping mall and a university campus — is a scalable indoor localization technique. In this paper, we report the current status of our investigation on the use of deep neural networks (DNNs) for scalable building/floor classification and floor-level position estimation based on Wi-Fi fingerprinting. Exploiting the hierarchical nature of the building/floor estimation and floor-level coordinates estimation of a location, we propose a new DNN architecture consisting of a stacked autoencoder for the reduction of feature space dimension and a feed-forward classifier for multi-label classification of building/floor/location, on which the multi-building and multi-floor indoor localization system based on Wi-Fi fingerprinting is built. Experimental results for the performance of building/floor estimation and floor-level coordinates estimation of a given location demonstrate the feasibility of the proposed DNN-based indoor localization system, which can provide near state-of-the-art performance using a single DNN, for the implementation with lower complexity and energy consumption at mobile devices.
Tasks	Multi-Label Classification
Published	2017-12-06
URL	http://arxiv.org/abs/1712.01990v1
PDF	http://arxiv.org/pdf/1712.01990v1.pdf
PWC	https://paperswithcode.com/paper/a-scalable-deep-neural-network-architecture
Repo	https://github.com/vohoaiviet/indoor_localization
Framework	none

Mapping to Declarative Knowledge for Word Problem Solving


Title	Mapping to Declarative Knowledge for Word Problem Solving
Authors	Subhro Roy, Dan Roth
Abstract	Math word problems form a natural abstraction to a range of quantitative reasoning problems, such as understanding financial news, sports results, and casualties of war. Solving such problems requires the understanding of several mathematical concepts such as dimensional analysis, subset relationships, etc. In this paper, we develop declarative rules which govern the translation of natural language description of these concepts to math expressions. We then present a framework for incorporating such declarative knowledge into word problem solving. Our method learns to map arithmetic word problem text to math expressions, by learning to select the relevant declarative knowledge for each operation of the solution expression. This provides a way to handle multiple concepts in the same problem while, at the same time, support interpretability of the answer expression. Our method models the mapping to declarative knowledge as a latent variable, thus removing the need for expensive annotations. Experimental evaluation suggests that our domain knowledge based solver outperforms all other systems, and that it generalizes better in the realistic case where the training data it is exposed to is biased in a different way than the test data.
Tasks
Published	2017-12-26
URL	http://arxiv.org/abs/1712.09391v1
PDF	http://arxiv.org/pdf/1712.09391v1.pdf
PWC	https://paperswithcode.com/paper/mapping-to-declarative-knowledge-for-word
Repo	https://github.com/CogComp/arithmetic
Framework	none

Trainable back-propagated functional transfer matrices


Title	Trainable back-propagated functional transfer matrices
Authors	Cheng-Hao Cai, Yanyan Xu, Dengfeng Ke, Kaile Su, Jing Sun
Abstract	Connections between nodes of fully connected neural networks are usually represented by weight matrices. In this article, functional transfer matrices are introduced as alternatives to the weight matrices: Instead of using real weights, a functional transfer matrix uses real functions with trainable parameters to represent connections between nodes. Multiple functional transfer matrices are then stacked together with bias vectors and activations to form deep functional transfer neural networks. These neural networks can be trained within the framework of back-propagation, based on a revision of the delta rules and the error transmission rule for functional connections. In experiments, it is demonstrated that the revised rules can be used to train a range of functional connections: 20 different functions are applied to neural networks with up to 10 hidden layers, and most of them gain high test accuracies on the MNIST database. It is also demonstrated that a functional transfer matrix with a memory function can roughly memorise a non-cyclical sequence of 400 digits.
Tasks
Published	2017-10-28
URL	http://arxiv.org/abs/1710.10403v1
PDF	http://arxiv.org/pdf/1710.10403v1.pdf
PWC	https://paperswithcode.com/paper/trainable-back-propagated-functional-transfer
Repo	https://github.com/cchrewrite/Functional-Transfer-Neural-Networks
Framework	none

DSOD: Learning Deeply Supervised Object Detectors from Scratch


Title	DSOD: Learning Deeply Supervised Object Detectors from Scratch
Authors	Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, Xiangyang Xue
Abstract	We present Deeply Supervised Object Detector (DSOD), a framework that can learn object detectors from scratch. State-of-the-art object objectors rely heavily on the off-the-shelf networks pre-trained on large-scale classification datasets like ImageNet, which incurs learning bias due to the difference on both the loss functions and the category distributions between classification and detection tasks. Model fine-tuning for the detection task could alleviate this bias to some extent but not fundamentally. Besides, transferring pre-trained models from classification to detection between discrepant domains is even more difficult (e.g. RGB to depth images). A better solution to tackle these two critical problems is to train object detectors from scratch, which motivates our proposed DSOD. Previous efforts in this direction mostly failed due to much more complicated loss functions and limited training data in object detection. In DSOD, we contribute a set of design principles for training object detectors from scratch. One of the key findings is that deep supervision, enabled by dense layer-wise connections, plays a critical role in learning a good detector. Combining with several other principles, we develop DSOD following the single-shot detection (SSD) framework. Experiments on PASCAL VOC 2007, 2012 and MS COCO datasets demonstrate that DSOD can achieve better results than the state-of-the-art solutions with much more compact models. For instance, DSOD outperforms SSD on all three benchmarks with real-time detection speed, while requires only 1/2 parameters to SSD and 1/10 parameters to Faster RCNN. Our code and models are available at: https://github.com/szq0214/DSOD .
Tasks	Object Detection
Published	2017-08-03
URL	http://arxiv.org/abs/1708.01241v2
PDF	http://arxiv.org/pdf/1708.01241v2.pdf
PWC	https://paperswithcode.com/paper/dsod-learning-deeply-supervised-object
Repo	https://github.com/szq0214/GRP-DSOD
Framework	pytorch