February 2, 2020

3093 words 15 mins read

Paper Group AWR 51

Bayesian estimation of the latent dimension and communities in stochastic blockmodels. Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations. Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach. RRPN: Radar Regi …

Bayesian estimation of the latent dimension and communities in stochastic blockmodels


Title	Bayesian estimation of the latent dimension and communities in stochastic blockmodels
Authors	Francesco Sanna Passino, Nicholas A. Heard
Abstract	Spectral embedding of adjacency or Laplacian matrices of undirected graphs is a common technique for representing a network in a lower dimensional latent space, with optimal theoretical guarantees. The embedding can be used to estimate the community structure of the network, with strong consistency results in the stochastic blockmodel framework. One of the main practical limitations of standard algorithms for community detection from spectral embeddings is that the number of communities and the latent dimension of the embedding must be specified in advance. In this article, a novel Bayesian model for simultaneous and automatic selection of the appropriate dimension of the latent space and the number of blocks is proposed. Extensions to directed and bipartite graphs are discussed. The model is tested on simulated and real world network data, showing promising performance for recovering latent community structure.
Tasks	Community Detection
Published	2019-04-06
URL	http://arxiv.org/abs/1904.05333v2
PDF	http://arxiv.org/pdf/1904.05333v2.pdf
PWC	https://paperswithcode.com/paper/bayesian-estimation-of-the-latent-dimension
Repo	https://github.com/fraspass/sbm
Framework	none

Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations


Title	Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations
Authors	Vishnu Suresh Lokhande, Songwong Tasneeyapant, Abhay Venkatesh, Sathya N. Ravi, Vikas Singh
Abstract	Rectified Linear Units (ReLUs) are among the most widely used activation function in a broad variety of tasks in vision. Recent theoretical results suggest that despite their excellent practical performance, in various cases, a substitution with basis expansions (e.g., polynomials) can yield significant benefits from both the optimization and generalization perspective. Unfortunately, the existing results remain limited to networks with a couple of layers, and the practical viability of these results is not yet known. Motivated by some of these results, we explore the use of Hermite polynomial expansions as a substitute for ReLUs in deep networks. While our experiments with supervised learning do not provide a clear verdict, we find that this strategy offers considerable benefits in semi-supervised learning (SSL) / transductive learning settings. We carefully develop this idea and show how the use of Hermite polynomials based activations can yield improvements in pseudo-label accuracies and sizable financial savings (due to concurrent runtime benefits). Further, we show via theoretical analysis, that the networks (with Hermite activations) offer robustness to noise and other attractive mathematical properties.
Tasks
Published	2019-09-12
URL	https://arxiv.org/abs/1909.05479v2
PDF	https://arxiv.org/pdf/1909.05479v2.pdf
PWC	https://paperswithcode.com/paper/generating-accurate-pseudo-labels-via-hermite
Repo	https://github.com/lokhande-vishnu/DeepHermites
Framework	none

Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach


Title	Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach
Authors	Zhenyu Wu, Karthik Suresh, Priya Narayanan, Hongyu Xu, Heesung Kwon, Zhangyang Wang
Abstract	Object detection from images captured by Unmanned Aerial Vehicles (UAVs) is becoming increasingly useful. Despite the great success of the generic object detection methods trained on ground-to-ground images, a huge performance drop is observed when they are directly applied to images captured by UAVs. The unsatisfactory performance is owing to many UAV-specific nuisances, such as varying flying altitudes, adverse weather conditions, dynamically changing viewing angles, etc. Those nuisances constitute a large number of fine-grained domains, across which the detection model has to stay robust. Fortunately, UAVs will record meta-data that depict those varying attributes, which are either freely available along with the UAV images, or can be easily obtained. We propose to utilize those free meta-data in conjunction with associated UAV images to learn domain-robust features via an adversarial training framework dubbed Nuisance Disentangled Feature Transform (NDFT), for the specific challenging problem of object detection in UAV images, achieving a substantial gain in robustness to those nuisances. We demonstrate the effectiveness of our proposed algorithm, by showing state-of-the-art performance (single model) on two existing UAV-based object detection benchmarks. The code is available at https://github.com/TAMU-VITA/UAV-NDFT.
Tasks	Object Detection, Robust Object Detection
Published	2019-08-11
URL	https://arxiv.org/abs/1908.03856v1
PDF	https://arxiv.org/pdf/1908.03856v1.pdf
PWC	https://paperswithcode.com/paper/delving-into-robust-object-detection-from
Repo	https://github.com/TAMU-VITA/UAV-NDFT
Framework	pytorch

RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles


Title	RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles
Authors	Ramin Nabati, Hairong Qi
Abstract	Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object’s distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall. Code has been made publicly available at https://github.com/mrnabati/RRPN .
Tasks	Autonomous Driving, Autonomous Vehicles, Object Detection, Sensor Fusion
Published	2019-05-01
URL	https://arxiv.org/abs/1905.00526v2
PDF	https://arxiv.org/pdf/1905.00526v2.pdf
PWC	https://paperswithcode.com/paper/rrpn-radar-region-proposal-network-for-object
Repo	https://github.com/mrnabati/RRPN
Framework	none

Provable robustness against all adversarial $l_p$-perturbations for $p\geq 1$


Title	Provable robustness against all adversarial $l_p$-perturbations for $p\geq 1$
Authors	Francesco Croce, Matthias Hein
Abstract	In recent years several adversarial attacks and defenses have been proposed. Often seemingly robust models turn out to be non-robust when more sophisticated attacks are used. One way out of this dilemma are provable robustness guarantees. While provably robust models for specific $l_p$-perturbation models have been developed, they are still vulnerable to other $l_q$-perturbations. We propose a new regularization scheme, MMR-Universal, for ReLU networks which enforces robustness wrt $l_1$- and $l_\infty$-perturbations and show how that leads to provably robust models wrt any $l_p$-norm for $p\geq 1$.
Tasks
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11213v1
PDF	https://arxiv.org/pdf/1905.11213v1.pdf
PWC	https://paperswithcode.com/paper/provable-robustness-against-all-adversarial
Repo	https://github.com/fra31/mmr-universal
Framework	pytorch

Exploring the Performance of Deep Residual Networks in Crazyhouse Chess


Title	Exploring the Performance of Deep Residual Networks in Crazyhouse Chess
Authors	Sun-Yu Gordon Chi
Abstract	Crazyhouse is a chess variant that incorporates all of the classical chess rules, but allows users to drop pieces captured from the opponent as a normal move. Until 2018, all competitive computer engines for this board game made use of an alpha-beta pruning algorithm with a hand-crafted evaluation function for each position. Previous machine learning-based algorithms for just regular chess, such as NeuroChess and Giraffe, took hand-crafted evaluation features as input rather than a raw board representation. More recent projects, such as AlphaZero, reached massive success but required massive computational resources in order to reach its final strength. This paper describes the development of SixtyFour, an engine designed to compete in the chess variant of Crazyhouse with limited hardware. This specific variant poses a multitude of significant challenges due to its large branching factor, state-space complexity, and the multiple move types a player can make. We propose the novel creation of a neural network-based evaluation function for Crazyhouse. More importantly, we evaluate the effectiveness of an ensemble model, which allows the training time and datasets to be easily distributed on regular CPU hardware commodity. Early versions of the network have attained a playing level comparable to a strong amateur on online servers.
Tasks
Published	2019-08-25
URL	https://arxiv.org/abs/1908.09296v1
PDF	https://arxiv.org/pdf/1908.09296v1.pdf
PWC	https://paperswithcode.com/paper/exploring-the-performance-of-deep-residual
Repo	https://github.com/QueensGambit/CrazyAra
Framework	mxnet

Refining HTN Methods via Task Insertion with Preferences


Title	Refining HTN Methods via Task Insertion with Preferences
Authors	Zhanhao Xiao, Hai Wan, Hankui Hankz Zhuo, Andreas Herzig, Laurent Perrussel, Peilin Chen
Abstract	Hierarchical Task Network (HTN) planning is showing its power in real-world planning. Although domain experts have partial hierarchical domain knowledge, it is time-consuming to specify all HTN methods, leaving them incomplete. On the other hand, traditional HTN learning approaches focus only on declarative goals, omitting the hierarchical domain knowledge. In this paper, we propose a novel learning framework to refine HTN methods via task insertion with completely preserving the original methods. As it is difficult to identify incomplete methods without designating declarative goals for compound tasks, we introduce the notion of prioritized preference to capture the incompleteness possibility of methods. Specifically, the framework first computes the preferred completion profile w.r.t. the prioritized preference to refine the incomplete methods. Then it finds the minimal set of refined methods via a method substitution operation. Experimental analysis demonstrates that our approach is effective, especially in solving new HTN planning instances.
Tasks
Published	2019-11-29
URL	https://arxiv.org/abs/1911.12949v1
PDF	https://arxiv.org/pdf/1911.12949v1.pdf
PWC	https://paperswithcode.com/paper/refining-htn-methods-via-task-insertion-with
Repo	https://github.com/sysulic/MethodRefine
Framework	none

Using Database Rule for Weak Supervised Text-to-SQL Generation


Title	Using Database Rule for Weak Supervised Text-to-SQL Generation
Authors	Tong Guo, Huilin Gao
Abstract	We present a simple way to do the task of text-to-SQL problem with weak supervision. We call it Rule-SQL. Given the question and the answer from the database table without the SQL logic form, Rule-SQL use the rules based on table column names and question string for the SQL exploration first and then use the explored SQL for supervised training. We design several rules for reducing the exploration search space. For the deep model, we leverage BERT for the representation layer and separate the model to SELECT, AGG and WHERE parts. The experiment result on WikiSQL outperforms the strong baseline of full supervision and is comparable to the start-of-the-art weak supervised mothods.
Tasks	Text-To-Sql
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00620v6
PDF	https://arxiv.org/pdf/1907.00620v6.pdf
PWC	https://paperswithcode.com/paper/using-database-rule-for-weak-supervised-text
Repo	https://github.com/guotong1988/Rule-SQL
Framework	none

Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering


Title	Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answering
Authors	Casimiro Pio Carrino, Marta R. Costa-jussà, José A. R. Fonollosa
Abstract	Recently, multilingual question answering became a crucial research topic, and it is receiving increased interest in the NLP community. However, the unavailability of large-scale datasets makes it challenging to train multilingual QA systems with performance comparable to the English ones. In this work, we develop the Translate Align Retrieve (TAR) method to automatically translate the Stanford Question Answering Dataset (SQuAD) v1.1 to Spanish. We then used this dataset to train Spanish QA systems by fine-tuning a Multilingual-BERT model. Finally, we evaluated our QA models with the recently proposed MLQA and XQuAD benchmarks for cross-lingual Extractive QA. Experimental results show that our models outperform the previous Multilingual-BERT baselines achieving the new state-of-the-art value of 68.1 F1 points on the Spanish MLQA corpus and 77.6 F1 and 61.8 Exact Match points on the Spanish XQuAD corpus. The resulting, synthetically generated SQuAD-es v1.1 corpora, with almost 100% of data contained in the original English version, to the best of our knowledge, is the first large-scale QA training resource for Spanish.
Tasks	Question Answering
Published	2019-12-11
URL	https://arxiv.org/abs/1912.05200v2
PDF	https://arxiv.org/pdf/1912.05200v2.pdf
PWC	https://paperswithcode.com/paper/automatic-spanish-translation-of-the-squad
Repo	https://github.com/ccasimiro88/TranslateAlignRetrieve
Framework	none

Discovering Subdimensional Motifs of Different Lengths in Large-Scale Multivariate Time Series


Title	Discovering Subdimensional Motifs of Different Lengths in Large-Scale Multivariate Time Series
Authors	Yifeng Gao, Jessica Lin
Abstract	Detecting repeating patterns of different lengths in time series, also called variable-length motifs, has received a great amount of attention by researchers and practitioners. Despite the significant progress that has been made in recent single dimensional variable-length motif discovery work, detecting variable-length \textit{subdimensional motifs}—patterns that are simultaneously occurring only in a subset of dimensions in multivariate time series—remains a difficult task. The main challenge is scalability. On the one hand, the brute-force enumeration solution, which searches for motifs of all possible lengths, is very time consuming even in single dimensional time series. On the other hand, previous work show that index-based fixed-length approximate motif discovery algorithms such as random projection are not suitable for detecting variable-length motifs due to memory requirement. In this paper, we introduce an approximate variable-length subdimensional motif discovery algorithm called \textbf{C}ollaborative \textbf{HI}erarchy based \textbf{M}otif \textbf{E}numeration (CHIME) to efficiently detect variable-length subdimensional motifs given a minimum motif length in large-scale multivariate time series. We show that the memory cost of the approach is significantly smaller than that of random projection. Moreover, the speed of the proposed algorithm is significantly faster than that of the state-of-the-art algorithms. We demonstrate that CHIME can efficiently detect meaningful variable-length subdimensional motifs in large real world multivariate time series datasets.
Tasks	Time Series
Published	2019-11-20
URL	https://arxiv.org/abs/1911.09218v1
PDF	https://arxiv.org/pdf/1911.09218v1.pdf
PWC	https://paperswithcode.com/paper/discovering-subdimensional-motifs-of
Repo	https://github.com/flash121123/CHIME
Framework	none

Learning Graph Pooling and Hybrid Convolutional Operations for Text Representations


Title	Learning Graph Pooling and Hybrid Convolutional Operations for Text Representations
Authors	Hongyang Gao, Yongjun Chen, Shuiwang Ji
Abstract	With the development of graph convolutional networks (GCN), deep learning methods have started to be used on graph data. In additional to convolutional layers, pooling layers are another important components of deep learning. However, no effective pooling methods have been developed for graphs currently. In this work, we propose the graph pooling (gPool) layer, which employs a trainable projection vector to measure the importance of nodes in graphs. By selecting the k-most important nodes to form the new graph, gPool achieves the same objective as regular max pooling layers operating on images. Another limitation of GCN when used on graph-based text representation tasks is that, GCNs do not consider the order information of nodes in graph. To address this limitation, we propose the hybrid convolutional (hConv) layer that combines GCN and regular convolutional operations. The hConv layer is capable of increasing receptive fields quickly and computing features automatically. Based on the proposed gPool and hConv layers, we develop new deep networks for text categorization tasks. Our results show that the networks based on gPool and hConv layers achieves new state-of-the-art performance as compared to baseline methods.
Tasks	Text Categorization
Published	2019-01-21
URL	http://arxiv.org/abs/1901.06965v2
PDF	http://arxiv.org/pdf/1901.06965v2.pdf
PWC	https://paperswithcode.com/paper/learning-graph-pooling-and-hybrid
Repo	https://github.com/HongyangGao/hConv-gPool-Net
Framework	tf

Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems


Title	Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Authors	Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, Pascale Fung
Abstract	Over-dependence on domain ontology and lack of knowledge sharing across domains are two practical and yet less studied problems of dialogue state tracking. Existing approaches generally fall short in tracking unknown slot values during inference and often have difficulties in adapting to new domains. In this paper, we propose a Transferable Dialogue State Generator (TRADE) that generates dialogue states from utterances using a copy mechanism, facilitating knowledge transfer when predicting (domain, slot, value) triplets not encountered during training. Our model is composed of an utterance encoder, a slot gate, and a state generator, which are shared across domains. Empirical results demonstrate that TRADE achieves state-of-the-art joint goal accuracy of 48.62% for the five domains of MultiWOZ, a human-human dialogue dataset. In addition, we show its transferring ability by simulating zero-shot and few-shot dialogue state tracking for unseen domains. TRADE achieves 60.58% joint goal accuracy in one of the zero-shot domains, and is able to adapt to few-shot cases without forgetting already trained domains.
Tasks	Dialogue State Tracking, Task-Oriented Dialogue Systems, Transfer Learning
Published	2019-05-21
URL	https://arxiv.org/abs/1905.08743v2
PDF	https://arxiv.org/pdf/1905.08743v2.pdf
PWC	https://paperswithcode.com/paper/transferable-multi-domain-state-generator-for
Repo	https://github.com/jasonwu0731/trade-dst
Framework	pytorch

Phase Transition Behavior of Cardinality and XOR Constraints


Title	Phase Transition Behavior of Cardinality and XOR Constraints
Authors	Yash Pote, Saurabh Joshi, Kuldeep S. Meel
Abstract	The runtime performance of modern SAT solvers is deeply connected to the phase transition behavior of CNF formulas. While CNF solving has witnessed significant runtime improvement over the past two decades, the same does not hold for several other classes such as the conjunction of cardinality and XOR constraints, denoted as CARD-XOR formulas. The problem of determining the satisfiability of CARD-XOR formulas is a fundamental problem with a wide variety of applications ranging from discrete integration in the field of artificial intelligence to maximum likelihood decoding in coding theory. The runtime behavior of random CARD-XOR formulas is unexplored in prior work. In this paper, we present the first rigorous empirical study to characterize the runtime behavior of 1-CARD-XOR formulas. We show empirical evidence of a surprising phase-transition that follows a non-linear tradeoff between CARD and XOR constraints.
Tasks
Published	2019-10-22
URL	https://arxiv.org/abs/1910.09755v1
PDF	https://arxiv.org/pdf/1910.09755v1.pdf
PWC	https://paperswithcode.com/paper/phase-transition-behavior-of-cardinality-and
Repo	https://github.com/meelgroup/1-CARD-XOR
Framework	none

Using Pre-Training Can Improve Model Robustness and Uncertainty


Title	Using Pre-Training Can Improve Model Robustness and Uncertainty
Authors	Dan Hendrycks, Kimin Lee, Mantas Mazeika
Abstract	He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training. We show that although pre-training may not improve performance on traditional classification metrics, it improves model robustness and uncertainty estimates. Through extensive experiments on adversarial examples, label corruption, class imbalance, out-of-distribution detection, and confidence calibration, we demonstrate large gains from pre-training and complementary effects with task-specific methods. We introduce adversarial pre-training and show approximately a 10% absolute improvement over the previous state-of-the-art in adversarial robustness. In some cases, using pre-training without task-specific methods also surpasses the state-of-the-art, highlighting the need for pre-training when evaluating future methods on robustness and uncertainty tasks.
Tasks	Calibration, Out-of-Distribution Detection
Published	2019-01-28
URL	https://arxiv.org/abs/1901.09960v5
PDF	https://arxiv.org/pdf/1901.09960v5.pdf
PWC	https://paperswithcode.com/paper/using-pre-training-can-improve-model
Repo	https://github.com/hendrycks/pre-training
Framework	pytorch

RepPoints: Point Set Representation for Object Detection


Title	RepPoints: Point Set Representation for Object Detection
Authors	Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin
Abstract	Modern object detectors rely heavily on rectangular bounding boxes, such as anchors, proposals and the final predictions, to represent objects at various recognition stages. The bounding box is convenient to use but provides only a coarse localization of objects and leads to a correspondingly coarse extraction of object features. In this paper, we present \textbf{RepPoints} (representative points), a new finer representation of objects as a set of sample points useful for both localization and recognition. Given ground truth localization and recognition targets for training, RepPoints learn to automatically arrange themselves in a manner that bounds the spatial extent of an object and indicates semantically significant local areas. They furthermore do not require the use of anchors to sample a space of bounding boxes. We show that an anchor-free object detector based on RepPoints can be as effective as the state-of-the-art anchor-based detection methods, with 46.5 AP and 67.4 $AP_{50}$ on the COCO test-dev detection benchmark, using ResNet-101 model. Code is available at https://github.com/microsoft/RepPoints.
Tasks	Object Detection
Published	2019-04-25
URL	https://arxiv.org/abs/1904.11490v2
PDF	https://arxiv.org/pdf/1904.11490v2.pdf
PWC	https://paperswithcode.com/paper/reppoints-point-set-representation-for-object
Repo	https://github.com/microsoft/RepPoints
Framework	pytorch