April 3, 2020

3515 words 17 mins read

Paper Group AWR 12

Comparing recurrent and convolutional neural networks for predicting wave propagation. Entity Context and Relational Paths for Knowledge Graph Completion. Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study. Detection of 3D Bounding Boxes of Vehicles Using Perspective Transformation for Accurate Speed Measurement. P …

Comparing recurrent and convolutional neural networks for predicting wave propagation


Title	Comparing recurrent and convolutional neural networks for predicting wave propagation
Authors	Stathi Fotiadis, Eduardo Pignatelli, Mario Lino Valencia, Chris Cantwell, Amos Storkey, Anil A. Bharath
Abstract	Dynamical systems can be modelled by partial differential equations and numerical computations are used everywhere in science and engineering. In this work, we investigate the performance of recurrent and convolutional deep neural network architectures to predict the surface waves. The system is governed by the Saint-Venant equations. We improve on the long-term prediction over previous methods while keeping the inference time at a fraction of numerical simulations. We also show that convolutional networks perform at least as well as recurrent networks in this task. Finally, we assess the generalisation capability of each network by extrapolating in longer time-frames and in different physical settings.
Tasks
Published	2020-02-20
URL	https://arxiv.org/abs/2002.08981v2
PDF	https://arxiv.org/pdf/2002.08981v2.pdf
PWC	https://paperswithcode.com/paper/comparing-recurrent-and-convolutional-neural
Repo	https://github.com/stathius/wave_propagation
Framework	pytorch

Entity Context and Relational Paths for Knowledge Graph Completion


Title	Entity Context and Relational Paths for Knowledge Graph Completion
Authors	Hongwei Wang, Hongyu Ren, Jure Leskovec
Abstract	Knowledge graph completion aims to predict missing relations between entities in a knowledge graph. While many different methods have been proposed, there is a lack of a unifying framework that would lead to state-of-the-art results. Here we develop PathCon, a knowledge graph completion method that harnesses four novel insights to outperform existing methods. PathCon predicts relations between a pair of entities by: (1) Considering the Relational Context of each entity by capturing the relation types adjacent to the entity and modeled through a novel edge-based message passing scheme; (2) Considering the Relational Paths capturing all paths between the two entities; And, (3) adaptively integrating the Relational Context and Relational Path through a learnable attention mechanism. Importantly, (4) in contrast to conventional node-based representations, PathCon represents context and path only using the relation types, which makes it applicable in an inductive setting. Experimental results on knowledge graph benchmarks as well as our newly proposed dataset show that PathCon outperforms state-of-the-art knowledge graph completion methods by a large margin. Finally, PathCon is able to provide interpretable explanations by identifying relations that provide the context and paths that are important for a given predicted relation.
Tasks	Knowledge Graph Completion
Published	2020-02-17
URL	https://arxiv.org/abs/2002.06757v1
PDF	https://arxiv.org/pdf/2002.06757v1.pdf
PWC	https://paperswithcode.com/paper/entity-context-and-relational-paths-for
Repo	https://github.com/muhanzhang/IGPL
Framework	pytorch

Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study


Title	Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study
Authors	Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, Chengkai Li
Abstract	In the active research area of employing embedding models for knowledge graph completion, particularly for the task of link prediction, most prior studies used two benchmark datasets FB15k and WN18 in evaluating such models. Most triples in these and other datasets in such studies belong to reverse and duplicate relations which exhibit high data redundancy due to semantic duplication, correlation or data incompleteness. This is a case of excessive data leakage—a model is trained using features that otherwise would not be available when the model needs to be applied for real prediction. There are also Cartesian product relations for which every triple formed by the Cartesian product of applicable subjects and objects is a true fact. Link prediction on the aforementioned relations is easy and can be achieved with even better accuracy using straightforward rules instead of sophisticated embedding models. A more fundamental defect of these models is that the link prediction scenario, given such data, is non-existent in the real-world. This paper is the first systematic study with the main objective of assessing the true effectiveness of embedding models when the unrealistic triples are removed. Our experiment results show these models are much less accurate than what we used to perceive. Their poor accuracy renders link prediction a task without truly effective automated solution. Hence, we call for re-investigation of possible effective approaches.
Tasks	Knowledge Graph Completion, Link Prediction
Published	2020-03-18
URL	https://arxiv.org/abs/2003.08001v1
PDF	https://arxiv.org/pdf/2003.08001v1.pdf
PWC	https://paperswithcode.com/paper/realistic-re-evaluation-of-knowledge-graph
Repo	https://github.com/idirlab/kgcompletion
Framework	tf

Detection of 3D Bounding Boxes of Vehicles Using Perspective Transformation for Accurate Speed Measurement


Title	Detection of 3D Bounding Boxes of Vehicles Using Perspective Transformation for Accurate Speed Measurement
Authors	Viktor Kocur, Milan Ftáčnik
Abstract	Detection and tracking of vehicles captured by traffic surveillance cameras is a key component of intelligent transportation systems. We present an improved version of our algorithm for detection of 3D bounding boxes of vehicles, their tracking and subsequent speed estimation. Our algorithm utilizes the known geometry of vanishing points in the surveilled scene to construct a perspective transformation. The transformation enables an intuitive simplification of the problem of detecting 3D bounding boxes to detection of 2D bounding boxes with one additional parameter using a standard 2D object detector. Main contribution of this paper is an improved construction of the perspective transformation which is more robust and fully automatic and an extended experimental evaluation of speed estimation. We test our algorithm on the speed estimation task of the BrnoCompSpeed dataset. We evaluate our approach with different configurations to gauge the relationship between accuracy and computational costs and benefits of 3D bounding box detection over 2D detection. All of the tested configurations run in real-time and are fully automatic. Compared to other published state-of-the-art fully automatic results our algorithm reduces the mean absolute speed measurement error by 32% (1.10 km/h to 0.75 km/h) and the absolute median error by 40% (0.97 km/h to 0.58 km/h).
Tasks
Published	2020-03-29
URL	https://arxiv.org/abs/2003.13137v1
PDF	https://arxiv.org/pdf/2003.13137v1.pdf
PWC	https://paperswithcode.com/paper/detection-of-3d-bounding-boxes-of-vehicles
Repo	https://github.com/kocurvik/BCS_results
Framework	none

Pedestrian Detection: The Elephant In The Room


Title	Pedestrian Detection: The Elephant In The Room
Authors	Irtiza Hasan, Shengcai Liao, Jinpeng Li, Saad Ullah Akram, Ling Shao
Abstract	Pedestrian detection is used in many vision based applications ranging from video surveillance to autonomous driving. Despite achieving high performance, it is still largely unknown how well existing detectors generalize to unseen data. To this end, we conduct a comprehensive study in this paper, using a general principle of direct cross-dataset evaluation. Through this study, we find that existing state-of-the-art pedestrian detectors generalize poorly from one dataset to another. We demonstrate that there are two reasons for this trend. Firstly, they over-fit on popular datasets in a traditional single-dataset training and test pipeline. Secondly, the training source is generally not dense in pedestrians and diverse in scenarios. Accordingly, through experiments we find that a general purpose object detector works better in direct cross-dataset evaluation compared with state-of-the-art pedestrian detectors and we illustrate that diverse and dense datasets, collected by crawling the web, serve to be an efficient source of pre-training for pedestrian detection. Furthermore, we find that a progressive training pipeline works good for autonomous driving oriented detector. We improve upon previous state-of-the-art on reasonable/heavy subsets of CityPersons dataset by 1.3%/1.7% and on Caltech by 1.8%/14.9% in terms of log average miss rate (MR^2) points without any fine-tuning on the test set. Detector trained through proposed pipeline achieves top rank at the leaderborads of CityPersons [42] and ECP [4]. Code and models will be available at https://github.com/hasanirtiza/Pedestron.
Tasks	Autonomous Driving, Pedestrian Detection
Published	2020-03-19
URL	https://arxiv.org/abs/2003.08799v2
PDF	https://arxiv.org/pdf/2003.08799v2.pdf
PWC	https://paperswithcode.com/paper/pedestrian-detection-the-elephant-in-the-room
Repo	https://github.com/hasanirtiza/Pedestron
Framework	pytorch

On Positive-Unlabeled Classification in GAN


Title	On Positive-Unlabeled Classification in GAN
Authors	Tianyu Guo, Chang Xu, Jiajun Huang, Yunhe Wang, Boxin Shi, Chao Xu, Dacheng Tao
Abstract	This paper defines a positive and unlabeled classification problem for standard GANs, which then leads to a novel technique to stabilize the training of the discriminator in GANs. Traditionally, real data are taken as positive while generated data are negative. This positive-negative classification criterion was kept fixed all through the learning process of the discriminator without considering the gradually improved quality of generated data, even if they could be more realistic than real data at times. In contrast, it is more reasonable to treat the generated data as unlabeled, which could be positive or negative according to their quality. The discriminator is thus a classifier for this positive and unlabeled classification problem, and we derive a new Positive-Unlabeled GAN (PUGAN). We theoretically discuss the global optimality the proposed model will achieve and the equivalent optimization goal. Empirically, we find that PUGAN can achieve comparable or even better performance than those sophisticated discriminator stabilization methods.
Tasks
Published	2020-02-04
URL	https://arxiv.org/abs/2002.01136v1
PDF	https://arxiv.org/pdf/2002.01136v1.pdf
PWC	https://paperswithcode.com/paper/on-positive-unlabeled-classification-in-gan
Repo	https://github.com/huangjiadidi/PUGAN
Framework	pytorch

LiteMORT: A memory efficient gradient boosting tree system on adaptive compact distributions


Title	LiteMORT: A memory efficient gradient boosting tree system on adaptive compact distributions
Authors	Yingshi Chen
Abstract	Gradient boosted decision trees (GBDT) is the leading algorithm for many commercial and academic data applications. We give a deep analysis of this algorithm, especially the histogram technique, which is a basis for the regulized distribution with compact support. We present three new modifications. 1) Share memory technique to reduce memory usage. In many cases, it only need the data source itself and no extra memory. 2) Implicit merging for “merge overflow problem”.“merge overflow” means that merge some small datasets to huge datasets, which are too huge to be solved. By implicit merging, we just need the original small datasets to train the GBDT model. 3) Adaptive resize algorithm of histogram bins to improve accuracy. Experiments on two large Kaggle competitions verified our methods. They use much less memory than LightGBM and have higher accuracy. We have implemented these algorithms in an open-source package LiteMORT. The source codes are available at https://github.com/closest-git/LiteMORT
Tasks
Published	2020-01-26
URL	https://arxiv.org/abs/2001.09419v1
PDF	https://arxiv.org/pdf/2001.09419v1.pdf
PWC	https://paperswithcode.com/paper/litemort-a-memory-efficient-gradient-boosting
Repo	https://github.com/closest-git/LiteMORT
Framework	none

Diffusion bridges for stochastic Hamiltonian systems with applications to shape analysis


Title	Diffusion bridges for stochastic Hamiltonian systems with applications to shape analysis
Authors	Alexis Arnaudon, Frank van der Meulen, Moritz Schauer, Stefan Sommer
Abstract	Stochastically evolving geometric systems are studied in geometric mechanics for modelling turbulence parts of multi-scale fluid flows and in shape analysis for stochastic evolutions of shapes of e.g. human organs. Recently introduced models involve stochastic differential equations that govern the dynamics of a diffusion process $X$. In applications $X$ is only partially observed at times $0$ and $T>0$. Conditional on these observations, interest lies in inferring parameters in the dynamics of the diffusion and reconstructing the path $(X_t,, t\in [0,T])$. The latter problem is known as bridge simulation. We develop a general scheme for bridge sampling in the case of finite dimensional systems of shape landmarks and singular solutions in fluid dynamics. This scheme allows for subsequent statistical inference of properties of the fluid flow or the evolution of observed shapes. It covers stochastic landmark models for which no suitable simulation method has been proposed in the literature, that removes restrictions of earlier approaches, improves the handling of the nonlinearity of the configuration space leading to more effective sampling schemes and allows to generalise the common inexact matching scheme to the stochastic setting.
Tasks
Published	2020-02-03
URL	https://arxiv.org/abs/2002.00885v2
PDF	https://arxiv.org/pdf/2002.00885v2.pdf
PWC	https://paperswithcode.com/paper/diffusion-bridges-for-stochastic-hamiltonian
Repo	https://github.com/mschauer/BridgeLandmarks.jl
Framework	none

Auto-Encoding Twin-Bottleneck Hashing


Title	Auto-Encoding Twin-Bottleneck Hashing
Authors	Yuming Shen, Jie Qin, Jiaxin Chen, Mengyang Yu, Li Liu, Fan Zhu, Fumin Shen, Ling Shao
Abstract	Conventional unsupervised hashing methods usually take advantage of similarity graphs, which are either pre-computed in the high-dimensional space or obtained from random anchor points. On the one hand, existing methods uncouple the procedures of hash function learning and graph construction. On the other hand, graphs empirically built upon original data could introduce biased prior knowledge of data relevance, leading to sub-optimal retrieval performance. In this paper, we tackle the above problems by proposing an efficient and adaptive code-driven graph, which is updated by decoding in the context of an auto-encoder. Specifically, we introduce into our framework twin bottlenecks (i.e., latent variables) that exchange crucial information collaboratively. One bottleneck (i.e., binary codes) conveys the high-level intrinsic data structure captured by the code-driven graph to the other (i.e., continuous variables for low-level detail information), which in turn propagates the updated network feedback for the encoder to learn more discriminative binary codes. The auto-encoding learning objective literally rewards the code-driven graph to learn an optimal encoder. Moreover, the proposed model can be simply optimized by gradient descent without violating the binary constraints. Experiments on benchmarked datasets clearly show the superiority of our framework over the state-of-the-art hashing methods. Our source code can be found at https://github.com/ymcidence/TBH.
Tasks	graph construction
Published	2020-02-27
URL	https://arxiv.org/abs/2002.11930v2
PDF	https://arxiv.org/pdf/2002.11930v2.pdf
PWC	https://paperswithcode.com/paper/auto-encoding-twin-bottleneck-hashing
Repo	https://github.com/ymcidence/TBH
Framework	tf

Stabilizing Differentiable Architecture Search via Perturbation-based Regularization


Title	Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Authors	Xiangning Chen, Cho-Jui Hsieh
Abstract	Differentiable architecture search (DARTS) is a prevailing NAS solution to identify architectures. Based on the continuous relaxation of the architecture space, DARTS learns a differentiable architecture weight and largely reduces the search cost. However, its stability and generalizability have been challenged for yielding deteriorating architectures as the search proceeds. We find that the precipitous validation loss landscape, which leads to a dramatic performance drop when distilling the final architecture, is an essential factor that causes instability. Based on this observation, we propose a perturbation-based regularization, named SmoothDARTS (SDARTS), to smooth the loss landscape and improve the generalizability of DARTS. In particular, our new formulations stabilize DARTS by either random smoothing or adversarial attack. The search trajectory on NAS-Bench-1Shot1 demonstrates the effectiveness of our approach and due to the improved stability, we achieve performance gain across various search spaces on 4 datasets. Furthermore, we mathematically show that SDARTS implicitly regularizes the Hessian norm of the validation loss, which accounts for a smoother loss landscape and improved performance. The code is available at https://github.com/xiangning-chen/SmoothDARTS.
Tasks	Adversarial Attack
Published	2020-02-12
URL	https://arxiv.org/abs/2002.05283v1
PDF	https://arxiv.org/pdf/2002.05283v1.pdf
PWC	https://paperswithcode.com/paper/stabilizing-differentiable-architecture
Repo	https://github.com/xiangning-chen/SmoothDARTS
Framework	pytorch

On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering


Title	On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Authors	Xinyu Wang, Yuliang Liu, Chunhua Shen, Chun Chet Ng, Canjie Luo, Lianwen Jin, Chee Seng Chan, Anton van den Hengel, Liangwei Wang
Abstract	Visual Question Answering (VQA) methods have made incredible progress, but suffer from a failure to generalize. This is visible in the fact that they are vulnerable to learning coincidental correlations in the data rather than deeper relations between image content and ideas expressed in language. We present a dataset that takes a step towards addressing this problem in that it contains questions expressed in two languages, and an evaluation process that co-opts a well understood image-based metric to reflect the method’s ability to reason. Measuring reasoning directly encourages generalization by penalizing answers that are coincidentally correct. The dataset reflects the scene-text version of the VQA problem, and the reasoning evaluation can be seen as a text-based version of a referring expression challenge. Experiments and analysis are provided that show the value of the dataset.
Tasks	Question Answering, Visual Question Answering
Published	2020-02-24
URL	https://arxiv.org/abs/2002.10215v2
PDF	https://arxiv.org/pdf/2002.10215v2.pdf
PWC	https://paperswithcode.com/paper/on-the-general-value-of-evidence-and
Repo	https://github.com/xinke-wang/Awesome-Text-VQA
Framework	none

Segmentation with Residual Attention U-Net and an Edge-Enhancement Approach Preserves Cell Shape Features


Title	Segmentation with Residual Attention U-Net and an Edge-Enhancement Approach Preserves Cell Shape Features
Authors	Nanyan Zhu, Chen Liu, Zakary S. Singer, Tal Danino, Andrew F. Laine, Jia Guo
Abstract	The ability to extrapolate gene expression dynamics in living single cells requires robust cell segmentation, and one of the challenges is the amorphous or irregularly shaped cell boundaries. To address this issue, we modified the U-Net architecture to segment cells in fluorescence widefield microscopy images and quantitatively evaluated its performance. We also proposed a novel loss function approach that emphasizes the segmentation accuracy on cell boundaries and encourages shape feature preservation. With a 97% sensitivity, 93% specificity, 91% Jaccard similarity, and 95% Dice coefficient, our proposed method called Residual Attention U-Net with edge-enhancement surpassed the state-of-the-art U-Net in segmentation performance as evaluated by the traditional metrics. More remarkably, the same proposed candidate also performed the best in terms of the preservation of valuable shape features, namely area, eccentricity, major axis length, solidity and orientation. These improvements on shape feature preservation can serve as useful assets for downstream cell tracking and quantification of changes in cell statistics or features over time.
Tasks	Cell Segmentation
Published	2020-01-15
URL	https://arxiv.org/abs/2001.05548v1
PDF	https://arxiv.org/pdf/2001.05548v1.pdf
PWC	https://paperswithcode.com/paper/segmentation-with-residual-attention-u-net
Repo	https://github.com/SAIL-GuoLab/Cell_Segmentation_and_Tracking
Framework	pytorch

Improving Image Autoencoder Embeddings with Perceptual Loss


Title	Improving Image Autoencoder Embeddings with Perceptual Loss
Authors	Gustav Grund Pihlgren, Fredrik Sandin, Marcus Liwicki
Abstract	Autoencoders are commonly trained using element-wise loss. However, element-wise loss disregards high-level structures in the image which can lead to embeddings that disregard them as well. A recent improvement to autoencoders that help alleviate this problem is the use of perceptual loss. This work investigate perceptual loss from the perspective of encoder embeddings themselves. Autoencoders are trained to embed images from three different computer vision datasets using perceptual loss based on a pretrained model as well as pixel-wise loss. A host of different predictors are trained to perform object positioning and classification on the datasets given the embedded images as input. The two kinds of losses are evaluated by comparing how the predictors performed with embeddings from the differently trained autoencoders. The results show that, in the image domain, the embeddings generated by autoencoders trained with perceptual loss enable more accurate predictions than those trained with element-wise loss. Furthermore, the results show that, on the task of object-positioning of a small-scale feature, perceptual loss can improve the results by a factor 10. The experimental setup is available online: https://github.com/guspih/Perceptual-Autoencoders
Tasks
Published	2020-01-10
URL	https://arxiv.org/abs/2001.03444v1
PDF	https://arxiv.org/pdf/2001.03444v1.pdf
PWC	https://paperswithcode.com/paper/improving-image-autoencoder-embeddings-with
Repo	https://github.com/guspih/Perceptual-Autoencoders
Framework	pytorch

Recurrent Neural Networks with Longitudinal Pooling and Consistency Regularization


Title	Recurrent Neural Networks with Longitudinal Pooling and Consistency Regularization
Authors	Jiahong Ouyang, Qingyu Zhao, Edith V Sullivan, Adolf Pfefferbaum, Susan F. Tapert, Ehsan Adeli, Kilian M Pohl
Abstract	Most neurological diseases are characterized by gradual deterioration of brain structure and function. To identify the impact of such diseases, studies have been acquiring large longitudinal MRI datasets and applied deep-learning to predict diagnosis label(s). These learning models apply Convolutional Neural Networks (CNN) to extract informative features from each time point of the longitudinal MRI and Recurrent Neural Networks (RNN) to classify each time point based on those features. However, they neglect the progressive nature of the disease, which may result in clinically implausible predictions across visits. In this paper, we propose a framework that injects the extracted features from CNNs at each time point to the RNN cells considering the dependencies across different time points in the longitudinal data. On the feature level, we propose a novel longitudinal pooling layer to couple features of a visit with those of proceeding ones. On the prediction level, we add a consistency regularization to the classification objective in line with the nature of the disease progression across visits. We evaluate the proposed method on the longitudinal structural MRIs from three neuroimaging datasets: Alzheimer’s Disease Neuroimaging Initiative (ADNI, N=404), a dataset composed of 274 healthy controls and 329 patients with Alcohol Use Disorder (AUD), and 255 youths from the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA). All three experiments show that our method is superior to the widely used methods. The code is available at https://github.com/ouyangjiahong/longitudinal-pooling.
Tasks
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13958v1
PDF	https://arxiv.org/pdf/2003.13958v1.pdf
PWC	https://paperswithcode.com/paper/recurrent-neural-networks-with-longitudinal
Repo	https://github.com/ouyangjiahong/longitudinal-pooling
Framework	none

DeepGS: Deep Representation Learning of Graphs and Sequences for Drug-Target Binding Affinity Prediction


Title	DeepGS: Deep Representation Learning of Graphs and Sequences for Drug-Target Binding Affinity Prediction
Authors	Xuan Lin, Kaiqi Zhao, Tong Xiao, Zhe Quan, Zhi-Jie Wang, Philip S. Yu
Abstract	Accurately predicting drug-target binding affinity (DTA) in silico is a key task in drug discovery. Most of the conventional DTA prediction methods are simulation-based, which rely heavily on domain knowledge or the assumption of having the 3D structure of the targets, which are often difficult to obtain. Meanwhile, traditional machine learning-based methods apply various features and descriptors, and simply depend on the similarities between drug-target pairs. Recently, with the increasing amount of affinity data available and the success of deep representation learning models on various domains, deep learning techniques have been applied to DTA prediction. However, these methods consider either label/one-hot encodings or the topological structure of molecules, without considering the local chemical context of amino acids and SMILES sequences. Motivated by this, we propose a novel end-to-end learning framework, called DeepGS, which uses deep neural networks to extract the local chemical context from amino acids and SMILES sequences, as well as the molecular structure from the drugs. To assist the operations on the symbolic data, we propose to use advanced embedding techniques (i.e., Smi2Vec and Prot2Vec) to encode the amino acids and SMILES sequences to a distributed representation. Meanwhile, we suggest a new molecular structure modeling approach that works well under our framework. We have conducted extensive experiments to compare our proposed method with state-of-the-art models including KronRLS, SimBoost, DeepDTA and DeepCPI. Extensive experimental results demonstrate the superiorities and competitiveness of DeepGS.
Tasks	Drug Discovery, Representation Learning
Published	2020-03-31
URL	https://arxiv.org/abs/2003.13902v1
PDF	https://arxiv.org/pdf/2003.13902v1.pdf
PWC	https://paperswithcode.com/paper/deepgs-deep-representation-learning-of-graphs
Repo	https://github.com/jacklin18/DeepGS
Framework	pytorch