January 29, 2020

2936 words 14 mins read

Paper Group ANR 488

Region homogeneity in the Logarithmic Image Processing framework: application to region growing algorithms. Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding. Saec: Similarity-Aware Embedding Compression in Recommendation Systems. Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A F …

Region homogeneity in the Logarithmic Image Processing framework: application to region growing algorithms


Title	Region homogeneity in the Logarithmic Image Processing framework: application to region growing algorithms
Authors	Guillaume Noyel, Michel Jourlin
Abstract	In order to create an image segmentation method robust to lighting changes, two novel homogeneity criteria of an image region were studied. Both were defined using the Logarithmic Image Processing (LIP) framework whose laws model lighting changes. The first criterion estimates the LIP-additive homogeneity and is based on the LIP-additive law. It is theoretically insensitive to lighting changes caused by variations of the camera exposure-time or source intensity. The second, the LIP-multiplicative homogeneity criterion, is based on the LIP-multiplicative law and is insensitive to changes due to variations of the object thickness or opacity. Each criterion is then applied in Revol and Jourlin’s (1997) region growing method which is based on the homogeneity of an image region. The region growing method becomes therefore robust to the lighting changes specific to each criterion. Experiments on simulated and on real images presenting lighting variations prove the robustness of the criteria to those variations. Compared to a state-of the art method based on the image component-tree, ours is more robust. These results open the way to numerous applications where the lighting is uncontrolled or partially controlled.
Tasks	Semantic Segmentation
Published	2019-04-17
URL	http://arxiv.org/abs/1904.12597v1
PDF	http://arxiv.org/pdf/1904.12597v1.pdf
PWC	https://paperswithcode.com/paper/190412597
Repo
Framework

Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding


Title	Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding
Authors	Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Kunlin Cao, Qi Song, Shaoting Zhang, Siwei Lyu, Youbing Yin
Abstract	Modeling the sequential information of image sequences has been a vital step of various vision tasks and convolutional long short-term memory (ConvLSTM) has demonstrated its superb performance in such spatiotemporal problems. Nevertheless, the hierarchical data structures in a significant amount of tasks (e.g., human body parts and vessel/airway tree in biomedical images) cannot be properly modeled by sequential models. Thus, ConvLSTM is not suitable for tree-structured image data analysis. In order to address these limitations, we present tree-structured ConvLSTM models for tree-structured image analysis tasks which can be trained end-to-end. To demonstrate the effectiveness of the proposed tree-structured ConvLSTM model, we present a tree-structured segmentation framework which consists of a tree-structured ConvLSTM and an attention fully convolutional network (FCN) model. The proposed framework is extensively validated on four large-scale coronary artery datasets. The results demonstrate the effectiveness and efficiency of the proposed method.
Tasks
Published	2019-01-29
URL	http://arxiv.org/abs/1902.10053v1
PDF	http://arxiv.org/pdf/1902.10053v1.pdf
PWC	https://paperswithcode.com/paper/attention-driven-tree-structured
Repo
Framework

Saec: Similarity-Aware Embedding Compression in Recommendation Systems


Title	Saec: Similarity-Aware Embedding Compression in Recommendation Systems
Authors	Xiaorui Wu, Hong Xu, Honglin Zhang, Huaming Chen, Jian Wang
Abstract	Production recommendation systems rely on embedding methods to represent various features. An impeding challenge in practice is that the large embedding matrix incurs substantial memory footprint in serving as the number of features grows over time. We propose a similarity-aware embedding matrix compression method called Saec to address this challenge. Saec clusters similar features within a field to reduce the embedding matrix size. Saec also adopts a fast clustering optimization based on feature frequency to drastically improve clustering time. We implement and evaluate Saec on Numerous, the production distributed machine learning system in Tencent, with 10-day worth of feature data from QQ mobile browser. Testbed experiments show that Saec reduces the number of embedding vectors by two orders of magnitude, compresses the embedding size by ~27x, and delivers the same AUC and log loss performance.
Tasks	Recommendation Systems
Published	2019-02-26
URL	http://arxiv.org/abs/1903.00103v1
PDF	http://arxiv.org/pdf/1903.00103v1.pdf
PWC	https://paperswithcode.com/paper/saec-similarity-aware-embedding-compression
Repo
Framework

Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study


Title	Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study
Authors	James Barry, Joachim Wagner, Jennifer Foster
Abstract	Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, projecting from a single polyglot model which is trained on the combination of all source languages. Furthermore, we reproduce multi-source projection (Tyers et al., 2018), in which dependency trees of multiple sources are combined. Finally, we apply multi-treebank modelling to the projected treebanks, in addition to or alternatively to polyglot modelling on the source side. We find that polyglot training on the source languages produces an overall trend of better results on the target language but the single best result for the target language is obtained by projecting from monolingual source parsing models and then training multi-treebank POS tagging and parsing models on the target side.
Tasks	Dependency Parsing
Published	2019-10-17
URL	https://arxiv.org/abs/1910.07938v1
PDF	https://arxiv.org/pdf/1910.07938v1.pdf
PWC	https://paperswithcode.com/paper/cross-lingual-parsing-with-polyglot-training
Repo
Framework

Disentangling Factors of Variation Using Few Labels


Title	Disentangling Factors of Variation Using Few Labels
Authors	Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem
Abstract	Learning disentangled representations is considered a cornerstone problem in representation learning. Recently, Locatello et al. (2019) demonstrated that unsupervised disentanglement learning without inductive biases is theoretically impossible and that existing inductive biases and unsupervised methods do not allow to consistently learn disentangled representations. However, in many practical settings, one might have access to a limited amount of supervision, for example through manual labeling of (some) factors of variation in a few training examples. In this paper, we investigate the impact of such supervision on state-of-the-art disentanglement methods and perform a large scale study, training over 52000 models under well-defined and reproducible experimental conditions. We observe that a small number of labeled examples (0.01–0.5% of the data set), with potentially imprecise and incomplete labels, is sufficient to perform model selection on state-of-the-art unsupervised models. Further, we investigate the benefit of incorporating supervision into the training process. Overall, we empirically validate that with little and imprecise supervision it is possible to reliably learn disentangled representations.
Tasks	Model Selection, Representation Learning
Published	2019-05-03
URL	https://arxiv.org/abs/1905.01258v2
PDF	https://arxiv.org/pdf/1905.01258v2.pdf
PWC	https://paperswithcode.com/paper/disentangling-factors-of-variation-using-few
Repo
Framework

Optimal Transport, CycleGAN, and Penalized LS for Unsupervised Learning in Inverse Problems


Title	Optimal Transport, CycleGAN, and Penalized LS for Unsupervised Learning in Inverse Problems
Authors	Byeongsu Sim, Gyutaek Oh, Jeongsol Kim, Chanyong Jung, Jong Chul Ye
Abstract	The penalized least squares (PLS) is a classic method to inverse problems, where a regularization term is added to stabilize the solution. Optimal transport (OT) is another mathematical framework for computer vision tasks that provides means to transport one measure to another at minimal cost. The cycle-consistent generative adversarial network (cycleGAN) is a recent extension of GAN to learn target distributions with less mode collapsing behavior. Although similar in that no supervised training is required, the algorithms look different, so the mathematical relationship between these approaches is not clear. In this article, we provide an important advance to unveil the missing link. Specifically, we reveal that a cycleGAN architecture is originated from formulating a dual OT problem, by using the consistency constraint of PLS as a regularization term in the primal OT problem. This suggests that cycleGAN can be considered stochastic generalization of classical PLS approaches. Our derivation is so general that various types of cycleGAN architectures can be easily derived by merely changing the transport cost. As proofs of concept, we provide three distinct cycleGAN architecture for various biomedical imaging problems, such as accelerated magnetic resonance imaging (MRI), super-resolution microscopy, and low-dose x-ray computed tomography (CT). Experimental results confirm the efficacy and the flexibility of the theory.
Tasks	Computed Tomography (CT), Super-Resolution
Published	2019-09-25
URL	https://arxiv.org/abs/1909.12116v2
PDF	https://arxiv.org/pdf/1909.12116v2.pdf
PWC	https://paperswithcode.com/paper/optimal-transport-cyclegan-and-penalized-ls
Repo
Framework

Curriculum Loss: Robust Learning and Generalization against Label Corruption


Title	Curriculum Loss: Robust Learning and Generalization against Label Corruption
Authors	Yueming Lyu, Ivor W. Tsang
Abstract	Deep neural networks (DNNs) have great expressive power, which can even memorize samples with wrong labels. It is vitally important to reiterate robustness and generalization in DNNs against label corruption. To this end, this paper studies the 0-1 loss, which has a monotonic relationship with an empirical adversary (reweighted) risk~\citep{hu2016does}. Although the 0-1 loss has some robust properties, it is difficult to optimize. To efficiently optimize the 0-1 loss while keeping its robust properties, we propose a very simple and efficient loss, i.e. curriculum loss (CL). Our CL is a tighter upper bound of the 0-1 loss compared with conventional summation based surrogate losses. Moreover, CL can adaptively select samples for model training. As a result, our loss can be deemed as a novel perspective of curriculum sample selection strategy, which bridges a connection between curriculum learning and robust learning. Experimental results on benchmark datasets validate the robustness of the proposed loss.
Tasks
Published	2019-05-24
URL	https://arxiv.org/abs/1905.10045v3
PDF	https://arxiv.org/pdf/1905.10045v3.pdf
PWC	https://paperswithcode.com/paper/curriculum-loss-robust-learning-and
Repo
Framework

Robust Ordinal VAE: Employing Noisy Pairwise Comparisons for Disentanglement


Title	Robust Ordinal VAE: Employing Noisy Pairwise Comparisons for Disentanglement
Authors	Junxiang Chen, Kayhan Batmanghelich
Abstract	Recent work by Locatello et al. (2018) has shown that an inductive bias is required to disentangle factors of interest in Variational Autoencoder (VAE). Motivated by a real-world problem, we propose a setting where such bias is introduced by providing pairwise ordinal comparisons between instances, based on the desired factor to be disentangled. For example, a doctor compares pairs of patients based on the level of severity of their illnesses, and the desired factor is a quantitive level of the disease severity. In a real-world application, the pairwise comparisons are usually noisy. Our method, Robust Ordinal VAE (ROVAE), incorporates the noisy pairwise ordinal comparisons in the disentanglement task. We introduce non-negative random variables in ROVAE, such that it can automatically determine whether each pairwise ordinal comparison is trustworthy and ignore the noisy comparisons. Experimental results demonstrate that ROVAE outperforms existing methods and is more robust to noisy pairwise comparisons in both benchmark datasets and a real-world application.
Tasks
Published	2019-10-14
URL	https://arxiv.org/abs/1910.05898v1
PDF	https://arxiv.org/pdf/1910.05898v1.pdf
PWC	https://paperswithcode.com/paper/robust-ordinal-vae-employing-noisy-pairwise
Repo
Framework

Invasiveness Prediction of Pulmonary Adenocarcinomas Using Deep Feature Fusion Networks


Title	Invasiveness Prediction of Pulmonary Adenocarcinomas Using Deep Feature Fusion Networks
Authors	Xiang Li, Jiechao Ma, Hongwei Li
Abstract	Early diagnosis of pathological invasiveness of pulmonary adenocarcinomas using computed tomography (CT) imaging would alter the course of treatment of adenocarcinomas and subsequently improve the prognosis. Most of the existing systems use either conventional radiomics features or deep-learning features alone to predict the invasiveness. In this study, we explore the fusion of the two kinds of features and claim that radiomics features can be complementary to deep-learning features. An effective deep feature fusion network is proposed to exploit the complementarity between the two kinds of features, which improves the invasiveness prediction results. We collected a private dataset that contains lung CT scans of 676 patients categorized into four invasiveness types from a collaborating hospital. Evaluations on this dataset demonstrate the effectiveness of our proposal.
Tasks	Computed Tomography (CT)
Published	2019-09-21
URL	https://arxiv.org/abs/1909.09837v1
PDF	https://arxiv.org/pdf/1909.09837v1.pdf
PWC	https://paperswithcode.com/paper/190909837
Repo
Framework

Coarse-scale PDEs from fine-scale observations via machine learning


Title	Coarse-scale PDEs from fine-scale observations via machine learning
Authors	Seungjoon Lee, Mahdi Kooshkbaghi, Konstantinos Spiliotis, Constantinos I. Siettos, Ioannis G. Kevrekidis
Abstract	Complex spatiotemporal dynamics of physicochemical processes are often modeled at a microscopic level (through e.g. atomistic, agent-based or lattice models) based on first principles. Some of these processes can also be successfully modeled at the macroscopic level using e.g. partial differential equations (PDEs) describing the evolution of the right few macroscopic observables (e.g. concentration and momentum fields). Deriving good macroscopic descriptions (the so-called “closure problem”) is often a time-consuming process requiring deep understanding/intuition about the system of interest. Recent developments in data science provide alternative ways to effectively extract/learn accurate macroscopic descriptions approximating the underlying microscopic observations. In this paper, we introduce a data-driven framework for the identification of unavailable coarse-scale PDEs from microscopic observations via machine learning algorithms. Specifically, using Gaussian Processes, Artificial Neural Networks, and/or Diffusion Maps, the proposed framework uncovers the relation between the relevant macroscopic space fields and their time evolution (the right-hand-side of the explicitly unavailable macroscopic PDE). Interestingly, several choices equally representative of the data can be discovered. The framework will be illustrated through the data-driven discovery of macroscopic, concentration-level PDEs resulting from a fine-scale, Lattice Boltzmann level model of a reaction/transport process. Once the coarse evolution law is identified, it can be simulated to produce long-term macroscopic predictions. Different features (pros as well as cons) of alternative machine learning algorithms for performing this task (Gaussian Processes and Artificial Neural Networks), are presented and discussed.
Tasks	Gaussian Processes
Published	2019-09-12
URL	https://arxiv.org/abs/1909.05707v1
PDF	https://arxiv.org/pdf/1909.05707v1.pdf
PWC	https://paperswithcode.com/paper/coarse-scale-pdes-from-fine-scale
Repo
Framework

Z-Net: an Anisotropic 3D DCNN for Medical CT Volume Segmentation


Title	Z-Net: an Anisotropic 3D DCNN for Medical CT Volume Segmentation
Authors	Peichao Li, Xiao-Yun Zhou, Zhao-Yang Wang, Guang-Zhong Yang
Abstract	Accurate volume segmentation from the Computed Tomography (CT) scan is a common prerequisite for pre-operative planning, intra-operative guidance and quantitative assessment of therapeutic outcomes in robot-assisted Minimally Invasive Surgery (MIS). 3D Deep Convolutional Neural Network (DCNN) is a viable solution for this task, but is memory intensive. Small isotropic patches are cropped from the original and large CT volume to mitigate this issue in practice, but it may cause discontinuities between the adjacent patches and severe class-imbalances within individual sub-volumes. This paper presents a new 3D DCNN framework, namely Z-Net, to tackle the discontinuity and class-imbalance issue by preserving a full field-of-view of the objects in the XY planes using anisotropic spatial separable convolutions. The proposed Z-Net can be seamlessly integrated into existing 3D DCNNs with isotropic convolutions such as 3D U-Net and V-Net, with improved volume segmentation Intersection over Union (IoU) - up to $12.6%$. Detailed validation of Z-Net is provided for CT aortic, liver and lung segmentation, demonstrating the effectiveness and practical value of Z-Net for intra-operative 3D navigation in robot-assisted MIS.
Tasks	Computed Tomography (CT)
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07480v2
PDF	https://arxiv.org/pdf/1909.07480v2.pdf
PWC	https://paperswithcode.com/paper/z-net-an-asymmetric-3d-dcnn-for-medical-ct
Repo
Framework

Action-Sensitive Phonological Dependencies


Title	Action-Sensitive Phonological Dependencies
Authors	Yiding Hao, Dustin Bowers
Abstract	This paper defines a subregular class of functions called the tier-based synchronized strictly local (TSSL) functions. These functions are similar to the the tier-based input-output strictly local (TIOSL) functions, except that the locality condition is enforced not on the input and output streams, but on the computation history of the minimal subsequential finite-state transducer. We show that TSSL functions naturally describe rhythmic syncope while TIOSL functions cannot, and we argue that TSSL functions provide a more restricted characterization of rhythmic syncope than existing treatments within Optimality Theory.
Tasks
Published	2019-06-12
URL	https://arxiv.org/abs/1906.06464v1
PDF	https://arxiv.org/pdf/1906.06464v1.pdf
PWC	https://paperswithcode.com/paper/action-sensitive-phonological-dependencies
Repo
Framework

Multi-level Domain Adaptive learning for Cross-Domain Detection


Title	Multi-level Domain Adaptive learning for Cross-Domain Detection
Authors	Rongchang Xie, Fei Yu, Jiachao Wang, Yizhou Wang, Li Zhang
Abstract	In recent years, object detection has shown impressive results using supervised deep learning, but it remains challenging in a cross-domain environment. The variations of illumination, style, scale, and appearance in different domains can seriously affect the performance of detection models. Previous works use adversarial training to align global features across the domain shift and to achieve image information transfer. However, such methods do not effectively match the distribution of local features, resulting in limited improvement in cross-domain object detection. To solve this problem, we propose a multi-level domain adaptive model to simultaneously align the distributions of local-level features and global-level features. We evaluate our method with multiple experiments, including adverse weather adaptation, synthetic data adaptation, and cross camera adaptation. In most object categories, the proposed method achieves superior performance against state-of-the-art techniques, which demonstrates the effectiveness and robustness of our method.
Tasks	Object Detection
Published	2019-07-26
URL	https://arxiv.org/abs/1907.11484v2
PDF	https://arxiv.org/pdf/1907.11484v2.pdf
PWC	https://paperswithcode.com/paper/multi-level-domain-adaptive-learning-for
Repo
Framework

ENIGMAWatch: ProofWatch Meets ENIGMA


Title	ENIGMAWatch: ProofWatch Meets ENIGMA
Authors	Zarathustra Goertzel, Jan Jakubův, Josef Urban
Abstract	In this work we describe a new learning-based proof guidance – ENIGMAWatch – for saturation-style first-order theorem provers. ENIGMAWatch combines two guiding approaches for the given-clause selection implemented for the E ATP system: ProofWatch and ENIGMA. ProofWatch is motivated by the watchlist (hints) method and based on symbolic matching of multiple related proofs, while ENIGMA is based on statistical machine learning. The two methods are combined by using the evolving information about symbolic proof matching as an additional information that characterizes the saturation-style proof search for the statistical learning methods. The new system is experimentally evaluated on a large set of problems from the Mizar Library. We show that the added proof-matching information is considered important by the statistical machine learners, and that it leads to improvements in E’s Performance over ProofWatch and ENIGMA.
Tasks
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09565v2
PDF	https://arxiv.org/pdf/1905.09565v2.pdf
PWC	https://paperswithcode.com/paper/enigmawatch-proofwatch-meets-enigma
Repo
Framework

A K-means-based Multi-subpopulation Particle Swarm Optimization for Neural Network Ensemble


Title	A K-means-based Multi-subpopulation Particle Swarm Optimization for Neural Network Ensemble
Authors	Hui Yu
Abstract	This paper presents a k-means-based multi-subpopulation particle swarm optimization, denoted as KMPSO, for training the neural network ensemble. In the proposed KMPSO, particles are dynamically partitioned into clusters via the k-means clustering algorithm at every iteration, and each of the resulting clusters is responsible for training a component neural network. The performance of the KMPSO has been evaluated on several benchmark problems. Our results show that the proposed method can effectively control the trade-off between the diversity and accuracy in the ensemble, thus achieving competitive results in comparison with related algorithms.
Tasks
Published	2019-06-12
URL	https://arxiv.org/abs/1907.03743v1
PDF	https://arxiv.org/pdf/1907.03743v1.pdf
PWC	https://paperswithcode.com/paper/a-k-means-based-multi-subpopulation-particle
Repo
Framework