January 26, 2020

3283 words 16 mins read

Paper Group ANR 1369

High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes. Action Anticipation with RBF Kernelized Feature Mapping RNN. Can adversarial training learn image captioning ?. Towards Interlingua Neural Machine Translation. Event-Based Modeling with High-Dimensional Imaging Biom …

High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes


Title	High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes
Authors	Maolin Shi, Wei Sun, Xueguan Song, Hongyou Li
Abstract	Support vector regression (SVR) has been widely used to reduce the high computational cost of computer simulation. SVR assumes the input parameters have equal sample sizes, but unequal sample sizes are often encountered in engineering practices. To solve this issue, a new prediction approach based on SVR, namely as high-low-level SVR approach (HL-SVR) is proposed for data modeling of input parameters of unequal sample sizes in this paper. The proposed approach is consisted of low-level SVR models for the input parameters of larger sample sizes and high-level SVR model for the input parameters of smaller sample sizes. For each training point of the input parameters of smaller sample sizes, one low-level SVR model is built based on its corresponding input parameters of larger sample sizes and their responses of interest. The high-level SVR model is built based on the obtained responses from the low-level SVR models and the input parameters of smaller sample sizes. Several numerical examples are used to validate the performance of HL-SVR. The experimental results indicate that HL-SVR can produce more accurate prediction results than conventional SVR. The proposed approach is applied on the stress analysis of dental implant, which the structural parameters have massive samples but the material of implant can only be selected from several Ti and its alloys. The prediction performance of the proposed approach is much better than the conventional SVR. The proposed approach can be used for the design, optimization and analysis of engineering systems with input parameters of unequal sample sizes.
Tasks
Published	2019-05-31
URL	https://arxiv.org/abs/1906.05777v1
PDF	https://arxiv.org/pdf/1906.05777v1.pdf
PWC	https://paperswithcode.com/paper/high-low-level-support-vector-regression
Repo
Framework

Action Anticipation with RBF Kernelized Feature Mapping RNN


Title	Action Anticipation with RBF Kernelized Feature Mapping RNN
Authors	Yuge Shi, Basura Fernando, Richard Hartley
Abstract	We introduce a novel Recurrent Neural Network-based algorithm for future video feature generation and action anticipation called feature mapping RNN. Our novel RNN architecture builds upon three effective principles of machine learning, namely parameter sharing, Radial Basis Function kernels and adversarial training. Using only some of the earliest frames of a video, the feature mapping RNN is able to generate future features with a fraction of the parameters needed in traditional RNN. By feeding these future features into a simple multi-layer perceptron facilitated with an RBF kernel layer, we are able to accurately predict the action in the video. In our experiments, we obtain 18% improvement on JHMDB-21 dataset, 6% on UCF101-24 and 13% improvement on UT-Interaction datasets over prior state-of-the-art for action anticipation.
Tasks
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07806v2
PDF	https://arxiv.org/pdf/1911.07806v2.pdf
PWC	https://paperswithcode.com/paper/action-anticipation-with-rbf
Repo
Framework

Can adversarial training learn image captioning ?


Title	Can adversarial training learn image captioning ?
Authors	Jean-Benoit Delbrouck, Bastien Vanderplaetse, Stéphane Dupont
Abstract	Recently, generative adversarial networks (GAN) have gathered a lot of interest. Their efficiency in generating unseen samples of high quality, especially images, has improved over the years. In the field of Natural Language Generation (NLG), the use of the adversarial setting to generate meaningful sentences has shown to be difficult for two reasons: the lack of existing architectures to produce realistic sentences and the lack of evaluation tools. In this paper, we propose an adversarial architecture related to the conditional GAN (cGAN) that generates sentences according to a given image (also called image captioning). This attempt is the first that uses no pre-training or reinforcement methods. We also explain why our experiment settings can be safely evaluated and interpreted for further works.
Tasks	Image Captioning, Text Generation
Published	2019-10-31
URL	https://arxiv.org/abs/1910.14609v1
PDF	https://arxiv.org/pdf/1910.14609v1.pdf
PWC	https://paperswithcode.com/paper/can-adversarial-training-learn-image
Repo
Framework

Towards Interlingua Neural Machine Translation


Title	Towards Interlingua Neural Machine Translation
Authors	Carlos Escolano, Marta R. Costa-jussà, José A. R. Fonollosa
Abstract	Common intermediate language representation in neural machine translation can be used to extend bilingual to multilingual systems by incremental training. In this paper, we propose a new architecture based on introducing an interlingual loss as an additional training objective. By adding and forcing this interlingual loss, we are able to train multiple encoders and decoders for each language, sharing a common intermediate representation. Translation results on the low-resourced tasks (Turkish-English and Kazakh-English tasks, from the popular Workshop on Machine Translation benchmark) show the following BLEU improvements up to 2.8. However, results on a larger dataset (Russian-English and Kazakh-English, from the same baselines) show BLEU loses if the same amount. While our system is only providing improvements for the low-resourced tasks in terms of translation quality, our system is capable of quickly deploying new language pairs without retraining the rest of the system, which may be a game-changer in some situations (i.e. in a disaster crisis where international help is required towards a small region or to develop some translation system for a client). Precisely, what is most relevant from our architecture is that it is capable of: (1) reducing the number of production systems, with respect to the number of languages, from quadratic to linear (2) incrementally adding a new language in the system without retraining languages previously there and (3) allowing for translations from the new language to all the others present in the system
Tasks	Machine Translation
Published	2019-05-15
URL	https://arxiv.org/abs/1905.06831v2
PDF	https://arxiv.org/pdf/1905.06831v2.pdf
PWC	https://paperswithcode.com/paper/towards-interlingua-neural-machine
Repo
Framework

Event-Based Modeling with High-Dimensional Imaging Biomarkers for Estimating Spatial Progression of Dementia


Title	Event-Based Modeling with High-Dimensional Imaging Biomarkers for Estimating Spatial Progression of Dementia
Authors	Vikram Venkatraghavan, Florian Dubost, Esther E. Bron, Wiro J. Niessen, Marleen de Bruijne, Stefan Klein
Abstract	Event-based models (EBM) are a class of disease progression models that can be used to estimate temporal ordering of neuropathological changes from cross-sectional data. Current EBMs only handle scalar biomarkers, such as regional volumes, as inputs. However, regional aggregates are a crude summary of the underlying high-resolution images, potentially limiting the accuracy of EBM. Therefore, we propose a novel method that exploits high-dimensional voxel-wise imaging biomarkers: n-dimensional discriminative EBM (nDEBM). nDEBM is based on an insight that mixture modeling, which is a key element of conventional EBMs, can be replaced by a more scalable semi-supervised support vector machine (SVM) approach. This SVM is used to estimate the degree of abnormality of each region which is then used to obtain subject-specific disease progression patterns. These patterns are in turn used for estimating the mean ordering by fitting a generalized Mallows model. In order to validate the biomarker ordering obtained using nDEBM, we also present a framework for Simulation of Imaging Biomarkers’ Temporal Evolution (SImBioTE) that mimics neurodegeneration in brain regions. SImBioTE trains variational auto-encoders (VAE) in different brain regions independently to simulate images at varying stages of disease progression. We also validate nDEBM clinically using data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). In both experiments, nDEBM using high-dimensional features gave better performance than state-of-the-art EBM methods using regional volume biomarkers. This suggests that nDEBM is a promising approach for disease progression modeling.
Tasks
Published	2019-03-08
URL	http://arxiv.org/abs/1903.03386v1
PDF	http://arxiv.org/pdf/1903.03386v1.pdf
PWC	https://paperswithcode.com/paper/event-based-modeling-with-high-dimensional
Repo
Framework

A Hierarchical Optimizer for Recommendation System Based on Shortest Path Algorithm


Title	A Hierarchical Optimizer for Recommendation System Based on Shortest Path Algorithm
Authors	Jiacheng Dai, Zhifeng Jia, Xiaofeng Gao, Guihai Chen
Abstract	Top-k Nearest Geosocial Keyword (T-kNGK) query on geosocial network is defined to give users k recommendations based on some keywords and designated spatial range, and can be realized by shortest path algorithms. However, shortest path algorithm cannot provide convincing recommendations, so we design a hierarchical optimizer consisting of classifiers and a constant optimizer to optimize the result by some features of the service providers.
Tasks
Published	2019-11-07
URL	https://arxiv.org/abs/1911.08994v1
PDF	https://arxiv.org/pdf/1911.08994v1.pdf
PWC	https://paperswithcode.com/paper/a-hierarchical-optimizer-for-recommendation
Repo
Framework

Sparse regular variation


Title	Sparse regular variation
Authors	Nicolas Meyer, Olivier Wintenberger
Abstract	Regular variation provides a convenient theoretical framework to study large events. In the multivariate setting, the dependence structure of the positive extremes is characterized by a measure - the spectral measure - defined on the positive orthant of the unit sphere. This measure gathers information on the localization of extreme events and is often sparse since severe events do not simultaneously occur in all directions. Unfortunately, it is defined through weak convergence which does not provide a natural way to capture this sparsity structure.In this paper, we introduce the notion of sparse regular variation which allows to better learn the dependence structure of extreme events. This concept is based on the Euclidean projection onto the simplex for which efficient algorithms are known. We show several results for sparsely regularly varying random vectors and prove that under mild assumptions sparse regular variation and regular variation are two equivalent notions. Finally, we provide numerical evidence of our theoretical findings and compare our method with a recent one developed by Goix et al. (2017).
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00686v2
PDF	https://arxiv.org/pdf/1907.00686v2.pdf
PWC	https://paperswithcode.com/paper/sparse-regular-variation
Repo
Framework

Compressing Weight-updates for Image Artifacts Removal Neural Networks


Title	Compressing Weight-updates for Image Artifacts Removal Neural Networks
Authors	Yat Hong Lam, Alireza Zare, Caglar Aytekin, Francesco Cricri, Jani Lainema, Emre Aksu, Miska Hannuksela
Abstract	In this paper, we present a novel approach for fine-tuning a decoder-side neural network in the context of image compression, such that the weight-updates are better compressible. At encoder side, we fine-tune a pre-trained artifact removal network on target data by using a compression objective applied on the weight-update. In particular, the compression objective encourages weight-updates which are sparse and closer to quantized values. This way, the final weight-update can be compressed more efficiently by pruning and quantization, and can be included into the encoded bitstream together with the image bitstream of a traditional codec. We show that this approach achieves reconstruction quality which is on-par or slightly superior to a traditional codec, at comparable bitrates. To our knowledge, this is the first attempt to combine image compression and neural network’s weight update compression.
Tasks	Image Compression, Quantization
Published	2019-05-10
URL	https://arxiv.org/abs/1905.04079v2
PDF	https://arxiv.org/pdf/1905.04079v2.pdf
PWC	https://paperswithcode.com/paper/compressing-weight-updates-for-image
Repo
Framework

Improving Automatic Jazz Melody Generation by Transfer Learning Techniques


Title	Improving Automatic Jazz Melody Generation by Transfer Learning Techniques
Authors	Hsiao-Tzu Hung, Chung-Yang Wang, Yi-Hsuan Yang, Hsin-Min Wang
Abstract	In this paper, we tackle the problem of transfer learning for Jazz automatic generation. Jazz is one of representative types of music, but the lack of Jazz data in the MIDI format hinders the construction of a generative model for Jazz. Transfer learning is an approach aiming to solve the problem of data insufficiency, so as to transfer the common feature from one domain to another. In view of its success in other machine learning problems, we investigate whether, and how much, it can help improve automatic music generation for under-resourced musical genres. Specifically, we use a recurrent variational autoencoder as the generative model, and use a genre-unspecified dataset as the source dataset and a Jazz-only dataset as the target dataset. Two transfer learning methods are evaluated using six levels of source-to-target data ratios. The first method is to train the model on the source dataset, and then fine-tune the resulting model parameters on the target dataset. The second method is to train the model on both the source and target datasets at the same time, but add genre labels to the latent vectors and use a genre classifier to improve Jazz generation. The evaluation results show that the second method seems to perform better overall, but it cannot take full advantage of the genre-unspecified dataset.
Tasks	Music Generation, Transfer Learning
Published	2019-08-26
URL	https://arxiv.org/abs/1908.09484v1
PDF	https://arxiv.org/pdf/1908.09484v1.pdf
PWC	https://paperswithcode.com/paper/improving-automatic-jazz-melody-generation-by
Repo
Framework

Inexact Primal-Dual Gradient Projection Methods for Nonlinear Optimization on Convex Set


Title	Inexact Primal-Dual Gradient Projection Methods for Nonlinear Optimization on Convex Set
Authors	Fan Zhang, Hao Wang, Jiashan Wang, Kai Yang
Abstract	In this paper, we propose a novel primal-dual inexact gradient projection method for nonlinear optimization problems with convex-set constraint. This method only needs inexact computation of the projections onto the convex set for each iteration, consequently reducing the computational cost for projections per iteration. This feature is attractive especially for solving problems where the projections are computationally not easy to calculate. Global convergence guarantee and O(1/k) ergodic convergence rate of the optimality residual are provided under loose assumptions. We apply our proposed strategy to l1-ball constrained problems. Numerical results exhibit that our inexact gradient projection methods for solving l1-ball constrained problems are more efficient than the exact methods.
Tasks
Published	2019-11-18
URL	https://arxiv.org/abs/1911.07758v1
PDF	https://arxiv.org/pdf/1911.07758v1.pdf
PWC	https://paperswithcode.com/paper/inexact-primal-dual-gradient-projection
Repo
Framework

An Exponential Learning Rate Schedule for Deep Learning


Title	An Exponential Learning Rate Schedule for Deep Learning
Authors	Zhiyuan Li, Sanjeev Arora
Abstract	Intriguing empirical evidence exists that deep learning can work well with exoticschedules for varying the learning rate. This paper suggests that the phenomenon may be due to Batch Normalization or BN, which is ubiquitous and provides benefits in optimization and generalization across all standard architectures. The following new results are shown about BN with weight decay and momentum (in other words, the typical use case which was not considered in earlier theoretical analyses of stand-alone BN. 1. Training can be done using SGD with momentum and an exponentially increasing learning rate schedule, i.e., learning rate increases by some $(1 +\alpha)$ factor in every epoch for some $\alpha >0$. (Precise statement in the paper.) To the best of our knowledge this is the first time such a rate schedule has been successfully used, let alone for highly successful architectures. As expected, such training rapidly blows up network weights, but the net stays well-behaved due to normalization. 2. Mathematical explanation of the success of the above rate schedule: a rigorous proof that it is equivalent to the standard setting of BN + SGD + StandardRate Tuning + Weight Decay + Momentum. This equivalence holds for other normalization layers as well, Group Normalization, LayerNormalization, Instance Norm, etc. 3. A worked-out toy example illustrating the above linkage of hyper-parameters. Using either weight decay or BN alone reaches global minimum, but convergence fails when both are used.
Tasks
Published	2019-10-16
URL	https://arxiv.org/abs/1910.07454v3
PDF	https://arxiv.org/pdf/1910.07454v3.pdf
PWC	https://paperswithcode.com/paper/an-exponential-learning-rate-schedule-for-1
Repo
Framework

Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification


Title	Analysis of Video Feature Learning in Two-Stream CNNs on the Example of Zebrafish Swim Bout Classification
Authors	Bennet Breier, Arno Onken
Abstract	Semmelhack et al. (2014) have achieved high classification accuracy in distinguishing swim bouts of zebrafish using a Support Vector Machine (SVM). Convolutional Neural Networks (CNNs) have reached superior performance in various image recognition tasks over SVMs, but these powerful networks remain a black box. Reaching better transparency helps to build trust in their classifications and makes learned features interpretable to experts. Using a recently developed technique called Deep Taylor Decomposition, we generated heatmaps to highlight input regions of high relevance for predictions. We find that our CNN makes predictions by analyzing the steadiness of the tail’s trunk, which markedly differs from the manually extracted features used by Semmelhack et al. (2014). We further uncovered that the network paid attention to experimental artifacts. Removing these artifacts ensured the validity of predictions. After correction, our best CNN beats the SVM by 6.12%, achieving a classification accuracy of 96.32%. Our work thus demonstrates the utility of AI explainability for CNNs.
Tasks
Published	2019-12-20
URL	https://arxiv.org/abs/1912.09857v1
PDF	https://arxiv.org/pdf/1912.09857v1.pdf
PWC	https://paperswithcode.com/paper/analysis-of-video-feature-learning-in-two-1
Repo
Framework

Dual-domain Cascade of U-nets for Multi-channel Magnetic Resonance Image Reconstruction


Title	Dual-domain Cascade of U-nets for Multi-channel Magnetic Resonance Image Reconstruction
Authors	Roberto Souza, Mariana Bento, Nikita Nogovitsyn, Kevin J. Chung, R. Marc Lebel, Richard Frayne
Abstract	The U-net is a deep-learning network model that has been used to solve a number of inverse problems. In this work, the concatenation of two-element U-nets, termed the W-net, operating in k-space (K) and image (I) domains, were evaluated for multi-channel magnetic resonance (MR) image reconstruction. The two element network combinations were evaluated for the four possible image-k-space domain configurations: a) W-net II, b) W-net KK, c) W-net IK, and d) W-net KI were evaluated. Selected promising four element networks (WW-nets) were also examined. Two configurations of each network were compared: 1) Each coil channel processed independently, and 2) all channels processed simultaneously. One hundred and eleven volumetric, T1-weighted, 12-channel coil k-space datasets were used in the experiments. Normalized root mean squared error, peak signal to noise ratio, visual information fidelity and visual inspection were used to assess the reconstructed images against the fully sampled reference images. Our results indicated that networks that operate solely in the image domain are better suited when processing individual channels of multi-channel data independently. Dual domain methods are more advantageous when simultaneously reconstructing all channels of multi-channel data. Also, the appropriate cascade of U-nets compared favorably (p < 0.01) to the previously published, state-of-the-art Deep Cascade model in in three out of four experiments.
Tasks	Image Reconstruction
Published	2019-11-04
URL	https://arxiv.org/abs/1911.01458v1
PDF	https://arxiv.org/pdf/1911.01458v1.pdf
PWC	https://paperswithcode.com/paper/dual-domain-cascade-of-u-nets-for-multi
Repo
Framework

Inherent Tradeoffs in Learning Fair Representations


Title	Inherent Tradeoffs in Learning Fair Representations
Authors	Han Zhao, Geoffrey J. Gordon
Abstract	With the prevalence of machine learning in high-stakes applications, especially the ones regulated by anti-discrimination laws or societal norms, it is crucial to ensure that the predictive models do not propagate any existing bias or discrimination. Due to the ability of deep neural nets to learn rich representations, recent advances in algorithmic fairness have focused on learning fair representations with adversarial techniques to reduce bias in data while preserving utility simultaneously. In this paper, through the lens of information theory, we provide the first result that quantitatively characterizes the tradeoff between demographic parity and the joint utility across different population groups. Specifically, when the base rates differ between groups, we show that any method aiming to learn fair representations admits an information-theoretic lower bound on the joint error across these groups. To complement our negative results, we also prove that if the optimal decision functions across different groups are close, then learning fair representations leads to an alternative notion of fairness, known as the accuracy parity, which states that the error rates are close between groups. Finally, our theoretical findings are also confirmed empirically on real-world datasets.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08386v2
PDF	https://arxiv.org/pdf/1906.08386v2.pdf
PWC	https://paperswithcode.com/paper/inherent-tradeoffs-in-learning-fair
Repo
Framework

Pick-and-Learn: Automatic Quality Evaluation for Noisy-Labeled Image Segmentation


Title	Pick-and-Learn: Automatic Quality Evaluation for Noisy-Labeled Image Segmentation
Authors	Haidong Zhu, Jialin Shi, Ji Wu
Abstract	Deep learning methods have achieved promising performance in many areas, but they are still struggling with noisy-labeled images during the training process. Considering that the annotation quality indispensably relies on great expertise, the problem is even more crucial in the medical image domain. How to eliminate the disturbance from noisy labels for segmentation tasks without further annotations is still a significant challenge. In this paper, we introduce our label quality evaluation strategy for deep neural networks automatically assessing the quality of each label, which is not explicitly provided, and training on clean-annotated ones. We propose a solution for network automatically evaluating the relative quality of the labels in the training set and using good ones to tune the network parameters. We also design an overfitting control module to let the network maximally learn from the precise annotations during the training process. Experiments on the public biomedical image segmentation dataset have proved the method outperforms baseline methods and retains both high accuracy and good generalization at different noise levels.
Tasks	Semantic Segmentation
Published	2019-07-27
URL	https://arxiv.org/abs/1907.11835v1
PDF	https://arxiv.org/pdf/1907.11835v1.pdf
PWC	https://paperswithcode.com/paper/pick-and-learn-automatic-quality-evaluation
Repo
Framework