October 21, 2019

3439 words 17 mins read

Paper Group AWR 55

Molecular generative model based on conditional variational autoencoder for de novo molecular design. The Importance of Being Recurrent for Modeling Hierarchical Structure. A Differential Privacy Mechanism Design Under Matrix-Valued Query. Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning. Proximity Forest: An effective …

Molecular generative model based on conditional variational autoencoder for de novo molecular design


Title	Molecular generative model based on conditional variational autoencoder for de novo molecular design
Authors	Jaechang Lim, Seongok Ryu, Jin Woo Kim, Woo Youn Kim
Abstract	We propose a molecular generative model based on the conditional variational autoencoder for de novo molecular design. It is specialized to control multiple molecular properties simultaneously by imposing them on a latent space. As a proof of concept, we demonstrate that it can be used to generate drug-like molecules with five target properties. We were also able to adjust a single property without changing the others and to manipulate it beyond the range of the dataset.
Tasks
Published	2018-06-15
URL	http://arxiv.org/abs/1806.05805v1
PDF	http://arxiv.org/pdf/1806.05805v1.pdf
PWC	https://paperswithcode.com/paper/molecular-generative-model-based-on
Repo	https://github.com/jaechanglim/CVAE
Framework	tf

The Importance of Being Recurrent for Modeling Hierarchical Structure


Title	The Importance of Being Recurrent for Modeling Hierarchical Structure
Authors	Ke Tran, Arianna Bisazza, Christof Monz
Abstract	Recent work has shown that recurrent neural networks (RNNs) can implicitly capture and exploit hierarchical information when trained to solve common natural language processing tasks such as language modeling (Linzen et al., 2016) and neural machine translation (Shi et al., 2016). In contrast, the ability to model structured data with non-recurrent neural networks has received little attention despite their success in many NLP tasks (Gehring et al., 2017; Vaswani et al., 2017). In this work, we compare the two architectures—recurrent versus non-recurrent—with respect to their ability to model hierarchical structure and find that recurrency is indeed important for this purpose.
Tasks	Language Modelling, Machine Translation
Published	2018-03-09
URL	http://arxiv.org/abs/1803.03585v2
PDF	http://arxiv.org/pdf/1803.03585v2.pdf
PWC	https://paperswithcode.com/paper/the-importance-of-being-recurrent-for
Repo	https://github.com/ketranm/fan_vs_rnn
Framework	pytorch

A Differential Privacy Mechanism Design Under Matrix-Valued Query


Title	A Differential Privacy Mechanism Design Under Matrix-Valued Query
Authors	Thee Chanyaswad, Alex Dytso, H. Vincent Poor, Prateek Mittal
Abstract	Traditionally, differential privacy mechanism design has been tailored for a scalar-valued query function. Although many mechanisms such as the Laplace and Gaussian mechanisms can be extended to a matrix-valued query function by adding i.i.d. noise to each element of the matrix, this method is often sub-optimal as it forfeits an opportunity to exploit the structural characteristics typically associated with matrix analysis. In this work, we consider the design of differential privacy mechanism specifically for a matrix-valued query function. The proposed solution is to utilize a matrix-variate noise, as opposed to the traditional scalar-valued noise. Particularly, we propose a novel differential privacy mechanism called the Matrix-Variate Gaussian (MVG) mechanism, which adds a matrix-valued noise drawn from a matrix-variate Gaussian distribution. We prove that the MVG mechanism preserves $(\epsilon,\delta)$-differential privacy, and show that it allows the structural characteristics of the matrix-valued query function to naturally be exploited. Furthermore, due to the multi-dimensional nature of the MVG mechanism and the matrix-valued query, we introduce the concept of directional noise, which can be utilized to mitigate the impact the noise has on the utility of the query. Finally, we demonstrate the performance of the MVG mechanism and the advantages of directional noise using three matrix-valued queries on three privacy-sensitive datasets. We find that the MVG mechanism notably outperforms four previous state-of-the-art approaches, and provides comparable utility to the non-private baseline. Our work thus presents a promising prospect for both future research and implementation of differential privacy for matrix-valued query functions.
Tasks
Published	2018-02-26
URL	http://arxiv.org/abs/1802.10077v1
PDF	http://arxiv.org/pdf/1802.10077v1.pdf
PWC	https://paperswithcode.com/paper/a-differential-privacy-mechanism-design-under
Repo	https://github.com/inspire-group/MVG-Mechansim
Framework	none

Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning


Title	Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning
Authors	Vinod Kumar Chauhan, Anuj Sharma, Kalpana Dahiya
Abstract	Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this paper, we have proposed a novel Stochastic Trust RegiOn Inexact Newton method, called as STRON, to solve large-scale learning problems which uses conjugate gradient (CG) to inexactly solve trust region subproblem. The method uses progressive subsampling in the calculation of gradient and Hessian values to take the advantage of both, stochastic and full-batch regimes. We have extended STRON using existing variance reduction techniques to deal with the noisy gradients and using preconditioned conjugate gradient (PCG) as subproblem solver, and empirically proved that they do not work as expected, for the large-scale learning problems. Finally, our empirical results prove efficacy of the proposed method against existing methods with bench marked datasets.
Tasks
Published	2018-12-26
URL	https://arxiv.org/abs/1812.10426v3
PDF	https://arxiv.org/pdf/1812.10426v3.pdf
PWC	https://paperswithcode.com/paper/stochastic-trust-region-inexact-newton-method
Repo	https://github.com/jmdvinodjmd/LIBS2ML
Framework	none

Proximity Forest: An effective and scalable distance-based classifier for time series


Title	Proximity Forest: An effective and scalable distance-based classifier for time series
Authors	Benjamin Lucas, Ahmed Shifaz, Charlotte Pelletier, Lachlan O’Neill, Nayyar Zaidi, Bart Goethals, Francois Petitjean, Geoffrey I. Webb
Abstract	Research into the classification of time series has made enormous progress in the last decade. The UCR time series archive has played a significant role in challenging and guiding the development of new learners for time series classification. The largest dataset in the UCR archive holds 10 thousand time series only; which may explain why the primary research focus has been in creating algorithms that have high accuracy on relatively small datasets. This paper introduces Proximity Forest, an algorithm that learns accurate models from datasets with millions of time series, and classifies a time series in milliseconds. The models are ensembles of highly randomized Proximity Trees. Whereas conventional decision trees branch on attribute values (and usually perform poorly on time series), Proximity Trees branch on the proximity of time series to one exemplar time series or another; allowing us to leverage the decades of work into developing relevant measures for time series. Proximity Forest gains both efficiency and accuracy by stochastic selection of both exemplars and similarity measures. Our work is motivated by recent time series applications that provide orders of magnitude more time series than the UCR benchmarks. Our experiments demonstrate that Proximity Forest is highly competitive on the UCR archive: it ranks among the most accurate classifiers while being significantly faster. We demonstrate on a 1M time series Earth observation dataset that Proximity Forest retains this accuracy on datasets that are many orders of magnitude greater than those in the UCR repository, while learning its models at least 100,000 times faster than current state of the art models Elastic Ensemble and COTE.
Tasks	Time Series, Time Series Classification
Published	2018-08-31
URL	http://arxiv.org/abs/1808.10594v2
PDF	http://arxiv.org/pdf/1808.10594v2.pdf
PWC	https://paperswithcode.com/paper/proximity-forest-an-effective-and-scalable
Repo	https://github.com/fpetitjean/ProximityForest
Framework	none

Interpretable Time Series Classification using All-Subsequence Learning and Symbolic Representations in Time and Frequency Domains


Title	Interpretable Time Series Classification using All-Subsequence Learning and Symbolic Representations in Time and Frequency Domains
Authors	Thach Le Nguyen, Severin Gsponer, Iulia Ilie, Georgiana Ifrim
Abstract	The time series classification literature has expanded rapidly over the last decade, with many new classification approaches published each year. The research focus has mostly been on improving the accuracy and efficiency of classifiers, while their interpretability has been somewhat neglected. Classifier interpretability has become a critical constraint for many application domains and the introduction of the ‘right to explanation’ GDPR EU legislation in May 2018 is likely to further emphasize the importance of explainable learning algorithms. In this work we analyse the state-of-the-art for time series classification, and propose new algorithms that aim to maintain the classifier accuracy and efficiency, but keep interpretability as a key design constraint. We present new time series classification algorithms that advance the state-of-the-art by implementing the following three key ideas: (1) Multiple resolutions of symbolic approximations: we combine symbolic representations obtained using different parameters; (2) Multiple domain representations: we combine symbolic approximations in time (e.g., SAX) and frequency (e.g., SFA) domains; (3) Efficient navigation of a huge symbolic-words space: we adapt a symbolic sequence classifier named SEQL, to make it work with multiple domain representations (e.g., SAX-SEQL, SFA-SEQL), and use its greedy feature selection strategy to effectively filter the best features for each representation. We show that a multi-resolution multi-domain linear classifier, SAX-SFA-SEQL, achieves a similar accuracy to the state-of-the-art COTE ensemble, and to a recent deep learning method (FCN), but uses a fraction of the time required by either COTE or FCN. We discuss the accuracy, efficiency and interpretability of our proposed algorithms. To further analyse the interpretability aspect of our classifiers, we present a case study on an ecology benchmark.
Tasks	Feature Selection, Time Series, Time Series Classification
Published	2018-08-12
URL	http://arxiv.org/abs/1808.04022v1
PDF	http://arxiv.org/pdf/1808.04022v1.pdf
PWC	https://paperswithcode.com/paper/interpretable-time-series-classification
Repo	https://github.com/lnthach/Mr-SEQL
Framework	none

Y-Net: A deep Convolutional Neural Network for Polyp Detection


Title	Y-Net: A deep Convolutional Neural Network for Polyp Detection
Authors	Ahmed Mohammed, Sule Yildirim, Ivar Farup, Marius Pedersen, Øistein Hovde
Abstract	Colorectal polyps are important precursors to colon cancer, the third most common cause of cancer mortality for both men and women. It is a disease where early detection is of crucial importance. Colonoscopy is commonly used for early detection of cancer and precancerous pathology. It is a demanding procedure requiring significant amount of time from specialized physicians and nurses, in addition to a significant miss-rates of polyps by specialists. Automated polyp detection in colonoscopy videos has been demonstrated to be a promising way to handle this problem. {However, polyps detection is a challenging problem due to the availability of limited amount of training data and large appearance variations of polyps. To handle this problem, we propose a novel deep learning method Y-Net that consists of two encoder networks with a decoder network. Our proposed Y-Net method} relies on efficient use of pre-trained and un-trained models with novel sum-skip-concatenation operations. Each of the encoders are trained with encoder specific learning rate along the decoder. Compared with the previous methods employing hand-crafted features or 2-D/3-D convolutional neural network, our approach outperforms state-of-the-art methods for polyp detection with 7.3% F1-score and 13% recall improvement.
Tasks
Published	2018-06-05
URL	http://arxiv.org/abs/1806.01907v1
PDF	http://arxiv.org/pdf/1806.01907v1.pdf
PWC	https://paperswithcode.com/paper/y-net-a-deep-convolutional-neural-network-for
Repo	https://github.com/ahme0307/Ynet
Framework	none

Refining interaction search through signed iterative Random Forests


Title	Refining interaction search through signed iterative Random Forests
Authors	Karl Kumbier, Sumanta Basu, James B. Brown, Susan Celniker, Bin Yu
Abstract	Advances in supervised learning have enabled accurate prediction in biological systems governed by complex interactions among biomolecules. However, state-of-the-art predictive algorithms are typically black-boxes, learning statistical interactions that are difficult to translate into testable hypotheses. The iterative Random Forest algorithm took a step towards bridging this gap by providing a computationally tractable procedure to identify the stable, high-order feature interactions that drive the predictive accuracy of Random Forests (RF). Here we refine the interactions identified by iRF to explicitly map responses as a function of interacting features. Our method, signed iRF, describes subsets of rules that frequently occur on RF decision paths. We refer to these rule subsets as signed interactions. Signed interactions share not only the same set of interacting features but also exhibit similar thresholding behavior, and thus describe a consistent functional relationship between interacting features and responses. We describe stable and predictive importance metrics to rank signed interactions. For each SPIM, we define null importance metrics that characterize its expected behavior under known structure. We evaluate our proposed approach in biologically inspired simulations and two case studies: predicting enhancer activity and spatial gene expression patterns. In the case of enhancer activity, s-iRF recovers one of the few experimentally validated high-order interactions and suggests novel enhancer elements where this interaction may be active. In the case of spatial gene expression patterns, s-iRF recovers all 11 reported links in the gap gene network. By refining the process of interaction recovery, our approach has the potential to guide mechanistic inquiry into systems whose scale and complexity is beyond human comprehension.
Tasks
Published	2018-10-16
URL	http://arxiv.org/abs/1810.07287v1
PDF	http://arxiv.org/pdf/1810.07287v1.pdf
PWC	https://paperswithcode.com/paper/refining-interaction-search-through-signed
Repo	https://github.com/sumbose/iRF
Framework	none

TD-Regularized Actor-Critic Methods


Title	TD-Regularized Actor-Critic Methods
Authors	Simone Parisi, Voot Tangkaratt, Jan Peters, Mohammad Emtiyaz Khan
Abstract	Actor-critic methods can achieve incredible performance on difficult reinforcement learning problems, but they are also prone to instability. This is partly due to the interaction between the actor and critic during learning, e.g., an inaccurate step taken by one of them might adversely affect the other and destabilize the learning. To avoid such issues, we propose to regularize the learning objective of the actor by penalizing the temporal difference (TD) error of the critic. This improves stability by avoiding large steps in the actor update whenever the critic is highly inaccurate. The resulting method, which we call the TD-regularized actor-critic method, is a simple plug-and-play approach to improve stability and overall performance of the actor-critic methods. Evaluations on standard benchmarks confirm this.
Tasks
Published	2018-12-19
URL	http://arxiv.org/abs/1812.08288v3
PDF	http://arxiv.org/pdf/1812.08288v3.pdf
PWC	https://paperswithcode.com/paper/td-regularized-actor-critic-methods
Repo	https://github.com/sparisi/td-reg
Framework	tf

Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments


Title	Autonomous Deep Learning: Continual Learning Approach for Dynamic Environments
Authors	Andri Ashfahani, Mahardhika Pratama
Abstract	The feasibility of deep neural networks (DNNs) to address data stream problems still requires intensive study because of the static and offline nature of conventional deep learning approaches. A deep continual learning algorithm, namely autonomous deep learning (ADL), is proposed in this paper. Unlike traditional deep learning methods, ADL features a flexible structure where its network structure can be constructed from scratch with the absence of an initial network structure via the self-constructing network structure. ADL specifically addresses catastrophic forgetting by having a different-depth structure which is capable of achieving a trade-off between plasticity and stability. Network significance (NS) formula is proposed to drive the hidden nodes growing and pruning mechanism. Drift detection scenario (DDS) is put forward to signal distributional changes in data streams which induce the creation of a new hidden layer. The maximum information compression index (MICI) method plays an important role as a complexity reduction module eliminating redundant layers. The efficacy of ADL is numerically validated under the prequential test-then-train procedure in lifelong environments using nine popular data stream problems. The numerical results demonstrate that ADL consistently outperforms recent continual learning methods while characterizing the automatic construction of network structures.
Tasks	Continual Learning
Published	2018-10-17
URL	https://arxiv.org/abs/1810.07348v4
PDF	https://arxiv.org/pdf/1810.07348v4.pdf
PWC	https://paperswithcode.com/paper/autonomous-deep-learning-continual-learning
Repo	https://github.com/SeptivianaSavitri/adl_python
Framework	none

An Evaluation of Methods for Real-Time Anomaly Detection using Force Measurements from the Turning Process


Title	An Evaluation of Methods for Real-Time Anomaly Detection using Force Measurements from the Turning Process
Authors	Yuanzhi Huang, Eamonn Ahearne, Szymon Baron, Andrew Parnell
Abstract	We examined the use of three conventional anomaly detection methods and assess their potential for on-line tool wear monitoring. Through efficient data processing and transformation of the algorithm proposed here, in a real-time environment, these methods were tested for fast evaluation of cutting tools on CNC machines. The three-dimensional force data streams we used were extracted from a turning experiment of 21 runs for which a tool was run until it generally satisfied an end-of-life criterion. Our real-time anomaly detection algorithm was scored and optimised according to how precisely it can predict the progressive wear of the tool flank. Most of our tool wear predictions were accurate and reliable as illustrated in our off-line simulation results. Particularly when the multivariate analysis was applied, the algorithm we develop was found to be very robust across different scenarios and against parameter changes. It shall be reasonably easy to apply our approach elsewhere for real-time tool wear analytics.
Tasks	Anomaly Detection
Published	2018-12-20
URL	http://arxiv.org/abs/1812.09178v1
PDF	http://arxiv.org/pdf/1812.09178v1.pdf
PWC	https://paperswithcode.com/paper/an-evaluation-of-methods-for-real-time
Repo	https://github.com/Yuanzhi-H/Real-Time-Detection-Methods
Framework	none

A Closed-form Solution to Photorealistic Image Stylization


Title	A Closed-form Solution to Photorealistic Image Stylization
Authors	Yijun Li, Ming-Yu Liu, Xueting Li, Ming-Hsuan Yang, Jan Kautz
Abstract	Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closed-form solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle .
Tasks	Image Stylization
Published	2018-02-19
URL	http://arxiv.org/abs/1802.06474v5
PDF	http://arxiv.org/pdf/1802.06474v5.pdf
PWC	https://paperswithcode.com/paper/a-closed-form-solution-to-photorealistic
Repo	https://github.com/smaranjitghose/DeepHoli
Framework	none

Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition


Title	Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition
Authors	Ming Sun, Yuchen Yuan, Feng Zhou, Errui Ding
Abstract	Attention-based learning for fine-grained image recognition remains a challenging task, where most of the existing methods treat each object part in isolation, while neglecting the correlations among them. In addition, the multi-stage or multi-scale mechanisms involved make the existing methods less efficient and hard to be trained end-to-end. In this paper, we propose a novel attention-based convolutional neural network (CNN) which regulates multiple object parts among different input images. Our method first learns multiple attention region features of each input image through the one-squeeze multi-excitation (OSME) module, and then apply the multi-attention multi-class constraint (MAMC) in a metric learning framework. For each anchor feature, the MAMC functions by pulling same-attention same-class features closer, while pushing different-attention or different-class features away. Our method can be easily trained end-to-end, and is highly efficient which requires only one training stage. Moreover, we introduce Dogs-in-the-Wild, a comprehensive dog species dataset that surpasses similar existing datasets by category coverage, data volume and annotation quality. This dataset will be released upon acceptance to facilitate the research of fine-grained image recognition. Extensive experiments are conducted to show the substantial improvements of our method on four benchmark datasets.
Tasks	Fine-Grained Image Recognition, Metric Learning
Published	2018-06-14
URL	http://arxiv.org/abs/1806.05372v1
PDF	http://arxiv.org/pdf/1806.05372v1.pdf
PWC	https://paperswithcode.com/paper/multi-attention-multi-class-constraint-for
Repo	https://github.com/xcnkx/fine_grained_classification
Framework	tf

Toward Characteristic-Preserving Image-based Virtual Try-On Network


Title	Toward Characteristic-Preserving Image-based Virtual Try-On Network
Authors	Bochao Wang, Huabin Zheng, Xiaodan Liang, Yimin Chen, Liang Lin, Meng Yang
Abstract	Image-based virtual try-on systems for fitting new in-shop clothes into a person image have attracted increasing research attention, yet is still challenging. A desirable pipeline should not only transform the target clothes into the most fitting shape seamlessly but also preserve well the clothes identity in the generated image, that is, the key characteristics (e.g. texture, logo, embroidery) that depict the original clothes. However, previous image-conditioned generation works fail to meet these critical requirements towards the plausible virtual try-on performance since they fail to handle large spatial misalignment between the input image and target clothes. Prior work explicitly tackled spatial deformation using shape context matching, but failed to preserve clothing details due to its coarse-to-fine strategy. In this work, we propose a new fully-learnable Characteristic-Preserving Virtual Try-On Network(CP-VTON) for addressing all real-world challenges in this task. First, CP-VTON learns a thin-plate spline transformation for transforming the in-shop clothes into fitting the body shape of the target person via a new Geometric Matching Module (GMM) rather than computing correspondences of interest points as prior works did. Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness. Extensive experiments on a fashion dataset demonstrate our CP-VTON achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.
Tasks
Published	2018-07-20
URL	http://arxiv.org/abs/1807.07688v3
PDF	http://arxiv.org/pdf/1807.07688v3.pdf
PWC	https://paperswithcode.com/paper/toward-characteristic-preserving-image-based
Repo	https://github.com/sergeywong/cp-vton
Framework	pytorch

Adversarial Regularizers in Inverse Problems


Title	Adversarial Regularizers in Inverse Problems
Authors	Sebastian Lunz, Ozan Öktem, Carola-Bibiane Schönlieb
Abstract	Inverse Problems in medical imaging and computer vision are traditionally solved using purely model-based methods. Among those variational regularization models are one of the most popular approaches. We propose a new framework for applying data-driven approaches to inverse problems, using a neural network as a regularization functional. The network learns to discriminate between the distribution of ground truth images and the distribution of unregularized reconstructions. Once trained, the network is applied to the inverse problem by solving the corresponding variational problem. Unlike other data-based approaches for inverse problems, the algorithm can be applied even if only unsupervised training data is available. Experiments demonstrate the potential of the framework for denoising on the BSDS dataset and for computed tomography reconstruction on the LIDC dataset.
Tasks	Denoising
Published	2018-05-29
URL	http://arxiv.org/abs/1805.11572v2
PDF	http://arxiv.org/pdf/1805.11572v2.pdf
PWC	https://paperswithcode.com/paper/adversarial-regularizers-in-inverse-problems
Repo	https://github.com/lunz-s/DeepAdverserialRegulariser
Framework	tf