October 19, 2019

3346 words 16 mins read

Paper Group ANR 106

DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score. Compositional Stochastic Average Gradient for Machine Learning and Related Applications. Towards a Semantic Perceptual Image Metric. SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation. Feasibility of Colon …

DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score


Title	DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score
Authors	Yuma Koizumi, Kenta Niwa, Yusuke Hioka, Kazunori Kobayashi, Yoichi Haneda
Abstract	We propose a training method for deep neural network (DNN)-based source enhancement to increase objective sound quality assessment (OSQA) scores such as the perceptual evaluation of speech quality (PESQ). In many conventional studies, DNNs have been used as a mapping function to estimate time-frequency masks and trained to minimize an analytically tractable objective function such as the mean squared error (MSE). Since OSQA scores have been used widely for sound-quality evaluation, constructing DNNs to increase OSQA scores would be better than using the minimum-MSE to create high-quality output signals. However, since most OSQA scores are not analytically tractable, \textit{i.e.}, they are black boxes, the gradient of the objective function cannot be calculated by simply applying back-propagation. To calculate the gradient of the OSQA-based objective function, we formulated a DNN optimization scheme on the basis of \textit{black-box optimization}, which is used for training a computer that plays a game. For a black-box-optimization scheme, we adopt the policy gradient method for calculating the gradient on the basis of a sampling algorithm. To simulate output signals using the sampling algorithm, DNNs are used to estimate the probability density function of the output signals that maximize OSQA scores. The OSQA scores are calculated from the simulated output signals, and the DNNs are trained to increase the probability of generating the simulated output signals that achieve high OSQA scores. Through several experiments, we found that OSQA scores significantly increased by applying the proposed method, even though the MSE was not minimized.
Tasks
Published	2018-10-22
URL	http://arxiv.org/abs/1810.09137v1
PDF	http://arxiv.org/pdf/1810.09137v1.pdf
PWC	https://paperswithcode.com/paper/dnn-based-source-enhancement-to-increase
Repo
Framework


Title	Compositional Stochastic Average Gradient for Machine Learning and Related Applications
Authors	Tsung-Yu Hsieh, Yasser EL-Manzalawy, Yiwei Sun, Vasant Honavar
Abstract	Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compositional optimization problems (FS-CEVF). Compositional stochastic variance reduced gradient (C-SVRG) methods that combine stochastic compositional gradient descent (SCGD) and stochastic variance reduced gradient descent (SVRG) methods are the state-of-the-art methods for FS-CEVF problems. We introduce compositional stochastic average gradient descent (C-SAG) a novel extension of the stochastic average gradient method (SAG) to minimize composition of finite-sum functions. C-SAG, like SAG, estimates gradient by incorporating memory of previous gradient information. We present theoretical analyses of C-SAG which show that C-SAG, like SAG, and C-SVRG, achieves a linear convergence rate when the objective function is strongly convex; However, C-CAG achieves lower oracle query complexity per iteration than C-SVRG. Finally, we present results of experiments showing that C-SAG converges substantially faster than full gradient (FG), as well as C-SVRG.
Tasks	Portfolio Optimization
Published	2018-09-04
URL	http://arxiv.org/abs/1809.01225v2
PDF	http://arxiv.org/pdf/1809.01225v2.pdf
PWC	https://paperswithcode.com/paper/compositional-stochastic-average-gradient-for
Repo
Framework

Towards a Semantic Perceptual Image Metric


Title	Towards a Semantic Perceptual Image Metric
Authors	Troy Chinen, Johannes Ballé, Chunhui Gu, Sung Jin Hwang, Sergey Ioffe, Nick Johnston, Thomas Leung, David Minnen, Sean O’Malley, Charles Rosenberg, George Toderici
Abstract	We present a full reference, perceptual image metric based on VGG-16, an artificial neural network trained on object classification. We fit the metric to a new database based on 140k unique images annotated with ground truth by human raters who received minimal instruction. The resulting metric shows competitive performance on TID 2013, a database widely used to assess image quality assessments methods. More interestingly, it shows strong responses to objects potentially carrying semantic relevance such as faces and text, which we demonstrate using a visualization technique and ablation experiments. In effect, the metric appears to model a higher influence of semantic context on judgments, which we observe particularly in untrained raters. As the vast majority of users of image processing systems are unfamiliar with Image Quality Assessment (IQA) tasks, these findings may have significant impact on real-world applications of perceptual metrics.
Tasks	Image Quality Assessment, Object Classification
Published	2018-08-01
URL	http://arxiv.org/abs/1808.00447v1
PDF	http://arxiv.org/pdf/1808.00447v1.pdf
PWC	https://paperswithcode.com/paper/towards-a-semantic-perceptual-image-metric
Repo
Framework

SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation


Title	SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation
Authors	Abel S. Zacarias, Luís A. Alexandre
Abstract	Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous learned tasks. In this paper we present a method to overcome catastrophic forgetting on convolutional neural networks, that learns new tasks and preserves the performance on old tasks without accessing the data of the original model, by selective network augmentation. The experiment results showed that SeNA-CNN, in some scenarios, outperforms the state-of-art Learning without Forgetting algorithm. Results also showed that in some situations it is better to use SeNA-CNN instead of training a neural network using isolated learning.
Tasks
Published	2018-02-22
URL	http://arxiv.org/abs/1802.08250v2
PDF	http://arxiv.org/pdf/1802.08250v2.pdf
PWC	https://paperswithcode.com/paper/sena-cnn-overcoming-catastrophic-forgetting
Repo
Framework

Feasibility of Colon Cancer Detection in Confocal Laser Microscopy Images Using Convolution Neural Networks


Title	Feasibility of Colon Cancer Detection in Confocal Laser Microscopy Images Using Convolution Neural Networks
Authors	Nils Gessert, Lukas Wittig, Daniel Drömann, Tobias Keck, Alexander Schlaefer, David B. Ellebrecht
Abstract	Histological evaluation of tissue samples is a typical approach to identify colorectal cancer metastases in the peritoneum. For immediate assessment, reliable and real-time in-vivo imaging would be required. For example, intraoperative confocal laser microscopy has been shown to be suitable for distinguishing organs and also malignant and benign tissue. So far, the analysis is done by human experts. We investigate the feasibility of automatic colon cancer classification from confocal laser microscopy images using deep learning models. We overcome very small dataset sizes through transfer learning with state-of-the-art architectures. We achieve an accuracy of 89.1% for cancer detection in the peritoneum which indicates viability as an intraoperative decision support system.
Tasks	Colon Cancer Detection In Confocal Laser Microscopy Images, Transfer Learning
Published	2018-12-04
URL	http://arxiv.org/abs/1812.01464v2
PDF	http://arxiv.org/pdf/1812.01464v2.pdf
PWC	https://paperswithcode.com/paper/feasibility-of-colon-cancer-detection-in
Repo
Framework

To Reverse the Gradient or Not: An Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition


Title	To Reverse the Gradient or Not: An Empirical Comparison of Adversarial and Multi-task Learning in Speech Recognition
Authors	Yossi Adi, Neil Zeghidour, Ronan Collobert, Nicolas Usunier, Vitaliy Liptchinsky, Gabriel Synnaeve
Abstract	Transcribed datasets typically contain speaker identity for each instance in the data. We investigate two ways to incorporate this information during training: Multi-Task Learning and Adversarial Learning. In multi-task learning, the goal is speaker prediction; we expect a performance improvement with this joint training if the two tasks of speech recognition and speaker recognition share a common set of underlying features. In contrast, adversarial learning is a means to learn representations invariant to the speaker. We then expect better performance if this learnt invariance helps generalizing to new speakers. While the two approaches seem natural in the context of speech recognition, they are incompatible because they correspond to opposite gradients back-propagated to the model. In order to better understand the effect of these approaches in terms of error rates, we compare both strategies in controlled settings. Moreover, we explore the use of additional untranscribed data in a semi-supervised, adversarial learning manner to improve error rates. Our results show that deep models trained on big datasets already develop invariant representations to speakers without any auxiliary loss. When considering adversarial learning and multi-task learning, the impact on the acoustic model seems minor. However, models trained in a semi-supervised manner can improve error-rates.
Tasks	Multi-Task Learning, Speaker Recognition, Speech Recognition
Published	2018-12-09
URL	http://arxiv.org/abs/1812.03483v3
PDF	http://arxiv.org/pdf/1812.03483v3.pdf
PWC	https://paperswithcode.com/paper/to-reverse-the-gradient-or-not-an-empirical
Repo
Framework

(Nearly) Efficient Algorithms for the Graph Matching Problem on Correlated Random Graphs


Title	(Nearly) Efficient Algorithms for the Graph Matching Problem on Correlated Random Graphs
Authors	Boaz Barak, Chi-Ning Chou, Zhixian Lei, Tselil Schramm, Yueqi Sheng
Abstract	We give a quasipolynomial time algorithm for the graph matching problem (also known as noisy or robust graph isomorphism) on correlated random graphs. Specifically, for every $\gamma>0$, we give a $n^{O(\log n)}$ time algorithm that given a pair of $\gamma$-correlated $G(n,p)$ graphs $G_0,G_1$ with average degree between $n^{\varepsilon}$ and $n^{1/153}$ for $\varepsilon = o(1)$, recovers the “ground truth” permutation $\pi\in S_n$ that matches the vertices of $G_0$ to the vertices of $G_n$ in the way that minimizes the number of mismatched edges. We also give a recovery algorithm for a denser regime, and a polynomial-time algorithm for distinguishing between correlated and uncorrelated graphs. Prior work showed that recovery is information-theoretically possible in this model as long the average degree was at least $\log n$, but sub-exponential time algorithms were only known in the dense case (i.e., for $p > n^{-o(1)}$). Moreover, “Percolation Graph Matching”, which is the most common heuristic for this problem, has been shown to require knowledge of $n^{\Omega(1)}$ “seeds” (i.e., input/output pairs of the permutation $\pi$) to succeed in this regime. In contrast our algorithms require no seed and succeed for $p$ which is as low as $n^{o(1)-1}$.
Tasks	Graph Matching
Published	2018-05-07
URL	http://arxiv.org/abs/1805.02349v2
PDF	http://arxiv.org/pdf/1805.02349v2.pdf
PWC	https://paperswithcode.com/paper/nearly-efficient-algorithms-for-the-graph
Repo
Framework

Estimating Network Structure from Incomplete Event Data


Title	Estimating Network Structure from Incomplete Event Data
Authors	Benjamin Mark, Garvesh Raskutti, Rebecca Willett
Abstract	Multivariate Bernoulli autoregressive (BAR) processes model time series of events in which the likelihood of current events is determined by the times and locations of past events. These processes can be used to model nonlinear dynamical systems corresponding to criminal activity, responses of patients to different medical treatment plans, opinion dynamics across social networks, epidemic spread, and more. Past work examines this problem under the assumption that the event data is complete, but in many cases only a fraction of events are observed. Incomplete observations pose a significant challenge in this setting because the unobserved events still govern the underlying dynamical system. In this work, we develop a novel approach to estimating the parameters of a BAR process in the presence of unobserved events via an unbiased estimator of the complete data log-likelihood function. We propose a computationally efficient estimation algorithm which approximates this estimator via Taylor series truncation and establish theoretical results for both the statistical error and optimization error of our algorithm. We further justify our approach by testing our method on both simulated data and a real data set consisting of crimes recorded by the city of Chicago.
Tasks	Time Series
Published	2018-11-07
URL	http://arxiv.org/abs/1811.02979v1
PDF	http://arxiv.org/pdf/1811.02979v1.pdf
PWC	https://paperswithcode.com/paper/estimating-network-structure-from-incomplete
Repo
Framework

TriResNet: A Deep Triple-stream Residual Network for Histopathology Grading


Title	TriResNet: A Deep Triple-stream Residual Network for Histopathology Grading
Authors	Rene Bidart, Alexander Wong
Abstract	While microscopic analysis of histopathological slides is generally considered as the gold standard method for performing cancer diagnosis and grading, the current method for analysis is extremely time consuming and labour intensive as it requires pathologists to visually inspect tissue samples in a detailed fashion for the presence of cancer. As such, there has been significant recent interest in computer aided diagnosis systems for analysing histopathological slides for cancer grading to aid pathologists to perform cancer diagnosis and grading in a more efficient, accurate, and consistent manner. In this work, we investigate and explore a deep triple-stream residual network (TriResNet) architecture for the purpose of tile-level histopathology grading, which is the critical first step to computer-aided whole-slide histopathology grading. In particular, the design mentality behind the proposed TriResNet network architecture is to facilitate for the learning of a more diverse set of quantitative features to better characterize the complex tissue characteristics found in histopathology samples. Experimental results on two widely-used computer-aided histopathology benchmark datasets (CAMELYON16 dataset and Invasive Ductal Carcinoma (IDC) dataset) demonstrated that the proposed TriResNet network architecture was able to achieve noticeably improved accuracies when compared with two other state-of-the-art deep convolutional neural network architectures. Based on these promising results, the hope is that the proposed TriResNet network architecture could become a useful tool to aiding pathologists increase the consistency, speed, and accuracy of the histopathology grading process.
Tasks
Published	2018-06-22
URL	http://arxiv.org/abs/1806.08463v1
PDF	http://arxiv.org/pdf/1806.08463v1.pdf
PWC	https://paperswithcode.com/paper/triresnet-a-deep-triple-stream-residual
Repo
Framework

Um Sistema Multiagente no Combate ao Braqueamento de Capitais


Title	Um Sistema Multiagente no Combate ao Braqueamento de Capitais
Authors	Claudio Alexandre, João Balsa
Abstract	Money laundering is a crime that makes it possible to finance other crimes, for this reason, it is important for criminal organizations and their combat is prioritized by nations around the world. The anti-money laundering process has not evolved as expected because it has prioritized only the signaling of suspicious transactions. The constant increasing in the volume of transactions has overloaded the indispensable human work of final evaluation of the suspicions. This article presents a multiagent system that aims to go beyond the capture of suspicious transactions, seeking to assist the human expert in the analysis of suspicions. The agents created use data mining techniques to create transactional behavioral profiles; apply rules generated in learning process in conjunction with specific rules based on legal aspects and profiles created to capture suspicious transactions; and analyze these suspicious transactions indicating to the human expert those that require more detailed analysis.
Tasks
Published	2018-01-02
URL	http://arxiv.org/abs/1801.00743v1
PDF	http://arxiv.org/pdf/1801.00743v1.pdf
PWC	https://paperswithcode.com/paper/um-sistema-multiagente-no-combate-ao
Repo
Framework

Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning


Title	Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning
Authors	Yao Zhang, Andrew M. Saxe, Madhu S. Advani, Alpha A. Lee
Abstract	Finding parameters that minimise a loss function is at the core of many machine learning methods. The Stochastic Gradient Descent algorithm is widely used and delivers state of the art results for many problems. Nonetheless, Stochastic Gradient Descent typically cannot find the global minimum, thus its empirical effectiveness is hitherto mysterious. We derive a correspondence between parameter inference and free energy minimisation in statistical physics. The degree of undersampling plays the role of temperature. Analogous to the energy-entropy competition in statistical physics, wide but shallow minima can be optimal if the system is undersampled, as is typical in many applications. Moreover, we show that the stochasticity in the algorithm has a non-trivial correlation structure which systematically biases it towards wide minima. We illustrate our argument with two prototypical models: image classification using deep learning, and a linear neural network where we can analytically reveal the relationship between entropy and out-of-sample error.
Tasks	Image Classification
Published	2018-03-05
URL	http://arxiv.org/abs/1803.01927v1
PDF	http://arxiv.org/pdf/1803.01927v1.pdf
PWC	https://paperswithcode.com/paper/energy-entropy-competition-and-the
Repo
Framework

Amobee at IEST 2018: Transfer Learning from Language Models


Title	Amobee at IEST 2018: Transfer Learning from Language Models
Authors	Alon Rozental, Daniel Fleischer, Zohar Kelrich
Abstract	This paper describes the system developed at Amobee for the WASSA 2018 implicit emotions shared task (IEST). The goal of this task was to predict the emotion expressed by missing words in tweets without an explicit mention of those words. We developed an ensemble system consisting of language models together with LSTM-based networks containing a CNN attention mechanism. Our approach represents a novel use of language models (specifically trained on a large Twitter dataset) to predict and classify emotions. Our system reached 1st place with a macro $\text{F}_1$ score of 0.7145.
Tasks	Transfer Learning
Published	2018-08-27
URL	http://arxiv.org/abs/1808.08782v2
PDF	http://arxiv.org/pdf/1808.08782v2.pdf
PWC	https://paperswithcode.com/paper/amobee-at-iest-2018-transfer-learning-from
Repo
Framework

Hierarchical stochastic graphlet embedding for graph-based pattern recognition


Title	Hierarchical stochastic graphlet embedding for graph-based pattern recognition
Authors	Anjan Dutta, Pau Riba, Josep Lladós, Alicia Fornés
Abstract	Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties enabling the use of standard machine learning techniques. However, it is well known that graph embedding functions usually suffer from the loss of structural information. In this paper, we consider the hierarchical structure of a graph as a way to mitigate this loss of information. The hierarchical structure is constructed by topologically clustering the graph nodes, and considering each cluster as a node in the upper hierarchical level. Once this hierarchical structure is constructed, we consider several configurations to define the mapping into a vector space given a classical graph embedding, in particular, we propose to make use of the Stochastic Graphlet Embedding (SGE). Broadly speaking, SGE produces a distribution of uniformly sampled low to high order graphlets as a way to embed graphs into the vector space. In what follows, the coarse-to-fine structure of a graph hierarchy and the statistics fetched by the SGE complements each other and includes important structural information with varied contexts. Altogether, these two techniques substantially cope with the usual information loss involved in graph embedding techniques, obtaining a more robust graph representation. This fact has been corroborated through a detailed experimental evaluation on various benchmark graph datasets, where we outperform the state-of-the-art methods.
Tasks	Graph Embedding
Published	2018-07-08
URL	https://arxiv.org/abs/1807.02839v2
PDF	https://arxiv.org/pdf/1807.02839v2.pdf
PWC	https://paperswithcode.com/paper/hierarchical-stochastic-graphlet-embedding
Repo
Framework


Title	Covariate Shift Estimation based Adaptive Ensemble Learning for Handling Non-Stationarity in Motor Imagery related EEG-based Brain-Computer Interface
Authors	Haider Raza, Dheeraj Rathee, ShangMing Zhou, Hubert Cecotti, Girijesh Prasad
Abstract	The non-stationary nature of electroencephalography (EEG) signals makes an EEG-based brain-computer interface (BCI) a dynamic system, thus improving its performance is a challenging task. In addition, it is well-known that due to non-stationarity based covariate shifts, the input data distributions of EEG-based BCI systems change during inter- and intra-session transitions, which poses great difficulty for developments of online adaptive data-driven systems. Ensemble learning approaches have been used previously to tackle this challenge. However, passive scheme based implementation leads to poor efficiency while increasing high computational cost. This paper presents a novel integration of covariate shift estimation and unsupervised adaptive ensemble learning (CSE-UAEL) to tackle non-stationarity in motor-imagery (MI) related EEG classification. The proposed method first employs an exponentially weighted moving average model to detect the covariate shifts in the common spatial pattern features extracted from MI related brain responses. Then, a classifier ensemble was created and updated over time to account for changes in streaming input data distribution wherein new classifiers are added to the ensemble in accordance with estimated shifts. Furthermore, using two publicly available BCI-related EEG datasets, the proposed method was extensively compared with the state-of-the-art single-classifier based passive scheme, single-classifier based active scheme and ensemble based passive schemes. The experimental results show that the proposed active scheme based ensemble learning algorithm significantly enhances the BCI performance in MI classifications.
Tasks	EEG
Published	2018-05-02
URL	http://arxiv.org/abs/1805.01044v1
PDF	http://arxiv.org/pdf/1805.01044v1.pdf
PWC	https://paperswithcode.com/paper/covariate-shift-estimation-based-adaptive
Repo
Framework

An Ontology for Satellite Databases


Title	An Ontology for Satellite Databases
Authors	Robert J. Rovetto
Abstract	This paper demonstrates the development of ontology for satellite databases. First, I create a computational ontology for the Union of Concerned Scientists (UCS) Satellite Database (UCSSD for short), called the UCS Satellite Ontology (or UCSSO). Second, in developing UCSSO I show that The Space Situational Awareness Ontology (SSAO) (Rovetto and Kelso 2016)–an existing space domain reference ontology–and related ontology work by the author (Rovetto 2015, 2016) can be used either (i) with a database-specific local ontology such as UCSSO, or (ii) in its stead. In case (i), local ontologies such as UCSSO can reuse SSAO terms, perform term mappings, or extend it. In case (ii), the author’s orbital space ontology work, such as the SSAO, is usable by the UCSSD and organizations with other space object catalogs, as a reference ontology suite providing a common semantically-rich domain model. The SSAO, UCSSO, and the broader Orbital Space Environment Domain Ontology project is online at http://purl.org/space-ontology and GitHub. This ontology effort aims, in part, to provide accurate formal representations of the domain for various applications. Ontology engineering has the potential to facilitate the sharing and integration of satellite data from federated databases and sensors for safer spaceflight.
Tasks
Published	2018-01-06
URL	http://arxiv.org/abs/1801.02940v1
PDF	http://arxiv.org/pdf/1801.02940v1.pdf
PWC	https://paperswithcode.com/paper/an-ontology-for-satellite-databases
Repo
Framework