January 27, 2020

2963 words 14 mins read

Paper Group ANR 1193

$\mathcal{G}$-softmax: Improving Intra-class Compactness and Inter-class Separability of Features. Adaptive Segmentation of Knee Radiographs for Selecting the Optimal ROI in Texture Analysis. Response of Selective Attention in Middle Temporal Area. Learning Numeral Embeddings. Weighted Mean Curvature. Augmented Utilitarianism for AGI Safety. An Eff …

$\mathcal{G}$-softmax: Improving Intra-class Compactness and Inter-class Separability of Features


Title	$\mathcal{G}$-softmax: Improving Intra-class Compactness and Inter-class Separability of Features
Authors	Yan Luo, Yongkang Wong, Mohan Kankanhalli, Qi Zhao
Abstract	Intra-class compactness and inter-class separability are crucial indicators to measure the effectiveness of a model to produce discriminative features, where intra-class compactness indicates how close the features with the same label are to each other and inter-class separability indicates how far away the features with different labels are. In this work, we investigate intra-class compactness and inter-class separability of features learned by convolutional networks and propose a Gaussian-based softmax ($\mathcal{G}$-softmax) function that can effectively improve intra-class compactness and inter-class separability. The proposed function is simple to implement and can easily replace the softmax function. We evaluate the proposed $\mathcal{G}$-softmax function on classification datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) and on multi-label classification datasets (i.e., MS COCO and NUS-WIDE). The experimental results show that the proposed $\mathcal{G}$-softmax function improves the state-of-the-art models across all evaluated datasets. In addition, analysis of the intra-class compactness and inter-class separability demonstrates the advantages of the proposed function over the softmax function, which is consistent with the performance improvement. More importantly, we observe that high intra-class compactness and inter-class separability are linearly correlated to average precision on MS COCO and NUS-WIDE. This implies that improvement of intra-class compactness and inter-class separability would lead to improvement of average precision.
Tasks	Multi-Label Classification
Published	2019-04-08
URL	https://arxiv.org/abs/1904.04317v2
PDF	https://arxiv.org/pdf/1904.04317v2.pdf
PWC	https://paperswithcode.com/paper/mathcalg-softmax-improving-intra-class
Repo
Framework

Adaptive Segmentation of Knee Radiographs for Selecting the Optimal ROI in Texture Analysis


Title	Adaptive Segmentation of Knee Radiographs for Selecting the Optimal ROI in Texture Analysis
Authors	Neslihan Bayramoglu, Aleksei Tiulpin, Jukka Hirvasniemi, Miika T. Nieminen, Simo Saarakkala
Abstract	The purposes of this study were to investigate: 1) the effect of placement of region-of-interest (ROI) for texture analysis of subchondral bone in knee radiographs, and 2) the ability of several texture descriptors to distinguish between the knees with and without radiographic osteoarthritis (OA). Bilateral posterior-anterior knee radiographs were analyzed from the baseline of OAI and MOST datasets. A fully automatic method to locate the most informative region from subchondral bone using adaptive segmentation was developed. We used an oversegmentation strategy for partitioning knee images into the compact regions that follow natural texture boundaries. LBP, Fractal Dimension (FD), Haralick features, Shannon entropy, and HOG methods were computed within the standard ROI and within the proposed adaptive ROIs. Subsequently, we built logistic regression models to identify and compare the performances of each texture descriptor and each ROI placement method using 5-fold cross validation setting. Importantly, we also investigated the generalizability of our approach by training the models on OAI and testing them on MOST dataset.We used area under the receiver operating characteristic (ROC) curve (AUC) and average precision (AP) obtained from the precision-recall (PR) curve to compare the results. We found that the adaptive ROI improves the classification performance (OA vs. non-OA) over the commonly used standard ROI (up to 9% percent increase in AUC). We also observed that, from all texture parameters, LBP yielded the best performance in all settings with the best AUC of 0.840 [0.825, 0.852] and associated AP of 0.804 [0.786, 0.820]. Compared to the current state-of-the-art approaches, our results suggest that the proposed adaptive ROI approach in texture analysis of subchondral bone can increase the diagnostic performance for detecting the presence of radiographic OA.
Tasks	Texture Classification
Published	2019-08-21
URL	https://arxiv.org/abs/1908.07736v1
PDF	https://arxiv.org/pdf/1908.07736v1.pdf
PWC	https://paperswithcode.com/paper/190807736
Repo
Framework

Response of Selective Attention in Middle Temporal Area


Title	Response of Selective Attention in Middle Temporal Area
Authors	Linda Wang
Abstract	The primary visual cortex processes a large amount of visual information, however, due to its large receptive fields, when multiple stimuli fall within one receptive field, there are computational problems. To solve this problem, the visual system uses selective attention, which allocates resources to a specific spatial location, to attend to one of the stimuli in the receptive field. During this process, the center and width of the attending receptive field change. The model presented in the paper, which is extended and altered from Bobier et al., simulates the selective attention between the primary visual cortex, V1, and middle temporal (MT) area. The responses of the MT columns, which encode the target stimulus, are compared to the results of an experiment conducted by Womelsdorf et al. on the receptive field shift and shrinkage in macaque MT area from selective attention. Based on the results, the responses in the MT area are similar to the Gaussian shaped receptive fields found in the experiment. As well, the responses of the MT columns are also measured for accuracy of representing the target visual stimulus and is found to represent the stimulus with a root mean squared error around 0.17 to 0.18. The paper also explores varying model parameters, such as the membrane time constant and maximum firing rates, and how those affect the measurement. This model is a start to modeling the responses of selective attention, however there are still improvements that can be made to better compare with the experiment, produce more accurate responses and incorporate more biologically plausible features.
Tasks
Published	2019-04-16
URL	http://arxiv.org/abs/1904.07952v1
PDF	http://arxiv.org/pdf/1904.07952v1.pdf
PWC	https://paperswithcode.com/paper/response-of-selective-attention-in-middle
Repo
Framework

Learning Numeral Embeddings


Title	Learning Numeral Embeddings
Authors	Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, Kewei Tu
Abstract	Word embedding is an essential building block for deep learning methods for natural language processing. Although word embedding has been extensively studied over the years, the problem of how to effectively embed numerals, a special subset of words, is still underexplored. Existing word embedding methods do not learn numeral embeddings well because there are an infinite number of numerals and their individual appearances in training corpora are highly scarce. In this paper, we propose two novel numeral embedding methods that can handle the out-of-vocabulary (OOV) problem for numerals. We first induce a finite set of prototype numerals using either a self-organizing map or a Gaussian mixture model. We then represent the embedding of a numeral as a weighted average of the prototype number embeddings. Numeral embeddings represented in this manner can be plugged into existing word embedding learning approaches such as skip-gram for training. We evaluated our methods and showed its effectiveness on four intrinsic and extrinsic tasks: word similarity, embedding numeracy, numeral prediction, and sequence labeling.
Tasks
Published	2019-12-28
URL	https://arxiv.org/abs/2001.00003v3
PDF	https://arxiv.org/pdf/2001.00003v3.pdf
PWC	https://paperswithcode.com/paper/learning-numeral-embeddings
Repo
Framework

Weighted Mean Curvature


Title	Weighted Mean Curvature
Authors	Yuanhao Gong, Orcun Goksel
Abstract	In image processing tasks, spatial priors are essential for robust computations, regularization, algorithmic design and Bayesian inference. In this paper, we introduce weighted mean curvature (WMC) as a novel image prior and present an efficient computation scheme for its discretization in practical image processing applications. We first demonstrate the favorable properties of WMC, such as sampling invariance, scale invariance, and contrast invariance with Gaussian noise model; and we show the relation of WMC to area regularization. We further propose an efficient computation scheme for discretized WMC, which is demonstrated herein to process over 33.2 giga-pixels/second on GPU. This scheme yields itself to a convolutional neural network representation. Finally, WMC is evaluated on synthetic and real images, showing its superiority quantitatively to total-variation and mean curvature.
Tasks	Bayesian Inference
Published	2019-03-17
URL	https://arxiv.org/abs/1903.07189v2
PDF	https://arxiv.org/pdf/1903.07189v2.pdf
PWC	https://paperswithcode.com/paper/weighted-mean-curvature
Repo
Framework

Augmented Utilitarianism for AGI Safety


Title	Augmented Utilitarianism for AGI Safety
Authors	Nadisha-Marie Aliman, Leon Kester
Abstract	In the light of ongoing progresses of research on artificial intelligent systems exhibiting a steadily increasing problem-solving ability, the identification of practicable solutions to the value alignment problem in AGI Safety is becoming a matter of urgency. In this context, one preeminent challenge that has been addressed by multiple researchers is the adequate formulation of utility functions or equivalents reliably capturing human ethical conceptions. However, the specification of suitable utility functions harbors the risk of “perverse instantiation” for which no final consensus on responsible proactive countermeasures has been achieved so far. Amidst this background, we propose a novel socio-technological ethical framework denoted Augmented Utilitarianism which directly alleviates the perverse instantiation problem. We elaborate on how augmented by AI and more generally science and technology, it might allow a society to craft and update ethical utility functions while jointly undergoing a dynamical ethical enhancement. Further, we elucidate the need to consider embodied simulations in the design of utility functions for AGIs aligned with human values. Finally, we discuss future prospects regarding the usage of the presented scientifically grounded ethical framework and mention possible challenges.
Tasks
Published	2019-04-02
URL	http://arxiv.org/abs/1904.01540v1
PDF	http://arxiv.org/pdf/1904.01540v1.pdf
PWC	https://paperswithcode.com/paper/augmented-utilitarianism-for-agi-safety
Repo
Framework

An Efficient Augmented Lagrangian Based Method for Constrained Lasso


Title	An Efficient Augmented Lagrangian Based Method for Constrained Lasso
Authors	Zengde Deng, Anthony Man-Cho So
Abstract	Variable selection is one of the most important tasks in statistics and machine learning. To incorporate more prior information about the regression coefficients, the constrained Lasso model has been proposed in the literature. In this paper, we present an inexact augmented Lagrangian method to solve the Lasso problem with linear equality constraints. By fully exploiting second-order sparsity of the problem, we are able to greatly reduce the computational cost and obtain highly efficient implementations. Furthermore, numerical results on both synthetic data and real data show that our algorithm is superior to existing first-order methods in terms of both running time and solution accuracy.
Tasks
Published	2019-03-12
URL	http://arxiv.org/abs/1903.05006v1
PDF	http://arxiv.org/pdf/1903.05006v1.pdf
PWC	https://paperswithcode.com/paper/an-efficient-augmented-lagrangian-based
Repo
Framework

Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates


Title	Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates
Authors	Cong Xie, Oluwasanmi Koyejo, Indranil Gupta, Haibin Lin
Abstract	Recent years have witnessed the growth of large-scale distributed machine learning algorithms – specifically designed to accelerate model training by distributing computation across multiple machines. When scaling distributed training in this way, the communication overhead is often the bottleneck. In this paper, we study the local distributed Stochastic Gradient Descent~(SGD) algorithm, which reduces the communication overhead by decreasing the frequency of synchronization. While SGD with adaptive learning rates is a widely adopted strategy for training neural networks, it remains unknown how to implement adaptive learning rates in local SGD. To this end, we propose a novel SGD variant with reduced communication and adaptive learning rates, with provable convergence. Empirical results show that the proposed algorithm has fast convergence and efficiently reduces the communication overhead.
Tasks
Published	2019-11-20
URL	https://arxiv.org/abs/1911.09030v1
PDF	https://arxiv.org/pdf/1911.09030v1.pdf
PWC	https://paperswithcode.com/paper/local-adaalter-communication-efficient
Repo
Framework

Block-Coordinate Minimization for Large SDPs with Block-Diagonal Constraints


Title	Block-Coordinate Minimization for Large SDPs with Block-Diagonal Constraints
Authors	Yulun Tian, Kasra Khosoussi, Jonathan P. How
Abstract	The so-called Burer-Monteiro method is a well-studied technique for solving large-scale semidefinite programs (SDPs) via low-rank factorization. The main idea is to solve rank-restricted, albeit non-convex, surrogates instead of the SDP. Recent works have shown that, in an important class of SDPs with elegant geometric structure, one can find globally optimal solutions to the SDP by finding rank-deficient second-order critical points of an unconstrained Riemannian optimization problem. Hence, in such problems, the Burer-Monteiro approach can provide a scalable and reliable alternative to interior-point methods that scale poorly. Among various Riemannian optimization methods proposed, block-coordinate minimization (BCM) is of particular interest due to its simplicity. Erdogdu et al. in their recent work proposed BCM for problems over the Cartesian product of unit spheres and provided global convergence rate estimates for the algorithm. This report extends the BCM algorithm and the global convergence rate analysis of Erdogdu et al. from problems over the Cartesian product of unit spheres to the Cartesian product of Stiefel manifolds. The latter more general setting has important applications such as synchronization over the special orthogonal (SO) and special Euclidean (SE) groups.
Tasks
Published	2019-03-02
URL	https://arxiv.org/abs/1903.00597v4
PDF	https://arxiv.org/pdf/1903.00597v4.pdf
PWC	https://paperswithcode.com/paper/block-coordinate-minimization-for-large-sdps
Repo
Framework

Greenery Segmentation In Urban Images By Deep Learning


Title	Greenery Segmentation In Urban Images By Deep Learning
Authors	Artur A. M. Oliveira, Nina S. T. Hirata, Roberto Hirata Jr
Abstract	Vegetation is a relevant feature in the urban scenery and its awareness can be measured in an image by the Green View Index (GVI). Previous approaches to estimate the GVI were based upon heuristics image processing approaches and recently by deep learning networks (DLN). By leveraging some recent DLN architectures tuned to the image segmentation problem and exploiting a weighting strategy in the loss function (LF) we improved previously reported results in similar datasets.
Tasks	Semantic Segmentation
Published	2019-12-12
URL	https://arxiv.org/abs/1912.06199v1
PDF	https://arxiv.org/pdf/1912.06199v1.pdf
PWC	https://paperswithcode.com/paper/greenery-segmentation-in-urban-images-by-deep
Repo
Framework

Adaptive Gaussian Copula ABC


Title	Adaptive Gaussian Copula ABC
Authors	Yanzhi Chen, Michael U. Gutmann
Abstract	Approximate Bayesian computation (ABC) is a set of techniques for Bayesian inference when the likelihood is intractable but sampling from the model is possible. This work presents a simple yet effective ABC algorithm based on the combination of two classical ABC approaches — regression ABC and sequential ABC. The key idea is that rather than learning the posterior directly, we first target another auxiliary distribution that can be learned accurately by existing methods, through which we then subsequently learn the desired posterior with the help of a Gaussian copula. During this process, the complexity of the model changes adaptively according to the data at hand. Experiments on a synthetic dataset as well as three real-world inference tasks demonstrates that the proposed method is fast, accurate, and easy to use.
Tasks	Bayesian Inference
Published	2019-02-27
URL	http://arxiv.org/abs/1902.10704v1
PDF	http://arxiv.org/pdf/1902.10704v1.pdf
PWC	https://paperswithcode.com/paper/adaptive-gaussian-copula-abc
Repo
Framework

Robustness to Capitalization Errors in Named Entity Recognition


Title	Robustness to Capitalization Errors in Named Entity Recognition
Authors	Sravan Bodapati, Hyokun Yun, Yaser Al-Onaizan
Abstract	Robustness to capitalization errors is a highly desirable characteristic of named entity recognizers, yet we find standard models for the task are surprisingly brittle to such noise. Existing methods to improve robustness to the noise completely discard given orthographic information, mwhich significantly degrades their performance on well-formed text. We propose a simple alternative approach based on data augmentation, which allows the model to \emph{learn} to utilize or ignore orthographic information depending on its usefulness in the context. It achieves competitive robustness to capitalization errors while making negligible compromise to its performance on well-formed text and significantly improving generalization power on noisy user-generated text. Our experiments clearly and consistently validate our claim across different types of machine learning models, languages, and dataset sizes.
Tasks	Data Augmentation, Named Entity Recognition
Published	2019-11-13
URL	https://arxiv.org/abs/1911.05241v1
PDF	https://arxiv.org/pdf/1911.05241v1.pdf
PWC	https://paperswithcode.com/paper/robustness-to-capitalization-errors-in-named-1
Repo
Framework

Image Super-Resolution Using a Wavelet-based Generative Adversarial Network


Title	Image Super-Resolution Using a Wavelet-based Generative Adversarial Network
Authors	Qi Zhang, Huafeng Wang, Sichen Yang
Abstract	In this paper, we consider the problem of super-resolution recons-truction. This is a hot topic because super-resolution reconstruction has a wide range of applications in the medical field, remote sensing monitoring, and criminal investigation. Compared with traditional algorithms, the current super-resolution reconstruction algorithm based on deep learning greatly improves the clarity of reconstructed pictures. Existing work like Super-Resolution Using a Generative Adversarial Network (SRGAN) can effectively restore the texture details of the image. However, experimentally verified that the texture details of the image recovered by the SRGAN are not robust. In order to get super-resolution reconstructed images with richer high-frequency details, we improve the network structure and propose a super-resolution reconstruction algorithm combining wavelet transform and Generative Adversarial Network. The proposed algorithm can efficiently reconstruct high-resolution images with rich global information and local texture details. We have trained our model by PyTorch framework and VOC2012 dataset, and tested it by Set5, Set14, BSD100 and Urban100 test datasets.
Tasks	Image Super-Resolution, Super-Resolution
Published	2019-07-24
URL	https://arxiv.org/abs/1907.10213v1
PDF	https://arxiv.org/pdf/1907.10213v1.pdf
PWC	https://paperswithcode.com/paper/image-super-resolution-using-a-wavelet-based
Repo
Framework

Speech2Face: Learning the Face Behind a Voice


Title	Speech2Face: Learning the Face Behind a Voice
Authors	Tae-Hyun Oh, Tali Dekel, Changil Kim, Inbar Mosseri, William T. Freeman, Michael Rubinstein, Wojciech Matusik
Abstract	How much can we infer about a person’s looks from the way they speak? In this paper, we study the task of reconstructing a facial image of a person from a short audio recording of that person speaking. We design and train a deep neural network to perform this task using millions of natural Internet/YouTube videos of people speaking. During training, our model learns voice-face correlations that allow it to produce images that capture various physical attributes of the speakers such as age, gender and ethnicity. This is done in a self-supervised manner, by utilizing the natural co-occurrence of faces and speech in Internet videos, without the need to model attributes explicitly. We evaluate and numerically quantify how–and in what manner–our Speech2Face reconstructions, obtained directly from audio, resemble the true face images of the speakers.
Tasks
Published	2019-05-23
URL	https://arxiv.org/abs/1905.09773v1
PDF	https://arxiv.org/pdf/1905.09773v1.pdf
PWC	https://paperswithcode.com/paper/190509773
Repo
Framework

Interpreting, axiomatising and representing coherent choice functions in terms of desirability


Title	Interpreting, axiomatising and representing coherent choice functions in terms of desirability
Authors	Jasper De Bock, Gert de Cooman
Abstract	Choice functions constitute a simple, direct and very general mathematical framework for modelling choice under uncertainty. In particular, they are able to represent the set-valued choices that appear in imprecise-probabilistic decision making. We provide these choice functions with a clear interpretation in terms of desirability, use this interpretation to derive a set of basic coherence axioms, and show that this notion of coherence leads to a representation in terms of sets of strict preference orders. By imposing additional properties such as totality, the mixing property and Archimedeanity, we obtain representation in terms of sets of strict total orders, lexicographic probability systems, coherent lower previsions or linear previsions.
Tasks	Decision Making
Published	2019-02-28
URL	https://arxiv.org/abs/1903.00336v2
PDF	https://arxiv.org/pdf/1903.00336v2.pdf
PWC	https://paperswithcode.com/paper/an-alternative-approach-to-coherent-choice
Repo
Framework