January 25, 2020

3221 words 16 mins read

Paper Group ANR 1670

Machine Translation in Pronunciation Space. Deep Personalized Re-targeting. Decision Set Optimization and Energy-Efficient MIMO Communications. IoT Network Security from the Perspective of Adversarial Deep Learning. Restyling Data: Application to Unsupervised Domain Adaptation. Fast and Flexible Image Blind Denoising via Competition of Experts. Met …

Machine Translation in Pronunciation Space


Title	Machine Translation in Pronunciation Space
Authors	Hairong Liu, Mingbo Ma, Liang Huang
Abstract	The research in machine translation community focus on translation in text space. However, humans are in fact also good at direct translation in pronunciation space. Some existing translation systems, such as simultaneous machine translation, are inherently more natural and thus potentially more robust by directly translating in pronunciation space. In this paper, we conduct large scale experiments on a self-built dataset with about $20$M En-Zh pairs of text sentences and corresponding pronunciation sentences. We proposed three new categories of translations: $1)$ translating a pronunciation sentence in source language into a pronunciation sentence in target language (P2P-Tran), $2)$ translating a text sentence in source language into a pronunciation sentence in target language (T2P-Tran), and $3)$ translating a pronunciation sentence in source language into a text sentence in target language (P2T-Tran), and compare them with traditional text translation (T2T-Tran). Our experiments clearly show that all $4$ categories of translations have comparable performances, with small and sometimes ignorable differences.
Tasks	Machine Translation
Published	2019-11-03
URL	https://arxiv.org/abs/1911.00932v1
PDF	https://arxiv.org/pdf/1911.00932v1.pdf
PWC	https://paperswithcode.com/paper/machine-translation-in-pronunciation-space
Repo
Framework

Deep Personalized Re-targeting


Title	Deep Personalized Re-targeting
Authors	Meisam Hejazinia, Pavlos Mitsoulis-Ntompos, Serena Zhang
Abstract	Predicting booking probability and value at the traveler level plays a central role in computational advertising for massive two-sided vacation rental marketplaces. These marketplaces host millions of travelers with long shopping cycles, spending a lot of time in the discovery phase. The footprint of the travelers in their discovery is a useful data source to help these marketplaces to predict shopping probability and value. However, there is no one-size-fits-all solution for this purpose. In this paper, we propose a hybrid model that infuses deep and shallow neural network embeddings into a gradient boosting tree model. This approach allows the latent preferences of millions of travelers to be automatically learned from sparse session logs. In addition, we present the architecture that we deployed into our production system. We find that there is a pragmatic sweet spot between expensive complex deep neural networks and simple shallow neural networks that can increase the prediction performance of a model by seven percent, based on offline analysis.
Tasks
Published	2019-07-03
URL	https://arxiv.org/abs/1907.02822v2
PDF	https://arxiv.org/pdf/1907.02822v2.pdf
PWC	https://paperswithcode.com/paper/deep-personalized-re-targeting
Repo
Framework

Decision Set Optimization and Energy-Efficient MIMO Communications


Title	Decision Set Optimization and Energy-Efficient MIMO Communications
Authors	Hang Zou, Chao Zhang, Samson Lasaulce, Lucas Saludjian, Patrick Panciatici
Abstract	Assuming that the number of possible decisions for a transmitter (e.g., the number of possible beamforming vectors) has to be finite and is given, this paper investigates for the first time the problem of determining the best decision set when energy-efficiency maximization is pursued. We propose a framework to find a good (finite) decision set which induces a minimal performance loss w.r.t. to the continuous case. We exploit this framework for a scenario of energy-efficient MIMO communications in which transmit power and beamforming vectors have to be adapted jointly to the channel given under finite-rate feedback. To determine a good decision set we propose an algorithm which combines the approach of Invasive Weed Optimization (IWO) and an Evolutionary Algorithm (EA). We provide a numerical analysis which illustrates the benefits of our point of view. In particular, given a performance loss level, the feedback rate can by reduced by 2 when the transmit decision set has been designed properly by using our algorithm. The impact on energy-efficiency is also seen to be significant.
Tasks
Published	2019-09-16
URL	https://arxiv.org/abs/1909.07172v1
PDF	https://arxiv.org/pdf/1909.07172v1.pdf
PWC	https://paperswithcode.com/paper/decision-set-optimization-and-energy
Repo
Framework

IoT Network Security from the Perspective of Adversarial Deep Learning


Title	IoT Network Security from the Perspective of Adversarial Deep Learning
Authors	Yalin E. Sagduyu, Yi Shi, Tugba Erpek
Abstract	Machine learning finds rich applications in Internet of Things (IoT) networks such as information retrieval, traffic management, spectrum sensing, and signal authentication. While there is a surge of interest to understand the security issues of machine learning, their implications have not been understood yet for wireless applications such as those in IoT systems that are susceptible to various attacks due the open and broadcast nature of wireless communications. To support IoT systems with heterogeneous devices of different priorities, we present new techniques built upon adversarial machine learning and apply them to three types of over-the-air (OTA) wireless attacks, namely jamming, spectrum poisoning, and priority violation attacks. By observing the spectrum, the adversary starts with an exploratory attack to infer the channel access algorithm of an IoT transmitter by building a deep neural network classifier that predicts the transmission outcomes. Based on these prediction results, the wireless attack continues to either jam data transmissions or manipulate sensing results over the air (by transmitting during the sensing phase) to fool the transmitter into making wrong transmit decisions in the test phase (corresponding to an evasion attack). When the IoT transmitter collects sensing results as training data to retrain its channel access algorithm, the adversary launches a causative attack to manipulate the input data to the transmitter over the air. We show that these attacks with different levels of energy consumption and stealthiness lead to significant loss in throughput and success ratio in wireless communications for IoT systems. Then we introduce a defense mechanism that systematically increases the uncertainty of the adversary at the inference stage and improves the performance. Results provide new insights on how to attack and defend IoT networks using deep learning.
Tasks	Information Retrieval
Published	2019-05-31
URL	https://arxiv.org/abs/1906.00076v1
PDF	https://arxiv.org/pdf/1906.00076v1.pdf
PWC	https://paperswithcode.com/paper/190600076
Repo
Framework

Restyling Data: Application to Unsupervised Domain Adaptation


Title	Restyling Data: Application to Unsupervised Domain Adaptation
Authors	Vasileios Gkitsas, Antonis Karakottas, Nikolaos Zioulis, Dimitrios Zarpalas, Petros Daras
Abstract	Machine learning is driven by data, yet while their availability is constantly increasing, training data require laborious, time consuming and error-prone labelling or ground truth acquisition, which in some cases is very difficult or even impossible. Recent works have resorted to synthetic data generation, but the inferior performance of models trained on synthetic data when applied to the real world, introduced the challenge of unsupervised domain adaptation. In this work we investigate an unsupervised domain adaptation technique that descends from another perspective, in order to avoid the complexity of adversarial training and cycle consistencies. We exploit the recent advances in photorealistic style transfer and take a fully data driven approach. While this concept is already implicitly formulated within the intricate objectives of domain adaptation GANs, we take an explicit approach and apply it directly as data pre-processing. The resulting technique is scalable, efficient and easy to implement, offers competitive performance to the complex state-of-the-art alternatives and can open up new pathways for domain adaptation.
Tasks	Domain Adaptation, Style Transfer, Synthetic Data Generation, Unsupervised Domain Adaptation
Published	2019-09-24
URL	https://arxiv.org/abs/1909.10900v1
PDF	https://arxiv.org/pdf/1909.10900v1.pdf
PWC	https://paperswithcode.com/paper/restyling-data-application-to-unsupervised
Repo
Framework


Title	Fast and Flexible Image Blind Denoising via Competition of Experts
Authors	Shunta Maeda
Abstract	Fast and flexible processing are two essential requirements for a number of practical applications of image denoising. Current state-of-the-art methods, however, still require either high computational cost or limited scopes of the target. We introduce an efficient ensemble network trained via a competition of expert networks, as an application for image blind denoising. We realize automatic division of unlabeled noisy datasets into clusters respectively optimized to enhance denoising performance. The architecture is scalable, can be extended to deal with diverse noise sources/levels without increasing the computation time. Taking advantage of this method, we save up to approximately 90% of computational cost without sacrifice of the denoising performance compared to single network models with identical architectures. We also compare the proposed method with several existing algorithms and observe significant outperformance over prior arts in terms of computational efficiency.
Tasks	Denoising, Image Denoising
Published	2019-11-20
URL	https://arxiv.org/abs/1911.08724v1
PDF	https://arxiv.org/pdf/1911.08724v1.pdf
PWC	https://paperswithcode.com/paper/fast-and-flexible-image-blind-denoising-via
Repo
Framework

Method for Constructing Artificial Intelligence Player with Abstraction to Markov Decision Processes in Multiplayer Game of Mahjong


Title	Method for Constructing Artificial Intelligence Player with Abstraction to Markov Decision Processes in Multiplayer Game of Mahjong
Authors	Moyuru Kurita, Kunihito Hoki
Abstract	We propose a method for constructing artificial intelligence (AI) of mahjong, which is a multiplayer imperfect information game. Since the size of the game tree is huge, constructing an expert-level AI player of mahjong is challenging. We define multiple Markov decision processes (MDPs) as abstractions of mahjong to construct effective search trees. We also introduce two methods of inferring state values of the original mahjong using these MDPs. We evaluated the effectiveness of our method using gameplays vis-`{a}-vis the current strongest AI player.
Tasks
Published	2019-04-16
URL	http://arxiv.org/abs/1904.07491v1
PDF	http://arxiv.org/pdf/1904.07491v1.pdf
PWC	https://paperswithcode.com/paper/method-for-constructing-artificial
Repo
Framework

Practical Compositional Fairness: Understanding Fairness in Multi-Task ML Systems


Title	Practical Compositional Fairness: Understanding Fairness in Multi-Task ML Systems
Authors	Xuezhi Wang, Nithum Thain, Anu Sinha, Ed H. Chi, Jilin Chen, Alex Beutel
Abstract	Most literature in fairness has focused on improving fairness with respect to one single model or one single objective. However, real-world machine learning systems are usually composed of many different components. Unfortunately, recent research has shown that even if each component is “fair”, the overall system can still be “unfair”. In this paper, we focus on how well fairness composes over multiple components in real systems. We consider two recently proposed fairness metrics for rankings: exposure and pairwise ranking accuracy gap. We provide theory that demonstrates a set of conditions under which fairness of individual models does compose. We then present an analytical framework for both understanding whether a system’s signals can achieve compositional fairness, and diagnosing which of these signals lowers the overall system’s end-to-end fairness the most. Despite previously bleak theoretical results, on multiple data-sets – including a large-scale real-world recommender system – we find that the overall system’s end-to-end fairness is largely achievable by improving fairness in individual components.
Tasks	Recommendation Systems
Published	2019-11-05
URL	https://arxiv.org/abs/1911.01916v2
PDF	https://arxiv.org/pdf/1911.01916v2.pdf
PWC	https://paperswithcode.com/paper/practical-compositional-fairness
Repo
Framework

Post-Training 4-bit Quantization on Embedding Tables


Title	Post-Training 4-bit Quantization on Embedding Tables
Authors	Hui Guan, Andrey Malevich, Jiyan Yang, Jongsoo Park, Hector Yuen
Abstract	Continuous representations have been widely adopted in recommender systems where a large number of entities are represented using embedding vectors. As the cardinality of the entities increases, the embedding components can easily contain millions of parameters and become the bottleneck in both storage and inference due to large memory consumption. This work focuses on post-training 4-bit quantization on the continuous embeddings. We propose row-wise uniform quantization with greedy search and codebook-based quantization that consistently outperforms state-of-the-art quantization approaches on reducing accuracy degradation. We deploy our uniform quantization technique on a production model in Facebook and demonstrate that it can reduce the model size to only 13.89% of the single-precision version while the model quality stays neutral.
Tasks	Quantization, Recommendation Systems
Published	2019-11-05
URL	https://arxiv.org/abs/1911.02079v1
PDF	https://arxiv.org/pdf/1911.02079v1.pdf
PWC	https://paperswithcode.com/paper/post-training-4-bit-quantization-on-embedding
Repo
Framework

Evaluation of an AI System for the Detection of Diabetic Retinopathy from Images Captured with a Handheld Portable Fundus Camera: the MAILOR AI study


Title	Evaluation of an AI System for the Detection of Diabetic Retinopathy from Images Captured with a Handheld Portable Fundus Camera: the MAILOR AI study
Authors	T W Rogers, J Gonzalez-Bueno, R Garcia Franco, E Lopez Star, D Méndez Marín, J Vassallo, V C Lansingh, S Trikha, N Jaccard
Abstract	Objectives: To evaluate the performance of an Artificial Intelligence (AI) system (Pegasus, Visulytix Ltd., UK), at the detection of Diabetic Retinopathy (DR) from images captured by a handheld portable fundus camera. Methods: A cohort of 6,404 patients (~80% with diabetes mellitus) was screened for retinal diseases using a handheld portable fundus camera (Pictor Plus, Volk Optical Inc., USA) at the Mexican Advanced Imaging Laboratory for Ocular Research. The images were graded for DR by specialists according to the Scottish DR grading scheme. The performance of the AI system was evaluated, retrospectively, in assessing Referable DR (RDR) and Proliferative DR (PDR) and compared to the performance on a publicly available desktop camera benchmark dataset. Results: For RDR detection, Pegasus performed with an 89.4% (95% CI: 88.0-90.7) Area Under the Receiver Operating Characteristic (AUROC) curve for the MAILOR cohort, compared to an AUROC of 98.5% (95% CI: 97.8-99.2) on the benchmark dataset. This difference was statistically significant. Moreover, no statistically significant difference was found in performance for PDR detection with Pegasus achieving an AUROC of 94.3% (95% CI: 91.0-96.9) on the MAILOR cohort and 92.2% (95% CI: 89.4-94.8) on the benchmark dataset. Conclusions: Pegasus showed good transferability for the detection of PDR from a curated desktop fundus camera dataset to real-world clinical practice with a handheld portable fundus camera. However, there was a substantial, and statistically significant, decrease in the diagnostic performance for RDR when using the handheld device.
Tasks
Published	2019-08-18
URL	https://arxiv.org/abs/1908.06399v1
PDF	https://arxiv.org/pdf/1908.06399v1.pdf
PWC	https://paperswithcode.com/paper/evaluation-of-an-ai-system-for-the-detection
Repo
Framework

Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach


Title	Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach
Authors	Haoran Sun, Songtao Lu, Mingyi Hong
Abstract	Many modern large-scale machine learning problems benefit from decentralized and stochastic optimization. Recent works have shown that utilizing both decentralized computing and local stochastic gradient estimates can outperform state-of-the-art centralized algorithms, in applications involving highly non-convex problems, such as training deep neural networks. In this work, we propose a decentralized stochastic algorithm to deal with certain smooth non-convex problems where there are $m$ nodes in the system, and each node has a large number of samples (denoted as $n$). Differently from the majority of the existing decentralized learning algorithms for either stochastic or finite-sum problems, our focus is given to both reducing the total communication rounds among the nodes, while accessing the minimum number of local data samples. In particular, we propose an algorithm named D-GET (decentralized gradient estimation and tracking), which jointly performs decentralized gradient estimation (which estimates the local gradient using a subset of local samples) and gradient tracking (which tracks the global full gradient using local estimates). We show that, to achieve certain $\epsilon$ stationary solution of the deterministic finite sum problem, the proposed algorithm achieves an $\mathcal{O}(mn^{1/2}\epsilon^{-1})$ sample complexity and an $\mathcal{O}(\epsilon^{-1})$ communication complexity. These bounds significantly improve upon the best existing bounds of $\mathcal{O}(mn\epsilon^{-1})$ and $\mathcal{O}(\epsilon^{-1})$, respectively. Similarly, for online problems, the proposed method achieves an $\mathcal{O}(m \epsilon^{-3/2})$ sample complexity and an $\mathcal{O}(\epsilon^{-1})$ communication complexity, while the best existing bounds are $\mathcal{O}(m\epsilon^{-2})$ and $\mathcal{O}(\epsilon^{-2})$, respectively.
Tasks	Stochastic Optimization
Published	2019-10-13
URL	https://arxiv.org/abs/1910.05857v1
PDF	https://arxiv.org/pdf/1910.05857v1.pdf
PWC	https://paperswithcode.com/paper/improving-the-sample-and-communication
Repo
Framework

Multi-Scale Geometric Consistency Guided Multi-View Stereo


Title	Multi-Scale Geometric Consistency Guided Multi-View Stereo
Authors	Qingshan Xu, Wenbing Tao
Abstract	In this paper, we propose an efficient multi-scale geometric consistency guided multi-view stereo method for accurate and complete depth map estimation. We first present our basic multi-view stereo method with Adaptive Checkerboard sampling and Multi-Hypothesis joint view selection (ACMH). It leverages structured region information to sample better candidate hypotheses for propagation and infer the aggregation view subset at each pixel. For the depth estimation of low-textured areas, we further propose to combine ACMH with multi-scale geometric consistency guidance (ACMM) to obtain the reliable depth estimates for low-textured areas at coarser scales and guarantee that they can be propagated to finer scales. To correct the erroneous estimates propagated from the coarser scales, we present a novel detail restorer. Experiments on extensive datasets show our method achieves state-of-the-art performance, recovering the depth estimation not only in low-textured areas but also in details.
Tasks	Depth Estimation
Published	2019-04-17
URL	http://arxiv.org/abs/1904.08103v1
PDF	http://arxiv.org/pdf/1904.08103v1.pdf
PWC	https://paperswithcode.com/paper/190408103
Repo
Framework

Beyond image classification: zooplankton identification with deep vector space embeddings


Title	Beyond image classification: zooplankton identification with deep vector space embeddings
Authors	Ketil Malde, Hyeongji Kim
Abstract	Zooplankton images, like many other real world data types, have intrinsic properties that make the design of effective classification systems difficult. For instance, the number of classes encountered in practical settings is potentially very large, and classes can be ambiguous or overlap. In addition, the choice of taxonomy often differs between researchers and between institutions. Although high accuracy has been achieved in benchmarks using standard classifier architectures, biases caused by an inflexible classification scheme can have profound effects when the output is used in ecosystem assessments and monitoring. Here, we propose using a deep convolutional network to construct a vector embedding of zooplankton images. The system maps (embeds) each image into a high-dimensional Euclidean space so that distances between vectors reflect semantic relationships between images. We show that the embedding can be used to derive classifications with comparable accuracy to a specific classifier, but that it simultaneously reveals important structures in the data. Furthermore, we apply the embedding to new classes previously unseen by the system, and evaluate its classification performance in such cases. Traditional neural network classifiers perform well when the classes are clearly defined a priori and have sufficiently large labeled data sets available. For practical cases in ecology as well as in many other fields this is not the case, and we argue that the vector embedding method presented here is a more appropriate approach.
Tasks	Image Classification
Published	2019-09-25
URL	https://arxiv.org/abs/1909.11380v1
PDF	https://arxiv.org/pdf/1909.11380v1.pdf
PWC	https://paperswithcode.com/paper/beyond-image-classification-zooplankton
Repo
Framework

Seismic Bayesian evidential learning: Estimation and uncertainty quantification of sub-resolution reservoir properties


Title	Seismic Bayesian evidential learning: Estimation and uncertainty quantification of sub-resolution reservoir properties
Authors	Anshuman Pradhan, Tapan Mukerji
Abstract	We present a framework that enables estimation of low-dimensional sub-resolution reservoir properties directly from seismic data, without requiring the solution of a high dimensional seismic inverse problem. Our workflow is based on the Bayesian evidential learning approach and exploits learning the direct relation between seismic data and reservoir properties to efficiently estimate reservoir properties. The theoretical framework we develop allows incorporation of non-linear statistical models for seismic estimation problems. Uncertainty quantification is performed with Approximate Bayesian Computation. With the help of a synthetic example of estimation of reservoir net-to-gross and average fluid saturations in sub-resolution thin-sand reservoir, several nuances are foregrounded regarding the applicability of unsupervised and supervised learning methods for seismic estimation problems. Finally, we demonstrate the efficacy of our approach by estimating posterior uncertainty of reservoir net-to-gross in sub-resolution thin-sand reservoir from an offshore delta dataset using 3D pre-stack seismic data.
Tasks
Published	2019-05-14
URL	https://arxiv.org/abs/1905.05508v1
PDF	https://arxiv.org/pdf/1905.05508v1.pdf
PWC	https://paperswithcode.com/paper/seismic-bayesian-evidential-learning
Repo
Framework

Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat


Title	Unsupervised and interpretable scene discovery with Discrete-Attend-Infer-Repeat
Authors	Duo Wang, Mateja Jamnik, Pietro Lio
Abstract	In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a Recurrent Auto-Encoder with structured latent distributions containing discrete categorical distributions, continuous attribute distributions, and factorised spatial attention. While inspired by the original AIR model andretaining AIR model’s capability in identifying objects in an image, Discrete-AIR provides direct interpretability of the latent codes. We show that for Multi-MNIST and a multiple-objects version of dSprites dataset, the Discrete-AIR model needs just one categorical latent variable, one attribute variable (for Multi-MNIST only), together with spatial attention variables, for efficient inference. We perform analysis to show that the learnt categorical distributions effectively capture the categories of objects in the scene for Multi-MNIST and for Multi-Sprites.
Tasks
Published	2019-03-14
URL	http://arxiv.org/abs/1903.06581v1
PDF	http://arxiv.org/pdf/1903.06581v1.pdf
PWC	https://paperswithcode.com/paper/unsupervised-and-interpretable-scene
Repo
Framework