Paper Group ANR 226
Ranking a set of objects: a graph based least-square approach. MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning. Active Preference Elicitation via Adjustable Robust Optimization. Multi-Task Reinforcement Learning with Soft Modularization. Learned Weight Sharing for Deep Multi-Task Learning by Natural Evo …
Ranking a set of objects: a graph based least-square approach
| Title | Ranking a set of objects: a graph based least-square approach |
| Authors | Evgenia Christoforou, Alessandro Nordio, Alberto Tarable, Emilio Leonardi |
| Abstract | We consider the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of equal workers. We assume that objects are endowed with intrinsic qualities and that the probability with which an object is preferred to another depends only on the difference between the qualities of the two competitors. We propose a class of non-adaptive ranking algorithms that rely on a least-squares optimization criterion for the estimation of qualities. Such algorithms are shown to be asymptotically optimal (i.e., they require $O(\frac{N}{\epsilon^2}\log \frac{N}{\delta})$ comparisons to be $(\epsilon, \delta)$-PAC). Numerical results show that our schemes are very efficient also in many non-asymptotic scenarios exhibiting a performance similar to the maximum-likelihood algorithm. Moreover, we show how they can be extended to adaptive schemes and test them on real-world datasets. |
| Tasks | |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11590v1 |
| https://arxiv.org/pdf/2002.11590v1.pdf | |
| PWC | https://paperswithcode.com/paper/ranking-a-set-of-objects-a-graph-based-least |
| Repo | |
| Framework | |
MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
| Title | MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning |
| Authors | Yuan Gao, Haoping Bai, Zequn Jie, Jiayi Ma, Kui Jia, Wei Liu |
| Abstract | We propose to incorporate neural architecture search (NAS) into general-purpose multi-task learning (GP-MTL). Existing NAS methods typically define different search spaces according to different tasks. In order to adapt to different task combinations (i.e., task sets), we disentangle the GP-MTL networks into single-task backbones (optionally encode the task priors), and a hierarchical and layerwise features sharing/fusing scheme across them. This enables us to design a novel and general task-agnostic search space, which inserts cross-task edges (i.e., feature fusion connections) into fixed single-task network backbones. Moreover, we also propose a novel single-shot gradient-based search algorithm that closes the performance gap between the searched architectures and the final evaluation architecture. This is realized with a minimum entropy regularization on the architecture weights during the search phase, which makes the architecture weights converge to near-discrete values and therefore achieves a single model. As a result, our searched model can be directly used for evaluation without (re-)training from scratch. We perform extensive experiments using different single-task backbones on various task sets, demonstrating the promising performance obtained by exploiting the hierarchical and layerwise features, as well as the desirable generalizability to different i) task sets and ii) single-task backbones. The code of our paper is available at https://github.com/bhpfelix/MTLNAS. |
| Tasks | Multi-Task Learning, Neural Architecture Search |
| Published | 2020-03-31 |
| URL | https://arxiv.org/abs/2003.14058v1 |
| https://arxiv.org/pdf/2003.14058v1.pdf | |
| PWC | https://paperswithcode.com/paper/mtl-nas-task-agnostic-neural-architecture |
| Repo | |
| Framework | |
Active Preference Elicitation via Adjustable Robust Optimization
| Title | Active Preference Elicitation via Adjustable Robust Optimization |
| Authors | Phebe Vayanos, Duncan McElfresh, Yingxiao Ye, John Dickerson, Eric Rice |
| Abstract | We consider the problem faced by a recommender system which seeks to offer a user with unknown preferences an item. Before making a recommendation, the system has the opportunity to elicit the user’s preferences by making queries. Each query corresponds to a pairwise comparison between items. We take the point of view of either a risk averse or regret averse recommender system which only possess set-based information on the user utility function. We investigate: a) an offline elicitation setting, where all queries are made at once, and b) an online elicitation setting, where queries are selected sequentially over time. We propose exact robust optimization formulations of these problems which integrate the elicitation and recommendation phases and study the complexity of these problems. For the offline case, where the problem takes the form of a two-stage robust optimization problem with decision-dependent information discovery, we provide an enumeration-based algorithm and also an equivalent reformulation in the form of a mixed-binary linear program which we solve via column-and-constraint generation. For the online setting, where the problem takes the form of a multi-stage robust optimization problem with decision-dependent information discovery, we propose a conservative solution approach. We evaluate the performance of our methods on both synthetic data and real data from the Homeless Management Information System. We simulate elicitation of the preferences of policy-makers in terms of characteristics of housing allocation policies to better match individuals experiencing homelessness to scarce housing resources. Our framework is shown to outperform the state-of-the-art techniques from the literature. |
| Tasks | Recommendation Systems |
| Published | 2020-03-04 |
| URL | https://arxiv.org/abs/2003.01899v1 |
| https://arxiv.org/pdf/2003.01899v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-preference-elicitation-via-adjustable |
| Repo | |
| Framework | |
Multi-Task Reinforcement Learning with Soft Modularization
| Title | Multi-Task Reinforcement Learning with Soft Modularization |
| Authors | Ruihan Yang, Huazhe Xu, Yi Wu, Xiaolong Wang |
| Abstract | Multi-task learning is a very challenging problem in reinforcement learning. While training multiple tasks jointly allow the policies to share parameters across different tasks, the optimization problem becomes non-trivial: It is unclear what parameters in the network should be reused across tasks, and the gradients from different tasks may interfere with each other. Thus, instead of naively sharing parameters across tasks, we introduce an explicit modularization technique on policy representation to alleviate this optimization issue. Given a base policy network, we design a routing network which estimates different routing strategies to reconfigure the base network for each task. Instead of creating a concrete route for each task, our task-specific policy is represented by a soft combination of all possible routes. We name this approach soft modularization. We experiment with multiple robotics manipulation tasks in simulation and show our method improves sample efficiency and performance over baselines by a large margin. |
| Tasks | Multi-Task Learning |
| Published | 2020-03-30 |
| URL | https://arxiv.org/abs/2003.13661v1 |
| https://arxiv.org/pdf/2003.13661v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-task-reinforcement-learning-with-soft |
| Repo | |
| Framework | |
Learned Weight Sharing for Deep Multi-Task Learning by Natural Evolution Strategy and Stochastic Gradient Descent
| Title | Learned Weight Sharing for Deep Multi-Task Learning by Natural Evolution Strategy and Stochastic Gradient Descent |
| Authors | Jonas Prellberg, Oliver Kramer |
| Abstract | In deep multi-task learning, weights of task-specific networks are shared between tasks to improve performance on each single one. Since the question, which weights to share between layers, is difficult to answer, human-designed architectures often share everything but a last task-specific layer. In many cases, this simplistic approach severely limits performance. Instead, we propose an algorithm to learn the assignment between a shared set of weights and task-specific layers. To optimize the non-differentiable assignment and at the same time train the differentiable weights, learning takes place via a combination of natural evolution strategy and stochastic gradient descent. The end result are task-specific networks that share weights but allow independent inference. They achieve lower test errors than baselines and methods from literature on three multi-task learning datasets. |
| Tasks | Multi-Task Learning |
| Published | 2020-03-23 |
| URL | https://arxiv.org/abs/2003.10159v1 |
| https://arxiv.org/pdf/2003.10159v1.pdf | |
| PWC | https://paperswithcode.com/paper/learned-weight-sharing-for-deep-multi-task |
| Repo | |
| Framework | |
Automatic Identification of Types of Alterations in Historical Manuscripts
| Title | Automatic Identification of Types of Alterations in Historical Manuscripts |
| Authors | David Lassner, Anne Baillot, Sergej Dogadov, Klaus-Robert Müller, Shinichi Nakajima |
| Abstract | Alterations in historical manuscripts such as letters represent a promising field of research. On the one hand, they help understand the construction of text. On the other hand, topics that are being considered sensitive at the time of the manuscript gain coherence and contextuality when taking alterations into account, especially in the case of deletions. The analysis of alterations in manuscripts, though, is a traditionally very tedious work. In this paper, we present a machine learning-based approach to help categorize alterations in documents. In particular, we present a new probabilistic model (Alteration Latent Dirichlet Allocation, alterLDA in the following) that categorizes content-related alterations. The method proposed here is developed based on experiments carried out on the digital scholarly edition Berlin Intellectuals, for which alterLDA achieves high performance in the recognition of alterations on labelled data. On unlabelled data, applying alterLDA leads to interesting new insights into the alteration behavior of authors, editors and other manuscript contributors, as well as insights into sensitive topics in the correspondence of Berlin intellectuals around 1800. In addition to the findings based on the digital scholarly edition Berlin Intellectuals, we present a general framework for the analysis of text genesis that can be used in the context of other digital resources representing document variants. To that end, we present in detail the methodological steps that are to be followed in order to achieve such results, giving thereby a prime example of an Machine Learning application the Digital Humanities. |
| Tasks | |
| Published | 2020-03-20 |
| URL | https://arxiv.org/abs/2003.09136v2 |
| https://arxiv.org/pdf/2003.09136v2.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-identification-of-types-of |
| Repo | |
| Framework | |
Multivariate Gaussian Variational Inference by Natural Gradient Descent
| Title | Multivariate Gaussian Variational Inference by Natural Gradient Descent |
| Authors | Timothy D. Barfoot |
| Abstract | This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there are some advantages to choosing a parameterization comprising the mean and inverse covariance matrix and provide a simple NGD update that accounts for the symmetric (and sparse) nature of the inverse covariance matrix. |
| Tasks | |
| Published | 2020-01-27 |
| URL | https://arxiv.org/abs/2001.10025v1 |
| https://arxiv.org/pdf/2001.10025v1.pdf | |
| PWC | https://paperswithcode.com/paper/multivariate-gaussian-variational-inference |
| Repo | |
| Framework | |
Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation
| Title | Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation |
| Authors | Shawn Mathew, Saad Nadeem, Sruti Kumari, Arie Kaufman |
| Abstract | Colorectal cancer screening modalities, such as optical colonoscopy (OC) and virtual colonoscopy (VC), are critical for diagnosing and ultimately removing polyps (precursors of colon cancer). The non-invasive VC is normally used to inspect a 3D reconstructed colon (from CT scans) for polyps and if found, the OC procedure is performed to physically traverse the colon via endoscope and remove these polyps. In this paper, we present a deep learning framework, Extended and Directional CycleGAN, for lossy unpaired image-to-image translation between OC and VC to augment OC video sequences with scale-consistent depth information from VC, and augment VC with patient-specific textures, color and specular highlights from OC (e.g, for realistic polyp synthesis). Both OC and VC contain structural information, but it is obscured in OC by additional patient-specific texture and specular highlights, hence making the translation from OC to VC lossy. The existing CycleGAN approaches do not handle lossy transformations. To address this shortcoming, we introduce an extended cycle consistency loss, which compares the geometric structures from OC in the VC domain. This loss removes the need for the CycleGAN to embed OC information in the VC domain. To handle a stronger removal of the textures and lighting, a Directional Discriminator is introduced to differentiate the direction of translation (by creating paired information for the discriminator), as opposed to the standard CycleGAN which is direction-agnostic. Combining the extended cycle consistency loss and the Directional Discriminator, we show state-of-the-art results on scale-consistent depth inference for phantom, textured VC and for real polyp and normal colon video sequences. We also present results for realistic pendunculated and flat polyp synthesis from bumps introduced in 3D VC models. |
| Tasks | Image-to-Image Translation |
| Published | 2020-03-27 |
| URL | https://arxiv.org/abs/2003.12473v1 |
| https://arxiv.org/pdf/2003.12473v1.pdf | |
| PWC | https://paperswithcode.com/paper/augmenting-colonoscopy-using-extended-and |
| Repo | |
| Framework | |
GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling
| Title | GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling |
| Authors | Yahui Liu, Marco De Nadai, Jian Yao, Nicu Sebe, Bruno Lepri, Xavier Alameda-Pineda |
| Abstract | Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images. Recent studies have shown remarkable success for multiple domains but they suffer from two main limitations: they are either built from several two-domain mappings that are required to be learned independently, or they generate low-diversity results, a problem known as mode collapse. To overcome these limitations, we propose a method named GMM-UNIT, which is based on a content-attribute disentangled representation where the attribute space is fitted with a GMM. Each GMM component represents a domain, and this simple assumption has two prominent advantages. First, it can be easily extended to most multi-domain and multi-modal image-to-image translation tasks. Second, the continuous domain encoding allows for interpolation between domains and for extrapolation to unseen domains and translations. Additionally, we show how GMM-UNIT can be constrained down to different methods in the literature, meaning that GMM-UNIT is a unifying framework for unsupervised image-to-image translation. |
| Tasks | Image-to-Image Translation, Unsupervised Image-To-Image Translation |
| Published | 2020-03-15 |
| URL | https://arxiv.org/abs/2003.06788v2 |
| https://arxiv.org/pdf/2003.06788v2.pdf | |
| PWC | https://paperswithcode.com/paper/gmm-unit-unsupervised-multi-domain-and-multi-1 |
| Repo | |
| Framework | |
Trained Trajectory based Automated Parking System using Visual SLAM
| Title | Trained Trajectory based Automated Parking System using Visual SLAM |
| Authors | Nivedita Tripathi, Senthil Yogamani |
| Abstract | Automated Parking is becoming a standard feature in modern vehicles. Existing parking systems build a local map to be able to plan for maneuvering towards a detected slot. Next generation parking systems have an use case where they build a persistent map of the environment where the car is frequently parked, say for example, home parking or office parking. The pre-built map helps in re-localizing the vehicle better when its trying to park the next time. This is achieved by augmenting the parking system with a Visual SLAM pipeline and the feature is called trained trajectory parking. In this paper, we discuss the use cases, design and implementation of a trained trajectory automated parking system. To encourage further research, we release a dataset of 50 video sequences comprising of over 100,000 images with the associated ground truth as a companion to our WoodScape dataset \cite{Yogamani_2019_ICCV}. To the best of the authors’ knowledge, this is the first public dataset for trained trajectory parking system scenarios. |
| Tasks | |
| Published | 2020-01-07 |
| URL | https://arxiv.org/abs/2001.02161v2 |
| https://arxiv.org/pdf/2001.02161v2.pdf | |
| PWC | https://paperswithcode.com/paper/trained-trajectory-based-automated-parking |
| Repo | |
| Framework | |
Variational Auto-Encoder: not all failures are equal
| Title | Variational Auto-Encoder: not all failures are equal |
| Authors | Michele Sebag, Victor Berger, Michèle Sebag |
| Abstract | We claim that a source of severe failures for Variational Auto-Encoders is the choice of the distribution class used for the observation model.A first theoretical and experimental contribution of the paper is to establish that even in the large sample limit with arbitrarily powerful neural architectures and latent space, the VAE failsif the sharpness of the distribution class does not match the scale of the data.Our second claim is that the distribution sharpness must preferably be learned by the VAE (as opposed to, fixed and optimized offline): Autonomously adjusting this sharpness allows the VAE to dynamically control the trade-off between the optimization of the reconstruction loss and the latent compression. A second empirical contribution is to show how the control of this trade-off is instrumental in escaping poor local optima, akin a simulated annealing schedule.Both claims are backed upon experiments on artificial data, MNIST and CelebA, showing how sharpness learning addresses the notorious VAE blurriness issue. |
| Tasks | |
| Published | 2020-03-04 |
| URL | https://arxiv.org/abs/2003.01972v1 |
| https://arxiv.org/pdf/2003.01972v1.pdf | |
| PWC | https://paperswithcode.com/paper/variational-auto-encoder-not-all-failures-are |
| Repo | |
| Framework | |
Depthwise-STFT based separable Convolutional Neural Networks
| Title | Depthwise-STFT based separable Convolutional Neural Networks |
| Authors | Sudhakar Kumawat, Shanmuganathan Raman |
| Abstract | In this paper, we propose a new convolutional layer called Depthwise-STFT Separable layer that can serve as an alternative to the standard depthwise separable convolutional layer. The construction of the proposed layer is inspired by the fact that the Fourier coefficients can accurately represent important features such as edges in an image. It utilizes the Fourier coefficients computed (channelwise) in the 2D local neighborhood (e.g., 3x3) of each position of the input map to obtain the feature maps. The Fourier coefficients are computed using 2D Short Term Fourier Transform (STFT) at multiple fixed low frequency points in the 2D local neighborhood at each position. These feature maps at different frequency points are then linearly combined using trainable pointwise (1x1) convolutions. We show that the proposed layer outperforms the standard depthwise separable layer-based models on the CIFAR-10 and CIFAR-100 image classification datasets with reduced space-time complexity. |
| Tasks | Image Classification |
| Published | 2020-01-27 |
| URL | https://arxiv.org/abs/2001.09912v1 |
| https://arxiv.org/pdf/2001.09912v1.pdf | |
| PWC | https://paperswithcode.com/paper/depthwise-stft-based-separable-convolutional |
| Repo | |
| Framework | |
Partial Queries for Constraint Acquisition
| Title | Partial Queries for Constraint Acquisition |
| Authors | Christian Bessiere, Clement Carbonnel, Anton Dries, Emmanuel Hebrard, George Katsirelos, Nadjib Lazaar, Nina Narodytska, Claude-Guy Quimper, Kostas Stergiou, Dimosthenis C. Tsouros, Toby Walsh |
| Abstract | Learning constraint networks is known to require a number of membership queries exponential in the number of variables. In this paper, we learn constraint networks by asking the user partial queries. That is, we ask the user to classify assignments to subsets of the variables as positive or negative. We provide an algorithm, called QUACQ, that, given a negative example, focuses onto a constraint of the target network in a number of queries logarithmic in the size of the example. The whole constraint network can then be learned with a polynomial number of partial queries. We give information theoretic lower bounds for learning some simple classes of constraint networks and show that our generic algorithm is optimal in some cases. |
| Tasks | |
| Published | 2020-03-14 |
| URL | https://arxiv.org/abs/2003.06649v1 |
| https://arxiv.org/pdf/2003.06649v1.pdf | |
| PWC | https://paperswithcode.com/paper/partial-queries-for-constraint-acquisition |
| Repo | |
| Framework | |
Teacher-Student Domain Adaptation for Biosensor Models
| Title | Teacher-Student Domain Adaptation for Biosensor Models |
| Authors | Lawrence G. Phillips, David B. Grimes, Yihan Jessie Li |
| Abstract | We present an approach to domain adaptation, addressing the case where data from the source domain is abundant, labelled data from the target domain is limited or non-existent, and a small amount of paired source-target data is available. The method is designed for developing deep learning models that detect the presence of medical conditions based on data from consumer-grade portable biosensors. It addresses some of the key problems in this area, namely, the difficulty of acquiring large quantities of clinically labelled data from the biosensor, and the noise and ambiguity that can affect the clinical labels. The idea is to pre-train an expressive model on a large dataset of labelled recordings from a sensor modality for which data is abundant, and then to adapt the model’s lower layers so that its predictions on the target modality are similar to the original model’s on paired examples from the source modality. We show that the pre-trained model’s predictions provide a substantially better learning signal than the clinician-provided labels, and that this teacher-student technique significantly outperforms both a naive application of supervised deep learning and a label-supervised version of domain adaptation on a synthetic dataset and in a real-world case study on sleep apnea. By reducing the volume of data required and obviating the need for labels, our approach should reduce the cost associated with developing high-performance deep learning models for biosensors. |
| Tasks | Domain Adaptation |
| Published | 2020-03-17 |
| URL | https://arxiv.org/abs/2003.07896v2 |
| https://arxiv.org/pdf/2003.07896v2.pdf | |
| PWC | https://paperswithcode.com/paper/teacher-student-domain-adaptation-for |
| Repo | |
| Framework | |
Amateur Drones Detection: A machine learning approach utilizing the acoustic signals in the presence of strong interference
| Title | Amateur Drones Detection: A machine learning approach utilizing the acoustic signals in the presence of strong interference |
| Authors | Zahoor Uddin, Muhammad Altaf, Muhammad Bilal, Lewis Nkenyereye, Ali Kashif Bashir |
| Abstract | Owing to small size, sensing capabilities and autonomous nature, the Unmanned Air Vehicles (UAVs) have enormous applications in various areas, e.g., remote sensing, navigation, archaeology, journalism, environmental science, and agriculture. However, the unmonitored deployment of UAVs called the amateur drones (AmDr) can lead to serious security threats and risk to human life and infrastructure. Therefore, timely detection of the AmDr is essential for the protection and security of sensitive organizations, human life and other vital infrastructure. AmDrs can be detected using different techniques based on sound, video, thermal, and radio frequencies. However, the performance of these techniques is limited in sever atmospheric conditions. In this paper, we propose an efficient unsupervise machine learning approach of independent component analysis (ICA) to detect various acoustic signals i.e., sounds of bird, airplanes, thunderstorm, rain, wind and the UAVs in practical scenario. After unmixing the signals, the features like Mel Frequency Cepstral Coefficients (MFCC), the power spectral density (PSD) and the Root Mean Square Value (RMS) of the PSD are extracted by using ICA. The PSD and the RMS of PSD signals are extracted by first passing the signals from octave band filter banks. Based on the above features the signals are classified using Support Vector Machines (SVM) and K Nearest Neighbor (KNN) to detect the presence or absence of AmDr. Unique feature of the proposed technique is the detection of a single or multiple AmDrs at a time in the presence of multiple acoustic interfering signals. The proposed technique is verified through extensive simulations and it is observed that the RMS values of PSD with KNN performs better than the MFCC with KNN and SVM. |
| Tasks | |
| Published | 2020-02-28 |
| URL | https://arxiv.org/abs/2003.01519v1 |
| https://arxiv.org/pdf/2003.01519v1.pdf | |
| PWC | https://paperswithcode.com/paper/amateur-drones-detection-a-machine-learning |
| Repo | |
| Framework | |
