Paper Group ANR 1140
Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating. Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels. STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks. Seeker: Real-Time Interactive Search. Combining SMT and NMT Back-Translat …
Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating
| Title | Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating |
| Authors | Xin Yao, Tianchi Huang, Rui-Xiao Zhang, Ruiyu Li, Lifeng Sun |
| Abstract | Federated Averaging (FedAvg) serves as the fundamental framework in Federated Learning (FL) settings. However, we argue that 1) the multiple steps of local updating will result in gradient biases and 2) there is an inconsistency between the target distribution and the optimization objectives following the training paradigm in FedAvg. To tackle these problems, we first propose an unbiased gradient aggregation algorithm with the keep-trace gradient descent and gradient evaluation strategy. Then we introduce a meta updating procedure with a controllable meta training set to provide a clear and consistent optimization objective. Experimental results demonstrate that the proposed methods outperform compared ones with various network architectures in both the IID and non-IID FL settings. |
| Tasks | |
| Published | 2019-10-18 |
| URL | https://arxiv.org/abs/1910.08234v2 |
| https://arxiv.org/pdf/1910.08234v2.pdf | |
| PWC | https://paperswithcode.com/paper/federated-learning-with-unbiased-gradient |
| Repo | |
| Framework | |
Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels
| Title | Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels |
| Authors | Sebastian Guendel, Florin C. Ghesu, Sasa Grbic, Eli Gibson, Bogdan Georgescu, Andreas Maier, Dorin Comaniciu |
| Abstract | Chest X-ray (CXR) is the most common X-ray examination performed in daily clinical practice for the diagnosis of various heart and lung abnormalities. The large amount of data to be read and reported, with 100+ studies per day for a single radiologist, poses a challenge in maintaining consistently high interpretation accuracy. In this work, we propose a method for the classification of different abnormalities based on CXR scans of the human body. The system is based on a novel multi-task deep learning architecture that in addition to the abnormality classification, supports the segmentation of the lungs and heart and classification of regions where the abnormality is located. We demonstrate that by training these tasks concurrently, one can increase the classification performance of the model. Experiments were performed on an extensive collection of 297,541 chest X-ray images from 86,876 patients, leading to a state-of-the-art performance level of 0.883 AUC on average for 12 different abnormalities. We also conducted a detailed performance analysis and compared the accuracy of our system with 3 board-certified radiologists. In this context, we highlight the high level of label noise inherent to this problem. On a reduced subset containing only cases with high confidence reference labels based on the consensus of the 3 radiologists, our system reached an average AUC of 0.945. |
| Tasks | Multi-Task Learning |
| Published | 2019-05-15 |
| URL | https://arxiv.org/abs/1905.06362v1 |
| https://arxiv.org/pdf/1905.06362v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-task-learning-for-chest-x-ray |
| Repo | |
| Framework | |
STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks
| Title | STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks |
| Authors | Shuochao Yao, Ailing Piao, Wenjun Jiang, Yiran Zhao, Huajie Shao, Shengzhong Liu, Dongxin Liu, Jinyang Li, Tianshi Wang, Shaohan Hu, Lu Su, Jiawei Han, Tarek Abdelzaher |
| Abstract | Recent advances in deep learning motivate the use of deep neural networks in Internet-of-Things (IoT) applications. These networks are modelled after signal processing in the human brain, thereby leading to significant advantages at perceptual tasks such as vision and speech recognition. IoT applications, however, often measure physical phenomena, where the underlying physics (such as inertia, wireless signal propagation, or the natural frequency of oscillation) are fundamentally a function of signal frequencies, offering better features in the frequency domain. This observation leads to a fundamental question: For IoT applications, can one develop a new brand of neural network structures that synthesize features inspired not only by the biology of human perception but also by the fundamental nature of physics? Hence, in this paper, instead of using conventional building blocks (e.g., convolutional and recurrent layers), we propose a new foundational neural network building block, the Short-Time Fourier Neural Network (STFNet). It integrates a widely-used time-frequency analysis method, the Short-Time Fourier Transform, into data processing to learn features directly in the frequency domain, where the physics of underlying phenomena leave better foot-prints. STFNets bring additional flexibility to time-frequency analysis by offering novel nonlinear learnable operations that are spectral-compatible. Moreover, STFNets show that transforming signals to a domain that is more connected to the underlying physics greatly simplifies the learning process. We demonstrate the effectiveness of STFNets with extensive experiments. STFNets significantly outperform the state-of-the-art deep learning models in all experiments. A STFNet, therefore, demonstrates superior capability as the fundamental building block of deep neural networks for IoT applications for various sensor inputs. |
| Tasks | Speech Recognition |
| Published | 2019-02-21 |
| URL | http://arxiv.org/abs/1902.07849v1 |
| http://arxiv.org/pdf/1902.07849v1.pdf | |
| PWC | https://paperswithcode.com/paper/stfnets-learning-sensing-signals-from-the |
| Repo | |
| Framework | |
Seeker: Real-Time Interactive Search
| Title | Seeker: Real-Time Interactive Search |
| Authors | Ari Biswas, Thai T Pham, Michael Vogelsong, Benjamin Snyder, Houssam Nassif |
| Abstract | This paper introduces Seeker, a system that allows users to interactively refine search rankings in real time, through feedback in the form of likes and dislikes. When searching online, users may not know how to accurately describe their product of choice in words. An alternative approach is to search an embedding space, allowing the user to query using a representation of the item (like a tune for a song, or a picture for an object). However, this approach requires the user to possess an example representation of their desired item. Additionally, most current search systems do not allow the user to dynamically adapt the results with further feedback. On the other hand, users often have a mental picture of the desired item and are able to answer ordinal questions of the form: “Is this item similar to what you have in mind?” With this assumption, our algorithm allows for users to provide sequential feedback on search results to adapt the search feed. We show that our proposed approach works well both qualitatively and quantitatively. Unlike most previous representation-based search systems, we can quantify the quality of our algorithm by evaluating humans-in-the-loop experiments. |
| Tasks | |
| Published | 2019-05-17 |
| URL | https://arxiv.org/abs/1905.13125v1 |
| https://arxiv.org/pdf/1905.13125v1.pdf | |
| PWC | https://paperswithcode.com/paper/190513125 |
| Repo | |
| Framework | |
Combining SMT and NMT Back-Translated Data for Efficient NMT
| Title | Combining SMT and NMT Back-Translated Data for Efficient NMT |
| Authors | Alberto Poncelas, Maja Popovic, Dimitar Shterionov, Gideon Maillette de Buy Wenniger, Andy Way |
| Abstract | Neural Machine Translation (NMT) models achieve their best performance when large sets of parallel data are used for training. Consequently, techniques for augmenting the training set have become popular recently. One of these methods is back-translation (Sennrich et al., 2016), which consists on generating synthetic sentences by translating a set of monolingual, target-language sentences using a Machine Translation (MT) model. Generally, NMT models are used for back-translation. In this work, we analyze the performance of models when the training data is extended with synthetic data using different MT approaches. In particular we investigate back-translated data generated not only by NMT but also by Statistical Machine Translation (SMT) models and combinations of both. The results reveal that the models achieve the best performances when the training set is augmented with back-translated data created by merging different MT approaches. |
| Tasks | Machine Translation |
| Published | 2019-09-09 |
| URL | https://arxiv.org/abs/1909.03750v1 |
| https://arxiv.org/pdf/1909.03750v1.pdf | |
| PWC | https://paperswithcode.com/paper/combining-smt-and-nmt-back-translated-data |
| Repo | |
| Framework | |
Debugging Frame Semantic Role Labeling
| Title | Debugging Frame Semantic Role Labeling |
| Authors | Alexandre Kabbach |
| Abstract | We propose a quantitative and qualitative analysis of the performances of statistical models for frame semantic structure extraction. We report on a replication study on FrameNet 1.7 data and show that preprocessing toolkits play a major role in argument identification performances, observing gains similar in their order of magnitude to those reported by recent models for frame semantic parsing. We report on the robustness of a recent statistical classifier for frame semantic parsing to lexical configurations of predicate-argument structures, relying on an artificially augmented dataset generated using a rule-based algorithm combining valence pattern matching and lexical substitution. We prove that syntactic pre-processing plays a major role in the performances of statistical classifiers to argument identification, and discuss the core reasons of syntactic mismatch between dependency parsers output and FrameNet syntactic formalism. Finally, we suggest new leads for improving statistical models for frame semantic parsing, including joint syntax-semantic parsing relying on FrameNet syntactic formalism, latent classes inference via split-and-merge algorithms and neural network architectures relying on rich input representations of words. |
| Tasks | Semantic Parsing, Semantic Role Labeling |
| Published | 2019-01-22 |
| URL | http://arxiv.org/abs/1901.07475v1 |
| http://arxiv.org/pdf/1901.07475v1.pdf | |
| PWC | https://paperswithcode.com/paper/debugging-frame-semantic-role-labeling |
| Repo | |
| Framework | |
Learning Optimal Linear Regularizers
| Title | Learning Optimal Linear Regularizers |
| Authors | Matthew Streeter |
| Abstract | We present algorithms for efficiently learning regularizers that improve generalization. Our approach is based on the insight that regularizers can be viewed as upper bounds on the generalization gap, and that reducing the slack in the bound can improve performance on test data. For a broad class of regularizers, the hyperparameters that give the best upper bound can be computed using linear programming. Under certain Bayesian assumptions, solving the LP lets us “jump” to the optimal hyperparameters given very limited data. This suggests a natural algorithm for tuning regularization hyperparameters, which we show to be effective on both real and synthetic data. |
| Tasks | |
| Published | 2019-02-19 |
| URL | http://arxiv.org/abs/1902.07234v2 |
| http://arxiv.org/pdf/1902.07234v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-optimal-linear-regularizers |
| Repo | |
| Framework | |
Recovering Facial Reflectance and Geometry from Multi-view Images
| Title | Recovering Facial Reflectance and Geometry from Multi-view Images |
| Authors | Guoxian Song, Jianmin Zheng, Jianfei Cai, Tat-Jen Cham |
| Abstract | While the problem of estimating shapes and diffuse reflectances of human faces from images has been extensively studied, there is relatively less work done on recovering the specular albedo. This paper presents a lightweight solution for inferring photorealistic facial reflectance and geometry. Our system processes video streams from two views of a subject, and outputs two reflectance maps for diffuse and specular albedos, as well as a vector map of surface normals. A model-based optimization approach is used, consisting of the three stages of multi-view face model fitting, facial reflectance inference and facial geometry refinement. Our approach is based on a novel formulation built upon the 3D morphable model (3DMM) for representing 3D textured faces in conjunction with the Blinn-Phong reflection model. It has the advantage of requiring only a simple setup with two video streams, and is able to exploit the interaction between the diffuse and specular reflections across multiple views as well as time frames. As a result, the method is able to reliably recover high-fidelity facial reflectance and geometry, which facilitates various applications such as generating photorealistic facial images under new viewpoints or illumination conditions. |
| Tasks | |
| Published | 2019-11-27 |
| URL | https://arxiv.org/abs/1911.11999v1 |
| https://arxiv.org/pdf/1911.11999v1.pdf | |
| PWC | https://paperswithcode.com/paper/recovering-facial-reflectance-and-geometry |
| Repo | |
| Framework | |
Harnessing Slow Dynamics in Neuromorphic Computation
| Title | Harnessing Slow Dynamics in Neuromorphic Computation |
| Authors | Tianlin Liu |
| Abstract | Neuromorphic Computing is a nascent research field in which models and devices are designed to process information by emulating biological neural systems. Thanks to their superior energy efficiency, analog neuromorphic systems are highly promising for embedded, wearable, and implantable systems. However, optimizing neural networks deployed on these systems is challenging. One main challenge is the so-called timescale mismatch: Dynamics of analog circuits tend to be too fast to process real-time sensory inputs. In this thesis, we propose a few working solutions to slow down dynamics of on-chip spiking neural networks. We empirically show that, by harnessing slow dynamics, spiking neural networks on analog neuromorphic systems can gain non-trivial performance boosts on a battery of real-time signal processing tasks. |
| Tasks | |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.12116v1 |
| https://arxiv.org/pdf/1905.12116v1.pdf | |
| PWC | https://paperswithcode.com/paper/harnessing-slow-dynamics-in-neuromorphic |
| Repo | |
| Framework | |
Maximum Entropy Based Significance of Itemsets
| Title | Maximum Entropy Based Significance of Itemsets |
| Authors | Nikolaj Tatti |
| Abstract | We consider the problem of defining the significance of an itemset. We say that the itemset is significant if we are surprised by its frequency when compared to the frequencies of its sub-itemsets. In other words, we estimate the frequency of the itemset from the frequencies of its sub-itemsets and compute the deviation between the real value and the estimate. For the estimation we use Maximum Entropy and for measuring the deviation we use Kullback-Leibler divergence. A major advantage compared to the previous methods is that we are able to use richer models whereas the previous approaches only measure the deviation from the independence model. We show that our measure of significance goes to zero for derivable itemsets and that we can use the rank as a statistical test. Our empirical results demonstrate that for our real datasets the independence assumption is too strong but applying more flexible models leads to good results. |
| Tasks | |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10632v2 |
| http://arxiv.org/pdf/1904.10632v2.pdf | |
| PWC | https://paperswithcode.com/paper/maximum-entropy-based-significance-of |
| Repo | |
| Framework | |
Relation-Aware Pyramid Network (RapNet) for temporal action proposal
| Title | Relation-Aware Pyramid Network (RapNet) for temporal action proposal |
| Authors | Jialin Gao, Zhixiang Shi, Jiani Li, Yufeng Yuan, Jiwei Li, Xi Zhou |
| Abstract | In this technical report, we describe our solution to temporal action proposal (task 1) in ActivityNet Challenge 2019. First, we fine-tune a ResNet-50-C3D CNN on ActivityNet v1.3 based on Kinetics pretrained model to extract snippet-level video representations and then we design a Relation-Aware Pyramid Network (RapNet) to generate temporal multiscale proposals with confidence score. After that, we employ a two-stage snippet-level boundary adjustment scheme to re-rank the order of generated proposals. Ensemble methods are also been used to improve the performance of our solution, which helps us achieve 2nd place. |
| Tasks | |
| Published | 2019-08-09 |
| URL | https://arxiv.org/abs/1908.03448v1 |
| https://arxiv.org/pdf/1908.03448v1.pdf | |
| PWC | https://paperswithcode.com/paper/relation-aware-pyramid-network-rapnet-for |
| Repo | |
| Framework | |
A Web Page Classifier Library Based on Random Image Content Analysis Using Deep Learning
| Title | A Web Page Classifier Library Based on Random Image Content Analysis Using Deep Learning |
| Authors | Leonardo Espinosa Leal, Kaj-Mikael Björk, Amaury Lendasse, Anton Akusok |
| Abstract | In this paper, we present a methodology and the corresponding Python library 1 for the classification of webpages. Our method retrieves a fixed number of images from a given webpage, and based on them classifies the webpage into a set of established classes with a given probability. The library trains a random forest model build upon the features extracted from images by a pre-trained deep network. The implementation is tested by recognizing weapon class webpages in a curated list of 3859 websites. The results show that the best method of classifying a webpage into the studies classes is to assign the class according to the maximum probability of any image belonging to this (weapon) class being above the threshold, across all the retrieved images. Further research explores the possibilities for the developed methodology to also apply in image classification for healthcare applications. |
| Tasks | Image Classification |
| Published | 2019-12-18 |
| URL | https://arxiv.org/abs/1912.08644v1 |
| https://arxiv.org/pdf/1912.08644v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-web-page-classifier-library-based-on-random |
| Repo | |
| Framework | |
Discovery of Useful Questions as Auxiliary Tasks
| Title | Discovery of Useful Questions as Auxiliary Tasks |
| Authors | Vivek Veeriah, Matteo Hessel, Zhongwen Xu, Richard Lewis, Janarthanan Rajendran, Junhyuk Oh, Hado van Hasselt, David Silver, Satinder Singh |
| Abstract | Arguably, intelligent agents ought to be able to discover their own questions so that in learning answers for them they learn unanticipated useful knowledge and skills; this departs from the focus in much of machine learning on agents learning answers to externally defined questions. We present a novel method for a reinforcement learning (RL) agent to discover questions formulated as general value functions or GVFs, a fairly rich form of knowledge representation. Specifically, our method uses non-myopic meta-gradients to learn GVF-questions such that learning answers to them, as an auxiliary task, induces useful representations for the main task faced by the RL agent. We demonstrate that auxiliary tasks based on the discovered GVFs are sufficient, on their own, to build representations that support main task learning, and that they do so better than popular hand-designed auxiliary tasks from the literature. Furthermore, we show, in the context of Atari 2600 videogames, how such auxiliary tasks, meta-learned alongside the main task, can improve the data efficiency of an actor-critic agent. |
| Tasks | |
| Published | 2019-09-10 |
| URL | https://arxiv.org/abs/1909.04607v1 |
| https://arxiv.org/pdf/1909.04607v1.pdf | |
| PWC | https://paperswithcode.com/paper/discovery-of-useful-questions-as-auxiliary |
| Repo | |
| Framework | |
NodeDrop: A Condition for Reducing Network Size without Effect on Output
| Title | NodeDrop: A Condition for Reducing Network Size without Effect on Output |
| Authors | Louis Jensen, Jacob Harer, Sang Chin |
| Abstract | Determining an appropriate number of features for each layer in a neural network is an important and difficult task. This task is especially important in applications on systems with limited memory or processing power. Many current approaches to reduce network size either utilize iterative procedures, which can extend training time significantly, or require very careful tuning of algorithm parameters to achieve reasonable results. In this paper we propose NodeDrop, a new method for eliminating features in a network. With NodeDrop, we define a condition to identify and guarantee which nodes carry no information, and then use regularization to encourage nodes to meet this condition. We find that NodeDrop drastically reduces the number of features in a network while maintaining high performance, reducing the number of parameters by a factor of 114x for a VGG like network on CIFAR10 without a drop in accuracy. |
| Tasks | |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.01026v2 |
| https://arxiv.org/pdf/1906.01026v2.pdf | |
| PWC | https://paperswithcode.com/paper/nodedrop-a-condition-for-reducing-network |
| Repo | |
| Framework | |
Bibliothèque de la communauté assomptionniste : saisie informatique et classement Dewey
| Title | Bibliothèque de la communauté assomptionniste : saisie informatique et classement Dewey |
| Authors | Benoit Soubeyran |
| Abstract | The Library of Saint Peter in Gallicantu has had an eventful history and different phases of classification. It was constituted by the contribution of various private libraries of Religious of the Holy Land. Its history is intimately linked to the Assumptionist presence in Jerusalem. The computerization work carried out from 2018 onwards made it possible to clarify the classification framework based on Dewey’s decimal classification and to use the databases - Wikidata, VIAF - to improve the BNF catalogue. |
| Tasks | |
| Published | 2019-09-18 |
| URL | https://arxiv.org/abs/1909.08756v1 |
| https://arxiv.org/pdf/1909.08756v1.pdf | |
| PWC | https://paperswithcode.com/paper/bibliotheque-de-la-communaute-assomptionniste |
| Repo | |
| Framework | |
