Paper Group ANR 897
SaaS: Speed as a Supervisor for Semi-supervised Learning. Generalized Batch Normalization: Towards Accelerating Deep Neural Networks. Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images. Handling Adversarial Concept Drift in Streaming Data. Superconducting Optoelectronic Neurons IV: Transmitter Circ …
SaaS: Speed as a Supervisor for Semi-supervised Learning
| Title | SaaS: Speed as a Supervisor for Semi-supervised Learning |
| Authors | Safa Cicek, Alhussein Fawzi, Stefano Soatto |
| Abstract | We introduce the SaaS Algorithm for semi-supervised learning, which uses learning speed during stochastic gradient descent in a deep neural network to measure the quality of an iterative estimate of the posterior probability of unknown labels. Training speed in supervised learning correlates strongly with the percentage of correct labels, so we use it as an inference criterion for the unknown labels, without attempting to infer the model parameters at first. Despite its simplicity, SaaS achieves state-of-the-art results in semi-supervised learning benchmarks. |
| Tasks | |
| Published | 2018-05-02 |
| URL | http://arxiv.org/abs/1805.00980v1 |
| http://arxiv.org/pdf/1805.00980v1.pdf | |
| PWC | https://paperswithcode.com/paper/saas-speed-as-a-supervisor-for-semi |
| Repo | |
| Framework | |
Generalized Batch Normalization: Towards Accelerating Deep Neural Networks
| Title | Generalized Batch Normalization: Towards Accelerating Deep Neural Networks |
| Authors | Xiaoyong Yuan, Zheng Feng, Matthew Norton, Xiaolin Li |
| Abstract | Utilizing recently introduced concepts from statistics and quantitative risk management, we present a general variant of Batch Normalization (BN) that offers accelerated convergence of Neural Network training compared to conventional BN. In general, we show that mean and standard deviation are not always the most appropriate choice for the centering and scaling procedure within the BN transformation, particularly if ReLU follows the normalization step. We present a Generalized Batch Normalization (GBN) transformation, which can utilize a variety of alternative deviation measures for scaling and statistics for centering, choices which naturally arise from the theory of generalized deviation measures and risk theory in general. When used in conjunction with the ReLU non-linearity, the underlying risk theory suggests natural, arguably optimal choices for the deviation measure and statistic. Utilizing the suggested deviation measure and statistic, we show experimentally that training is accelerated more so than with conventional BN, often with improved error rate as well. Overall, we propose a more flexible BN transformation supported by a complimentary theoretical framework that can potentially guide design choices. |
| Tasks | |
| Published | 2018-12-08 |
| URL | http://arxiv.org/abs/1812.03271v1 |
| http://arxiv.org/pdf/1812.03271v1.pdf | |
| PWC | https://paperswithcode.com/paper/generalized-batch-normalization-towards |
| Repo | |
| Framework | |
Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images
| Title | Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images |
| Authors | Jing Zhang, Yang Cao, Yang Wang, Chenglin Wen, Chang Wen Chen |
| Abstract | Modeling statistical regularity plays an essential role in ill-posed image processing problems. Recently, deep learning based methods have been presented to implicitly learn statistical representation of pixel distributions in natural images and leverage it as a constraint to facilitate subsequent tasks, such as color constancy and image dehazing. However, the existing CNN architecture is prone to variability and diversity of pixel intensity within and between local regions, which may result in inaccurate statistical representation. To address this problem, this paper presents a novel fully point-wise CNN architecture for modeling statistical regularities in natural images. Specifically, we propose to randomly shuffle the pixels in the origin images and leverage the shuffled image as input to make CNN more concerned with the statistical properties. Moreover, since the pixels in the shuffled image are independent identically distributed, we can replace all the large convolution kernels in CNN with point-wise ($1*1$) convolution kernels while maintaining the representation ability. Experimental results on two applications: color constancy and image dehazing, demonstrate the superiority of our proposed network over the existing architectures, i.e., using 1/10$\sim$1/100 network parameters and computational cost while achieving comparable performance. |
| Tasks | Color Constancy, Image Dehazing |
| Published | 2018-01-19 |
| URL | http://arxiv.org/abs/1801.06302v3 |
| http://arxiv.org/pdf/1801.06302v3.pdf | |
| PWC | https://paperswithcode.com/paper/fully-point-wise-convolutional-neural-network |
| Repo | |
| Framework | |
Handling Adversarial Concept Drift in Streaming Data
| Title | Handling Adversarial Concept Drift in Streaming Data |
| Authors | Tegjyot Singh Sethi, Mehmed Kantardzic |
| Abstract | Classifiers operating in a dynamic, real world environment, are vulnerable to adversarial activity, which causes the data distribution to change over time. These changes are traditionally referred to as concept drift, and several approaches have been developed in literature to deal with the problem of drift handling and detection. However, most concept drift handling techniques, approach it as a domain independent task, to make them applicable to a wide gamut of reactive systems. These techniques were developed from an adversarial agnostic perspective, where they are naive and assume that drift is a benign change, which can be fixed by updating the model. However, this is not the case when an active adversary is trying to evade the deployed classification system. In such an environment, the properties of concept drift are unique, as the drift is intended to degrade the system and at the same time designed to avoid detection by traditional concept drift detection techniques. This special category of drift is termed as adversarial drift, and this paper analyzes its characteristics and impact, in a streaming environment. A novel framework for dealing with adversarial concept drift is proposed, called the Predict-Detect streaming framework. Experimental evaluation of the framework, on generated adversarial drifting data streams, demonstrates that this framework is able to provide reliable unsupervised indication of drift, and is able to recover from drifts swiftly. While traditional partially labeled concept drift detection methodologies fail to detect adversarial drifts, the proposed framework is able to detect such drifts and operates with <6% labeled data, on average. Also, the framework provides benefits for active learning over imbalanced data streams, by innately providing for feature space honeypots, where minority class adversarial samples may be captured. |
| Tasks | Active Learning |
| Published | 2018-03-24 |
| URL | http://arxiv.org/abs/1803.09160v1 |
| http://arxiv.org/pdf/1803.09160v1.pdf | |
| PWC | https://paperswithcode.com/paper/handling-adversarial-concept-drift-in |
| Repo | |
| Framework | |
Superconducting Optoelectronic Neurons IV: Transmitter Circuits
| Title | Superconducting Optoelectronic Neurons IV: Transmitter Circuits |
| Authors | Jeffrey M. Shainline, Adam N. McCaughan, Sonia M. Buckley, Richard P. Mirin, Sae Woo Nam |
| Abstract | A superconducting optoelectronic neuron will produce a small current pulse upon reaching threshold. We present an amplifier chain that converts this small current pulse to a voltage pulse sufficient to produce light from a semiconductor diode. This light is the signal used to communicate between neurons in the network. The amplifier chain comprises a thresholding Josephson junction, a relaxation oscillator Josephson junction, a superconducting thin-film current-gated current amplifier, and a superconducting thin-film current-gated voltage amplifier. We analyze the performance of the elements in the amplifier chain in the time domain to calculate the energy consumption per photon created for several values of light-emitting diode capacitance and efficiency. The speed of the amplification sequence allows neuronal firing up to at least 20,MHz with power density low enough to be cooled easily with standard $^4$He cryogenic systems operating at 4.2,K. |
| Tasks | |
| Published | 2018-05-04 |
| URL | http://arxiv.org/abs/1805.01941v2 |
| http://arxiv.org/pdf/1805.01941v2.pdf | |
| PWC | https://paperswithcode.com/paper/superconducting-optoelectronic-neurons-iv |
| Repo | |
| Framework | |
Influence of the Event Rate on Discrimination Abilities of Bankruptcy Prediction Models
| Title | Influence of the Event Rate on Discrimination Abilities of Bankruptcy Prediction Models |
| Authors | Lili Zhang, Jennifer Priestley, Xuelei Ni |
| Abstract | In bankruptcy prediction, the proportion of events is very low, which is often oversampled to eliminate this bias. In this paper, we study the influence of the event rate on discrimination abilities of bankruptcy prediction models. First the statistical association and significance of public records and firmographics indicators with the bankruptcy were explored. Then the event rate was oversampled from 0.12% to 10%, 20%, 30%, 40%, and 50%, respectively. Seven models were developed, including Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, Support Vector Machine, Bayesian Network, and Neural Network. Under different event rates, models were comprehensively evaluated and compared based on Kolmogorov-Smirnov Statistic, accuracy, F1 score, Type I error, Type II error, and ROC curve on the hold-out dataset with their best probability cut-offs. Results show that Bayesian Network is the most insensitive to the event rate, while Support Vector Machine is the most sensitive. |
| Tasks | |
| Published | 2018-03-10 |
| URL | http://arxiv.org/abs/1803.03756v1 |
| http://arxiv.org/pdf/1803.03756v1.pdf | |
| PWC | https://paperswithcode.com/paper/influence-of-the-event-rate-on-discrimination |
| Repo | |
| Framework | |
A Hybrid Approach to Privacy-Preserving Federated Learning
| Title | A Hybrid Approach to Privacy-Preserving Federated Learning |
| Authors | Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, Yi Zhou |
| Abstract | Federated learning facilitates the collaborative training of models without the sharing of raw data. However, recent attacks demonstrate that simply maintaining data locality during training processes does not provide sufficient privacy guarantees. Rather, we need a federated learning system capable of preventing inference over both the messages exchanged during training and the final trained model while ensuring the resulting model also has acceptable predictive accuracy. Existing federated learning approaches either use secure multiparty computation (SMC) which is vulnerable to inference or differential privacy which can lead to low accuracy given a large number of parties with relatively small amounts of data each. In this paper, we present an alternative approach that utilizes both differential privacy and SMC to balance these trade-offs. Combining differential privacy with secure multiparty computation enables us to reduce the growth of noise injection as the number of parties increases without sacrificing privacy while maintaining a pre-defined rate of trust. Our system is therefore a scalable approach that protects against inference threats and produces models with high accuracy. Additionally, our system can be used to train a variety of machine learning models, which we validate with experimental results on 3 different machine learning algorithms. Our experiments demonstrate that our approach out-performs state of the art solutions. |
| Tasks | |
| Published | 2018-12-07 |
| URL | https://arxiv.org/abs/1812.03224v2 |
| https://arxiv.org/pdf/1812.03224v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-hybrid-approach-to-privacy-preserving |
| Repo | |
| Framework | |
Survival prediction using ensemble tumor segmentation and transfer learning
| Title | Survival prediction using ensemble tumor segmentation and transfer learning |
| Authors | Mariano Cabezas, Sergi Valverde, Sandra González-Villà, Albert Clérigues, Mostafa Salem, Kaisar Kushibar, Jose Bernal, Arnau Oliver, Xavier Lladó |
| Abstract | Segmenting tumors and their subregions is a challenging task as demonstrated by the annual BraTS challenge. Moreover, predicting the survival of the patient using mainly imaging features, while being a desirable outcome to evaluate the treatment of the patient, it is also a difficult task. In this paper, we present a cascaded pipeline to segment the tumor and its subregions and then we use these results and other clinical features together with image features coming from a pretrained VGG-16 network to predict the survival of the patient. Preliminary results with the training and validation dataset show a promising start in terms of segmentation, while the prediction values could be improved with further testing on the feature extraction part of the network. |
| Tasks | Transfer Learning |
| Published | 2018-10-04 |
| URL | http://arxiv.org/abs/1810.04274v1 |
| http://arxiv.org/pdf/1810.04274v1.pdf | |
| PWC | https://paperswithcode.com/paper/survival-prediction-using-ensemble-tumor |
| Repo | |
| Framework | |
Wasserstein Measure Coresets
| Title | Wasserstein Measure Coresets |
| Authors | Sebastian Claici, Aude Genevay, Justin Solomon |
| Abstract | The proliferation of large data sets and Bayesian inference techniques motivates demand for better data sparsification. Coresets provide a principled way of summarizing a large dataset via a smaller one that is guaranteed to match the performance of the full data set on specific problems. Classical coresets, however, neglect the underlying data distribution, which is often continuous. We address this oversight by introducing Wasserstein measure coresets, an extension of coresets which by definition takes into account generalization. Our formulation of the problem, which essentially consists in minimizing the Wasserstein distance, is solvable via stochastic gradient descent. This yields an algorithm which simply requires sample access to the data distribution and is able to handle large data streams in an online manner. We validate our construction for inference and clustering. |
| Tasks | Bayesian Inference |
| Published | 2018-05-18 |
| URL | https://arxiv.org/abs/1805.07412v2 |
| https://arxiv.org/pdf/1805.07412v2.pdf | |
| PWC | https://paperswithcode.com/paper/wasserstein-coresets-for-lipschitz-costs |
| Repo | |
| Framework | |
Towards Audio to Scene Image Synthesis using Generative Adversarial Network
| Title | Towards Audio to Scene Image Synthesis using Generative Adversarial Network |
| Authors | Chia-Hung Wan, Shun-Po Chuang, Hung-Yi Lee |
| Abstract | Humans can imagine a scene from a sound. We want machines to do so by using conditional generative adversarial networks (GANs). By applying the techniques including spectral norm, projection discriminator and auxiliary classifier, compared with naive conditional GAN, the model can generate images with better quality in terms of both subjective and objective evaluations. Almost three-fourth of people agree that our model have the ability to generate images related to sounds. By inputting different volumes of the same sound, our model output different scales of changes based on the volumes, showing that our model truly knows the relationship between sounds and images to some extent. |
| Tasks | Image Generation |
| Published | 2018-08-13 |
| URL | http://arxiv.org/abs/1808.04108v1 |
| http://arxiv.org/pdf/1808.04108v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-audio-to-scene-image-synthesis-using |
| Repo | |
| Framework | |
Variable Selection for Nonparametric Learning with Power Series Kernels
| Title | Variable Selection for Nonparametric Learning with Power Series Kernels |
| Authors | Kota Matsui, Wataru Kumagai, Kenta Kanamori, Mitsuaki Nishikimi, Takafumi Kanamori |
| Abstract | In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the proposed method can be applied to various kernel nonparametric estimation such as kernel ridge regression, kernel-based density and density-ratio estimation. We prove that the proposed method has the property of the variable selection consistency when the power series kernel is used. This result is regarded as an extension of the variable selection consistency for the non-negative garrote to the kernel-based estimators. Several experiments including simulation studies and real data applications show the effectiveness of the proposed method. |
| Tasks | |
| Published | 2018-06-02 |
| URL | http://arxiv.org/abs/1806.00569v2 |
| http://arxiv.org/pdf/1806.00569v2.pdf | |
| PWC | https://paperswithcode.com/paper/variable-selection-for-nonparametric-learning |
| Repo | |
| Framework | |
Crowdsourcing with Fairness, Diversity and Budget Constraints
| Title | Crowdsourcing with Fairness, Diversity and Budget Constraints |
| Authors | Naman Goel, Boi Faltings |
| Abstract | Recent studies have shown that the labels collected from crowdworkers can be discriminatory with respect to sensitive attributes such as gender and race. This raises questions about the suitability of using crowdsourced data for further use, such as for training machine learning algorithms. In this work, we address the problem of fair and diverse data collection from a crowd under budget constraints. We propose a novel algorithm which maximizes the expected accuracy of the collected data, while ensuring that the errors satisfy desired notions of fairness. We provide guarantees on the performance of our algorithm and show that the algorithm performs well in practice through experiments on a real dataset. |
| Tasks | |
| Published | 2018-10-31 |
| URL | http://arxiv.org/abs/1810.13314v2 |
| http://arxiv.org/pdf/1810.13314v2.pdf | |
| PWC | https://paperswithcode.com/paper/crowdsourcing-with-fairness-diversity-and |
| Repo | |
| Framework | |
Improving Deep Models of Person Re-identification for Cross-Dataset Usage
| Title | Improving Deep Models of Person Re-identification for Cross-Dataset Usage |
| Authors | Sergey Rodionov, Alexey Potapov, Hugo Latapie, Enzo Fenoglio, Maxim Peterson |
| Abstract | Person re-identification (Re-ID) is the task of matching humans across cameras with non-overlapping views that has important applications in visual surveillance. Like other computer vision tasks, this task has gained much with the utilization of deep learning methods. However, existing solutions based on deep learning are usually trained and tested on samples taken from same datasets, while in practice one need to deploy Re-ID systems for new sets of cameras for which labeled data is unavailable. Here, we mitigate this problem for one state-of-the-art model, namely, metric embedding trained with the use of the triplet loss function, although our results can be extended to other models. The contribution of our work consists in developing a method of training the model on multiple datasets, and a method for its online practically unsupervised fine-tuning. These methods yield up to 19.1% improvement in Rank-1 score in the cross-dataset evaluation. |
| Tasks | Person Re-Identification |
| Published | 2018-07-23 |
| URL | http://arxiv.org/abs/1807.08526v1 |
| http://arxiv.org/pdf/1807.08526v1.pdf | |
| PWC | https://paperswithcode.com/paper/improving-deep-models-of-person-re |
| Repo | |
| Framework | |
Ear Recognition With Score-Level Fusion Based On CMC In Long-Wave Infrared Spectrum
| Title | Ear Recognition With Score-Level Fusion Based On CMC In Long-Wave Infrared Spectrum |
| Authors | Umit Kacar, Murvet Kirci |
| Abstract | Only a few studies have been reported regarding human ear recognition in long wave infrared band. Thus, we have created ear database based on long wave infrared band. We have called that the database is long wave infrared band MIDAS consisting of 2430 records of 81 subjects. Thermal band provides seamless operation both night and day, robust against spoofing with understanding live ear and invariant to illumination conditions for human ear recognition. We have proposed to use different algorithms to reveal the distinctive features. Then, we have reduced the number of dimensions using subspace methods. Finally, the dimension of data is reduced in accordance with the classifier methods. After this, the decision is determined by the best sores or combining some of the best scores with matching fusion. The results have showed that the fusion technique was successful. We have reached 97.71% for rank-1 with 567 test probes. Furthermore, we have defined the perfect rank which is rank number when recognition rate reaches 100% in cumulative matching curve. This evaluation is important for especially forensics, for example corpse identification, criminal investigation etc. |
| Tasks | |
| Published | 2018-01-27 |
| URL | http://arxiv.org/abs/1801.09054v1 |
| http://arxiv.org/pdf/1801.09054v1.pdf | |
| PWC | https://paperswithcode.com/paper/ear-recognition-with-score-level-fusion-based |
| Repo | |
| Framework | |
More Effective Ontology Authoring with Test-Driven Development
| Title | More Effective Ontology Authoring with Test-Driven Development |
| Authors | C. Maria Keet, Kieren Davies, Agnieszka Lawrynowicz |
| Abstract | Ontology authoring is a complex process, where commonly the automated reasoner is invoked for verification of newly introduced changes, therewith amounting to a time-consuming test-last approach. Test-Driven Development (TDD) for ontology authoring is a recent {\em test-first} approach that aims to reduce authoring time and increase authoring efficiency. Current TDD testing falls short on coverage of OWL features and possible test outcomes, the rigorous foundation thereof, and evaluations to ascertain its effectiveness. We aim to address these issues in one instantiation of TDD for ontology authoring. We first propose a succinct, logic-based model of TDD testing and present novel TDD algorithms so as to cover also any OWL 2 class expression for the TBox and for the principal ABox assertions, and prove their correctness. The algorithms use methods from the OWL API directly such that reclassification is not necessary for test execution, therewith reducing ontology authoring time. The algorithms were implemented in TDDonto2, a Prot'eg'e plugin. TDDonto2 was evaluated on editing efficiency and by users. The editing efficiency study demonstrated that it is faster than a typical ontology authoring interface, especially for medium size and large ontologies. The user evaluation demonstrated that modellers make significantly less errors with TDDonto2 compared to the standard Prot'eg'e interface and complete their tasks better using less time. Thus, the results indicate that Test-Driven Development is a promising approach in an ontology development methodology. |
| Tasks | |
| Published | 2018-12-14 |
| URL | http://arxiv.org/abs/1812.06015v1 |
| http://arxiv.org/pdf/1812.06015v1.pdf | |
| PWC | https://paperswithcode.com/paper/more-effective-ontology-authoring-with-test |
| Repo | |
| Framework | |
