Paper Group ANR 96
Exponential Separations in Local Differential Privacy. Semantic filtering through deep source separation on microscopy images. Adaptive Robust Optimization with Nearly Submodular Structure. Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE). Localized Debiased Machine Learning: Efficient Estim …
Exponential Separations in Local Differential Privacy
| Title | Exponential Separations in Local Differential Privacy |
| Authors | Matthew Joseph, Jieming Mao, Aaron Roth |
| Abstract | We prove a general connection between the communication complexity of two-player games and the sample complexity of their multi-player locally private analogues. We use this connection to prove sample complexity lower bounds for locally differentially private protocols as straightforward corollaries of results from communication complexity. In particular, we 1) use a communication lower bound for the hidden layers problem to prove an exponential sample complexity separation between sequentially and fully interactive locally private protocols, and 2) use a communication lower bound for the pointer chasing problem to prove an exponential sample complexity separation between $k$ round and $k+1$ round sequentially interactive locally private protocols, for every $k$. |
| Tasks | |
| Published | 2019-07-01 |
| URL | https://arxiv.org/abs/1907.00813v3 |
| https://arxiv.org/pdf/1907.00813v3.pdf | |
| PWC | https://paperswithcode.com/paper/exponential-separations-in-local-differential |
| Repo | |
| Framework | |
Semantic filtering through deep source separation on microscopy images
| Title | Semantic filtering through deep source separation on microscopy images |
| Authors | Avelino Javer, Jens Rittscher |
| Abstract | By their very nature microscopy images of cells and tissues consist of a limited number of object types or components. In contrast to most natural scenes, the composition is known a priori. Decomposing biological images into semantically meaningful objects and layers is the aim of this paper. Building on recent approaches to image de-noising we present a framework that achieves state-of-the-art segmentation results requiring little or no manual annotations. Here, synthetic images generated by adding cell crops are sufficient to train the model. Extensive experiments on cellular images, a histology data set, and small animal videos demonstrate that our approach generalizes to a broad range of experimental settings. As the proposed methodology does not require densely labelled training images and is capable of resolving the partially overlapping objects it holds the promise of being of use in a number of different applications. |
| Tasks | |
| Published | 2019-09-02 |
| URL | https://arxiv.org/abs/1909.00691v1 |
| https://arxiv.org/pdf/1909.00691v1.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-filtering-through-deep-source |
| Repo | |
| Framework | |
Adaptive Robust Optimization with Nearly Submodular Structure
| Title | Adaptive Robust Optimization with Nearly Submodular Structure |
| Authors | Shaojie Tang, Jing Yuan |
| Abstract | Constrained submodular maximization has been extensively studied in the recent years. In this paper, we study adaptive robust optimization with nearly submodular structure (ARONSS). Our objective is to randomly select a subset of items that maximizes the worst-case value of several reward functions simultaneously. Our work differs from existing studies in two ways: (1) we study the robust optimization problem under the adaptive setting, i.e., one needs to adaptively select items based on the feedback collected from picked items, and (2) our results apply to a broad range of reward functions characterized by $\epsilon$-nearly submodular function. We first analyze the adaptvity gap of ARONSS and show that the gap between the best adaptive solution and the best non-adaptive solution is bounded. Then we propose a approximate solution to this problem when all reward functions are submodular. Our algorithm achieves approximation ratio $(1-1/e)$ when considering matroid constraint. At last, we present two heuristics for the general case. All proposed solutions are non-adaptive which are easy to implement. |
| Tasks | |
| Published | 2019-05-14 |
| URL | https://arxiv.org/abs/1905.05339v3 |
| https://arxiv.org/pdf/1905.05339v3.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-robust-optimization-with-nearly |
| Repo | |
| Framework | |
Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE)
| Title | Analysis and Improvement of Adversarial Training in DQN Agents With Adversarially-Guided Exploration (AGE) |
| Authors | Vahid Behzadan, William Hsu |
| Abstract | This paper investigates the effectiveness of adversarial training in enhancing the robustness of Deep Q-Network (DQN) policies to state-space perturbations. We first present a formal analysis of adversarial training in DQN agents and its performance with respect to the proportion of adversarial perturbations to nominal observations used for training. Next, we consider the sample-inefficiency of current adversarial training techniques, and propose a novel Adversarially-Guided Exploration (AGE) mechanism based on a modified hybrid of the $\epsilon$-greedy algorithm and Boltzmann exploration. We verify the feasibility of this exploration mechanism through experimental evaluation of its performance in comparison with the traditional decaying $\epsilon$-greedy and parameter-space noise exploration algorithms. |
| Tasks | |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.01119v1 |
| https://arxiv.org/pdf/1906.01119v1.pdf | |
| PWC | https://paperswithcode.com/paper/analysis-and-improvement-of-adversarial |
| Repo | |
| Framework | |
Localized Debiased Machine Learning: Efficient Estimation of Quantile Treatment Effects, Conditional Value at Risk, and Beyond
| Title | Localized Debiased Machine Learning: Efficient Estimation of Quantile Treatment Effects, Conditional Value at Risk, and Beyond |
| Authors | Nathan Kallus, Xiaojie Mao, Masatoshi Uehara |
| Abstract | We consider the efficient estimation of a low-dimensional parameter in the presence of very high-dimensional nuisances that may depend on the parameter of interest. An important example is the quantile treatment effect (QTE) in causal inference, where the efficient estimation equation involves as a nuisance the conditional cumulative distribution evaluated at the quantile to be estimated. Debiased machine learning (DML) is a data-splitting approach to address the need to estimate nuisances using flexible machine learning methods that may not satisfy strong metric entropy conditions, but applying it to problems with estimand-dependent nuisances would require estimating too many nuisances to be practical. For the QTE estimation, DML requires we learn the whole conditional cumulative distribution function, which may be challenging in practice and stands in contrast to only needing to estimate just two regression functions as in the efficient estimation of average treatment effects. Instead, we propose localized debiased machine learning (LDML), a new three-way data-splitting approach that avoids this burdensome step and needs only estimate the nuisances at a single initial bad guess for the parameters. In particular, under a Frechet-derivative orthogonality condition, we show the oracle estimation equation is asymptotically equivalent to one where the nuisance is evaluated at the true parameter value and we provide a strategy to target this alternative formulation. In the case of QTE estimation, this involves only learning two binary regression models, for which many standard, time-tested machine learning methods exist. We prove that under certain lax rate conditions, our estimator has the same favorable asymptotic behavior as the infeasible oracle estimator that solves the estimating equation with the true nuisance functions. |
| Tasks | Causal Inference |
| Published | 2019-12-30 |
| URL | https://arxiv.org/abs/1912.12945v1 |
| https://arxiv.org/pdf/1912.12945v1.pdf | |
| PWC | https://paperswithcode.com/paper/localized-debiased-machine-learning-efficient |
| Repo | |
| Framework | |
Deep Reinforcement Learning for Imbalanced Classification
| Title | Deep Reinforcement Learning for Imbalanced Classification |
| Authors | Enlu Lin, Qiong Chen, Xiaoming Qi |
| Abstract | Data in real-world application often exhibit skewed class distribution which poses an intense challenge for machine learning. Conventional classification algorithms are not effective in the case of imbalanced data distribution, and may fail when the data distribution is highly imbalanced. To address this issue, we propose a general imbalanced classification model based on deep reinforcement learning. We formulate the classification problem as a sequential decision-making process and solve it by deep Q-learning network. The agent performs a classification action on one sample at each time step, and the environment evaluates the classification action and returns a reward to the agent. The reward from minority class sample is larger so the agent is more sensitive to the minority class. The agent finally finds an optimal classification policy in imbalanced data under the guidance of specific reward function and beneficial learning environment. Experiments show that our proposed model outperforms the other imbalanced classification algorithms, and it can identify more minority samples and has great classification performance. |
| Tasks | Decision Making, Q-Learning |
| Published | 2019-01-05 |
| URL | http://arxiv.org/abs/1901.01379v1 |
| http://arxiv.org/pdf/1901.01379v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-reinforcement-learning-for-imbalanced |
| Repo | |
| Framework | |
Domain Adaptation for Vehicle Detection from Bird’s Eye View LiDAR Point Cloud Data
| Title | Domain Adaptation for Vehicle Detection from Bird’s Eye View LiDAR Point Cloud Data |
| Authors | Khaled Saleh, Ahmed Abobakr, Mohammed Attia, Julie Iskander, Darius Nahavandi, Mohammed Hossny |
| Abstract | Point cloud data from 3D LiDAR sensors are one of the most crucial sensor modalities for versatile safety-critical applications such as self-driving vehicles. Since the annotations of point cloud data is an expensive and time-consuming process, therefore recently the utilisation of simulated environments and 3D LiDAR sensors for this task started to get some popularity. With simulated sensors and environments, the process for obtaining an annotated synthetic point cloud data became much easier. However, the generated synthetic point cloud data are still missing the artefacts usually exist in point cloud data from real 3D LiDAR sensors. As a result, the performance of the trained models on this data for perception tasks when tested on real point cloud data is degraded due to the domain shift between simulated and real environments. Thus, in this work, we are proposing a domain adaptation framework for bridging this gap between synthetic and real point cloud data. Our proposed framework is based on the deep cycle-consistent generative adversarial networks (CycleGAN) architecture. We have evaluated the performance of our proposed framework on the task of vehicle detection from a bird’s eye view (BEV) point cloud images coming from real 3D LiDAR sensors. The framework has shown competitive results with an improvement of more than 7% in average precision score over other baseline approaches when tested on real BEV point cloud images. |
| Tasks | Domain Adaptation, Unsupervised Domain Adaptation |
| Published | 2019-05-22 |
| URL | https://arxiv.org/abs/1905.08955v1 |
| https://arxiv.org/pdf/1905.08955v1.pdf | |
| PWC | https://paperswithcode.com/paper/domain-adaptation-for-vehicle-detection-from |
| Repo | |
| Framework | |
Learning Credible Deep Neural Networks with Rationale Regularization
| Title | Learning Credible Deep Neural Networks with Rationale Regularization |
| Authors | Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu |
| Abstract | Recent explainability related studies have shown that state-of-the-art DNNs do not always adopt correct evidences to make decisions. It not only hampers their generalization but also makes them less likely to be trusted by end-users. In pursuit of developing more credible DNNs, in this paper we propose CREX, which encourages DNN models to focus more on evidences that actually matter for the task at hand, and to avoid overfitting to data-dependent bias and artifacts. Specifically, CREX regularizes the training process of DNNs with rationales, i.e., a subset of features highlighted by domain experts as justifications for predictions, to enforce DNNs to generate local explanations that conform with expert rationales. Even when rationales are not available, CREX still could be useful by requiring the generated explanations to be sparse. Experimental results on two text classification datasets demonstrate the increased credibility of DNNs trained with CREX. Comprehensive analysis further shows that while CREX does not always improve prediction accuracy on the held-out test set, it significantly increases DNN accuracy on new and previously unseen data beyond test set, highlighting the advantage of the increased credibility. |
| Tasks | Text Classification |
| Published | 2019-08-13 |
| URL | https://arxiv.org/abs/1908.05601v1 |
| https://arxiv.org/pdf/1908.05601v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-credible-deep-neural-networks-with |
| Repo | |
| Framework | |
Diagnosis of Celiac Disease and Environmental Enteropathy on Biopsy Images Using Color Balancing on Convolutional Neural Networks
| Title | Diagnosis of Celiac Disease and Environmental Enteropathy on Biopsy Images Using Color Balancing on Convolutional Neural Networks |
| Authors | Kamran Kowsari, Rasoul Sali, Marium N. Khan, William Adorno, S. Asad Ali, Sean R. Moore, Beatrice C. Amadi, Paul Kelly, Sana Syed, Donald E. Brown |
| Abstract | Celiac Disease (CD) and Environmental Enteropathy (EE) are common causes of malnutrition and adversely impact normal childhood development. CD is an autoimmune disorder that is prevalent worldwide and is caused by an increased sensitivity to gluten. Gluten exposure destructs the small intestinal epithelial barrier, resulting in nutrient mal-absorption and childhood under-nutrition. EE also results in barrier dysfunction but is thought to be caused by an increased vulnerability to infections. EE has been implicated as the predominant cause of under-nutrition, oral vaccine failure, and impaired cognitive development in low-and-middle-income countries. Both conditions require a tissue biopsy for diagnosis, and a major challenge of interpreting clinical biopsy images to differentiate between these gastrointestinal diseases is striking histopathologic overlap between them. In the current study, we propose a convolutional neural network (CNN) to classify duodenal biopsy images from subjects with CD, EE, and healthy controls. We evaluated the performance of our proposed model using a large cohort containing 1000 biopsy images. Our evaluations show that the proposed model achieves an area under ROC of 0.99, 1.00, and 0.97 for CD, EE, and healthy controls, respectively. These results demonstrate the discriminative power of the proposed model in duodenal biopsies classification. |
| Tasks | |
| Published | 2019-04-10 |
| URL | https://arxiv.org/abs/1904.05773v5 |
| https://arxiv.org/pdf/1904.05773v5.pdf | |
| PWC | https://paperswithcode.com/paper/diagnosis-of-celiac-disease-and-environmental |
| Repo | |
| Framework | |
Optimal Use of Experience in First Person Shooter Environments
| Title | Optimal Use of Experience in First Person Shooter Environments |
| Authors | Matthew Aitchison |
| Abstract | Although reinforcement learning has made great strides recently, a continuing limitation is that it requires an extremely high number of interactions with the environment. In this paper, we explore the effectiveness of reusing experience from the experience replay buffer in the Deep Q-Learning algorithm. We test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment and show first, this requires a change in the learning rate, and second that it does not improve the performance of the agent. Furthermore, we show that updating less frequently is effective up to a ratio of 4:1, after which performance degrades significantly. These results quantitatively confirm the widespread practice of performing learning updates every 4th environmental step. |
| Tasks | Q-Learning |
| Published | 2019-06-24 |
| URL | https://arxiv.org/abs/1906.09734v1 |
| https://arxiv.org/pdf/1906.09734v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-use-of-experience-in-first-person |
| Repo | |
| Framework | |
On Intra-Class Variance for Deep Learning of Classifiers
| Title | On Intra-Class Variance for Deep Learning of Classifiers |
| Authors | Rafal Pilarczyk, Wladyslaw Skarbek |
| Abstract | A novel technique for deep learning of image classifiers is presented. The learned CNN models offer better separation of deep features (also known as embedded vectors) measured by Euclidean proximity and also no deterioration of the classification results by class membership probability. The latter feature can be used for enhancing image classifiers having the classes at the model’s exploiting stage different from from classes during the training stage. While the Shannon information of SoftMax probability for target class is extended for mini-batch by the intra-class variance, the trained network itself is extended by the Hadamard layer with the parameters representing the class centers. Contrary to the existing solutions, this extra neural layer enables interfacing of the training algorithm to the standard stochastic gradient optimizers, e.g. AdaM algorithm. Moreover, this approach makes the computed centroids immediately adapting to the updating embedded vectors and finally getting the comparable accuracy in less epochs. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1901.11186v2 |
| http://arxiv.org/pdf/1901.11186v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-intra-class-variance-for-deep-learning-of |
| Repo | |
| Framework | |
Few Labeled Atlases are Necessary for Deep-Learning-Based Segmentation
| Title | Few Labeled Atlases are Necessary for Deep-Learning-Based Segmentation |
| Authors | Hyeon Woo Lee, Mert R. Sabuncu, Adrian V. Dalca |
| Abstract | We tackle biomedical image segmentation in the scenario of only a few labeled brain MR images. This is an important and challenging task in medical applications, where manual annotations are time-consuming. Current multi-atlas based segmentation methods use image registration to warp segments from labeled images onto a new scan. In a different paradigm, supervised learning-based segmentation strategies have gained popularity. These method consistently use relatively large sets of labeled training data, and their behavior in the regime of a few labeled biomedical images has not been thoroughly evaluated. In this work, we provide two important results for segmentation in the scenario where few labeled images are available. First, we propose a straightforward implementation of efficient semi-supervised learning-based registration method, which we showcase in a multi-atlas segmentation framework. Second, through an extensive empirical study, we evaluate the performance of a supervised segmentation approach, where the training images are augmented via random deformations. Surprisingly, we find that in both paradigms, accurate segmentation is generally possible even in the context of few labeled images. |
| Tasks | Deformable Medical Image Registration, Image Registration, Medical Image Registration, Semantic Segmentation, Semi-Supervised Semantic Segmentation |
| Published | 2019-08-13 |
| URL | https://arxiv.org/abs/1908.04466v4 |
| https://arxiv.org/pdf/1908.04466v4.pdf | |
| PWC | https://paperswithcode.com/paper/few-labeled-atlases-are-necessary-for-deep |
| Repo | |
| Framework | |
RNN-Test: Adversarial Testing Framework for Recurrent Neural Network Systems
| Title | RNN-Test: Adversarial Testing Framework for Recurrent Neural Network Systems |
| Authors | Jianmin Guo, Yue Zhao, Xueying Han, Yu Jiang, Jiaguang Sun |
| Abstract | While huge efforts have been investigated in the adversarial testing of convolutional neural networks (CNN), the testing for recurrent neural networks (RNN) is still limited to the classification context and leave threats for vast sequential application domains. In this work, we propose a generic adversarial testing framework RNN-Test. First, based on the distinctive structure of RNNs, we define three novel coverage metrics to measure the testing completeness and guide the generation of adversarial inputs. Second, we propose the state inconsistency orientation to generate the perturbations by maximizing the inconsistency of the hidden states of RNN cells. Finally, we combine orientations with coverage guidance to produce minute perturbations. Given the RNN model and the sequential inputs, RNN-Test will modify one character or one word out of the whole inputs based on the perturbations obtained, so as to lead the RNN to produce wrong outputs. For evaluation, we apply RNN-Test on two models of common RNN structure - the PTB language model and the spell checker model. RNN-Test efficiently reduces the performance of the PTB language model by increasing its test perplexity by 58.11%, and finds numbers of incorrect behaviors of the spell checker model with the success rate of 73.44% on average. With our customization, RNN-Test using the redefined neuron coverage as guidance could achieve 35.71% higher perplexity than original strategy of DeepXplore. |
| Tasks | Language Modelling |
| Published | 2019-11-11 |
| URL | https://arxiv.org/abs/1911.06155v1 |
| https://arxiv.org/pdf/1911.06155v1.pdf | |
| PWC | https://paperswithcode.com/paper/rnn-test-adversarial-testing-framework-for |
| Repo | |
| Framework | |
Reinforcement Learning-based Automatic Diagnosis of Acute Appendicitis in Abdominal CT
| Title | Reinforcement Learning-based Automatic Diagnosis of Acute Appendicitis in Abdominal CT |
| Authors | Walid Abdullah Al, Il Dong Yun, Kyong Joon Lee |
| Abstract | Acute appendicitis characterized by a painful inflammation of the vermiform appendix is one of the most common surgical emergencies. Localizing the appendix is challenging due to its unclear anatomy amidst the complex colon-structure as observed in the conventional CT views, resulting in a time-consuming diagnosis. End-to-end learning of a convolutional neural network (CNN) is also not likely to be useful because of the negligible size of the appendix compared with the abdominal CT volume. With no prior computational approaches to the best of our knowledge, we propose the first computerized automation for acute appendicitis diagnosis. In our approach, we utilize a reinforcement learning agent deployed in the lower abdominal region to obtain the appendix location first to reduce the search space for diagnosis. Then, we obtain the classification scores (i.e., the likelihood of acute appendicitis) for the local neighborhood around the localized position, using a CNN trained only on a small appendix patch per volume. From the spatial representation of the resultant scores, we finally define a region of low-entropy (RLE) to choose the optimal diagnosis score, which helps improve the classification accuracy showing robustness even under high appendix localization error cases. In our experiment with 319 abdominal CT volumes, the proposed RLE-based decision with prior localization showed significant improvement over the standard CNN-based diagnosis approaches. |
| Tasks | |
| Published | 2019-09-02 |
| URL | https://arxiv.org/abs/1909.00617v1 |
| https://arxiv.org/pdf/1909.00617v1.pdf | |
| PWC | https://paperswithcode.com/paper/reinforcement-learning-based-automatic |
| Repo | |
| Framework | |
Intrinsically Sparse Long Short-Term Memory Networks
| Title | Intrinsically Sparse Long Short-Term Memory Networks |
| Authors | Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy |
| Abstract | Long Short-Term Memory (LSTM) has achieved state-of-the-art performances on a wide range of tasks. Its outstanding performance is guaranteed by the long-term memory ability which matches the sequential data perfectly and the gating structure controlling the information flow. However, LSTMs are prone to be memory-bandwidth limited in realistic applications and need an unbearable period of training and inference time as the model size is ever-increasing. To tackle this problem, various efficient model compression methods have been proposed. Most of them need a big and expensive pre-trained model which is a nightmare for resource-limited devices where the memory budget is strictly limited. To remedy this situation, in this paper, we incorporate the Sparse Evolutionary Training (SET) procedure into LSTM, proposing a novel model dubbed SET-LSTM. Rather than starting with a fully-connected architecture, SET-LSTM has a sparse topology and dramatically fewer parameters in both phases, training and inference. Considering the specific architecture of LSTMs, we replace the LSTM cells and embedding layers with sparse structures and further on, use an evolutionary strategy to adapt the sparse connectivity to the data. Additionally, we find that SET-LSTM can provide many different good combinations of sparse connectivity to substitute the overparameterized optimization problem of dense neural networks. Evaluated on four sentiment analysis classification datasets, the results demonstrate that our proposed model is able to achieve usually better performance than its fully connected counterpart while having less than 4% of its parameters. |
| Tasks | Model Compression, Sentiment Analysis |
| Published | 2019-01-26 |
| URL | http://arxiv.org/abs/1901.09208v1 |
| http://arxiv.org/pdf/1901.09208v1.pdf | |
| PWC | https://paperswithcode.com/paper/intrinsically-sparse-long-short-term-memory |
| Repo | |
| Framework | |
