January 31, 2020

3366 words 16 mins read

Paper Group ANR 81

Paper Group ANR 81

On the convex geometry of blind deconvolution and matrix completion. A Knowledge Transfer Framework for Differentially Private Sparse Learning. Scalable Semi-Supervised SVM via Triply Stochastic Gradients. Pipelines for Procedural Information Extraction from Scientific Literature: Towards Recipes using Machine Learning and Data Science. Detecting A …

On the convex geometry of blind deconvolution and matrix completion

Title On the convex geometry of blind deconvolution and matrix completion
Authors Felix Krahmer, Dominik Stöger
Abstract Low-rank matrix recovery from structured measurements has been a topic of intense study in the last decade and many important problems like matrix completion and blind deconvolution have been formulated in this framework. An important benchmark method to solve these problems is to minimize the nuclear norm, a convex proxy for the rank. A common approach to establish recovery guarantees for this convex program relies on the construction of a so-called approximate dual certificate. However, this approach provides only limited insight in various respects. Most prominently, the noise bounds exhibit seemingly suboptimal dimension factors. In this paper we take a novel, more geometric viewpoint to analyze both the matrix completion and the blind deconvolution scenario. We find that for both these applications the dimension factors in the noise bounds are not an artifact of the proof, but the problems are intrinsically badly conditioned. We show, however, that bad conditioning only arises for very small noise levels: Under mild assumptions that include many realistic noise levels we derive near-optimal error estimates for blind deconvolution under adversarial noise.
Tasks Matrix Completion
Published 2019-02-28
URL http://arxiv.org/abs/1902.11156v2
PDF http://arxiv.org/pdf/1902.11156v2.pdf
PWC https://paperswithcode.com/paper/on-the-convex-geometry-of-blind-deconvolution
Repo
Framework

A Knowledge Transfer Framework for Differentially Private Sparse Learning

Title A Knowledge Transfer Framework for Differentially Private Sparse Learning
Authors Lingxiao Wang, Quanquan Gu
Abstract We study the problem of estimating high dimensional models with underlying sparse structures while preserving the privacy of each training example. We develop a differentially private high-dimensional sparse learning framework using the idea of knowledge transfer. More specifically, we propose to distill the knowledge from a “teacher” estimator trained on a private dataset, by creating a new dataset from auxiliary features, and then train a differentially private “student” estimator using this new dataset. In addition, we establish the linear convergence rate as well as the utility guarantee for our proposed method. For sparse linear regression and sparse logistic regression, our method achieves improved utility guarantees compared with the best known results (Kifer et al., 2012; Wang and Gu, 2019). We further demonstrate the superiority of our framework through both synthetic and real-world data experiments.
Tasks Sparse Learning, Transfer Learning
Published 2019-09-13
URL https://arxiv.org/abs/1909.06322v1
PDF https://arxiv.org/pdf/1909.06322v1.pdf
PWC https://paperswithcode.com/paper/a-knowledge-transfer-framework-for
Repo
Framework

Scalable Semi-Supervised SVM via Triply Stochastic Gradients

Title Scalable Semi-Supervised SVM via Triply Stochastic Gradients
Authors Xiang Geng, Bin Gu, Xiang Li, Wanli Shi, Guansheng Zheng, Heng Huang
Abstract Semi-supervised learning (SSL) plays an increasingly important role in the big data era because a large number of unlabeled samples can be used effectively to improve the performance of the classifier. Semi-supervised support vector machine (S$^3$VM) is one of the most appealing methods for SSL, but scaling up S$^3$VM for kernel learning is still an open problem. Recently, a doubly stochastic gradient (DSG) algorithm has been proposed to achieve efficient and scalable training for kernel methods. However, the algorithm and theoretical analysis of DSG are developed based on the convexity assumption which makes them incompetent for non-convex problems such as S$^3$VM. To address this problem, in this paper, we propose a triply stochastic gradient algorithm for S$^3$VM, called TSGS$^3$VM. Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient. We use the approximated gradient to update the solution. More importantly, we establish new theoretic analysis for TSGS$^3$VM which guarantees that TSGS$^3$VM can converge to a stationary point. Extensive experimental results on a variety of datasets demonstrate that TSGS$^3$VM is much more efficient and scalable than existing S$^3$VM algorithms.
Tasks
Published 2019-07-26
URL https://arxiv.org/abs/1907.11584v1
PDF https://arxiv.org/pdf/1907.11584v1.pdf
PWC https://paperswithcode.com/paper/scalable-semi-supervised-svm-via-triply
Repo
Framework

Pipelines for Procedural Information Extraction from Scientific Literature: Towards Recipes using Machine Learning and Data Science

Title Pipelines for Procedural Information Extraction from Scientific Literature: Towards Recipes using Machine Learning and Data Science
Authors Huichen Yang, Carlos A. Aguirre, Maria F. De La Torre, Derek Christensen, Luis Bobadilla, Emily Davich, Jordan Roth, Lei Luo, Yihong Theis, Alice Lam, T. Yong-Jin Han, David Buttler, William H. Hsu
Abstract This paper describes a machine learning and data science pipeline for structured information extraction from documents, implemented as a suite of open-source tools and extensions to existing tools. It centers around a methodology for extracting procedural information in the form of recipes, stepwise procedures for creating an artifact (in this case synthesizing a nanomaterial), from published scientific literature. From our overall goal of producing recipes from free text, we derive the technical objectives of a system consisting of pipeline stages: document acquisition and filtering, payload extraction, recipe step extraction as a relationship extraction task, recipe assembly, and presentation through an information retrieval interface with question answering (QA) functionality. This system meets computational information and knowledge management (CIKM) requirements of metadata-driven payload extraction, named entity extraction, and relationship extraction from text. Functional contributions described in this paper include semi-supervised machine learning methods for PDF filtering and payload extraction tasks, followed by structured extraction and data transformation tasks beginning with section extraction, recipe steps as information tuples, and finally assembled recipes. Measurable objective criteria for extraction quality include precision and recall of recipe steps, ordering constraints, and QA accuracy, precision, and recall. Results, key novel contributions, and significant open problems derived from this work center around the attribution of these holistic quality measures to specific machine learning and inference stages of the pipeline, each with their performance measures. The desired recipes contain identified preconditions, material inputs, and operations, and constitute the overall output generated by our computational information and knowledge management (CIKM) system.
Tasks Entity Extraction, Information Retrieval, Question Answering
Published 2019-12-16
URL https://arxiv.org/abs/1912.07747v1
PDF https://arxiv.org/pdf/1912.07747v1.pdf
PWC https://paperswithcode.com/paper/pipelines-for-procedural-information
Repo
Framework

Detecting AI Trojans Using Meta Neural Analysis

Title Detecting AI Trojans Using Meta Neural Analysis
Authors Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A. Gunter, Bo Li
Abstract Machine learning models, especially neural networks (NNs), have achieved outstanding performance on diverse and complex applications. However, recent work has found that they are vulnerable to Trojan attacks where an adversary trains a corrupted model with poisoned data or directly manipulates its parameters in a stealthy way. Such Trojaned models can obtain good performance on normal data during test time while predicting incorrectly on the adversarially manipulated data samples. This paper aims to develop ways to detect Trojaned models. We mainly explore the idea of meta neural analysis, a technique involving training a meta NN model that can be used to predict whether or not a target NN model has certain properties. We develop a novel pipeline Meta Neural Trojaned model Detection (MNTD) system to predict if a given NN is Trojaned via meta neural analysis on a set of trained shadow models. We propose two ways to train the meta-classifier without knowing the Trojan attacker’s strategies. The first one, one-class learning, will fit a novel detection meta-classifier using only benign neural networks. The second one, called jumbo learning, will approximate a general distribution of Trojaned models and sample a “jumbo” set of Trojaned models to train the meta-classifier and evaluate on the unseen Trojan strategies. Extensive experiments demonstrate the effectiveness of MNTD in detecting different Trojan attacks in diverse areas such as vision, speech, tabular data, and natural language processing. We show that MNTD reaches an average of 97% detection AUC (Area Under the ROC Curve) score and outperforms existing approaches. Furthermore, we design and evaluate MNTD system to defend against strong adaptive attackers who have exactly the knowledge of the detection, which demonstrates the robustness of MNTD.
Tasks
Published 2019-10-08
URL https://arxiv.org/abs/1910.03137v2
PDF https://arxiv.org/pdf/1910.03137v2.pdf
PWC https://paperswithcode.com/paper/detecting-ai-trojans-using-meta-neural
Repo
Framework

Identification of primary angle-closure on AS-OCT images with Convolutional Neural Networks

Title Identification of primary angle-closure on AS-OCT images with Convolutional Neural Networks
Authors Chenglang Yuan, Cheng Bian, Hongjian Kang, Shu Liang, Kai Ma, Yefeng Zheng
Abstract Primary angle-closure disease (PACD) is a severe retinal disease, which might cause irreversible vision loss. In clinic, accurate identification of angle-closure and localization of the scleral spur’s position on anterior segment optical coherence tomography (AS-OCT) is essential for the diagnosis of PACD. However, manual delineation might confine in low accuracy and low efficiency. In this paper, we propose an efficient and accurate end-to-end architecture for angle-closure classification and scleral spur localization. Specifically, we utilize a revised ResNet152 as our backbone to improve the accuracy of the angle-closure identification. For scleral spur localization, we adopt EfficientNet as encoder because of its powerful feature extraction potential. By combining the skip-connect module and pyramid pooling module, the network is able to collect semantic cues in feature maps from multiple dimensions and scales. Afterward, we propose a novel keypoint registration loss to constrain the model’s attention to the intensity and location of the scleral spur area. Several experiments are extensively conducted to evaluate our method on the angle-closure glaucoma evaluation (AGE) Challenge dataset. The results show that our proposed architecture ranks the first place of the classification task on the test dataset and achieves the average Euclidean distance error of 12.00 pixels in the scleral spur localization task.
Tasks
Published 2019-10-23
URL https://arxiv.org/abs/1910.10414v1
PDF https://arxiv.org/pdf/1910.10414v1.pdf
PWC https://paperswithcode.com/paper/identification-of-primary-angle-closure-on-as
Repo
Framework

Scene Understanding for Autonomous Manipulation with Deep Learning

Title Scene Understanding for Autonomous Manipulation with Deep Learning
Authors Anh Nguyen
Abstract Over the past few years, deep learning techniques have achieved tremendous success in many visual understanding tasks such as object detection, image segmentation, and caption generation. Despite this thriving in computer vision and natural language processing, deep learning has not yet shown significant impact in robotics. Due to the gap between theory and application, there are many challenges when applying the results of deep learning to the real robotic systems. In this study, our long-term goal is to bridge the gap between computer vision and robotics by developing visual methods that can be used in real robots. In particular, this work tackles two fundamental visual problems for autonomous robotic manipulation: affordance detection and fine-grained action understanding. Theoretically, we propose different deep architectures to further improves the state of the art in each problem. Empirically, we show that the outcomes of our proposed methods can be applied in real robots and allow them to perform useful manipulation tasks.
Tasks Object Detection, Scene Understanding, Semantic Segmentation
Published 2019-03-23
URL http://arxiv.org/abs/1903.09761v1
PDF http://arxiv.org/pdf/1903.09761v1.pdf
PWC https://paperswithcode.com/paper/scene-understanding-for-autonomous
Repo
Framework

Visualizing Movement Control Optimization Landscapes

Title Visualizing Movement Control Optimization Landscapes
Authors Perttu Hämäläinen, Juuso Toikka, C. Karen Liu
Abstract A large body of animation research focuses on optimization of movement control, either as action sequences or policy parameters. However, as closed-form expressions of the objective functions are often not available, our understanding of the optimization problems is limited. Building on recent work on analyzing neural network training, we contribute novel visualizations of high-dimensional control optimization landscapes; this yields insights into why control optimization is hard and why common practices like early termination and spline-based action parameterizations make optimization easier. For example, our experiments show how trajectory optimization can become increasingly ill-conditioned with longer trajectories, but parameterizing control as partial target states - e.g., target angles converted to torques using a PD-controller - can act as an efficient preconditioner. Both our visualizations and quantitative empirical data also indicate that neural network policy optimization scales better than trajectory optimization for long planning horizons. Our work advances the understanding of movement optimization and our visualizations should also provide value in educational use.
Tasks
Published 2019-09-17
URL https://arxiv.org/abs/1909.07869v2
PDF https://arxiv.org/pdf/1909.07869v2.pdf
PWC https://paperswithcode.com/paper/visualizing-movement-control-optimization
Repo
Framework

Learning Longer-term Dependencies via Grouped Distributor Unit

Title Learning Longer-term Dependencies via Grouped Distributor Unit
Authors Wei Luo, Feng Yu
Abstract Learning long-term dependencies still remains difficult for recurrent neural networks (RNNs) despite their success in sequence modeling recently. In this paper, we propose a novel gated RNN structure, which contains only one gate. Hidden states in the proposed grouped distributor unit (GDU) are partitioned into groups. For each group, the proportion of memory to be overwritten in each state transition is limited to a constant and is adaptively distributed to each group member. In other word, every separate group has a fixed overall update rate, yet all units are allowed to have different paces. Information is therefore forced to be latched in a flexible way, which helps the model to capture long-term dependencies in data. Besides having a simpler structure, GDU is demonstrated experimentally to outperform LSTM and GRU on tasks including both pathological problems and natural data set.
Tasks
Published 2019-04-29
URL https://arxiv.org/abs/1906.08856v1
PDF https://arxiv.org/pdf/1906.08856v1.pdf
PWC https://paperswithcode.com/paper/learning-longer-term-dependencies-via-grouped
Repo
Framework

Unifying Heterogeneous Classifiers with Distillation

Title Unifying Heterogeneous Classifiers with Distillation
Authors Jayakorn Vongkulbhisal, Phongtharin Vinayavekhin, Marco Visentini-Scarzanella
Abstract In this paper, we study the problem of unifying knowledge from a set of classifiers with different architectures and target classes into a single classifier, given only a generic set of unlabelled data. We call this problem Unifying Heterogeneous Classifiers (UHC). This problem is motivated by scenarios where data is collected from multiple sources, but the sources cannot share their data, e.g., due to privacy concerns, and only privately trained models can be shared. In addition, each source may not be able to gather data to train all classes due to data availability at each source, and may not be able to train the same classification model due to different computational resources. To tackle this problem, we propose a generalisation of knowledge distillation to merge HCs. We derive a probabilistic relation between the outputs of HCs and the probability over all classes. Based on this relation, we propose two classes of methods based on cross-entropy minimisation and matrix factorisation, which allow us to estimate soft labels over all classes from unlabelled samples and use them in lieu of ground truth labels to train a unified classifier. Our extensive experiments on ImageNet, LSUN, and Places365 datasets show that our approaches significantly outperform a naive extension of distillation and can achieve almost the same accuracy as classifiers that are trained in a centralised, supervised manner.
Tasks
Published 2019-04-12
URL http://arxiv.org/abs/1904.06062v1
PDF http://arxiv.org/pdf/1904.06062v1.pdf
PWC https://paperswithcode.com/paper/unifying-heterogeneous-classifiers-with
Repo
Framework

ROAM: Recurrently Optimizing Tracking Model

Title ROAM: Recurrently Optimizing Tracking Model
Authors Tianyu Yang, Pengfei Xu, Runbo Hu, Hua Chai, Antoni B. Chan
Abstract In this paper, we design a tracking model consisting of response generation and bounding box regression, where the first component produces a heat map to indicate the presence of the object at different positions and the second part regresses the relative bounding box shifts to anchors mounted on sliding-window locations. Thanks to the resizable convolutional filters used in both components to adapt to the shape changes of objects, our tracking model does not need to enumerate different sized anchors, thus saving model parameters. To effectively adapt the model to appearance variations, we propose to offline train a recurrent neural optimizer to update tracking model in a meta-learning setting, which can converge the model in a few gradient steps. This improves the convergence speed of updating the tracking model while achieving better performance. We extensively evaluate our trackers, ROAM and ROAM++, on the OTB, VOT, LaSOT, GOT-10K and TrackingNet benchmark and our methods perform favorably against state-of-the-art algorithms.
Tasks Meta-Learning
Published 2019-07-28
URL https://arxiv.org/abs/1907.12006v3
PDF https://arxiv.org/pdf/1907.12006v3.pdf
PWC https://paperswithcode.com/paper/roam-recurrently-optimizing-tracking-model
Repo
Framework
Title The Futility of Bias-Free Learning and Search
Authors George D. Montanez, Jonathan Hayase, Julius Lauw, Dominique Macias, Akshay Trikha, Julia Vendemiatti
Abstract Building on the view of machine learning as search, we demonstrate the necessity of bias in learning, quantifying the role of bias (measured relative to a collection of possible datasets, or more generally, information resources) in increasing the probability of success. For a given degree of bias towards a fixed target, we show that the proportion of favorable information resources is strictly bounded from above. Furthermore, we demonstrate that bias is a conserved quantity, such that no algorithm can be favorably biased towards many distinct targets simultaneously. Thus bias encodes trade-offs. The probability of success for a task can also be measured geometrically, as the angle of agreement between what holds for the actual task and what is assumed by the algorithm, represented in its bias. Lastly, finding a favorably biasing distribution over a fixed set of information resources is provably difficult, unless the set of resources itself is already favorable with respect to the given task and algorithm.
Tasks
Published 2019-07-13
URL https://arxiv.org/abs/1907.06010v1
PDF https://arxiv.org/pdf/1907.06010v1.pdf
PWC https://paperswithcode.com/paper/the-futility-of-bias-free-learning-and-search
Repo
Framework

Reasoning about disclosure in data integration in the presence of source constraints

Title Reasoning about disclosure in data integration in the presence of source constraints
Authors Michael Benedikt, Pierre Bourhis, Louis Jachiet, Michaël Thomazo
Abstract Data integration systems allow users to access data sitting in multiple sources by means of queries over a global schema, related to the sources via mappings. Data sources often contain sensitive information, and thus an analysis is needed to verify that a schema satisfies a privacy policy, given as a set of queries whose answers should not be accessible to users. Such an analysis should take into account not only knowledge that an attacker may have about the mappings, but also what they may know about the semantics of the sources. In this paper, we show that source constraints can have a dramatic impact on disclosure analysis. We study the problem of determining whether a given data integration system discloses a source query to an attacker in the presence of constraints, providing both lower and upper bounds on source-aware disclosure analysis.
Tasks
Published 2019-06-03
URL https://arxiv.org/abs/1906.00624v1
PDF https://arxiv.org/pdf/1906.00624v1.pdf
PWC https://paperswithcode.com/paper/190600624
Repo
Framework

Learning Symmetric and Asymmetric Steganography via Adversarial Training

Title Learning Symmetric and Asymmetric Steganography via Adversarial Training
Authors Zheng Li, Ge Han, Yunqing Wei, Shanqing Guo
Abstract Steganography refers to the art of concealing secret messages within multiple media carriers so that an eavesdropper is unable to detect the presence and content of the hidden messages. In this paper, we firstly propose a novel key-dependent steganographic scheme that achieves steganographic objectives with adversarial training. Symmetric (secret-key) and Asymmetric (public-key) steganographic scheme are separately proposed and each scheme is successfully designed and implemented. We show that these encodings produced by our scheme improve the invisibility by 20% than previous deep-leanring-based work, and further that perform competitively remarkable undetectability 25% better than classic steganographic algorithms. Finally, we simulated our scheme in a real situation where the decoder achieved an accuracy of more than 98% of the original message.
Tasks
Published 2019-03-13
URL https://arxiv.org/abs/1903.05297v2
PDF https://arxiv.org/pdf/1903.05297v2.pdf
PWC https://paperswithcode.com/paper/learning-symmetric-and-asymmetric
Repo
Framework

Improved low-count quantitative PET reconstruction with an iterative neural network

Title Improved low-count quantitative PET reconstruction with an iterative neural network
Authors Hongki Lim, Il Yong Chun, Yuni K. Dewaraja, Jeffrey A. Fessler
Abstract Image reconstruction in low-count PET is particularly challenging because gammas from natural radioactivity in Lu-based crystals cause high random fractions that lower the measurement signal-to-noise-ratio (SNR). In model-based image reconstruction (MBIR), using more iterations of an unregularized method may increase the noise, so incorporating regularization into the image reconstruction is desirable to control the noise. New regularization methods based on learned convolutional operators are emerging in MBIR. We modify the architecture of an iterative neural network, BCD-Net, for PET MBIR, and demonstrate the efficacy of the trained BCD-Net using XCAT phantom data that simulates the low true coincidence count-rates with high random fractions typical for Y-90 PET patient imaging after Y-90 microsphere radioembolization. Numerical results show that the proposed BCD-Net significantly improves PET reconstruction performance compared to MBIR methods using non-trained regularizers, total variation (TV) and non-local means (NLM). BCD-Net significantly improved CNR and RMSE compared to TV (NLM) regularized MBIR. Moreover, BCD-Net successfully generalizes to data that differs from training data. Improvements were also demonstrated for the clinically relevant phantom measurement data where we used training and testing datasets having very different activity distributions and count-levels.
Tasks Image Reconstruction
Published 2019-06-05
URL https://arxiv.org/abs/1906.02327v2
PDF https://arxiv.org/pdf/1906.02327v2.pdf
PWC https://paperswithcode.com/paper/improved-low-count-quantitative-pet
Repo
Framework
comments powered by Disqus