Paper Group ANR 134
Efficient PAC Learning from the Crowd. Regularizing deep networks using efficient layerwise adversarial training. On the Sample Complexity of the Linear Quadratic Regulator. Data Fusion and Machine Learning Integration for Transformer Loss of Life Estimation. Enhanced Particle Swarm Optimization Algorithms for Multiple-Input Multiple-Output System …
Efficient PAC Learning from the Crowd
| Title | Efficient PAC Learning from the Crowd |
| Authors | Pranjal Awasthi, Avrim Blum, Nika Haghtalab, Yishay Mansour |
| Abstract | In recent years crowdsourcing has become the method of choice for gathering labeled training data for learning algorithms. Standard approaches to crowdsourcing view the process of acquiring labeled data separately from the process of learning a classifier from the gathered data. This can give rise to computational and statistical challenges. For example, in most cases there are no known computationally efficient learning algorithms that are robust to the high level of noise that exists in crowdsourced data, and efforts to eliminate noise through voting often require a large number of queries per example. In this paper, we show how by interleaving the process of labeling and learning, we can attain computational efficiency with much less overhead in the labeling cost. In particular, we consider the realizable setting where there exists a true target function in $\mathcal{F}$ and consider a pool of labelers. When a noticeable fraction of the labelers are perfect, and the rest behave arbitrarily, we show that any $\mathcal{F}$ that can be efficiently learned in the traditional realizable PAC model can be learned in a computationally efficient manner by querying the crowd, despite high amounts of noise in the responses. Moreover, we show that this can be done while each labeler only labels a constant number of examples and the number of labels requested per example, on average, is a constant. When no perfect labelers exist, a related task is to find a set of the labelers which are good but not perfect. We show that we can identify all good labelers, when at least the majority of labelers are good. |
| Tasks | |
| Published | 2017-03-21 |
| URL | http://arxiv.org/abs/1703.07432v2 |
| http://arxiv.org/pdf/1703.07432v2.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-pac-learning-from-the-crowd |
| Repo | |
| Framework | |
Regularizing deep networks using efficient layerwise adversarial training
| Title | Regularizing deep networks using efficient layerwise adversarial training |
| Authors | Swami Sankaranarayanan, Arpit Jain, Rama Chellappa, Ser Nam Lim |
| Abstract | Adversarial training has been shown to regularize deep neural networks in addition to increasing their robustness to adversarial examples. However, its impact on very deep state of the art networks has not been fully investigated. In this paper, we present an efficient approach to perform adversarial training by perturbing intermediate layer activations and study the use of such perturbations as a regularizer during training. We use these perturbations to train very deep models such as ResNets and show improvement in performance both on adversarial and original test data. Our experiments highlight the benefits of perturbing intermediate layer activations compared to perturbing only the inputs. The results on CIFAR-10 and CIFAR-100 datasets show the merits of the proposed adversarial training approach. Additional results on WideResNets show that our approach provides significant improvement in classification accuracy for a given base model, outperforming dropout and other base models of larger size. |
| Tasks | |
| Published | 2017-05-22 |
| URL | http://arxiv.org/abs/1705.07819v2 |
| http://arxiv.org/pdf/1705.07819v2.pdf | |
| PWC | https://paperswithcode.com/paper/regularizing-deep-networks-using-efficient |
| Repo | |
| Framework | |
On the Sample Complexity of the Linear Quadratic Regulator
| Title | On the Sample Complexity of the Linear Quadratic Regulator |
| Authors | Sarah Dean, Horia Mania, Nikolai Matni, Benjamin Recht, Stephen Tu |
| Abstract | This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then designs a controller using both the model and uncertainty estimate. Our technique uses contemporary tools from random matrix theory to bound the error in the estimation procedure. We also employ a recently developed approach to control synthesis called System Level Synthesis that enables robust control design by solving a convex optimization problem. We provide end-to-end bounds on the relative error in control cost that are nearly optimal in the number of parameters and that highlight salient properties of the system to be controlled such as closed-loop sensitivity and optimal control magnitude. We show experimentally that the Coarse-ID approach enables efficient computation of a stabilizing controller in regimes where simple control schemes that do not take the model uncertainty into account fail to stabilize the true system. |
| Tasks | |
| Published | 2017-10-04 |
| URL | http://arxiv.org/abs/1710.01688v3 |
| http://arxiv.org/pdf/1710.01688v3.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-sample-complexity-of-the-linear |
| Repo | |
| Framework | |
Data Fusion and Machine Learning Integration for Transformer Loss of Life Estimation
| Title | Data Fusion and Machine Learning Integration for Transformer Loss of Life Estimation |
| Authors | Mohsen Mahoor, Amin Khodaei |
| Abstract | Rapid growth of machine learning methodologies and their applications offer new opportunity for improved transformer asset management. Accordingly, power system operators are currently looking for data-driven methods to make better-informed decisions in terms of network management. In this paper, machine learning and data fusion techniques are integrated to estimate transformer loss of life. Using IEEE Std. C57.91-2011, a data synthesis process is proposed based on hourly transformer loading and ambient temperature values. This synthesized data is employed to estimate transformer loss of life by using Adaptive Network-Based Fuzzy Inference System (ANFIS) and Radial Basis Function (RBF) network, which are further fused together with the objective of improving the estimation accuracy. Among various data fusion techniques, Ordered Weighted Averaging (OWA) and sequential Kalman filter are selected to fuse the output results of the estimated ANFIS and RBF. Simulation results demonstrate the merit and the effectiveness of the proposed method. |
| Tasks | |
| Published | 2017-11-08 |
| URL | http://arxiv.org/abs/1711.03398v1 |
| http://arxiv.org/pdf/1711.03398v1.pdf | |
| PWC | https://paperswithcode.com/paper/data-fusion-and-machine-learning-integration |
| Repo | |
| Framework | |
Enhanced Particle Swarm Optimization Algorithms for Multiple-Input Multiple-Output System Modelling using Convolved Gaussian Process Models
| Title | Enhanced Particle Swarm Optimization Algorithms for Multiple-Input Multiple-Output System Modelling using Convolved Gaussian Process Models |
| Authors | Gang Cao, Edmund M-K Lai, Fakhrul Alam |
| Abstract | Convolved Gaussian Process (CGP) is able to capture the correlations not only between inputs and outputs but also among the outputs. This allows a superior performance of using CGP than standard Gaussian Process (GP) in the modelling of Multiple-Input Multiple-Output (MIMO) systems when observations are missing for some of outputs. Similar to standard GP, a key issue of CGP is the learning of hyperparameters from a set of input-output observations. It typically performed by maximizing the Log-Likelihood (LL) function which leads to an unconstrained nonlinear and non-convex optimization problem. Algorithms such as Conjugate Gradient (CG) or Broyden-Fletcher-Goldfarb-Shanno (BFGS) are commonly used but they often get stuck in local optima, especially for CGP where there are more hyperparameters. In addition, the LL value is not a reliable indicator for judging the quality intermediate models in the optimization process. In this paper, we propose to use enhanced Particle Swarm Optimization (PSO) algorithms to solve this problem by minimizing the model output error instead. This optimization criterion enables the quality of intermediate solutions to be directly observable during the optimization process. Two enhancements to the standard PSO algorithm which make use of gradient information and the multi- start technique are proposed. Simulation results on the modelling of both linear and nonlinear systems demonstrate the effectiveness of minimizing the model output error to learn hyperparameters and the performance of using enhanced algorithms. |
| Tasks | |
| Published | 2017-07-12 |
| URL | http://arxiv.org/abs/1709.04319v1 |
| http://arxiv.org/pdf/1709.04319v1.pdf | |
| PWC | https://paperswithcode.com/paper/enhanced-particle-swarm-optimization |
| Repo | |
| Framework | |
Using LIP to Gloss Over Faces in Single-Stage Face Detection Networks
| Title | Using LIP to Gloss Over Faces in Single-Stage Face Detection Networks |
| Authors | Siqi Yang, Arnold Wiliem, Shaokang Chen, Brian C. Lovell |
| Abstract | This work shows that it is possible to fool/attack recent state-of-the-art face detectors which are based on the single-stage networks. Successfully attacking face detectors could be a serious malware vulnerability when deploying a smart surveillance system utilizing face detectors. We show that existing adversarial perturbation methods are not effective to perform such an attack, especially when there are multiple faces in the input image. This is because the adversarial perturbation specifically generated for one face may disrupt the adversarial perturbation for another face. In this paper, we call this problem the Instance Perturbation Interference (IPI) problem. This IPI problem is addressed by studying the relationship between the deep neural network receptive field and the adversarial perturbation. As such, we propose the Localized Instance Perturbation (LIP) that uses adversarial perturbation constrained to the Effective Receptive Field (ERF) of a target to perform the attack. Experiment results show the LIP method massively outperforms existing adversarial perturbation generation methods – often by a factor of 2 to 10. |
| Tasks | Face Detection |
| Published | 2017-12-22 |
| URL | http://arxiv.org/abs/1712.08263v2 |
| http://arxiv.org/pdf/1712.08263v2.pdf | |
| PWC | https://paperswithcode.com/paper/using-lip-to-gloss-over-faces-in-single-stage |
| Repo | |
| Framework | |
The NLTK FrameNet API: Designing for Discoverability with a Rich Linguistic Resource
| Title | The NLTK FrameNet API: Designing for Discoverability with a Rich Linguistic Resource |
| Authors | Nathan Schneider, Chuck Wooters |
| Abstract | A new Python API, integrated within the NLTK suite, offers access to the FrameNet 1.7 lexical database. The lexicon (structured in terms of frames) as well as annotated sentences can be processed programatically, or browsed with human-readable displays via the interactive Python prompt. |
| Tasks | |
| Published | 2017-03-21 |
| URL | http://arxiv.org/abs/1703.07438v2 |
| http://arxiv.org/pdf/1703.07438v2.pdf | |
| PWC | https://paperswithcode.com/paper/the-nltk-framenet-api-designing-for |
| Repo | |
| Framework | |
Low Rank Matrix Recovery with Simultaneous Presence of Outliers and Sparse Corruption
| Title | Low Rank Matrix Recovery with Simultaneous Presence of Outliers and Sparse Corruption |
| Authors | Mostafa Rahmani, George Atia |
| Abstract | We study a data model in which the data matrix D can be expressed as D = L + S + C, where L is a low rank matrix, S an element-wise sparse matrix and C a matrix whose non-zero columns are outlying data points. To date, robust PCA algorithms have solely considered models with either S or C, but not both. As such, existing algorithms cannot account for simultaneous element-wise and column-wise corruptions. In this paper, a new robust PCA algorithm that is robust to simultaneous types of corruption is proposed. Our approach hinges on the sparse approximation of a sparsely corrupted column so that the sparse expansion of a column with respect to the other data points is used to distinguish a sparsely corrupted inlier column from an outlying data point. We also develop a randomized design which provides a scalable implementation of the proposed approach. The core idea of sparse approximation is analyzed analytically where we show that the underlying ell_1-norm minimization can obtain the representation of an inlier in presence of sparse corruptions. |
| Tasks | |
| Published | 2017-02-07 |
| URL | http://arxiv.org/abs/1702.01847v1 |
| http://arxiv.org/pdf/1702.01847v1.pdf | |
| PWC | https://paperswithcode.com/paper/low-rank-matrix-recovery-with-simultaneous |
| Repo | |
| Framework | |
A Comparison of Public Causal Search Packages on Linear, Gaussian Data with No Latent Variables
| Title | A Comparison of Public Causal Search Packages on Linear, Gaussian Data with No Latent Variables |
| Authors | Joseph D. Ramsey, Bryan Andrews |
| Abstract | We compare Tetrad (Java) algorithms to the other public software packages BNT (Bayes Net Toolbox, Matlab), pcalg (R), bnlearn (R) on the \vanilla” task of recovering DAG structure to the extent possible from data generated recursively from linear, Gaussian structure equation models (SEMs) with no latent variables, for random graphs, with no additional knowledge of variable order or adjacency structure, and without additional specification of intervention information. Each one of the above packages offers at least one implementation suitable to this purpose. We compare them on adjacency and orientation accuracy as well as time performance, for fixed datasets. We vary the number of variables, the number of samples, and the density of graph, for a total of 27 combinations, averaging all statistics over 10 runs, for a total of 270 datasets. All runs are carried out on the same machine and on their native platforms. An interactive visualization tool is provided for the reader who wishes to know more than can be documented explicitly in this report. |
| Tasks | |
| Published | 2017-09-13 |
| URL | http://arxiv.org/abs/1709.04240v2 |
| http://arxiv.org/pdf/1709.04240v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-comparison-of-public-causal-search-packages |
| Repo | |
| Framework | |
Seeing Through Noise: Visually Driven Speaker Separation and Enhancement
| Title | Seeing Through Noise: Visually Driven Speaker Separation and Enhancement |
| Authors | Aviv Gabbay, Ariel Ephrat, Tavi Halperin, Shmuel Peleg |
| Abstract | Isolating the voice of a specific person while filtering out other voices or background noises is challenging when video is shot in noisy environments. We propose audio-visual methods to isolate the voice of a single speaker and eliminate unrelated sounds. First, face motions captured in the video are used to estimate the speaker’s voice, by passing the silent video frames through a video-to-speech neural network-based model. Then the speech predictions are applied as a filter on the noisy input audio. This approach avoids using mixtures of sounds in the learning process, as the number of such possible mixtures is huge, and would inevitably bias the trained model. We evaluate our method on two audio-visual datasets, GRID and TCD-TIMIT, and show that our method attains significant SDR and PESQ improvements over the raw video-to-speech predictions, and a well-known audio-only method. |
| Tasks | Speaker Separation |
| Published | 2017-08-22 |
| URL | http://arxiv.org/abs/1708.06767v3 |
| http://arxiv.org/pdf/1708.06767v3.pdf | |
| PWC | https://paperswithcode.com/paper/seeing-through-noise-visually-driven-speaker |
| Repo | |
| Framework | |
Concept Transfer Learning for Adaptive Language Understanding
| Title | Concept Transfer Learning for Adaptive Language Understanding |
| Authors | Su Zhu, Kai Yu |
| Abstract | Concept definition is important in language understanding (LU) adaptation since literal definition difference can easily lead to data sparsity even if different data sets are actually semantically correlated. To address this issue, in this paper, a novel concept transfer learning approach is proposed. Here, substructures within literal concept definition are investigated to reveal the relationship between concepts. A hierarchical semantic representation for concepts is proposed, where a semantic slot is represented as a composition of {\em atomic concepts}. Based on this new hierarchical representation, transfer learning approaches are developed for adaptive LU. The approaches are applied to two tasks: value set mismatch and domain adaptation, and evaluated on two LU benchmarks: ATIS and DSTC 2&3. Thorough empirical studies validate both the efficiency and effectiveness of the proposed method. In particular, we achieve state-of-the-art performance ($F_1$-score 96.08%) on ATIS by only using lexicon features. |
| Tasks | Domain Adaptation, Transfer Learning |
| Published | 2017-06-03 |
| URL | https://arxiv.org/abs/1706.00927v3 |
| https://arxiv.org/pdf/1706.00927v3.pdf | |
| PWC | https://paperswithcode.com/paper/concept-transfer-learning-for-adaptive |
| Repo | |
| Framework | |
Symbolic LTLf Synthesis
| Title | Symbolic LTLf Synthesis |
| Authors | Shufang Zhu, Lucas M. Tabajara, Jianwen Li, Geguang Pu, Moshe Y. Vardi |
| Abstract | LTLf synthesis is the process of finding a strategy that satisfies a linear temporal specification over finite traces. An existing solution to this problem relies on a reduction to a DFA game. In this paper, we propose a symbolic framework for LTLf synthesis based on this technique, by performing the computation over a representation of the DFA as a boolean formula rather than as an explicit graph. This approach enables strategy generation by utilizing the mechanism of boolean synthesis. We implement this symbolic synthesis method in a tool called Syft, and demonstrate by experiments on scalable benchmarks that the symbolic approach scales better than the explicit one. |
| Tasks | |
| Published | 2017-05-23 |
| URL | http://arxiv.org/abs/1705.08426v2 |
| http://arxiv.org/pdf/1705.08426v2.pdf | |
| PWC | https://paperswithcode.com/paper/symbolic-ltlf-synthesis |
| Repo | |
| Framework | |
What can you do with a rock? Affordance extraction via word embeddings
| Title | What can you do with a rock? Affordance extraction via word embeddings |
| Authors | Nancy Fulda, Daniel Ricks, Ben Murdoch, David Wingate |
| Abstract | Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance detection is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance most of the time. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent’s action selections begin to resemble those a human would choose. |
| Tasks | Word Embeddings |
| Published | 2017-03-09 |
| URL | http://arxiv.org/abs/1703.03429v1 |
| http://arxiv.org/pdf/1703.03429v1.pdf | |
| PWC | https://paperswithcode.com/paper/what-can-you-do-with-a-rock-affordance |
| Repo | |
| Framework | |
Statistical Latent Space Approach for Mixed Data Modelling and Applications
| Title | Statistical Latent Space Approach for Mixed Data Modelling and Applications |
| Authors | Tu Dinh Nguyen, Truyen Tran, Dinh Phung, Svetha Venkatesh |
| Abstract | The analysis of mixed data has been raising challenges in statistics and machine learning. One of two most prominent challenges is to develop new statistical techniques and methodologies to effectively handle mixed data by making the data less heterogeneous with minimum loss of information. The other challenge is that such methods must be able to apply in large-scale tasks when dealing with huge amount of mixed data. To tackle these challenges, we introduce parameter sharing and balancing extensions to our recent model, the mixed-variate restricted Boltzmann machine (MV.RBM) which can transform heterogeneous data into homogeneous representation. We also integrate structured sparsity and distance metric learning into RBM-based models. Our proposed methods are applied in various applications including latent patient profile modelling in medical data analysis and representation learning for image retrieval. The experimental results demonstrate the models perform better than baseline methods in medical data and outperform state-of-the-art rivals in image dataset. |
| Tasks | Image Retrieval, Metric Learning, Representation Learning |
| Published | 2017-08-18 |
| URL | http://arxiv.org/abs/1708.05594v1 |
| http://arxiv.org/pdf/1708.05594v1.pdf | |
| PWC | https://paperswithcode.com/paper/statistical-latent-space-approach-for-mixed |
| Repo | |
| Framework | |
Episode-Based Active Learning with Bayesian Neural Networks
| Title | Episode-Based Active Learning with Bayesian Neural Networks |
| Authors | Feras Dayoub, Niko Sünderhauf, Peter Corke |
| Abstract | We investigate different strategies for active learning with Bayesian deep neural networks. We focus our analysis on scenarios where new, unlabeled data is obtained episodically, such as commonly encountered in mobile robotics applications. An evaluation of different strategies for acquisition, updating, and final training on the CIFAR-10 dataset shows that incremental network updates with final training on the accumulated acquisition set are essential for best performance, while limiting the amount of required human labeling labor. |
| Tasks | Active Learning |
| Published | 2017-03-21 |
| URL | http://arxiv.org/abs/1703.07473v1 |
| http://arxiv.org/pdf/1703.07473v1.pdf | |
| PWC | https://paperswithcode.com/paper/episode-based-active-learning-with-bayesian |
| Repo | |
| Framework | |
