Paper Group AWR 91
Active Learning from Positive and Unlabeled Data. From Word Embeddings to Item Recommendation. Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs. Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks. Topic Aware Neural Response Generation. Node Embedding via Word Embedding for Network Com …
Active Learning from Positive and Unlabeled Data
| Title | Active Learning from Positive and Unlabeled Data |
| Authors | Alireza Ghasemi, Hamid R. Rabiee, Mohsen Fadaee, Mohammad T. Manzuri, Mohammad H. Rohban |
| Abstract | During recent years, active learning has evolved into a popular paradigm for utilizing user’s feedback to improve accuracy of learning algorithms. Active learning works by selecting the most informative sample among unlabeled data and querying the label of that point from user. Many different methods such as uncertainty sampling and minimum risk sampling have been utilized to select the most informative sample in active learning. Although many active learning algorithms have been proposed so far, most of them work with binary or multi-class classification problems and therefore can not be applied to problems in which only samples from one class as well as a set of unlabeled data are available. Such problems arise in many real-world situations and are known as the problem of learning from positive and unlabeled data. In this paper we propose an active learning algorithm that can work when only samples of one class as well as a set of unlabelled data are available. Our method works by separately estimating probability desnity of positive and unlabeled points and then computing expected value of informativeness to get rid of a hyper-parameter and have a better measure of informativeness./ Experiments and empirical analysis show promising results compared to other similar methods. |
| Tasks | Active Learning |
| Published | 2016-02-24 |
| URL | http://arxiv.org/abs/1602.07495v1 |
| http://arxiv.org/pdf/1602.07495v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-learning-from-positive-and-unlabeled |
| Repo | https://github.com/aghasemi/alpud |
| Framework | none |
From Word Embeddings to Item Recommendation
| Title | From Word Embeddings to Item Recommendation |
| Authors | Makbule Gulcin Ozsoy |
| Abstract | Social network platforms can use the data produced by their users to serve them better. One of the services these platforms provide is recommendation service. Recommendation systems can predict the future preferences of users using their past preferences. In the recommendation systems literature there are various techniques, such as neighborhood based methods, machine-learning based methods and matrix-factorization based methods. In this work, a set of well known methods from natural language processing domain, namely Word2Vec, is applied to recommendation systems domain. Unlike previous works that use Word2Vec for recommendation, this work uses non-textual features, the check-ins, and it recommends venues to visit/check-in to the target users. For the experiments, a Foursquare check-in dataset is used. The results show that use of continuous vector space representations of items modeled by techniques of Word2Vec is promising for making recommendations. |
| Tasks | Recommendation Systems, Word Embeddings |
| Published | 2016-01-07 |
| URL | http://arxiv.org/abs/1601.01356v3 |
| http://arxiv.org/pdf/1601.01356v3.pdf | |
| PWC | https://paperswithcode.com/paper/from-word-embeddings-to-item-recommendation |
| Repo | https://github.com/mgulcin/DL_Rec |
| Framework | none |
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs
| Title | Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs |
| Authors | Li Jing, Yichen Shen, Tena Dubček, John Peurifoy, Scott Skirlo, Yann LeCun, Max Tegmark, Marin Soljačić |
| Abstract | Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data. This approach appears particularly promising for Recurrent Neural Networks (RNNs). In this work, we present a new architecture for implementing an Efficient Unitary Neural Network (EUNNs); its main advantages can be summarized as follows. Firstly, the representation capacity of the unitary space in an EUNN is fully tunable, ranging from a subspace of SU(N) to the entire unitary space. Secondly, the computational complexity for training an EUNN is merely $\mathcal{O}(1)$ per parameter. Finally, we test the performance of EUNNs on the standard copying task, the pixel-permuted MNIST digit recognition benchmark as well as the Speech Prediction Test (TIMIT). We find that our architecture significantly outperforms both other state-of-the-art unitary RNNs and the LSTM architecture, in terms of the final performance and/or the wall-clock training speed. EUNNs are thus promising alternatives to RNNs and LSTMs for a wide variety of applications. |
| Tasks | |
| Published | 2016-12-15 |
| URL | http://arxiv.org/abs/1612.05231v3 |
| http://arxiv.org/pdf/1612.05231v3.pdf | |
| PWC | https://paperswithcode.com/paper/tunable-efficient-unitary-neural-networks |
| Repo | https://github.com/flaport/torch_eunn |
| Framework | pytorch |
Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks
| Title | Recursive Autoconvolution for Unsupervised Learning of Convolutional Neural Networks |
| Authors | Boris Knyazev, Erhardt Barth, Thomas Martinetz |
| Abstract | In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently. We show that the recursive autoconvolution operator, adopted from physics, boosts existing unsupervised methods by learning more discriminative filters. We take well established convolutional neural networks and train their filters layer-wise. In addition, based on previous works we design a network which extracts more than 600k features per sample, but with the total number of trainable parameters greatly reduced by introducing shared filters in higher layers. We evaluate our networks on the MNIST, CIFAR-10, CIFAR-100 and STL-10 image classification benchmarks and report several state of the art results among other unsupervised methods. |
| Tasks | Image Classification |
| Published | 2016-06-02 |
| URL | http://arxiv.org/abs/1606.00611v2 |
| http://arxiv.org/pdf/1606.00611v2.pdf | |
| PWC | https://paperswithcode.com/paper/recursive-autoconvolution-for-unsupervised |
| Repo | https://github.com/bknyaz/autocnn_unsup |
| Framework | none |
Topic Aware Neural Response Generation
| Title | Topic Aware Neural Response Generation |
| Authors | Chen Xing, Wei Wu, Yu Wu, Jie Liu, Yalou Huang, Ming Zhou, Wei-Ying Ma |
| Abstract | We consider incorporating topic information into the sequence-to-sequence framework to generate informative and interesting responses for chatbots. To this end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. The model utilizes topics to simulate prior knowledge of human that guides them to form informative and interesting responses in conversation, and leverages the topic information in generation by a joint attention mechanism and a biased generation probability. The joint attention mechanism summarizes the hidden vectors of an input message as context vectors by message attention, synthesizes topic vectors by topic attention from the topic words of the message obtained from a pre-trained LDA model, and let these vectors jointly affect the generation of words in decoding. To increase the possibility of topic words appearing in responses, the model modifies the generation probability of topic words by adding an extra probability item to bias the overall distribution. Empirical study on both automatic evaluation metrics and human annotations shows that TA-Seq2Seq can generate more informative and interesting responses, and significantly outperform the-state-of-the-art response generation models. |
| Tasks | |
| Published | 2016-06-21 |
| URL | http://arxiv.org/abs/1606.08340v2 |
| http://arxiv.org/pdf/1606.08340v2.pdf | |
| PWC | https://paperswithcode.com/paper/topic-aware-neural-response-generation |
| Repo | https://github.com/nouhadziri/THRED |
| Framework | tf |
Node Embedding via Word Embedding for Network Community Discovery
| Title | Node Embedding via Word Embedding for Network Community Discovery |
| Authors | Weicong Ding, Christy Lin, Prakash Ishwar |
| Abstract | Neural node embeddings have recently emerged as a powerful representation for supervised learning tasks involving graph-structured data. We leverage this recent advance to develop a novel algorithm for unsupervised community discovery in graphs. Through extensive experimental studies on simulated and real-world data, we demonstrate that the proposed approach consistently improves over the current state-of-the-art. Specifically, our approach empirically attains the information-theoretic limits for community recovery under the benchmark Stochastic Block Models for graph generation and exhibits better stability and accuracy over both Spectral Clustering and Acyclic Belief Propagation in the community recovery limits. |
| Tasks | Graph Generation |
| Published | 2016-11-09 |
| URL | http://arxiv.org/abs/1611.03028v3 |
| http://arxiv.org/pdf/1611.03028v3.pdf | |
| PWC | https://paperswithcode.com/paper/node-embedding-via-word-embedding-for-network |
| Repo | https://github.com/cy93lin/SBM_node_embedding |
| Framework | none |
Boda-RTC: Productive Generation of Portable, Efficient Code for Convolutional Neural Networks on Mobile Computing Platforms
| Title | Boda-RTC: Productive Generation of Portable, Efficient Code for Convolutional Neural Networks on Mobile Computing Platforms |
| Authors | Matthew Moskewicz, Forrest Iandola, Kurt Keutzer |
| Abstract | The popularity of neural networks (NNs) spans academia, industry, and popular culture. In particular, convolutional neural networks (CNNs) have been applied to many image based machine learning tasks and have yielded strong results. The availability of hardware/software systems for efficient training and deployment of large and/or deep CNN models has been, and continues to be, an important consideration for the field. Early systems for NN computation focused on leveraging existing dense linear algebra techniques and libraries. Current approaches use low-level machine specific programming and/or closed-source, purpose-built vendor libraries. In this work, we present an open source system that, compared to existing approaches, achieves competitive computational speed while achieving higher portability. We achieve this by targeting the vendor-neutral OpenCL platform using a code-generation approach. We argue that our approach allows for both: (1) the rapid development of new computational kernels for existing hardware targets, and (2) the rapid tuning of existing computational kernels for new hardware targets. Results are presented for a case study of targeting the Qualcomm Snapdragon 820 mobile computing platform for CNN deployment. |
| Tasks | Code Generation |
| Published | 2016-06-01 |
| URL | http://arxiv.org/abs/1606.00094v2 |
| http://arxiv.org/pdf/1606.00094v2.pdf | |
| PWC | https://paperswithcode.com/paper/boda-rtc-productive-generation-of-portable |
| Repo | https://github.com/moskewcz/boda |
| Framework | none |
Neurally-Guided Procedural Models: Amortized Inference for Procedural Graphics Programs using Neural Networks
| Title | Neurally-Guided Procedural Models: Amortized Inference for Procedural Graphics Programs using Neural Networks |
| Authors | Daniel Ritchie, Anna Thomas, Pat Hanrahan, Noah D. Goodman |
| Abstract | Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models in computer graphics, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks which control how the model makes random choices based on the output it has generated thus far. We call such models neurally-guided procedural models. As a pre-computation, we train these models to maximize the likelihood of example outputs generated via SMC. They are then used as efficient SMC importance samplers, generating high-quality results with very few samples. We evaluate our method on L-system-like models with image-based constraints. Given a desired quality threshold, neurally-guided models can generate satisfactory results up to 10x faster than unguided models. |
| Tasks | |
| Published | 2016-03-19 |
| URL | http://arxiv.org/abs/1603.06143v2 |
| http://arxiv.org/pdf/1603.06143v2.pdf | |
| PWC | https://paperswithcode.com/paper/neurally-guided-procedural-models-amortized |
| Repo | https://github.com/dritchie/adnn |
| Framework | none |
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
| Title | FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation |
| Authors | Judy Hoffman, Dequan Wang, Fisher Yu, Trevor Darrell |
| Abstract | Fully convolutional models for dense prediction have proven successful for a wide range of visual tasks. Such models perform well in a supervised setting, but performance can be surprisingly poor under domain shifts that appear mild to a human observer. For example, training on one city and testing on another in a different geographic region and/or weather condition may result in significantly degraded performance due to pixel-level distribution shift. In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems. Our method consists of both global and category specific adaptation techniques. Global domain alignment is performed using a novel semantic segmentation network with fully convolutional domain adversarial learning. This initially adapted space then enables category specific adaptation through a generalization of constrained weak learning, with explicit transfer of the spatial layout from the source to the target domains. Our approach outperforms baselines across different settings on multiple large-scale datasets, including adapting across various real city environments, different synthetic sub-domains, from simulated to real environments, and on a novel large-scale dash-cam dataset. |
| Tasks | Image-to-Image Translation, Semantic Segmentation, Synthetic-to-Real Translation |
| Published | 2016-12-08 |
| URL | http://arxiv.org/abs/1612.02649v1 |
| http://arxiv.org/pdf/1612.02649v1.pdf | |
| PWC | https://paperswithcode.com/paper/fcns-in-the-wild-pixel-level-adversarial-and |
| Repo | https://github.com/Wanger-SJTU/FCN-in-the-wild |
| Framework | pytorch |
Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics
| Title | Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics |
| Authors | Felix Berkenkamp, Andreas Krause, Angela P. Schoellig |
| Abstract | Robotic algorithms typically depend on various parameters, the choice of which significantly affects the robot’s performance. While an initial guess for the parameters may be obtained from dynamic models of the robot, parameters are usually tuned manually on the real system to achieve the best performance. Optimization algorithms, such as Bayesian optimization, have been used to automate this process. However, these methods may evaluate unsafe parameters during the optimization process that lead to safety-critical system failures. Recently, a safe Bayesian optimization algorithm, called SafeOpt, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in robotics. For example, high-gain controllers might achieve low average tracking error (performance), but can overshoot and violate input constraints. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle. |
| Tasks | |
| Published | 2016-02-14 |
| URL | http://arxiv.org/abs/1602.04450v2 |
| http://arxiv.org/pdf/1602.04450v2.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-optimization-with-safety-constraints |
| Repo | https://github.com/befelix/SafeOpt |
| Framework | none |
DSAC - Differentiable RANSAC for Camera Localization
| Title | DSAC - Differentiable RANSAC for Camera Localization |
| Authors | Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, Carsten Rother |
| Abstract | RANSAC is an important algorithm in robust optimization and a central building block for many computer vision applications. In recent years, traditionally hand-crafted pipelines have been replaced by deep learning pipelines, which can be trained in an end-to-end fashion. However, RANSAC has so far not been used as part of such deep learning pipelines, because its hypothesis selection procedure is non-differentiable. In this work, we present two different ways to overcome this limitation. The most promising approach is inspired by reinforcement learning, namely to replace the deterministic hypothesis selection by a probabilistic selection for which we can derive the expected loss w.r.t. to all learnable parameters. We call this approach DSAC, the differentiable counterpart of RANSAC. We apply DSAC to the problem of camera localization, where deep learning has so far failed to improve on traditional approaches. We demonstrate that by directly minimizing the expected loss of the output camera poses, robustly estimated by RANSAC, we achieve an increase in accuracy. In the future, any deep learning pipeline can use DSAC as a robust optimization component. |
| Tasks | Camera Localization |
| Published | 2016-11-17 |
| URL | http://arxiv.org/abs/1611.05705v4 |
| http://arxiv.org/pdf/1611.05705v4.pdf | |
| PWC | https://paperswithcode.com/paper/dsac-differentiable-ransac-for-camera |
| Repo | https://github.com/vislearn/esac |
| Framework | pytorch |
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
| Title | On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima |
| Authors | Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, Ping Tak Peter Tang |
| Abstract | The stochastic gradient descent (SGD) method and its variants are algorithms of choice for many Deep Learning tasks. These methods operate in a small-batch regime wherein a fraction of the training data, say $32$-$512$ data points, is sampled to compute an approximation to the gradient. It has been observed in practice that when using a larger batch there is a degradation in the quality of the model, as measured by its ability to generalize. We investigate the cause for this generalization drop in the large-batch regime and present numerical evidence that supports the view that large-batch methods tend to converge to sharp minimizers of the training and testing functions - and as is well known, sharp minima lead to poorer generalization. In contrast, small-batch methods consistently converge to flat minimizers, and our experiments support a commonly held view that this is due to the inherent noise in the gradient estimation. We discuss several strategies to attempt to help large-batch methods eliminate this generalization gap. |
| Tasks | |
| Published | 2016-09-15 |
| URL | http://arxiv.org/abs/1609.04836v2 |
| http://arxiv.org/pdf/1609.04836v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-large-batch-training-for-deep-learning |
| Repo | https://github.com/JaimeGomezRecasens/KC-Deep-Learning |
| Framework | none |
Semi-Supervised Radio Signal Identification
| Title | Semi-Supervised Radio Signal Identification |
| Authors | Timothy J. O’Shea, Nathan West, Matthew Vondal, T. Charles Clancy |
| Abstract | Radio emitter recognition in dense multi-user environments is an important tool for optimizing spectrum utilization, identifying and minimizing interference, and enforcing spectrum policy. Radio data is readily available and easy to obtain from an antenna, but labeled and curated data is often scarce making supervised learning strategies difficult and time consuming in practice. We demonstrate that semi-supervised learning techniques can be used to scale learning beyond supervised datasets, allowing for discerning and recalling new radio signals by using sparse signal representations based on both unsupervised and supervised methods for nonlinear feature learning and clustering methods. |
| Tasks | |
| Published | 2016-11-01 |
| URL | http://arxiv.org/abs/1611.00303v2 |
| http://arxiv.org/pdf/1611.00303v2.pdf | |
| PWC | https://paperswithcode.com/paper/semi-supervised-radio-signal-identification |
| Repo | https://github.com/mistic-lab/IPSW-RFI |
| Framework | pytorch |
Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images
| Title | Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images |
| Authors | Debapriya Maji, Anirban Santara, Pabitra Mitra, Debdoot Sheet |
| Abstract | Vision impairment due to pathological damage of the retina can largely be prevented through periodic screening using fundus color imaging. However the challenge with large scale screening is the inability to exhaustively detect fine blood vessels crucial to disease diagnosis. In this work we present a computational imaging framework using deep and ensemble learning for reliable detection of blood vessels in fundus color images. An ensemble of deep convolutional neural networks is trained to segment vessel and non-vessel areas of a color fundus image. During inference, the responses of the individual ConvNets of the ensemble are averaged to form the final segmentation. In experimental evaluation with the DRIVE database, we achieve the objective of vessel detection with maximum average accuracy of 94.7% and area under ROC curve of 0.9283. |
| Tasks | |
| Published | 2016-03-15 |
| URL | http://arxiv.org/abs/1603.04833v1 |
| http://arxiv.org/pdf/1603.04833v1.pdf | |
| PWC | https://paperswithcode.com/paper/ensemble-of-deep-convolutional-neural |
| Repo | https://github.com/qiaotian/VesselSeg |
| Framework | none |
Semi-Supervised Classification with Graph Convolutional Networks
| Title | Semi-Supervised Classification with Graph Convolutional Networks |
| Authors | Thomas N. Kipf, Max Welling |
| Abstract | We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin. |
| Tasks | Document Classification, Graph Classification, Graph Regression, Node Classification, Skeleton Based Action Recognition |
| Published | 2016-09-09 |
| URL | http://arxiv.org/abs/1609.02907v4 |
| http://arxiv.org/pdf/1609.02907v4.pdf | |
| PWC | https://paperswithcode.com/paper/semi-supervised-classification-with-graph |
| Repo | https://github.com/tkipf/ica |
| Framework | none |
