Paper Group ANR 264
Learning Concept Embeddings for Efficient Bag-of-Concepts Densification. End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks. General AI Challenge - Round One: Gradual Learning. Modeling Latent Attention Within Neural Networks. A Robust Multi-Batch L-BFGS Method for Machine Learn …
Learning Concept Embeddings for Efficient Bag-of-Concepts Densification
| Title | Learning Concept Embeddings for Efficient Bag-of-Concepts Densification |
| Authors | Walid Shalaby, Wlodek Zadrozny |
| Abstract | Explicit concept space models have proven efficacy for text representation in many natural language and text mining applications. The idea is to embed textual structures into a semantic space of concepts which captures the main ideas, objects, and the characteristics of these structures. The so called Bag of Concepts (BoC) representation suffers from data sparsity causing low similarity scores between similar texts due to low concept overlap. To address this problem, we propose two neural embedding models to learn continuous concept vectors. Once they are learned, we propose an efficient vector aggregation method to generate fully continuous BoC representations. We evaluate our concept embedding models on three tasks: 1) measuring entity semantic relatedness and ranking where we achieve 1.6% improvement in correlation scores, 2) dataless concept categorization where we achieve state-of-the-art performance and reduce the categorization error rate by more than 5% compared to five prior word and entity embedding models, and 3) dataless document classification where our models outperform the sparse BoC representations. In addition, by exploiting our efficient linear time vector aggregation method, we achieve better accuracy scores with much less concept dimensions compared to previous BoC densification methods which operate in polynomial time and require hundreds of dimensions in the BoC representation. |
| Tasks | Document Classification |
| Published | 2017-02-10 |
| URL | http://arxiv.org/abs/1702.03342v2 |
| http://arxiv.org/pdf/1702.03342v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-concept-embeddings-for-efficient-bag |
| Repo | |
| Framework | |
End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks
| Title | End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks |
| Authors | Umut Güçlü, Yağmur Güçlütürk, Meysam Madadi, Sergio Escalera, Xavier Baró, Jordi González, Rob van Lier, Marcel A. J. van Gerven |
| Abstract | Recent years have seen a sharp increase in the number of related yet distinct advances in semantic segmentation. Here, we tackle this problem by leveraging the respective strengths of these advances. That is, we formulate a conditional random field over a four-connected graph as end-to-end trainable convolutional and recurrent networks, and estimate them via an adversarial process. Importantly, our model learns not only unary potentials but also pairwise potentials, while aggregating multi-scale contexts and controlling higher-order inconsistencies. We evaluate our model on two standard benchmark datasets for semantic face segmentation, achieving state-of-the-art results on both of them. |
| Tasks | Semantic Segmentation |
| Published | 2017-03-09 |
| URL | http://arxiv.org/abs/1703.03305v1 |
| http://arxiv.org/pdf/1703.03305v1.pdf | |
| PWC | https://paperswithcode.com/paper/end-to-end-semantic-face-segmentation-with |
| Repo | |
| Framework | |
General AI Challenge - Round One: Gradual Learning
| Title | General AI Challenge - Round One: Gradual Learning |
| Authors | Jan Feyereisl, Matej Nikl, Martin Poliak, Martin Stransky, Michal Vlasak |
| Abstract | The General AI Challenge is an initiative to encourage the wider artificial intelligence community to focus on important problems in building intelligent machines with more general scope than is currently possible. The challenge comprises of multiple rounds, with the first round focusing on gradual learning, i.e. the ability to re-use already learned knowledge for efficiently learning to solve subsequent problems. In this article, we will present details of the first round of the challenge, its inspiration and aims. We also outline a more formal description of the challenge and present a preliminary analysis of its curriculum, based on ideas from computational mechanics. We believe, that such formalism will allow for a more principled approach towards investigating tasks in the challenge, building new curricula and for potentially improving consequent challenge rounds. |
| Tasks | |
| Published | 2017-08-17 |
| URL | http://arxiv.org/abs/1708.05346v1 |
| http://arxiv.org/pdf/1708.05346v1.pdf | |
| PWC | https://paperswithcode.com/paper/general-ai-challenge-round-one-gradual |
| Repo | |
| Framework | |
Modeling Latent Attention Within Neural Networks
| Title | Modeling Latent Attention Within Neural Networks |
| Authors | Christopher Grimm, Dilip Arumugam, Siddharth Karamcheti, David Abel, Lawson L. S. Wong, Michael L. Littman |
| Abstract | Deep neural networks are able to solve tasks across a variety of domains and modalities of data. Despite many empirical successes, we lack the ability to clearly understand and interpret the learned internal mechanisms that contribute to such effective behaviors or, more critically, failure modes. In this work, we present a general method for visualizing an arbitrary neural network’s inner mechanisms and their power and limitations. Our dataset-centric method produces visualizations of how a trained network attends to components of its inputs. The computed “attention masks” support improved interpretability by highlighting which input attributes are critical in determining output. We demonstrate the effectiveness of our framework on a variety of deep neural network architectures in domains from computer vision, natural language processing, and reinforcement learning. The primary contribution of our approach is an interpretable visualization of attention that provides unique insights into the network’s underlying decision-making process irrespective of the data modality. |
| Tasks | Decision Making |
| Published | 2017-06-02 |
| URL | http://arxiv.org/abs/1706.00536v2 |
| http://arxiv.org/pdf/1706.00536v2.pdf | |
| PWC | https://paperswithcode.com/paper/modeling-latent-attention-within-neural |
| Repo | |
| Framework | |
A Robust Multi-Batch L-BFGS Method for Machine Learning
| Title | A Robust Multi-Batch L-BFGS Method for Machine Learning |
| Authors | Albert S. Berahas, Martin Takáč |
| Abstract | This paper describes an implementation of the L-BFGS method designed to deal with two adversarial situations. The first occurs in distributed computing environments where some of the computational nodes devoted to the evaluation of the function and gradient are unable to return results on time. A similar challenge occurs in a multi-batch approach in which the data points used to compute function and gradients are purposely changed at each iteration to accelerate the learning process. Difficulties arise because L-BFGS employs gradient differences to update the Hessian approximations, and when these gradients are computed using different data points the updating process can be unstable. This paper shows how to perform stable quasi-Newton updating in the multi-batch setting, studies the convergence properties for both convex and nonconvex functions, and illustrates the behavior of the algorithm in a distributed computing platform on binary classification logistic regression and neural network training problems that arise in machine learning. |
| Tasks | |
| Published | 2017-07-26 |
| URL | https://arxiv.org/abs/1707.08552v3 |
| https://arxiv.org/pdf/1707.08552v3.pdf | |
| PWC | https://paperswithcode.com/paper/a-robust-multi-batch-l-bfgs-method-for |
| Repo | |
| Framework | |
Technical Problems With “Programmable self-assembly in a thousand-robot swarm”
| Title | Technical Problems With “Programmable self-assembly in a thousand-robot swarm” |
| Authors | Muaz A. Niazi |
| Abstract | Rubenstein et al. present an interesting system of programmable self-assembled structure formation using 1000 Kilobot robots. The paper claims to advance work in artificial swarms similar to capabilities of natural systems besides being highly robust. However, the system lacks in terms of matching motility and complex shapes with holes, thereby limiting practical similarity to self-assembly in living systems. |
| Tasks | |
| Published | 2017-08-10 |
| URL | http://arxiv.org/abs/1708.03341v1 |
| http://arxiv.org/pdf/1708.03341v1.pdf | |
| PWC | https://paperswithcode.com/paper/technical-problems-with-programmable-self |
| Repo | |
| Framework | |
Adaptive Graph Signal Processing: Algorithms and Optimal Sampling Strategies
| Title | Adaptive Graph Signal Processing: Algorithms and Optimal Sampling Strategies |
| Authors | Paolo Di Lorenzo, Paolo Banelli, Elvin Isufi, Sergio Barbarossa, Geert Leus |
| Abstract | The goal of this paper is to propose novel strategies for adaptive learning of signals defined over graphs, which are observed over a (randomly time-varying) subset of vertices. We recast two classical adaptive algorithms in the graph signal processing framework, namely, the least mean squares (LMS) and the recursive least squares (RLS) adaptive estimation strategies. For both methods, a detailed mean-square analysis illustrates the effect of random sampling on the adaptive reconstruction capability and the steady-state performance. Then, several probabilistic sampling strategies are proposed to design the sampling probability at each node in the graph, with the aim of optimizing the tradeoff between steady-state performance, graph sampling rate, and convergence rate of the adaptive algorithms. Finally, a distributed RLS strategy is derived and is shown to be convergent to its centralized counterpart. Numerical simulations carried out over both synthetic and real data illustrate the good performance of the proposed sampling and reconstruction strategies for (possibly distributed) adaptive learning of signals defined over graphs. |
| Tasks | |
| Published | 2017-09-12 |
| URL | http://arxiv.org/abs/1709.03726v1 |
| http://arxiv.org/pdf/1709.03726v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-graph-signal-processing-algorithms |
| Repo | |
| Framework | |
Anytime Exact Belief Propagation
| Title | Anytime Exact Belief Propagation |
| Authors | Gabriel Azevedo Ferreira, Quentin Bertrand, Charles Maussion, Rodrigo de Salvo Braz |
| Abstract | Statistical Relational Models and, more recently, Probabilistic Programming, have been making strides towards an integration of logic and probabilistic reasoning. A natural expectation for this project is that a probabilistic logic reasoning algorithm reduces to a logic reasoning algorithm when provided a model that only involves 0-1 probabilities, exhibiting all the advantages of logic reasoning such as short-circuiting, intelligibility, and the ability to provide proof trees for a query answer. In fact, we can take this further and require that these characteristics be present even for probabilistic models with probabilities \emph{near} 0 and 1, with graceful degradation as the model becomes more uncertain. We also seek inference that has amortized constant time complexity on a model’s size (even if still exponential in the induced width of a more directly relevant portion of it) so that it can be applied to huge knowledge bases of which only a relatively small portion is relevant to typical queries. We believe that, among the probabilistic reasoning algorithms, Belief Propagation is the most similar to logic reasoning: messages are propagated among neighboring variables, and the paths of message-passing are similar to proof trees. However, Belief Propagation is either only applicable to tree models, or approximate (and without guarantees) for precision and convergence. In this paper we present work in progress on an Anytime Exact Belief Propagation algorithm that is very similar to Belief Propagation but is exact even for graphical models with cycles, while exhibiting soft short-circuiting, amortized constant time complexity in the model size, and which can provide probabilistic proof trees. |
| Tasks | Probabilistic Programming |
| Published | 2017-07-27 |
| URL | http://arxiv.org/abs/1707.08704v1 |
| http://arxiv.org/pdf/1707.08704v1.pdf | |
| PWC | https://paperswithcode.com/paper/anytime-exact-belief-propagation |
| Repo | |
| Framework | |
Contextual Multi-armed Bandits under Feature Uncertainty
| Title | Contextual Multi-armed Bandits under Feature Uncertainty |
| Authors | Se-Young Yun, Jun Hyun Nam, Sangwoo Mo, Jinwoo Shin |
| Abstract | We study contextual multi-armed bandit problems under linear realizability on rewards and uncertainty (or noise) on features. For the case of identical noise on features across actions, we propose an algorithm, coined {\em NLinRel}, having $O\left(T^{\frac{7}{8}} \left(\log{(dT)}+K\sqrt{d}\right)\right)$ regret bound for $T$ rounds, $K$ actions, and $d$-dimensional feature vectors. Next, for the case of non-identical noise, we observe that popular linear hypotheses including {\em NLinRel} are impossible to achieve such sub-linear regret. Instead, under assumption of Gaussian feature vectors, we prove that a greedy algorithm has $O\left(T^{\frac23}\sqrt{\log d}\right)$ regret bound with respect to the optimal linear hypothesis. Utilizing our theoretical understanding on the Gaussian case, we also design a practical variant of {\em NLinRel}, coined {\em Universal-NLinRel}, for arbitrary feature distributions. It first runs {\em NLinRel} for finding the `true’ coefficient vector using feature uncertainties and then adjust it to minimize its regret using the statistical feature information. We justify the performance of {\em Universal-NLinRel} on both synthetic and real-world datasets. | |
| Tasks | Multi-Armed Bandits |
| Published | 2017-03-03 |
| URL | http://arxiv.org/abs/1703.01347v1 |
| http://arxiv.org/pdf/1703.01347v1.pdf | |
| PWC | https://paperswithcode.com/paper/contextual-multi-armed-bandits-under-feature |
| Repo | |
| Framework | |
Feature Generation for Robust Semantic Role Labeling
| Title | Feature Generation for Robust Semantic Role Labeling |
| Authors | Travis Wolfe, Mark Dredze, Benjamin Van Durme |
| Abstract | Hand-engineered feature sets are a well understood method for creating robust NLP models, but they require a lot of expertise and effort to create. In this work we describe how to automatically generate rich feature sets from simple units called featlets, requiring less engineering. Using information gain to guide the generation process, we train models which rival the state of the art on two standard Semantic Role Labeling datasets with almost no task or linguistic insight. |
| Tasks | Semantic Role Labeling |
| Published | 2017-02-22 |
| URL | http://arxiv.org/abs/1702.07046v1 |
| http://arxiv.org/pdf/1702.07046v1.pdf | |
| PWC | https://paperswithcode.com/paper/feature-generation-for-robust-semantic-role |
| Repo | |
| Framework | |
Deep Domain Adaptation Based Video Smoke Detection using Synthetic Smoke Images
| Title | Deep Domain Adaptation Based Video Smoke Detection using Synthetic Smoke Images |
| Authors | Gao Xu, Yongming Zhang, Qixing Zhang, Gaohua Lin, Jinjun Wang |
| Abstract | In this paper, a deep domain adaptation based method for video smoke detection is proposed to extract a powerful feature representation of smoke. Due to the smoke image samples limited in scale and diversity for deep CNN training, we systematically produced adequate synthetic smoke images with a wide variation in the smoke shape, background and lighting conditions. Considering that the appearance gap (dataset bias) between synthetic and real smoke images degrades significantly the performance of the trained model on the test set composed fully of real images, we build deep architectures based on domain adaptation to confuse the distributions of features extracted from synthetic and real smoke images. This approach expands the domain-invariant feature space for smoke image samples. With their approximate feature distribution off non-smoke images, the recognition rate of the trained model is improved significantly compared to the model trained directly on mixed dataset of synthetic and real images. Experimentally, several deep architectures with different design choices are applied to the smoke detector. The ultimate framework can get a satisfactory result on the test set. We believe that our approach is a start in the direction of utilizing deep neural networks enhanced with synthetic smoke images for video smoke detection. |
| Tasks | Domain Adaptation |
| Published | 2017-03-31 |
| URL | http://arxiv.org/abs/1703.10729v1 |
| http://arxiv.org/pdf/1703.10729v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-domain-adaptation-based-video-smoke |
| Repo | |
| Framework | |
SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes
| Title | SceneCut: Joint Geometric and Object Segmentation for Indoor Scenes |
| Authors | Trung Pham, Thanh-Toan Do, Niko Sünderhauf, Ian Reid |
| Abstract | This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut’s joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which either represent objects or scene surfaces. The decomposition is qualified by an unified energy function over objectness and geometric fitting. We show how this energy function can be optimized efficiently by utilizing hierarchical segmentation trees. Moreover, we leverage a pre-trained convolutional oriented boundary network to predict accurate boundaries from images, which are used to construct high-quality region hierarchies. We evaluate SceneCut on several different indoor environments, and the results show that SceneCut significantly outperforms all the existing methods. |
| Tasks | Semantic Segmentation |
| Published | 2017-09-21 |
| URL | http://arxiv.org/abs/1709.07158v2 |
| http://arxiv.org/pdf/1709.07158v2.pdf | |
| PWC | https://paperswithcode.com/paper/scenecut-joint-geometric-and-object |
| Repo | |
| Framework | |
Comparison-Based Choices
| Title | Comparison-Based Choices |
| Authors | Jon Kleinberg, Sendhil Mullainathan, Johan Ugander |
| Abstract | A broad range of on-line behaviors are mediated by interfaces in which people make choices among sets of options. A rich and growing line of work in the behavioral sciences indicate that human choices follow not only from the utility of alternatives, but also from the choice set in which alternatives are presented. In this work we study comparison-based choice functions, a simple but surprisingly rich class of functions capable of exhibiting so-called choice-set effects. Motivated by the challenge of predicting complex choices, we study the query complexity of these functions in a variety of settings. We consider settings that allow for active queries or passive observation of a stream of queries, and give analyses both at the granularity of individuals or populations that might exhibit heterogeneous choice behavior. Our main result is that any comparison-based choice function in one dimension can be inferred as efficiently as a basic maximum or minimum choice function across many query contexts, suggesting that choice-set effects need not entail any fundamental algorithmic barriers to inference. We also introduce a class of choice functions we call distance-comparison-based functions, and briefly discuss the analysis of such functions. The framework we outline provides intriguing connections between human choice behavior and a range of questions in the theory of sorting. |
| Tasks | |
| Published | 2017-05-16 |
| URL | http://arxiv.org/abs/1705.05735v1 |
| http://arxiv.org/pdf/1705.05735v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparison-based-choices |
| Repo | |
| Framework | |
Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition
| Title | Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition |
| Authors | Yao Wang, Jiangjun Peng, Qian Zhao, Deyu Meng, Yee Leung, Xi-Le Zhao |
| Abstract | Hyperspectral images (HSIs) are often corrupted by a mixture of several types of noise during the acquisition process, e.g., Gaussian noise, impulse noise, dead lines, stripes, and many others. Such complex noise could degrade the quality of the acquired HSIs, limiting the precision of the subsequent processing. In this paper, we present a novel tensor-based HSI restoration approach by fully identifying the intrinsic structures of the clean HSI part and the mixed noise part respectively. Specifically, for the clean HSI part, we use tensor Tucker decomposition to describe the global correlation among all bands, and an anisotropic spatial-spectral total variation (SSTV) regularization to characterize the piecewise smooth structure in both spatial and spectral domains. For the mixed noise part, we adopt the $\ell_1$ norm regularization to detect the sparse noise, including stripes, impulse noise, and dead pixels. Despite that TV regulariztion has the ability of removing Gaussian noise, the Frobenius norm term is further used to model heavy Gaussian noise for some real-world scenarios. Then, we develop an efficient algorithm for solving the resulting optimization problem by using the augmented Lagrange multiplier (ALM) method. Finally, extensive experiments on simulated and real-world noise HSIs are carried out to demonstrate the superiority of the proposed method over the existing state-of-the-art ones. |
| Tasks | Image Restoration |
| Published | 2017-07-08 |
| URL | http://arxiv.org/abs/1707.02477v1 |
| http://arxiv.org/pdf/1707.02477v1.pdf | |
| PWC | https://paperswithcode.com/paper/hyperspectral-image-restoration-via-total |
| Repo | |
| Framework | |
Transferring Agent Behaviors from Videos via Motion GANs
| Title | Transferring Agent Behaviors from Videos via Motion GANs |
| Authors | Ashley D. Edwards, Charles L. Isbell Jr |
| Abstract | A major bottleneck for developing general reinforcement learning agents is determining rewards that will yield desirable behaviors under various circumstances. We introduce a general mechanism for automatically specifying meaningful behaviors from raw pixels. In particular, we train a generative adversarial network to produce short sub-goals represented through motion templates. We demonstrate that this approach generates visually meaningful behaviors in unknown environments with novel agents and describe how these motions can be used to train reinforcement learning agents. |
| Tasks | |
| Published | 2017-11-21 |
| URL | http://arxiv.org/abs/1711.07676v1 |
| http://arxiv.org/pdf/1711.07676v1.pdf | |
| PWC | https://paperswithcode.com/paper/transferring-agent-behaviors-from-videos-via |
| Repo | |
| Framework | |
