May 7, 2019

3271 words 16 mins read

Paper Group ANR 130

Solving Combinatorial Games using Products, Projections and Lexicographically Optimal Bases. Very Simple Classifier: a Concept Binary Classifier toInvestigate Features Based on Subsampling and Localility. Scale Normalization. Stride Length Estimation with Deep Learning. B-spline Shape from Motion & Shading: An Automatic Free-form Surface Modeling f …

Solving Combinatorial Games using Products, Projections and Lexicographically Optimal Bases


Title	Solving Combinatorial Games using Products, Projections and Lexicographically Optimal Bases
Authors	Swati Gupta, Michel Goemans, Patrick Jaillet
Abstract	In order to find Nash-equilibria for two-player zero-sum games where each player plays combinatorial objects like spanning trees, matchings etc, we consider two online learning algorithms: the online mirror descent (OMD) algorithm and the multiplicative weights update (MWU) algorithm. The OMD algorithm requires the computation of a certain Bregman projection, that has closed form solutions for simple convex sets like the Euclidean ball or the simplex. However, for general polyhedra one often needs to exploit the general machinery of convex optimization. We give a novel primal-style algorithm for computing Bregman projections on the base polytopes of polymatroids. Next, in the case of the MWU algorithm, although it scales logarithmically in the number of pure strategies or experts $N$ in terms of regret, the algorithm takes time polynomial in $N$; this especially becomes a problem when learning combinatorial objects. We give a general recipe to simulate the multiplicative weights update algorithm in time polynomial in their natural dimension. This is useful whenever there exists a polynomial time generalized counting oracle (even if approximate) over these objects. Finally, using the combinatorial structure of symmetric Nash-equilibria (SNE) when both players play bases of matroids, we show that these can be found with a single projection or convex minimization (without using online learning).
Tasks
Published	2016-03-01
URL	http://arxiv.org/abs/1603.00522v1
PDF	http://arxiv.org/pdf/1603.00522v1.pdf
PWC	https://paperswithcode.com/paper/solving-combinatorial-games-using-products
Repo
Framework

Very Simple Classifier: a Concept Binary Classifier toInvestigate Features Based on Subsampling and Localility


Title	Very Simple Classifier: a Concept Binary Classifier toInvestigate Features Based on Subsampling and Localility
Authors	Luca Masera, Enrico Blanzieri
Abstract	We propose Very Simple Classifier (VSC) a novel method designed to incorporate the concepts of subsampling and locality in the definition of features to be used as the input of a perceptron. The rationale is that locality theoretically guarantees a bound on the generalization error. Each feature in VSC is a max-margin classifier built on randomly-selected pairs of samples. The locality in VSC is achieved by multiplying the value of the feature by a confidence measure that can be characterized in terms of the Chebichev inequality. The output of the layer is then fed in a output layer of neurons. The weights of the output layer are then determined by a regularized pseudoinverse. Extensive comparison of VSC against 9 competitors in the task of binary classification is carried out. Results on 22 benchmark datasets with fixed parameters show that VSC is competitive with the Multi Layer Perceptron (MLP) and outperforms the other competitors. An exploration of the parameter space shows VSC can outperform MLP.
Tasks
Published	2016-09-14
URL	http://arxiv.org/abs/1609.04321v1
PDF	http://arxiv.org/pdf/1609.04321v1.pdf
PWC	https://paperswithcode.com/paper/very-simple-classifier-a-concept-binary
Repo
Framework

Scale Normalization


Title	Scale Normalization
Authors	Henry Z. Lo, Kevin Amaral, Wei Ding
Abstract	One of the difficulties of training deep neural networks is caused by improper scaling between layers. Scaling issues introduce exploding / gradient problems, and have typically been addressed by careful scale-preserving initialization. We investigate the value of preserving scale, or isometry, beyond the initial weights. We propose two methods of maintaing isometry, one exact and one stochastic. Preliminary experiments show that for both determinant and scale-normalization effectively speeds up learning. Results suggest that isometry is important in the beginning of learning, and maintaining it leads to faster learning.
Tasks
Published	2016-04-26
URL	http://arxiv.org/abs/1604.07796v1
PDF	http://arxiv.org/pdf/1604.07796v1.pdf
PWC	https://paperswithcode.com/paper/scale-normalization
Repo
Framework

Stride Length Estimation with Deep Learning


Title	Stride Length Estimation with Deep Learning
Authors	Julius Hannink, Thomas Kautz, Cristian F. Pasluosta, Jens Barth, Samuel Schülein, Karl-Günter Gaßmann, Jochen Klucken, Bjoern M. Eskofier
Abstract	Accurate estimation of spatial gait characteristics is critical to assess motor impairments resulting from neurological or musculoskeletal disease. Currently, however, methodological constraints limit clinical applicability of state-of-the-art double integration approaches to gait patterns with a clear zero-velocity phase. We describe a novel approach to stride length estimation that uses deep convolutional neural networks to map stride-specific inertial sensor data to the resulting stride length. The model is trained on a publicly available and clinically relevant benchmark dataset consisting of 1220 strides from 101 geriatric patients. Evaluation is done in a 10-fold cross validation and for three different stride definitions. Even though best results are achieved with strides defined from mid-stance to mid-stance with average accuracy and precision of 0.01 $\pm$ 5.37 cm, performance does not strongly depend on stride definition. The achieved precision outperforms state-of-the-art methods evaluated on this benchmark dataset by 3.0 cm (36%). Due to the independence of stride definition, the proposed method is not subject to the methodological constrains that limit applicability of state-of-the-art double integration methods. Furthermore, precision on the benchmark dataset could be improved. With more precise mobile stride length estimation, new insights to the progression of neurological disease or early indications might be gained. Due to the independence of stride definition, previously uncharted diseases in terms of mobile gait analysis can now be investigated by re-training and applying the proposed method.
Tasks
Published	2016-09-12
URL	http://arxiv.org/abs/1609.03321v3
PDF	http://arxiv.org/pdf/1609.03321v3.pdf
PWC	https://paperswithcode.com/paper/stride-length-estimation-with-deep-learning
Repo
Framework

B-spline Shape from Motion & Shading: An Automatic Free-form Surface Modeling for Face Reconstruction


Title	B-spline Shape from Motion & Shading: An Automatic Free-form Surface Modeling for Face Reconstruction
Authors	Weilong Peng, Zhiyong Feng, Chao Xu
Abstract	Recently, many methods have been proposed for face reconstruction from multiple images, most of which involve fundamental principles of Shape from Shading and Structure from motion. However, a majority of the methods just generate discrete surface model of face. In this paper, B-spline Shape from Motion and Shading (BsSfMS) is proposed to reconstruct continuous B-spline surface for multi-view face images, according to an assumption that shading and motion information in the images contain 1st- and 0th-order derivative of B-spline face respectively. Face surface is expressed as a B-spline surface that can be reconstructed by optimizing B-spline control points. Therefore, normals and 3D feature points computed from shading and motion of images respectively are used as the 1st- and 0th- order derivative information, to be jointly applied in optimizing the B-spline face. Additionally, an IMLS (iterative multi-least-square) algorithm is proposed to handle the difficult control point optimization. Furthermore, synthetic samples and LFW dataset are introduced and conducted to verify the proposed approach, and the experimental results demonstrate the effectiveness with different poses, illuminations, expressions etc., even with wild images.
Tasks	Face Reconstruction
Published	2016-01-21
URL	http://arxiv.org/abs/1601.05644v1
PDF	http://arxiv.org/pdf/1601.05644v1.pdf
PWC	https://paperswithcode.com/paper/b-spline-shape-from-motion-shading-an
Repo
Framework

Deep FisherNet for Object Classification


Title	Deep FisherNet for Object Classification
Authors	Peng Tang, Xinggang Wang, Baoguang Shi, Xiang Bai, Wenyu Liu, Zhuowen Tu
Abstract	Despite the great success of convolutional neural networks (CNN) for the image classification task on datasets like Cifar and ImageNet, CNN’s representation power is still somewhat limited in dealing with object images that have large variation in size and clutter, where Fisher Vector (FV) has shown to be an effective encoding strategy. FV encodes an image by aggregating local descriptors with a universal generative Gaussian Mixture Model (GMM). FV however has limited learning capability and its parameters are mostly fixed after constructing the codebook. To combine together the best of the two worlds, we propose in this paper a neural network structure with FV layer being part of an end-to-end trainable system that is differentiable; we name our network FisherNet that is learnable using backpropagation. Our proposed FisherNet combines convolutional neural network training and Fisher Vector encoding in a single end-to-end structure. We observe a clear advantage of FisherNet over plain CNN and standard FV in terms of both classification accuracy and computational efficiency on the challenging PASCAL VOC object classification task.
Tasks	Image Classification, Object Classification
Published	2016-07-31
URL	http://arxiv.org/abs/1608.00182v1
PDF	http://arxiv.org/pdf/1608.00182v1.pdf
PWC	https://paperswithcode.com/paper/deep-fishernet-for-object-classification
Repo
Framework

Comparing Human and Automated Evaluation of Open-Ended Student Responses to Questions of Evolution


Title	Comparing Human and Automated Evaluation of Open-Ended Student Responses to Questions of Evolution
Authors	Michael J Wiser, Louise S Mead, James J Smith, Robert T Pennock
Abstract	Written responses can provide a wealth of data in understanding student reasoning on a topic. Yet they are time- and labor-intensive to score, requiring many instructors to forego them except as limited parts of summative assessments at the end of a unit or course. Recent developments in Machine Learning (ML) have produced computational methods of scoring written responses for the presence or absence of specific concepts. Here, we compare the scores from one particular ML program – EvoGrader – to human scoring of responses to structurally- and content-similar questions that are distinct from the ones the program was trained on. We find that there is substantial inter-rater reliability between the human and ML scoring. However, sufficient systematic differences remain between the human and ML scoring that we advise only using the ML scoring for formative, rather than summative, assessment of student reasoning.
Tasks
Published	2016-03-22
URL	http://arxiv.org/abs/1603.07029v1
PDF	http://arxiv.org/pdf/1603.07029v1.pdf
PWC	https://paperswithcode.com/paper/comparing-human-and-automated-evaluation-of
Repo
Framework

Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge


Title	Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge
Authors	Ryan J. Gallagher, Kyle Reing, David Kale, Greg Ver Steeg
Abstract	While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters. Such model complexity issues only compound when trying to generalize generative models to incorporate human input. We introduce Correlation Explanation (CorEx), an alternative approach to topic modeling that does not assume an underlying generative model, and instead learns maximally informative topics through an information-theoretic framework. This framework naturally generalizes to hierarchical and semi-supervised extensions with no additional modeling assumptions. In particular, word-level domain knowledge can be flexibly incorporated within CorEx through anchor words, allowing topic separability and representation to be promoted with minimal human intervention. Across a variety of datasets, metrics, and experiments, we demonstrate that CorEx produces topics that are comparable in quality to those produced by unsupervised and semi-supervised variants of LDA.
Tasks
Published	2016-11-30
URL	http://arxiv.org/abs/1611.10277v4
PDF	http://arxiv.org/pdf/1611.10277v4.pdf
PWC	https://paperswithcode.com/paper/anchored-correlation-explanation-topic
Repo
Framework

Learning What and Where to Draw


Title	Learning What and Where to Draw
Authors	Scott Reed, Zeynep Akata, Santosh Mohan, Samuel Tenka, Bernt Schiele, Honglak Lee
Abstract	Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location. We show high-quality 128 x 128 image synthesis on the Caltech-UCSD Birds dataset, conditioned on both informal text descriptions and also object location. Our system exposes control over both the bounding box around the bird and its constituent parts. By modeling the conditional distributions over part locations, our system also enables conditioning on arbitrary subsets of parts (e.g. only the beak and tail), yielding an efficient interface for picking part locations. We also show preliminary results on the more challenging domain of text- and location-controllable synthesis of images of human actions on the MPII Human Pose dataset.
Tasks	Image Generation, Text-to-Image Generation
Published	2016-10-08
URL	http://arxiv.org/abs/1610.02454v1
PDF	http://arxiv.org/pdf/1610.02454v1.pdf
PWC	https://paperswithcode.com/paper/learning-what-and-where-to-draw
Repo
Framework

Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices


Title	Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices
Authors	Tayfun Gokmen, Yurii Vlasov
Abstract	In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction, etc. Training of large DNNs, however, is universally considered as time consuming and computationally intensive task that demands datacenter-scale computational resources recruited for many days. Here we propose a concept of resistive processing unit (RPU) devices that can potentially accelerate DNN training by orders of magnitude while using much less power. The proposed RPU device can store and update the weight values locally thus minimizing data movement during training and allowing to fully exploit the locality and the parallelism of the training algorithm. We identify the RPU device and system specifications for implementation of an accelerator chip for DNN training in a realistic CMOS-compatible technology. For large DNNs with about 1 billion weights this massively parallel RPU architecture can achieve acceleration factors of 30,000X compared to state-of-the-art microprocessors while providing power efficiency of 84,000 GigaOps/s/W. Problems that currently require days of training on a datacenter-size cluster with thousands of machines can be addressed within hours on a single RPU accelerator. A system consisted of a cluster of RPU accelerators will be able to tackle Big Data problems with trillions of parameters that is impossible to address today like, for example, natural speech recognition and translation between all world languages, real-time analytics on large streams of business and scientific data, integration and analysis of multimodal sensory data flows from massive number of IoT (Internet of Things) sensors.
Tasks	Object Detection, Speech Recognition
Published	2016-03-23
URL	http://arxiv.org/abs/1603.07341v1
PDF	http://arxiv.org/pdf/1603.07341v1.pdf
PWC	https://paperswithcode.com/paper/acceleration-of-deep-neural-network-training
Repo
Framework

A modular architecture for transparent computation in Recurrent Neural Networks


Title	A modular architecture for transparent computation in Recurrent Neural Networks
Authors	Giovanni Sirio Carmantini, Peter beim Graben, Mathieu Desroches, Serafim Rodrigues
Abstract	Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as to transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Goedelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: i) the design of a Central Pattern Generator from a finite-state locomotive controller, and ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments.
Tasks
Published	2016-09-07
URL	http://arxiv.org/abs/1609.01926v1
PDF	http://arxiv.org/pdf/1609.01926v1.pdf
PWC	https://paperswithcode.com/paper/a-modular-architecture-for-transparent
Repo
Framework

Semantic Change Detection with Hypermaps


Title	Semantic Change Detection with Hypermaps
Authors	Teppei Suzuki, Soma Shirakabe, Yudai Miyashita, Akio Nakamura, Yutaka Satoh, Hirokatsu Kataoka
Abstract	Change detection is the study of detecting changes between two different images of a scene taken at different times. By the detected change areas, however, a human cannot understand how different the two images. Therefore, a semantic understanding is required in the change detection research such as disaster investigation. The paper proposes the concept of semantic change detection, which involves intuitively inserting semantic meaning into detected change areas. We mainly focus on the novel semantic segmentation in addition to a conventional change detection approach. In order to solve this problem and obtain a high-level of performance, we propose an improvement to the hypercolumns representation, hereafter known as hypermaps, which effectively uses convolutional maps obtained from convolutional neural networks (CNNs). We also employ multi-scale feature representation captured by different image patches. We applied our method to the TSUNAMI Panoramic Change Detection dataset, and re-annotated the changed areas of the dataset via semantic classes. The results show that our multi-scale hypermaps provided outstanding performance on the re-annotated TSUNAMI dataset.
Tasks	Semantic Segmentation
Published	2016-04-26
URL	http://arxiv.org/abs/1604.07513v2
PDF	http://arxiv.org/pdf/1604.07513v2.pdf
PWC	https://paperswithcode.com/paper/semantic-change-detection-with-hypermaps
Repo
Framework

Going off the Grid: Iterative Model Selection for Biclustered Matrix Completion


Title	Going off the Grid: Iterative Model Selection for Biclustered Matrix Completion
Authors	Eric Chi, Liuiyi Hu, Arvind K. Saibaba, Arvind U. K. Rao
Abstract	We consider the problem of performing matrix completion with side information on row-by-row and column-by-column similarities. We build upon recent proposals for matrix estimation with smoothness constraints with respect to row and column graphs. We present a novel iterative procedure for directly minimizing an information criterion in order to select an appropriate amount row and column smoothing, namely perform model selection. We also discuss how to exploit the special structure of the problem to scale up the estimation and model selection procedure via the Hutchinson estimator. We present simulation results and an application to predicting associations in imaging-genomics studies.
Tasks	Matrix Completion, Model Selection
Published	2016-10-18
URL	http://arxiv.org/abs/1610.05400v2
PDF	http://arxiv.org/pdf/1610.05400v2.pdf
PWC	https://paperswithcode.com/paper/going-off-the-grid-iterative-model-selection
Repo
Framework

Entities as topic labels: Improving topic interpretability and evaluability combining Entity Linking and Labeled LDA


Title	Entities as topic labels: Improving topic interpretability and evaluability combining Entity Linking and Labeled LDA
Authors	Federico Nanni, Pablo Ruiz Fabo
Abstract	In order to create a corpus exploration method providing topics that are easier to interpret than standard LDA topic models, here we propose combining two techniques called Entity linking and Labeled LDA. Our method identifies in an ontology a series of descriptive labels for each document in a corpus. Then it generates a specific topic for each label. Having a direct relation between topics and labels makes interpretation easier; using an ontology as background knowledge limits label ambiguity. As our topics are described with a limited number of clear-cut labels, they promote interpretability, and this may help quantitative evaluation. We illustrate the potential of the approach by applying it in order to define the most relevant topics addressed by each party in the European Parliament’s fifth mandate (1999-2004).
Tasks	Entity Linking, Topic Models
Published	2016-04-26
URL	http://arxiv.org/abs/1604.07809v1
PDF	http://arxiv.org/pdf/1604.07809v1.pdf
PWC	https://paperswithcode.com/paper/entities-as-topic-labels-improving-topic
Repo
Framework

Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content


Title	Hierarchical Neural Language Models for Joint Representation of Streaming Documents and their Content
Authors	Nemanja Djuric, Hao Wu, Vladan Radosavljevic, Mihajlo Grbovic, Narayan Bhamidipati
Abstract	We consider the problem of learning distributed representations for documents in data streams. The documents are represented as low-dimensional vectors and are jointly learned with distributed vector representations of word tokens using a hierarchical framework with two embedded neural language models. In particular, we exploit the context of documents in streams and use one of the language models to model the document sequences, and the other to model word sequences within them. The models learn continuous vector representations for both word tokens and documents such that semantically similar documents and words are close in a common vector space. We discuss extensions to our model, which can be applied to personalized recommendation and social relationship mining by adding further user layers to the hierarchy, thus learning user-specific vectors to represent individual preferences. We validated the learned representations on a public movie rating data set from MovieLens, as well as on a large-scale Yahoo News data comprising three months of user activity logs collected on Yahoo servers. The results indicate that the proposed model can learn useful representations of both documents and word tokens, outperforming the current state-of-the-art by a large margin.
Tasks
Published	2016-06-28
URL	http://arxiv.org/abs/1606.08689v1
PDF	http://arxiv.org/pdf/1606.08689v1.pdf
PWC	https://paperswithcode.com/paper/hierarchical-neural-language-models-for-joint
Repo
Framework