Paper Group ANR 522
Distributed Voting/Ranking with Optimal Number of States per Node. Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction. Boundary Crossing Probabilities for General Exponential Families. Manifold Alignment Determination: finding correspondences across different data views. Learning Deep ResNet Bloc …
Distributed Voting/Ranking with Optimal Number of States per Node
| Title | Distributed Voting/Ranking with Optimal Number of States per Node |
| Authors | Saber Salehkaleybar, Arsalan Sharif-Nassab, S. Jamaloddin Golestani |
| Abstract | Considering a network with $n$ nodes, where each node initially votes for one (or more) choices out of $K$ possible choices, we present a Distributed Multi-choice Voting/Ranking (DMVR) algorithm to determine either the choice with maximum vote (the voting problem) or to rank all the choices in terms of their acquired votes (the ranking problem). The algorithm consolidates node votes across the network by updating the states of interacting nodes using two key operations, the union and the intersection. The proposed algorithm is simple, independent from network size, and easily scalable in terms of the number of choices $K$, using only $K\times 2^{K-1}$ nodal states for voting, and $K\times K!$ nodal states for ranking. We prove the number of states to be optimal in the ranking case, this optimality is conjectured to also apply to the voting case. The time complexity of the algorithm is analyzed in complete graphs. We show that the time complexity for both ranking and voting is $O(\log(n))$ for given vote percentages, and is inversely proportional to the minimum of the vote percentage differences among various choices. |
| Tasks | |
| Published | 2017-03-26 |
| URL | http://arxiv.org/abs/1703.08838v1 |
| http://arxiv.org/pdf/1703.08838v1.pdf | |
| PWC | https://paperswithcode.com/paper/distributed-votingranking-with-optimal-number |
| Repo | |
| Framework | |
Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction
| Title | Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction |
| Authors | Akosua Busia, Navdeep Jaitly |
| Abstract | Recently developed deep learning techniques have significantly improved the accuracy of various speech and image recognition systems. In this paper we show how to adapt some of these techniques to create a novel chained convolutional architecture with next-step conditioning for improving performance on protein sequence prediction problems. We explore its value by demonstrating its ability to improve performance on eight-class secondary structure prediction. We first establish a state-of-the-art baseline by adapting recent advances in convolutional neural networks which were developed for vision tasks. This model achieves 70.0% per amino acid accuracy on the CB513 benchmark dataset without use of standard performance-boosting techniques such as ensembling or multitask learning. We then improve upon this state-of-the-art result using a novel chained prediction approach which frames the secondary structure prediction as a next-step prediction problem. This sequential model achieves 70.3% Q8 accuracy on CB513 with a single model; an ensemble of these models produces 71.4% Q8 accuracy on the same test set, improving upon the previous overall state of the art for the eight-class secondary structure problem. Our models are implemented using TensorFlow, an open-source machine learning software library available at TensorFlow.org; we aim to release the code for these experiments as part of the TensorFlow repository. |
| Tasks | Protein Secondary Structure Prediction |
| Published | 2017-02-13 |
| URL | http://arxiv.org/abs/1702.03865v1 |
| http://arxiv.org/pdf/1702.03865v1.pdf | |
| PWC | https://paperswithcode.com/paper/next-step-conditioned-deep-convolutional |
| Repo | |
| Framework | |
Boundary Crossing Probabilities for General Exponential Families
| Title | Boundary Crossing Probabilities for General Exponential Families |
| Authors | Odalric-Ambrym Maillard |
| Abstract | We consider parametric exponential families of dimension $K$ on the real line. We study a variant of \textit{boundary crossing probabilities} coming from the multi-armed bandit literature, in the case when the real-valued distributions form an exponential family of dimension $K$. Formally, our result is a concentration inequality that bounds the probability that $\mathcal{B}^\psi(\hat \theta_n,\theta^\star)\geq f(t/n)/n$, where $\theta^\star$ is the parameter of an unknown target distribution, $\hat \theta_n$ is the empirical parameter estimate built from $n$ observations, $\psi$ is the log-partition function of the exponential family and $\mathcal{B}^\psi$ is the corresponding Bregman divergence. From the perspective of stochastic multi-armed bandits, we pay special attention to the case when the boundary function $f$ is logarithmic, as it is enables to analyze the regret of the state-of-the-art \KLUCB\ and \KLUCBp\ strategies, whose analysis was left open in such generality. Indeed, previous results only hold for the case when $K=1$, while we provide results for arbitrary finite dimension $K$, thus considerably extending the existing results. Perhaps surprisingly, we highlight that the proof techniques to achieve these strong results already existed three decades ago in the work of T.L. Lai, and were apparently forgotten in the bandit community. We provide a modern rewriting of these beautiful techniques that we believe are useful beyond the application to stochastic multi-armed bandits. |
| Tasks | Multi-Armed Bandits |
| Published | 2017-05-24 |
| URL | http://arxiv.org/abs/1705.08814v1 |
| http://arxiv.org/pdf/1705.08814v1.pdf | |
| PWC | https://paperswithcode.com/paper/boundary-crossing-probabilities-for-general |
| Repo | |
| Framework | |
Manifold Alignment Determination: finding correspondences across different data views
| Title | Manifold Alignment Determination: finding correspondences across different data views |
| Authors | Andreas Damianou, Neil D. Lawrence, Carl Henrik Ek |
| Abstract | We present Manifold Alignment Determination (MAD), an algorithm for learning alignments between data points from multiple views or modalities. The approach is capable of learning correspondences between views as well as correspondences between individual data-points. The proposed method requires only a few aligned examples from which it is capable to recover a global alignment through a probabilistic model. The strong, yet flexible regularization provided by the generative model is sufficient to align the views. We provide experiments on both synthetic and real data to highlight the benefit of the proposed approach. |
| Tasks | |
| Published | 2017-01-12 |
| URL | http://arxiv.org/abs/1701.03449v1 |
| http://arxiv.org/pdf/1701.03449v1.pdf | |
| PWC | https://paperswithcode.com/paper/manifold-alignment-determination-finding |
| Repo | |
| Framework | |
Learning Deep ResNet Blocks Sequentially using Boosting Theory
| Title | Learning Deep ResNet Blocks Sequentially using Boosting Theory |
| Authors | Furong Huang, Jordan Ash, John Langford, Robert Schapire |
| Abstract | Deep neural networks are known to be difficult to train due to the instability of back-propagation. A deep \emph{residual network} (ResNet) with identity loops remedies this by stabilizing gradient computations. We prove a boosting theory for the ResNet architecture. We construct $T$ weak module classifiers, each contains two of the $T$ layers, such that the combined strong learner is a ResNet. Therefore, we introduce an alternative Deep ResNet training algorithm, \emph{BoostResNet}, which is particularly suitable in non-differentiable architectures. Our proposed algorithm merely requires a sequential training of $T$ “shallow ResNets” which are inexpensive. We prove that the training error decays exponentially with the depth $T$ if the \emph{weak module classifiers} that we train perform slightly better than some weak baseline. In other words, we propose a weak learning condition and prove a boosting theory for ResNet under the weak learning condition. Our results apply to general multi-class ResNets. A generalization error bound based on margin theory is proved and suggests ResNet’s resistant to overfitting under network with $l_1$ norm bounded weights. |
| Tasks | |
| Published | 2017-06-15 |
| URL | http://arxiv.org/abs/1706.04964v4 |
| http://arxiv.org/pdf/1706.04964v4.pdf | |
| PWC | https://paperswithcode.com/paper/learning-deep-resnet-blocks-sequentially |
| Repo | |
| Framework | |
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
| Title | DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning |
| Authors | Xiaodong Gu, Hongyu Zhang, Dongmei Zhang, Sunghun Kim |
| Abstract | Computer programs written in one language are often required to be ported to other languages to support multiple devices and environments. When programs use language specific APIs (Application Programming Interfaces), it is very challenging to migrate these APIs to the corresponding APIs written in other languages. Existing approaches mine API mappings from projects that have corresponding versions in two languages. They rely on the sparse availability of bilingual projects, thus producing a limited number of API mappings. In this paper, we propose an intelligent system called DeepAM for automatically mining API mappings from a large-scale code corpus without bilingual projects. The key component of DeepAM is based on the multimodal sequence to sequence learning architecture that aims to learn joint semantic representations of bilingual API sequences from big source code data. Experimental results indicate that DeepAM significantly increases the accuracy of API mappings as well as the number of API mappings, when compared with the state-of-the-art approaches. |
| Tasks | |
| Published | 2017-04-25 |
| URL | http://arxiv.org/abs/1704.07734v1 |
| http://arxiv.org/pdf/1704.07734v1.pdf | |
| PWC | https://paperswithcode.com/paper/deepam-migrate-apis-with-multi-modal-sequence |
| Repo | |
| Framework | |
Combining Contrast Invariant L1 Data Fidelities with Nonlinear Spectral Image Decomposition
| Title | Combining Contrast Invariant L1 Data Fidelities with Nonlinear Spectral Image Decomposition |
| Authors | Leonie Zeune, Stephan A. van Gils, Leon W. M. M. Terstappen, Christoph Brune |
| Abstract | This paper focuses on multi-scale approaches for variational methods and corresponding gradient flows. Recently, for convex regularization functionals such as total variation, new theory and algorithms for nonlinear eigenvalue problems via nonlinear spectral decompositions have been developed. Those methods open new directions for advanced image filtering. However, for an effective use in image segmentation and shape decomposition, a clear interpretation of the spectral response regarding size and intensity scales is needed but lacking in current approaches. In this context, $L^1$ data fidelities are particularly helpful due to their interesting multi-scale properties such as contrast invariance. Hence, the novelty of this work is the combination of $L^1$-based multi-scale methods with nonlinear spectral decompositions. We compare $L^1$ with $L^2$ scale-space methods in view of spectral image representation and decomposition. We show that the contrast invariant multi-scale behavior of $L^1-TV$ promotes sparsity in the spectral response providing more informative decompositions. We provide a numerical method and analyze synthetic and biomedical images at which decomposition leads to improved segmentation. |
| Tasks | Semantic Segmentation |
| Published | 2017-03-16 |
| URL | http://arxiv.org/abs/1703.05560v1 |
| http://arxiv.org/pdf/1703.05560v1.pdf | |
| PWC | https://paperswithcode.com/paper/combining-contrast-invariant-l1-data |
| Repo | |
| Framework | |
Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning
| Title | Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning |
| Authors | Sahil Sharma, Aravind S. Lakshminarayanan, Balaraman Ravindran |
| Abstract | Reinforcement Learning algorithms can learn complex behavioral patterns for sequential decision making tasks wherein an agent interacts with an environment and acquires feedback in the form of rewards sampled from it. Traditionally, such algorithms make decisions, i.e., select actions to execute, at every single time step of the agent-environment interactions. In this paper, we propose a novel framework, Fine Grained Action Repetition (FiGAR), which enables the agent to decide the action as well as the time scale of repeating it. FiGAR can be used for improving any Deep Reinforcement Learning algorithm which maintains an explicit policy estimate by enabling temporal abstractions in the action space. We empirically demonstrate the efficacy of our framework by showing performance improvements on top of three policy search algorithms in different domains: Asynchronous Advantage Actor Critic in the Atari 2600 domain, Trust Region Policy Optimization in Mujoco domain and Deep Deterministic Policy Gradients in the TORCS car racing domain. |
| Tasks | Car Racing, Decision Making |
| Published | 2017-02-20 |
| URL | http://arxiv.org/abs/1702.06054v1 |
| http://arxiv.org/pdf/1702.06054v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-repeat-fine-grained-action |
| Repo | |
| Framework | |
Recurrent Deterministic Policy Gradient Method for Bipedal Locomotion on Rough Terrain Challenge
| Title | Recurrent Deterministic Policy Gradient Method for Bipedal Locomotion on Rough Terrain Challenge |
| Authors | Doo Re Song, Chuanyu Yang, Christopher McGreavy, Zhibin Li |
| Abstract | This paper presents a deep learning framework that is capable of solving partially observable locomotion tasks based on our novel interpretation of Recurrent Deterministic Policy Gradient (RDPG). We study on bias of sampled error measure and its variance induced by the partial observability of environment and subtrajectory sampling, respectively. Three major improvements are introduced in our RDPG based learning framework: tail-step bootstrap of interpolated temporal difference, initialisation of hidden state using past trajectory scanning, and injection of external experiences learned by other agents. The proposed learning framework was implemented to solve the Bipedal-Walker challenge in OpenAI’s gym simulation environment where only partial state information is available. Our simulation study shows that the autonomous behaviors generated by the RDPG agent are highly adaptive to a variety of obstacles and enables the agent to effectively traverse rugged terrains for long distance with higher success rate than leading contenders. |
| Tasks | |
| Published | 2017-10-08 |
| URL | https://arxiv.org/abs/1710.02896v6 |
| https://arxiv.org/pdf/1710.02896v6.pdf | |
| PWC | https://paperswithcode.com/paper/recurrent-deterministic-policy-gradient |
| Repo | |
| Framework | |
Eigenlogic: Interpretable Quantum Observables with applications to Fuzzy Behavior of Vehicular Robots
| Title | Eigenlogic: Interpretable Quantum Observables with applications to Fuzzy Behavior of Vehicular Robots |
| Authors | Zeno Toffano, François Dubois |
| Abstract | This work proposes a formulation of propositional logic, named Eigenlogic, using quantum observables as propositions. The eigenvalues of these operators are the truth-values and the associated eigenvectors the interpretations of the propositional system. Fuzzy logic arises naturally when considering vectors outside the eigensystem, the fuzzy membership function is obtained by the Born rule of the logical observable.This approach is then applied in the context of quantum robots using simple behavioral agents represented by Braitenberg vehicles. Processing with non-classical logic such as multivalued logic, fuzzy logic and the quantum Eigenlogic permits to enlarge the behavior possibilities and the associated decisions of these simple agents. |
| Tasks | |
| Published | 2017-07-17 |
| URL | http://arxiv.org/abs/1707.05654v1 |
| http://arxiv.org/pdf/1707.05654v1.pdf | |
| PWC | https://paperswithcode.com/paper/eigenlogic-interpretable-quantum-observables |
| Repo | |
| Framework | |
Learning to Predict Indoor Illumination from a Single Image
| Title | Learning to Predict Indoor Illumination from a Single Image |
| Authors | Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagné, Jean-François Lalonde |
| Abstract | We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. We show that this can be accomplished in a three step process: 1) we train a robust lighting classifier to automatically annotate the location of light sources in a large dataset of LDR environment maps, 2) we use these annotations to train a deep neural network that predicts the location of lights in a scene from a single limited field-of-view photo, and 3) we fine-tune this network using a small dataset of HDR environment maps to predict light intensities. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve results that are photo-realistic, which is validated via a perceptual user study. |
| Tasks | |
| Published | 2017-04-01 |
| URL | http://arxiv.org/abs/1704.00090v3 |
| http://arxiv.org/pdf/1704.00090v3.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-predict-indoor-illumination-from |
| Repo | |
| Framework | |
C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset
| Title | C-VQA: A Compositional Split of the Visual Question Answering (VQA) v1.0 Dataset |
| Authors | Aishwarya Agrawal, Aniruddha Kembhavi, Dhruv Batra, Devi Parikh |
| Abstract | Visual Question Answering (VQA) has received a lot of attention over the past couple of years. A number of deep learning models have been proposed for this task. However, it has been shown that these models are heavily driven by superficial correlations in the training data and lack compositionality – the ability to answer questions about unseen compositions of seen concepts. This compositionality is desirable and central to intelligence. In this paper, we propose a new setting for Visual Question Answering where the test question-answer pairs are compositionally novel compared to training question-answer pairs. To facilitate developing models under this setting, we present a new compositional split of the VQA v1.0 dataset, which we call Compositional VQA (C-VQA). We analyze the distribution of questions and answers in the C-VQA splits. Finally, we evaluate several existing VQA models under this new setting and show that the performances of these models degrade by a significant amount compared to the original VQA setting. |
| Tasks | Question Answering, Visual Question Answering |
| Published | 2017-04-26 |
| URL | http://arxiv.org/abs/1704.08243v1 |
| http://arxiv.org/pdf/1704.08243v1.pdf | |
| PWC | https://paperswithcode.com/paper/c-vqa-a-compositional-split-of-the-visual |
| Repo | |
| Framework | |
IHT dies hard: Provable accelerated Iterative Hard Thresholding
| Title | IHT dies hard: Provable accelerated Iterative Hard Thresholding |
| Authors | Rajiv Khanna, Anastasios Kyrillidis |
| Abstract | We study –both in theory and practice– the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization criteria with non-convex constraints, under standard assumptions. In diverse scenaria, we observe that acceleration in IHT leads to significant improvements, compared to state of the art projected gradient descent and Frank-Wolfe variants. As a byproduct of our inspection, we study the impact of selecting the momentum parameter: similar to convex settings, two modes of behavior are observed –“rippling” and linear– depending on the level of momentum. |
| Tasks | |
| Published | 2017-12-26 |
| URL | https://arxiv.org/abs/1712.09379v2 |
| https://arxiv.org/pdf/1712.09379v2.pdf | |
| PWC | https://paperswithcode.com/paper/iht-dies-hard-provable-accelerated-iterative |
| Repo | |
| Framework | |
3D Face Reconstruction from Light Field Images: A Model-free Approach
| Title | 3D Face Reconstruction from Light Field Images: A Model-free Approach |
| Authors | Mingtao Feng, Syed Zulqarnain Gilani, Yaonan Wang, Ajmal Mian |
| Abstract | Reconstructing 3D facial geometry from a single RGB image has recently instigated wide research interest. However, it is still an ill-posed problem and most methods rely on prior models hence undermining the accuracy of the recovered 3D faces. In this paper, we exploit the Epipolar Plane Images (EPI) obtained from light field cameras and learn CNN models that recover horizontal and vertical 3D facial curves from the respective horizontal and vertical EPIs. Our 3D face reconstruction network (FaceLFnet) comprises a densely connected architecture to learn accurate 3D facial curves from low resolution EPIs. To train the proposed FaceLFnets from scratch, we synthesize photo-realistic light field images from 3D facial scans. The curve by curve 3D face estimation approach allows the networks to learn from only 14K images of 80 identities, which still comprises over 11 Million EPIs/curves. The estimated facial curves are merged into a single pointcloud to which a surface is fitted to get the final 3D face. Our method is model-free, requires only a few training samples to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single light field images under varying poses, expressions and lighting conditions. Comparison on the BU-3DFE and BU-4DFE datasets show that our method reduces reconstruction errors by over 20% compared to recent state of the art. |
| Tasks | 3D Face Reconstruction, Face Reconstruction |
| Published | 2017-11-16 |
| URL | http://arxiv.org/abs/1711.05953v4 |
| http://arxiv.org/pdf/1711.05953v4.pdf | |
| PWC | https://paperswithcode.com/paper/3d-face-reconstruction-from-light-field |
| Repo | |
| Framework | |
Build Fast and Accurate Lemmatization for Arabic
| Title | Build Fast and Accurate Lemmatization for Arabic |
| Authors | Hamdy Mubarak |
| Abstract | In this paper we describe the complexity of building a lemmatizer for Arabic which has a rich and complex derivational morphology, and we discuss the need for a fast and accurate lammatization to enhance Arabic Information Retrieval (IR) results. We also introduce a new data set that can be used to test lemmatization accuracy, and an efficient lemmatization algorithm that outperforms state-of-the-art Arabic lemmatization in terms of accuracy and speed. We share the data set and the code for public. |
| Tasks | Information Retrieval, Lemmatization |
| Published | 2017-10-18 |
| URL | http://arxiv.org/abs/1710.06700v1 |
| http://arxiv.org/pdf/1710.06700v1.pdf | |
| PWC | https://paperswithcode.com/paper/build-fast-and-accurate-lemmatization-for |
| Repo | |
| Framework | |
