Paper Group ANR 260
Analysis of Dropout in Online Learning. Deep Convolutional Neural Networks for Raman Spectrum Recognition: A Unified Solution. On Compiling DNNFs without Determinism. Improved Support Recovery Guarantees for the Group Lasso With Applications to Structural Health Monitoring. An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comp …
Analysis of Dropout in Online Learning
| Title | Analysis of Dropout in Online Learning |
| Authors | Kazuyuki Hara |
| Abstract | Deep learning is the state-of-the-art in fields such as visual object recognition and speech recognition. This learning uses a large number of layers and a huge number of units and connections. Therefore, overfitting is a serious problem with it, and the dropout which is a kind of regularization tool is used. However, in online learning, the effect of dropout is not well known. This paper presents our investigation on the effect of dropout in online learning. We analyzed the effect of dropout on convergence speed near the singular point. Our results indicated that dropout is effective in online learning. Dropout tends to avoid the singular point for convergence speed near that point. |
| Tasks | Object Recognition, Speech Recognition |
| Published | 2017-11-09 |
| URL | http://arxiv.org/abs/1711.03343v1 |
| http://arxiv.org/pdf/1711.03343v1.pdf | |
| PWC | https://paperswithcode.com/paper/analysis-of-dropout-in-online-learning |
| Repo | |
| Framework | |
Deep Convolutional Neural Networks for Raman Spectrum Recognition: A Unified Solution
| Title | Deep Convolutional Neural Networks for Raman Spectrum Recognition: A Unified Solution |
| Authors | Jinchao Liu, Margarita Osadchy, Lorna Ashton, Michael Foster, Christopher J. Solomon, Stuart J. Gibson |
| Abstract | Machine learning methods have found many applications in Raman spectroscopy, especially for the identification of chemical species. However, almost all of these methods require non-trivial preprocessing such as baseline correction and/or PCA as an essential step. Here we describe our unified solution for the identification of chemical species in which a convolutional neural network is trained to automatically identify substances according to their Raman spectrum without the need of ad-hoc preprocessing steps. We evaluated our approach using the RRUFF spectral database, comprising mineral sample data. Superior classification performance is demonstrated compared with other frequently used machine learning algorithms including the popular support vector machine. |
| Tasks | |
| Published | 2017-08-18 |
| URL | http://arxiv.org/abs/1708.09022v1 |
| http://arxiv.org/pdf/1708.09022v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-convolutional-neural-networks-for-raman |
| Repo | |
| Framework | |
On Compiling DNNFs without Determinism
| Title | On Compiling DNNFs without Determinism |
| Authors | Umut Oztok, Adnan Darwiche |
| Abstract | State-of-the-art knowledge compilers generate deterministic subsets of DNNF, which have been recently shown to be exponentially less succinct than DNNF. In this paper, we propose a new method to compile DNNFs without enforcing determinism necessarily. Our approach is based on compiling deterministic DNNFs with the addition of auxiliary variables to the input formula. These variables are then existentially quantified from the deterministic structure in linear time, which would lead to a DNNF that is equivalent to the input formula and not necessarily deterministic. On the theoretical side, we show that the new method could generate exponentially smaller DNNFs than deterministic ones, even by adding a single auxiliary variable. Further, we show that various existing techniques that introduce auxiliary variables to the input formulas can be employed in our framework. On the practical side, we empirically demonstrate that our new method can significantly advance DNNF compilation on certain benchmarks. |
| Tasks | |
| Published | 2017-09-20 |
| URL | http://arxiv.org/abs/1709.07092v1 |
| http://arxiv.org/pdf/1709.07092v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-compiling-dnnfs-without-determinism |
| Repo | |
| Framework | |
Improved Support Recovery Guarantees for the Group Lasso With Applications to Structural Health Monitoring
| Title | Improved Support Recovery Guarantees for the Group Lasso With Applications to Structural Health Monitoring |
| Authors | Mojtaba Kadkhodaie Elyaderani, Swayambhoo Jain, Jeffrey Druce, Stefano Gonella, Jarvis Haupt |
| Abstract | This paper considers the problem of estimating an unknown high dimensional signal from noisy linear measurements, {when} the signal is assumed to possess a \emph{group-sparse} structure in a {known,} fixed dictionary. We consider signals generated according to a natural probabilistic model, and establish new conditions under which the set of indices of the non-zero groups of the signal (called the group-level support) may be accurately estimated via the group Lasso. Our results strengthen existing coherence-based analyses that exhibit the well-known “square root” bottleneck, allowing for the number of recoverable nonzero groups to be nearly as large as the total number of groups. We also establish a sufficient recovery condition relating the number of nonzero groups and the signal to noise ratio (quantified in terms of the ratio of the squared Euclidean norms of nonzero groups and the variance of the random additive {measurement} noise), and validate this trend empirically. Finally, we examine the implications of our results in the context of a structural health monitoring application, where the group Lasso approach facilitates demixing of a propagating acoustic wavefield, acquired on the material surface by a scanning laser Doppler vibrometer, into antithetical components, one of which indicates the locations of internal material defects. |
| Tasks | |
| Published | 2017-08-29 |
| URL | http://arxiv.org/abs/1708.08826v2 |
| http://arxiv.org/pdf/1708.08826v2.pdf | |
| PWC | https://paperswithcode.com/paper/improved-support-recovery-guarantees-for-the |
| Repo | |
| Framework | |
An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks
| Title | An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks |
| Authors | Yelong Shen, Xiaodong Liu, Kevin Duh, Jianfeng Gao |
| Abstract | Reading comprehension (RC) is a challenging task that requires synthesis of information across sentences and multiple turns of reasoning. Using a state-of-the-art RC model, we empirically investigate the performance of single-turn and multiple-turn reasoning on the SQuAD and MS MARCO datasets. The RC model is an end-to-end neural network with iterative attention, and uses reinforcement learning to dynamically control the number of turns. We find that multiple-turn reasoning outperforms single-turn reasoning for all question and answer types; further, we observe that enabling a flexible number of turns generally improves upon a fixed multiple-turn strategy. %across all question types, and is particularly beneficial to questions with lengthy, descriptive answers. We achieve results competitive to the state-of-the-art on these two datasets. |
| Tasks | Reading Comprehension |
| Published | 2017-11-09 |
| URL | http://arxiv.org/abs/1711.03230v1 |
| http://arxiv.org/pdf/1711.03230v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-empirical-analysis-of-multiple-turn |
| Repo | |
| Framework | |
Dense Face Alignment
| Title | Dense Face Alignment |
| Authors | Yaojie Liu, Amin Jourabloo, William Ren, Xiaoming Liu |
| Abstract | Face alignment is a classic problem in the computer vision field. Previous works mostly focus on sparse alignment with a limited number of facial landmark points, i.e., facial landmark detection. In this paper, for the first time, we aim at providing a very dense 3D alignment for large-pose face images. To achieve this, we train a CNN to estimate the 3D face shape, which not only aligns limited facial landmarks but also fits face contours and SIFT feature points. Moreover, we also address the bottleneck of training CNN with multiple datasets, due to different landmark markups on different datasets, such as 5, 34, 68. Experimental results show our method not only provides high-quality, dense 3D face fitting but also outperforms the state-of-the-art facial landmark detection methods on the challenging datasets. Our model can run at real time during testing. |
| Tasks | 3D Face Reconstruction, Face Alignment, Facial Landmark Detection |
| Published | 2017-09-05 |
| URL | http://arxiv.org/abs/1709.01442v1 |
| http://arxiv.org/pdf/1709.01442v1.pdf | |
| PWC | https://paperswithcode.com/paper/dense-face-alignment |
| Repo | |
| Framework | |
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
| Title | 3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds |
| Authors | Fangyu Liu, Shuaipeng Li, Liqiang Zhang, Chenghu Zhou, Rongtian Ye, Yuebin Wang, Jiwen Lu |
| Abstract | Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately, which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN) for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmentation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods. |
| Tasks | Image Classification, Object Localization, Semantic Parsing |
| Published | 2017-07-21 |
| URL | http://arxiv.org/abs/1707.06783v1 |
| http://arxiv.org/pdf/1707.06783v1.pdf | |
| PWC | https://paperswithcode.com/paper/3dcnn-dqn-rnn-a-deep-reinforcement-learning |
| Repo | |
| Framework | |
Two-Phase Learning for Weakly Supervised Object Localization
| Title | Two-Phase Learning for Weakly Supervised Object Localization |
| Authors | Dahun Kim, Donghyeon Cho, Donggeun Yoo, In So Kweon |
| Abstract | Weakly supervised semantic segmentation and localiza- tion have a problem of focusing only on the most important parts of an image since they use only image-level annota- tions. In this paper, we solve this problem fundamentally via two-phase learning. Our networks are trained in two steps. In the first step, a conventional fully convolutional network (FCN) is trained to find the most discriminative parts of an image. In the second step, the activations on the most salient parts are suppressed by inference conditional feedback, and then the second learning is performed to find the area of the next most important parts. By combining the activations of both phases, the entire portion of the tar- get object can be captured. Our proposed training scheme is novel and can be utilized in well-designed techniques for weakly supervised semantic segmentation, salient region detection, and object location prediction. Detailed experi- ments demonstrate the effectiveness of our two-phase learn- ing in each task. |
| Tasks | Object Localization, Semantic Segmentation, Weakly-Supervised Object Localization, Weakly-Supervised Semantic Segmentation |
| Published | 2017-08-07 |
| URL | http://arxiv.org/abs/1708.02108v3 |
| http://arxiv.org/pdf/1708.02108v3.pdf | |
| PWC | https://paperswithcode.com/paper/two-phase-learning-for-weakly-supervised |
| Repo | |
| Framework | |
TexT - Text Extractor Tool for Handwritten Document Transcription and Annotation
| Title | TexT - Text Extractor Tool for Handwritten Document Transcription and Annotation |
| Authors | Anders Hast, Per Cullhed, Ekta Vats |
| Abstract | This paper presents a framework for semi-automatic transcription of large-scale historical handwritten documents and proposes a simple user-friendly text extractor tool, TexT for transcription. The proposed approach provides a quick and easy transcription of text using computer assisted interactive technique. The algorithm finds multiple occurrences of the marked text on-the-fly using a word spotting system. TexT is also capable of performing on-the-fly annotation of handwritten text with automatic generation of ground truth labels, and dynamic adjustment and correction of user generated bounding box annotations with the word being perfectly encapsulated. The user can view the document and the found words in the original form or with background noise removed for easier visualization of transcription results. The effectiveness of TexT is demonstrated on an archival manuscript collection from well-known publicly available dataset. |
| Tasks | |
| Published | 2017-11-22 |
| URL | http://arxiv.org/abs/1801.05367v1 |
| http://arxiv.org/pdf/1801.05367v1.pdf | |
| PWC | https://paperswithcode.com/paper/text-text-extractor-tool-for-handwritten |
| Repo | |
| Framework | |
Copy the dynamics using a learning machine
| Title | Copy the dynamics using a learning machine |
| Authors | Hong Zhao |
| Abstract | Is it possible to generally construct a dynamical system to simulate a black system without recovering the equations of motion of the latter? Here we show that this goal can be approached by a learning machine. Trained by a set of input-output responses or a segment of time series of a black system, a learning machine can be served as a copy system to mimic the dynamics of various black systems. It can not only behave as the black system at the parameter set that the training data are made, but also recur the evolution history of the black system. As a result, the learning machine provides an effective way for prediction, and enables one to probe the global dynamics of a black system. These findings have significance for practical systems whose equations of motion cannot be approached accurately. Examples of copying the dynamics of an artificial neural network, the Lorenz system, and a variable star are given. Our idea paves a possible way towards copy a living brain. |
| Tasks | Time Series |
| Published | 2017-07-24 |
| URL | http://arxiv.org/abs/1707.07637v1 |
| http://arxiv.org/pdf/1707.07637v1.pdf | |
| PWC | https://paperswithcode.com/paper/copy-the-dynamics-using-a-learning-machine |
| Repo | |
| Framework | |
Online Handwritten Mathematical Expressions Recognition System Using Fuzzy Neural Network
| Title | Online Handwritten Mathematical Expressions Recognition System Using Fuzzy Neural Network |
| Authors | E. Naderan |
| Abstract | The article describes developed information technology for online recognition of handwritten mathematical expressions that based on proposed approaches to handwritten symbols recognition and structural analysis. |
| Tasks | |
| Published | 2017-07-11 |
| URL | http://arxiv.org/abs/1707.03088v2 |
| http://arxiv.org/pdf/1707.03088v2.pdf | |
| PWC | https://paperswithcode.com/paper/online-handwritten-mathematical-expressions |
| Repo | |
| Framework | |
Divide-and-Conquer Reinforcement Learning
| Title | Divide-and-Conquer Reinforcement Learning |
| Authors | Dibya Ghosh, Avi Singh, Aravind Rajeswaran, Vikash Kumar, Sergey Levine |
| Abstract | Standard model-free deep reinforcement learning (RL) algorithms sample a new initial state for each trial, allowing them to optimize policies that can perform well even in highly stochastic environments. However, problems that exhibit considerable initial state variation typically produce high-variance gradient estimates for model-free RL, making direct policy or value function optimization challenging. In this paper, we develop a novel algorithm that instead partitions the initial state space into “slices”, and optimizes an ensemble of policies, each on a different slice. The ensemble is gradually unified into a single policy that can succeed on the whole state space. This approach, which we term divide-and-conquer RL, is able to solve complex tasks where conventional deep RL methods are ineffective. Our results show that divide-and-conquer RL greatly outperforms conventional policy gradient methods on challenging grasping, manipulation, and locomotion tasks, and exceeds the performance of a variety of prior methods. Videos of policies learned by our algorithm can be viewed at http://bit.ly/dnc-rl |
| Tasks | Policy Gradient Methods |
| Published | 2017-11-27 |
| URL | http://arxiv.org/abs/1711.09874v2 |
| http://arxiv.org/pdf/1711.09874v2.pdf | |
| PWC | https://paperswithcode.com/paper/divide-and-conquer-reinforcement-learning |
| Repo | |
| Framework | |
The Exact Solution to Rank-1 L1-norm TUCKER2 Decomposition
| Title | The Exact Solution to Rank-1 L1-norm TUCKER2 Decomposition |
| Authors | Panos P. Markopoulos, Dimitris G. Chachlakis, Evangelos E. Papalexakis |
| Abstract | We study rank-1 {L1-norm-based TUCKER2} (L1-TUCKER2) decomposition of 3-way tensors, treated as a collection of $N$ $D \times M$ matrices that are to be jointly decomposed. Our contributions are as follows. i) We prove that the problem is equivalent to combinatorial optimization over $N$ antipodal-binary variables. ii) We derive the first two algorithms in the literature for its exact solution. The first algorithm has cost exponential in $N$; the second one has cost polynomial in $N$ (under a mild assumption). Our algorithms are accompanied by formal complexity analysis. iii) We conduct numerical studies to compare the performance of exact L1-TUCKER2 (proposed) with standard HOSVD, HOOI, GLRAM, PCA, L1-PCA, and TPCA-L1. Our studies show that L1-TUCKER2 outperforms (in tensor approximation) all the above counterparts when the processed data are outlier corrupted. |
| Tasks | Combinatorial Optimization |
| Published | 2017-10-31 |
| URL | http://arxiv.org/abs/1710.11306v1 |
| http://arxiv.org/pdf/1710.11306v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-exact-solution-to-rank-1-l1-norm-tucker2 |
| Repo | |
| Framework | |
Occam’s razor is insufficient to infer the preferences of irrational agents
| Title | Occam’s razor is insufficient to infer the preferences of irrational agents |
| Authors | Stuart Armstrong, Sören Mindermann |
| Abstract | Inverse reinforcement learning (IRL) attempts to infer human rewards or preferences from observed behavior. Since human planning systematically deviates from rationality, several approaches have been tried to account for specific human shortcomings. However, the general problem of inferring the reward function of an agent of unknown rationality has received little attention. Unlike the well-known ambiguity problems in IRL, this one is practically relevant but cannot be resolved by observing the agent’s policy in enough environments. This paper shows (1) that a No Free Lunch result implies it is impossible to uniquely decompose a policy into a planning algorithm and reward function, and (2) that even with a reasonable simplicity prior/Occam’s razor on the set of decompositions, we cannot distinguish between the true decomposition and others that lead to high regret. To address this, we need simple `normative’ assumptions, which cannot be deduced exclusively from observations. | |
| Tasks | |
| Published | 2017-12-15 |
| URL | http://arxiv.org/abs/1712.05812v6 |
| http://arxiv.org/pdf/1712.05812v6.pdf | |
| PWC | https://paperswithcode.com/paper/occams-razor-is-insufficient-to-infer-the |
| Repo | |
| Framework | |
Global hard thresholding algorithms for joint sparse image representation and denoising
| Title | Global hard thresholding algorithms for joint sparse image representation and denoising |
| Authors | Reza Borhani, Jeremy Watt, Aggelos Katsaggelos |
| Abstract | Sparse coding of images is traditionally done by cutting them into small patches and representing each patch individually over some dictionary given a pre-determined number of nonzero coefficients to use for each patch. In lack of a way to effectively distribute a total number (or global budget) of nonzero coefficients across all patches, current sparse recovery algorithms distribute the global budget equally across all patches despite the wide range of differences in structural complexity among them. In this work we propose a new framework for joint sparse representation and recovery of all image patches simultaneously. We also present two novel global hard thresholding algorithms, based on the notion of variable splitting, for solving the joint sparse model. Experimentation using both synthetic and real data shows effectiveness of the proposed framework for sparse image representation and denoising tasks. Additionally, time complexity analysis of the proposed algorithms indicate high scalability of both algorithms, making them favorable to use on large megapixel images. |
| Tasks | Denoising |
| Published | 2017-05-27 |
| URL | http://arxiv.org/abs/1705.09816v1 |
| http://arxiv.org/pdf/1705.09816v1.pdf | |
| PWC | https://paperswithcode.com/paper/global-hard-thresholding-algorithms-for-joint |
| Repo | |
| Framework | |
