July 27, 2019

3033 words 15 mins read

Paper Group ANR 685

Use of Knowledge Graph in Rescoring the N-Best List in Automatic Speech Recognition. Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning. Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years. Pain-Free Random Differential Privacy with Sensitivity Sampling. Null Dyn …

Use of Knowledge Graph in Rescoring the N-Best List in Automatic Speech Recognition


Title	Use of Knowledge Graph in Rescoring the N-Best List in Automatic Speech Recognition
Authors	Ashwini Jaya Kumar, Camilo Morales, Maria-Esther Vidal, Christoph Schmidt, Sören Auer
Abstract	With the evolution of neural network based methods, automatic speech recognition (ASR) field has been advanced to a level where building an application with speech interface is a reality. In spite of these advances, building a real-time speech recogniser faces several problems such as low recognition accuracy, domain constraint, and out-of-vocabulary words. The low recognition accuracy problem is addressed by improving the acoustic model, language model, decoder and by rescoring the N-best list at the output of the decoder. We are considering the N-best list rescoring approach to improve the recognition accuracy. Most of the methods in the literature use the grammatical, lexical, syntactic and semantic connection between the words in a recognised sentence as a feature to rescore. In this paper, we have tried to see the semantic relatedness between the words in a sentence to rescore the N-best list. Semantic relatedness is computed using TransE~\cite{bordes2013translating}, a method for low dimensional embedding of a triple in a knowledge graph. The novelty of the paper is the application of semantic web to automatic speech recognition.
Tasks	Language Modelling, Speech Recognition
Published	2017-05-22
URL	http://arxiv.org/abs/1705.08018v1
PDF	http://arxiv.org/pdf/1705.08018v1.pdf
PWC	https://paperswithcode.com/paper/use-of-knowledge-graph-in-rescoring-the-n
Repo
Framework

Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning


Title	Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning
Authors	Lidong Bing, William W. Cohen, Bhuwan Dhingra
Abstract	We propose a general approach to modeling semi-supervised learning (SSL) algorithms. Specifically, we present a declarative language for modeling both traditional supervised classification tasks and many SSL heuristics, including both well-known heuristics such as co-training and novel domain-specific heuristics. In addition to representing individual SSL heuristics, we show that multiple heuristics can be automatically combined using Bayesian optimization methods. We experiment with two classes of tasks, link-based text classification and relation extraction. We show modest improvements on well-studied link-based classification benchmarks, and state-of-the-art results on relation-extraction tasks for two realistic domains.
Tasks	Relation Extraction, Text Classification
Published	2017-03-05
URL	http://arxiv.org/abs/1703.01557v2
PDF	http://arxiv.org/pdf/1703.01557v2.pdf
PWC	https://paperswithcode.com/paper/using-graphs-of-classifiers-to-impose
Repo
Framework

Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years


Title	Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years
Authors	Nathan Siegle Hartmann, Livia Cucatto, Danielle Brants, Sandra Aluísio
Abstract	Recent research shows that most Brazilian students have serious problems regarding their reading skills. The full development of this skill is key for the academic and professional future of every citizen. Tools for classifying the complexity of reading materials for children aim to improve the quality of the model of teaching reading and text comprehension. For English, Fengs work [11] is considered the state-of-art in grade level prediction and achieved 74% of accuracy in automatically classifying 4 levels of textual complexity for close school grades. There are no classifiers for nonfiction texts for close grades in Portuguese. In this article, we propose a scheme for manual annotation of texts in 5 grade levels, which will be used for customized reading to avoid the lack of interest by students who are more advanced in reading and the blocking of those that still need to make further progress. We obtained 52% of accuracy in classifying texts into 5 levels and 74% in 3 levels. The results prove to be promising when compared to the state-of-art work.9
Tasks	Reading Comprehension
Published	2017-04-10
URL	http://arxiv.org/abs/1704.03013v1
PDF	http://arxiv.org/pdf/1704.03013v1.pdf
PWC	https://paperswithcode.com/paper/automatic-classification-of-the-complexity-of
Repo
Framework

Pain-Free Random Differential Privacy with Sensitivity Sampling


Title	Pain-Free Random Differential Privacy with Sensitivity Sampling
Authors	Benjamin I. P. Rubinstein, Francesco Aldà
Abstract	Popular approaches to differential privacy, such as the Laplace and exponential mechanisms, calibrate randomised smoothing through global sensitivity of the target non-private function. Bounding such sensitivity is often a prohibitively complex analytic calculation. As an alternative, we propose a straightforward sampler for estimating sensitivity of non-private mechanisms. Since our sensitivity estimates hold with high probability, any mechanism that would be $(\epsilon,\delta)$-differentially private under bounded global sensitivity automatically achieves $(\epsilon,\delta,\gamma)$-random differential privacy (Hall et al., 2012), without any target-specific calculations required. We demonstrate on worked example learners how our usable approach adopts a naturally-relaxed privacy guarantee, while achieving more accurate releases even for non-private functions that are black-box computer programs.
Tasks
Published	2017-06-08
URL	http://arxiv.org/abs/1706.02562v1
PDF	http://arxiv.org/pdf/1706.02562v1.pdf
PWC	https://paperswithcode.com/paper/pain-free-random-differential-privacy-with
Repo
Framework

Null Dynamical State Models of Human Cognitive Dysfunction


Title	Null Dynamical State Models of Human Cognitive Dysfunction
Authors	M. J. Gagen
Abstract	The hard problem in artificial intelligence asks how the shuffling of syntactical symbols in a program can lead to systems which experience semantics and qualia. We address this question in three stages. First, we introduce a new class of human semantic symbols which appears when unexpected and drastic environmental change causes humans to become surprised, confused, uncertain, and in extreme cases, unresponsive, passive and dysfunctional. For this class of symbols, pre-learned programs become inoperative so these syntactical programs cannot be the source of experienced qualia. Second, we model the dysfunctional human response to a radically changed environment as being the natural response of any learning machine facing novel inputs from well outside its previous training set. In this situation, learning machines are unable to extract information from their input and will typically enter a dynamical state characterized by null outputs and a lack of response. This state immediately predicts and explains the characteristics of the semantic experiences of humans in similar circumstances. In the third stage, we consider learning machines trained to implement multiple functions in simple sequential programs using environmental data to specify subroutine names, control flow instructions, memory calls, and so on. Drastic change in any of these environmental inputs can again lead to inoperative programs. By examining changes specific to people or locations we can model human cognitive symbols featuring these dependencies, such as attachment and grief. Our approach links known dynamical machines states with human qualia and thus offers new insight into the hard problem of artificial intelligence.
Tasks
Published	2017-12-25
URL	http://arxiv.org/abs/1712.09014v1
PDF	http://arxiv.org/pdf/1712.09014v1.pdf
PWC	https://paperswithcode.com/paper/null-dynamical-state-models-of-human
Repo
Framework

Multi-View Kernels for Low-Dimensional Modeling of Seismic Events


Title	Multi-View Kernels for Low-Dimensional Modeling of Seismic Events
Authors	Ofir Lindenbaum, Yuri Bregman, Neta Rabin, Amir Averbuch
Abstract	The problem of learning from seismic recordings has been studied for years. There is a growing interest in developing automatic mechanisms for identifying the properties of a seismic event. One main motivation is the ability have a reliable identification of man-made explosions. The availability of multiple high-dimensional observations has increased the use of machine learning techniques in a variety of fields. In this work, we propose to use a kernel-fusion based dimensionality reduction framework for generating meaningful seismic representations from raw data. The proposed method is tested on 2023 events that were recorded in Israel and in Jordan. The method achieves promising results in classification of event type as well as in estimating the location of the event. The proposed fusion and dimensionality reduction tools may be applied to other types of geophysical data.
Tasks	Dimensionality Reduction
Published	2017-06-06
URL	http://arxiv.org/abs/1706.01750v1
PDF	http://arxiv.org/pdf/1706.01750v1.pdf
PWC	https://paperswithcode.com/paper/multi-view-kernels-for-low-dimensional
Repo
Framework

QuickEdit: Editing Text & Translations by Crossing Words Out


Title	QuickEdit: Editing Text & Translations by Crossing Words Out
Authors	David Grangier, Michael Auli
Abstract	We propose a framework for computer-assisted text editing. It applies to translation post-editing and to paraphrasing. Our proposal relies on very simple interactions: a human editor modifies a sentence by marking tokens they would like the system to change. Our model then generates a new sentence which reformulates the initial sentence by avoiding marked words. The approach builds upon neural sequence-to-sequence modeling and introduces a neural network which takes as input a sentence along with change markers. Our model is trained on translation bitext by simulating post-edits. We demonstrate the advantage of our approach for translation post-editing through simulated post-edits. We also evaluate our model for paraphrasing through a user study.
Tasks
Published	2017-11-13
URL	http://arxiv.org/abs/1711.04805v2
PDF	http://arxiv.org/pdf/1711.04805v2.pdf
PWC	https://paperswithcode.com/paper/quickedit-editing-text-translations-by
Repo
Framework

Leverage Score Sampling for Faster Accelerated Regression and ERM


Title	Leverage Score Sampling for Faster Accelerated Regression and ERM
Authors	Naman Agarwal, Sham Kakade, Rahul Kidambi, Yin Tat Lee, Praneeth Netrapalli, Aaron Sidford
Abstract	Given a matrix $\mathbf{A}\in\mathbb{R}^{n\times d}$ and a vector $b \in\mathbb{R}^{d}$, we show how to compute an $\epsilon$-approximate solution to the regression problem $ \min_{x\in\mathbb{R}^{d}}\frac{1}{2} \mathbf{A} x - b_{2}^{2} $ in time $ \tilde{O} ((n+\sqrt{d\cdot\kappa_{\text{sum}}})\cdot s\cdot\log\epsilon^{-1}) $ where $\kappa_{\text{sum}}=\mathrm{tr}\left(\mathbf{A}^{\top}\mathbf{A}\right)/\lambda_{\min}(\mathbf{A}^{T}\mathbf{A})$ and $s$ is the maximum number of non-zero entries in a row of $\mathbf{A}$. Our algorithm improves upon the previous best running time of $ \tilde{O} ((n+\sqrt{n \cdot\kappa_{\text{sum}}})\cdot s\cdot\log\epsilon^{-1})$. We achieve our result through a careful combination of leverage score sampling techniques, proximal point methods, and accelerated coordinate descent. Our method not only matches the performance of previous methods, but further improves whenever leverage scores of rows are small (up to polylogarithmic factors). We also provide a non-linear generalization of these results that improves the running time for solving a broader class of ERM problems.
Tasks
Published	2017-11-22
URL	http://arxiv.org/abs/1711.08426v1
PDF	http://arxiv.org/pdf/1711.08426v1.pdf
PWC	https://paperswithcode.com/paper/leverage-score-sampling-for-faster
Repo
Framework

Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning


Title	Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning
Authors	Haiyan Yin, Jianda Chen, Sinno Jialin Pan
Abstract	In deep reinforcement learning (RL) tasks, an efficient exploration mechanism should be able to encourage an agent to take actions that lead to less frequent states which may yield higher accumulative future return. However, both knowing about the future and evaluating the frequentness of states are non-trivial tasks, especially for deep RL domains, where a state is represented by high-dimensional image frames. In this paper, we propose a novel informed exploration framework for deep RL, where we build the capability for an RL agent to predict over the future transitions and evaluate the frequentness for the predicted future frames in a meaningful manner. To this end, we train a deep prediction model to predict future frames given a state-action pair, and a convolutional autoencoder model to hash over the seen frames. In addition, to utilize the counts derived from the seen frames to evaluate the frequentness for the predicted frames, we tackle the challenge of matching the predicted future frames and their corresponding seen frames at the latent feature level. In this way, we derive a reliable metric for evaluating the novelty of the future direction pointed by each action, and hence inform the agent to explore the least frequent one.
Tasks	Efficient Exploration
Published	2017-07-03
URL	http://arxiv.org/abs/1707.00524v2
PDF	http://arxiv.org/pdf/1707.00524v2.pdf
PWC	https://paperswithcode.com/paper/hashing-over-predicted-future-frames-for
Repo
Framework

Early prediction of the duration of protests using probabilistic Latent Dirichlet Allocation and Decision Trees


Title	Early prediction of the duration of protests using probabilistic Latent Dirichlet Allocation and Decision Trees
Authors	Satyakama Paul, Madhur Hasija, Tshilidzi Marwala
Abstract	Protests and agitations are an integral part of every democratic civil society. In recent years, South Africa has seen a large increase in its protests. The objective of this paper is to provide an early prediction of the duration of protests from its free flowing English text description. Free flowing descriptions of the protests help us in capturing its various nuances such as multiple causes, courses of actions etc. Next we use a combination of unsupervised learning (topic modeling) and supervised learning (decision trees) to predict the duration of the protests. Our results show a high degree (close to 90%) of accuracy in early prediction of the duration of protests.We expect the work to help police and other security services in planning and managing their resources in better handling protests in future.
Tasks
Published	2017-09-18
URL	http://arxiv.org/abs/1711.00462v1
PDF	http://arxiv.org/pdf/1711.00462v1.pdf
PWC	https://paperswithcode.com/paper/early-prediction-of-the-duration-of-protests
Repo
Framework

Three Factors Influencing Minima in SGD


Title	Three Factors Influencing Minima in SGD
Authors	Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, Amos Storkey
Abstract	We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rate, batch size and the properties of the final minima, such as width or generalization, remains an open question. In order to tackle this problem we investigate the previously proposed approximation of SGD by a stochastic differential equation (SDE). We theoretically argue that three factors - learning rate, batch size and gradient covariance - influence the minima found by SGD. In particular we find that the ratio of learning rate to batch size is a key determinant of SGD dynamics and of the width of the final minima, and that higher values of the ratio lead to wider minima and often better generalization. We confirm these findings experimentally. Further, we include experiments which show that learning rate schedules can be replaced with batch size schedules and that the ratio of learning rate to batch size is an important factor influencing the memorization process.
Tasks
Published	2017-11-13
URL	http://arxiv.org/abs/1711.04623v3
PDF	http://arxiv.org/pdf/1711.04623v3.pdf
PWC	https://paperswithcode.com/paper/three-factors-influencing-minima-in-sgd
Repo
Framework

Adaptive Learning Rule for Hardware-based Deep Neural Networks Using Electronic Synapse Devices


Title	Adaptive Learning Rule for Hardware-based Deep Neural Networks Using Electronic Synapse Devices
Authors	Suhwan Lim, Jong-Ho Bae, Jai-Ho Eum, Sungtae Lee, Chul-Heung Kim, Dongseok Kwon, Byung-Gook Park, Jong-Ho Lee
Abstract	In this paper, we propose a learning rule based on a back-propagation (BP) algorithm that can be applied to a hardware-based deep neural network (HW-DNN) using electronic devices that exhibit discrete and limited conductance characteristics. This adaptive learning rule, which enables forward, backward propagation, as well as weight updates in hardware, is helpful during the implementation of power-efficient and high-speed deep neural networks. In simulations using a three-layer perceptron network, we evaluate the learning performance according to various conductance responses of electronic synapse devices and weight-updating methods. It is shown that the learning accuracy is comparable to that obtained when using a software-based BP algorithm when the electronic synapse device has a linear conductance response with a high dynamic range. Furthermore, the proposed unidirectional weight-updating method is suitable for electronic synapse devices which have nonlinear and finite conductance responses. Because this weight-updating method can compensate the demerit of asymmetric weight updates, we can obtain better accuracy compared to other methods. This adaptive learning rule, which can be applied to full hardware implementation, can also compensate the degradation of learning accuracy due to the probable device-to-device variation in an actual electronic synapse device.
Tasks
Published	2017-07-20
URL	http://arxiv.org/abs/1707.06381v2
PDF	http://arxiv.org/pdf/1707.06381v2.pdf
PWC	https://paperswithcode.com/paper/adaptive-learning-rule-for-hardware-based
Repo
Framework

Robust Submodular Maximization: A Non-Uniform Partitioning Approach


Title	Robust Submodular Maximization: A Non-Uniform Partitioning Approach
Authors	Ilija Bogunovic, Slobodan Mitrović, Jonathan Scarlett, Volkan Cevher
Abstract	We study the problem of maximizing a monotone submodular function subject to a cardinality constraint $k$, with the added twist that a number of items $\tau$ from the returned set may be removed. We focus on the worst-case setting considered in (Orlin et al., 2016), in which a constant-factor approximation guarantee was given for $\tau = o(\sqrt{k})$. In this paper, we solve a key open problem raised therein, presenting a new Partitioned Robust (PRo) submodular maximization algorithm that achieves the same guarantee for more general $\tau = o(k)$. Our algorithm constructs partitions consisting of buckets with exponentially increasing sizes, and applies standard submodular optimization subroutines on the buckets in order to construct the robust solution. We numerically demonstrate the performance of PRo in data summarization and influence maximization, demonstrating gains over both the greedy algorithm and the algorithm of (Orlin et al., 2016).
Tasks	Data Summarization
Published	2017-06-15
URL	http://arxiv.org/abs/1706.04918v1
PDF	http://arxiv.org/pdf/1706.04918v1.pdf
PWC	https://paperswithcode.com/paper/robust-submodular-maximization-a-non-uniform
Repo
Framework

Automation of Android Applications Testing Using Machine Learning Activities Classification


Title	Automation of Android Applications Testing Using Machine Learning Activities Classification
Authors	Ariel Rosenfeld, Odaya Kardashov, Orel Zang
Abstract	Mobile applications are being used every day by more than half of the world’s population to perform a great variety of tasks. With the increasingly widespread usage of these applications, the need arises for efficient techniques to test them. Many frameworks allow automating the process of application testing, however existing frameworks mainly rely on the application developer for providing testing scripts for each developed application, thus preventing reuse of these tests for similar applications. In this paper, we present a novel approach for the automation of testing Android applications by leveraging machine learning techniques and reusing popular test scenarios. We discuss and demonstrate the potential benefits of our approach in an empirical study where we show that our developed testing tool, based on the proposed approach, outperforms standard methods in realistic settings.
Tasks
Published	2017-09-04
URL	http://arxiv.org/abs/1709.00928v1
PDF	http://arxiv.org/pdf/1709.00928v1.pdf
PWC	https://paperswithcode.com/paper/automation-of-android-applications-testing
Repo
Framework

MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension


Title	MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
Authors	Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, Xiaofei He
Abstract	Machine comprehension(MC) style question answering is a representative problem in natural language processing. Previous methods rarely spend time on the improvement of encoding layer, especially the embedding of syntactic information and name entity of the words, which are very crucial to the quality of encoding. Moreover, existing attention methods represent each query word as a vector or use a single vector to represent the whole query sentence, neither of them can handle the proper weight of the key words in query sentence. In this paper, we introduce a novel neural network architecture called Multi-layer Embedding with Memory Network(MEMEN) for machine reading task. In the encoding layer, we employ classic skip-gram model to the syntactic and semantic information of the words to train a new kind of embedding layer. We also propose a memory network of full-orientation matching of the query and passage to catch more pivotal information. Experiments show that our model has competitive results both from the perspectives of precision and efficiency in Stanford Question Answering Dataset(SQuAD) among all published results and achieves the state-of-the-art results on TriviaQA dataset.
Tasks	Question Answering, Reading Comprehension
Published	2017-07-28
URL	http://arxiv.org/abs/1707.09098v1
PDF	http://arxiv.org/pdf/1707.09098v1.pdf
PWC	https://paperswithcode.com/paper/memen-multi-layer-embedding-with-memory
Repo
Framework