Paper Group ANR 985
Improving OOV Detection and Resolution with External Language Models in Acoustic-to-Word ASR. Coding for Crowdsourced Classification with XOR Queries. Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling. Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block a …
Improving OOV Detection and Resolution with External Language Models in Acoustic-to-Word ASR
| Title | Improving OOV Detection and Resolution with External Language Models in Acoustic-to-Word ASR |
| Authors | Hirofumi Inaguma, Masato Mimura, Shinsuke Sakai, Tatsuya Kawahara |
| Abstract | Acoustic-to-word (A2W) end-to-end automatic speech recognition (ASR) systems have attracted attention because of an extremely simplified architecture and fast decoding. To alleviate data sparseness issues due to infrequent words, the combination with an acoustic-to-character (A2C) model is investigated. Moreover, the A2C model can be used to recover out-of-vocabulary (OOV) words that are not covered by the A2W model, but this requires accurate detection of OOV words. A2W models learn contexts with both acoustic and transcripts; therefore they tend to falsely recognize OOV words as words in the vocabulary. In this paper, we tackle this problem by using external language models (LM), which are trained only with transcriptions and have better linguistic information to detect OOV words. The A2C model is used to resolve these OOV words. Experimental evaluations show that external LMs have the effects of not only reducing errors but also increasing the number of detected OOV words, and the proposed method significantly improves performances in English conversational and Japanese lecture corpora, especially for out-of-domain scenario. We also investigate the impact of the vocabulary size of A2W models and the data size for training LMs. Moreover, our approach can reduce the vocabulary size several times with marginal performance degradation. |
| Tasks | Speech Recognition |
| Published | 2019-09-22 |
| URL | https://arxiv.org/abs/1909.09993v1 |
| https://arxiv.org/pdf/1909.09993v1.pdf | |
| PWC | https://paperswithcode.com/paper/190909993 |
| Repo | |
| Framework | |
Coding for Crowdsourced Classification with XOR Queries
| Title | Coding for Crowdsourced Classification with XOR Queries |
| Authors | James Chin-Jen Pang, Hessam Mahdavifar, S. Sandeep Pradhan |
| Abstract | This paper models the crowdsourced labeling/classification problem as a sparsely encoded source coding problem, where each query answer, regarded as a code bit, is the XOR of a small number of labels, as source information bits. In this paper we leverage the connections between this problem and well-studied codes with sparse representations for the channel coding problem to provide querying schemes with almost optimal number of queries, each of which involving only a constant number of labels. We also extend this scenario to the case where some workers can be unresponsive. For this case, we propose querying schemes where each query involves only log n items, where n is the total number of items to be labeled. Furthermore, we consider classification of two correlated labeling systems and provide two-stage querying schemes with almost optimal number of queries each involving a constant number of labels. |
| Tasks | |
| Published | 2019-06-25 |
| URL | https://arxiv.org/abs/1906.10637v2 |
| https://arxiv.org/pdf/1906.10637v2.pdf | |
| PWC | https://paperswithcode.com/paper/coding-for-crowdsourced-classification-with |
| Repo | |
| Framework | |
Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling
| Title | Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling |
| Authors | Debjit Paul, Mittul Singh, Michael A. Hedderich, Dietrich Klakow |
| Abstract | In this paper, we address the problem of effectively self-training neural networks in a low-resource setting. Self-training is frequently used to automatically increase the amount of training data. However, in a low-resource scenario, it is less effective due to unreliable annotations created using self-labeling of unlabeled data. We propose to combine self-training with noise handling on the self-labeled data. Directly estimating noise on the combined clean training set and self-labeled data can lead to corruption of the clean data and hence, performs worse. Thus, we propose the Clean and Noisy Label Neural Network which trains on clean and noisy self-labeled data simultaneously by explicitly modelling clean and noisy labels separately. In our experiments on Chunking and NER, this approach performs more robustly than the baselines. Complementary to this explicit approach, noise can also be handled implicitly with the help of an auxiliary learning task. To such a complementary approach, our method is more beneficial than other baseline methods and together provides the best performance overall. |
| Tasks | Auxiliary Learning, Chunking |
| Published | 2019-03-28 |
| URL | http://arxiv.org/abs/1903.12008v1 |
| http://arxiv.org/pdf/1903.12008v1.pdf | |
| PWC | https://paperswithcode.com/paper/handling-noisy-labels-for-robustly-learning |
| Repo | |
| Framework | |
Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols
| Title | Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols |
| Authors | Nicolas Loizou, Peter Richtárik |
| Abstract | In this work we present a new framework for the analysis and design of randomized gossip algorithms for solving the average consensus problem. We show how classical randomized iterative methods for solving linear systems can be interpreted as gossip algorithms when applied to special systems encoding the underlying network and explain in detail their decentralized nature. Our general framework recovers a comprehensive array of well-known gossip algorithms as special cases, including the pairwise randomized gossip algorithm and path averaging gossip, and allows for the development of provably faster variants. The flexibility of the new approach enables the design of a number of new specific gossip methods. For instance, we propose and analyze novel block and the first provably accelerated randomized gossip protocols, and dual randomized gossip algorithms. From a numerical analysis viewpoint, our work is the first that explores in depth the decentralized nature of randomized iterative methods for linear systems and proposes them as methods for solving the average consensus problem. We evaluate the performance of the proposed gossip protocols by performing extensive experimental testing on typical wireless network topologies. |
| Tasks | |
| Published | 2019-05-20 |
| URL | https://arxiv.org/abs/1905.08645v2 |
| https://arxiv.org/pdf/1905.08645v2.pdf | |
| PWC | https://paperswithcode.com/paper/revisiting-randomized-gossip-algorithms |
| Repo | |
| Framework | |
Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning
| Title | Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning |
| Authors | Samuel Kessler, Vu Nguyen, Stefan Zohren, Stephen Roberts |
| Abstract | We place an Indian Buffet process (IBP) prior over the structure of a Bayesian Neural Network (BNN), thus allowing the complexity of the BNN to increase and decrease automatically. We further extend this model such that the prior on the structure of each hidden layer is shared globally across all layers, using a Hierarchical-IBP (H-IBP). We apply this model to the problem of resource allocation in Continual Learning (CL) where new tasks occur and the network requires extra resources. Our model uses online variational inference with reparameterisation of the Bernoulli and Beta distributions which constitute the IBP and H-IBP priors. As we automatically learn the number of weights in each layer of the BNN, overfitting and underfitting problems are largely overcome. We show empirically that our approach offers a competitive edge over existing methods in CL. |
| Tasks | Continual Learning |
| Published | 2019-12-04 |
| URL | https://arxiv.org/abs/1912.02290v3 |
| https://arxiv.org/pdf/1912.02290v3.pdf | |
| PWC | https://paperswithcode.com/paper/indian-buffet-neural-networks-for-continual |
| Repo | |
| Framework | |
A lossless data hiding scheme in JPEG images with segment coding
| Title | A lossless data hiding scheme in JPEG images with segment coding |
| Authors | Mingming Zhang, Quan Zhou, Yanlang Hu |
| Abstract | In this paper, we propose a lossless data hiding scheme in JPEG images. After quantified DCT transform, coefficients have characteristics that distribution in high frequencies is relatively sparse and absolute values are small. To improve encoding efficiency, we put forward an encoding algorithm that searches for a high frequency as terminate point and recode the coefficients above, so spare space is reserved to embed secret data and appended data with no file expansion. Receiver can obtain terminate point through data analysis, extract additional data and recover original JPEG images lossless. Experimental results show that the proposed method has a larger capacity than state-of-the-art works. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1901.11203v1 |
| http://arxiv.org/pdf/1901.11203v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-lossless-data-hiding-scheme-in-jpeg-images |
| Repo | |
| Framework | |
Combinatorial Bandits with Relative Feedback
| Title | Combinatorial Bandits with Relative Feedback |
| Authors | Aadirupa Saha, Aditya Gopalan |
| Abstract | We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute. Specifically, we study two regret minimisation problems over subsets of a finite ground set $[n]$, with subset-wise relative preference information feedback according to the Multinomial logit choice model. In the first setting, the learner can play subsets of size bounded by a maximum size and receives top-$m$ rank-ordered feedback, while in the second setting the learner can play subsets of a fixed size $k$ with a full subset ranking observed as feedback. For both settings, we devise instance-dependent and order-optimal regret algorithms with regret $O(\frac{n}{m} \ln T)$ and $O(\frac{n}{k} \ln T)$, respectively. We derive fundamental limits on the regret performance of online learning with subset-wise preferences, proving the tightness of our regret guarantees. Our results also show the value of eliciting more general top-$m$ rank-ordered feedback over single winner feedback ($m=1$). Our theoretical results are corroborated with empirical evaluations. |
| Tasks | |
| Published | 2019-03-01 |
| URL | https://arxiv.org/abs/1903.00543v2 |
| https://arxiv.org/pdf/1903.00543v2.pdf | |
| PWC | https://paperswithcode.com/paper/regret-minimisation-in-multinomial-logit |
| Repo | |
| Framework | |
Recognizing Images with at most one Spike per Neuron
| Title | Recognizing Images with at most one Spike per Neuron |
| Authors | Christoph Stöckl, Wolfgang Maass |
| Abstract | In order to port the performance of trained artificial neural networks (ANNs) to spiking neural networks (SNNs), which can be implemented in neuromorphic hardware with a drastically reduced energy consumption, an efficient ANN to SNN conversion is needed. Previous conversion schemes focused on the representation of the analog output of a rectified linear (ReLU) gate in the ANN by the firing rate of a spiking neuron. But this is not possible for other commonly used ANN gates, and it reduces the throughput even for ReLU gates. We introduce a new conversion method where a gate in the ANN, which can basically be of any type, is emulated by a small circuit of spiking neurons, with At Most One Spike (AMOS) per neuron. We show that this AMOS conversion improves the accuracy of SNNs for ImageNet from 74.60% to 80.97%, thereby bringing it within reach of the best available ANN accuracy (85.0%). The Top5 accuracy of SNNs is raised to 95.82%, getting even closer to the best Top5 performance of 97.2% for ANNs. In addition, AMOS conversion improves latency and throughput of spike-based image classification by several orders of magnitude. Hence these results suggest that SNNs provide a viable direction for developing highly energy efficient hardware for AI that combines high performance with versatility of applications. |
| Tasks | Image Classification |
| Published | 2019-12-30 |
| URL | https://arxiv.org/abs/2001.01682v3 |
| https://arxiv.org/pdf/2001.01682v3.pdf | |
| PWC | https://paperswithcode.com/paper/recognizing-images-with-at-most-one-spike-per |
| Repo | |
| Framework | |
Reinforcement Learning-Driven Test Generation for Android GUI Applications using Formal Specifications
| Title | Reinforcement Learning-Driven Test Generation for Android GUI Applications using Formal Specifications |
| Authors | Yavuz Koroglu, Alper Sen |
| Abstract | There have been many studies on automated test generation for mobile Graphical User Interface (GUI) applications. These studies successfully demonstrate how to detect fatal exceptions and achieve high code and activity coverage with fully automated test generation engines. However, it is unclear how many GUI functions these engines manage to test. Furthermore, these engines implement only implicit test oracles. We propose Fully Automated Reinforcement LEArning-Driven Specification-Based Test Generator for Android (FARLEAD-Android). FARLEAD-Android accepts a GUI-level formal specification as a Linear-time Temporal Logic (LTL) formula. By dynamically executing the Application Under Test (AUT), it learns how to generate a test that satisfies the LTL formula using Reinforcement Learning (RL). The LTL formula does not just guide the test generation but also acts as a specified test oracle, enabling the developer to define automated test oracles for a wide variety of GUI functions by changing the formula. Our evaluation shows that FARLEAD-Android is more effective and achieves higher performance in generating tests for specified GUI functions than three known approaches, Random, Monkey, and QBEa. To the best of our knowledge, FARLEAD-Android is the first fully automated mobile GUI testing engine that uses formal specifications. |
| Tasks | |
| Published | 2019-11-13 |
| URL | https://arxiv.org/abs/1911.05403v2 |
| https://arxiv.org/pdf/1911.05403v2.pdf | |
| PWC | https://paperswithcode.com/paper/reinforcement-learning-driven-test-generation |
| Repo | |
| Framework | |
Towards Big data processing in IoT: network management for online edge data processing
| Title | Towards Big data processing in IoT: network management for online edge data processing |
| Authors | Shuo Wan, Jiaxun Lu, Pingyi Fan, Khaled B. Letaief |
| Abstract | Heavy data load and wide cover range have always been crucial problems for internet of things (IoT). However, in mobile-edge computing (MEC) network, the huge data can be partly processed at the edge. In this paper, a MEC-based big data analysis network is discussed. The raw data generated by distributed network terminals are collected and processed by edge servers. The edge servers split out a large sum of redundant data and transmit extracted information to the center cloud for further analysis. However, for consideration of limited edge computation ability, part of the raw data in huge data sources may be directly transmitted to the cloud. To manage limited resources online, we propose an algorithm based on Lyapunov optimization to jointly optimize the policy of edge processor frequency, transmission power and bandwidth allocation. The algorithm aims at stabilizing data processing delay and saving energy without knowing probability distributions of data sources. The proposed network management algorithm may contribute to big data processing in future IoT. |
| Tasks | |
| Published | 2019-05-05 |
| URL | https://arxiv.org/abs/1905.01663v1 |
| https://arxiv.org/pdf/1905.01663v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-big-data-processing-in-iot-network |
| Repo | |
| Framework | |
Autonomous Open-Ended Learning of Interdependent Tasks
| Title | Autonomous Open-Ended Learning of Interdependent Tasks |
| Authors | Vieri Giuliano Santucci, Emilio Cartoni, Bruno Castro da Silva, Gianluca Baldassarre |
| Abstract | Autonomy is fundamental for artificial agents acting in complex real-world scenarios. The acquisition of many different skills is pivotal to foster versatile autonomous behaviour and thus a main objective for robotics and machine learning. Intrinsic motivations have proven to properly generate a task-agnostic signal to drive the autonomous acquisition of multiple policies in settings requiring the learning of multiple tasks. However, in real world scenarios tasks may be interdependent so that some of them may constitute the precondition for learning other ones. Despite different strategies have been used to tackle the acquisition of interdependent/hierarchical tasks, fully autonomous open-ended learning in these scenarios is still an open question. Building on previous research within the framework of intrinsically-motivated open-ended learning, we propose an architecture for robot control that tackles this problem from the point of view of decision making, i.e. treating the selection of tasks as a Markov Decision Process where the system selects the policies to be trained in order to maximise its competence over all the tasks. The system is then tested with a humanoid robot solving interdependent multiple reaching tasks. |
| Tasks | Decision Making |
| Published | 2019-05-07 |
| URL | https://arxiv.org/abs/1905.02690v1 |
| https://arxiv.org/pdf/1905.02690v1.pdf | |
| PWC | https://paperswithcode.com/paper/autonomous-open-ended-learning-of |
| Repo | |
| Framework | |
Real-time Prediction of Automotive Collision Risk from Monocular Video
| Title | Real-time Prediction of Automotive Collision Risk from Monocular Video |
| Authors | Derek J. Phillips, Juan Carlos Aragon, Anjali Roychowdhury, Regina Madigan, Sunil Chintakindi, Mykel J. Kochenderfer |
| Abstract | Many automotive applications, such as Advanced Driver Assistance Systems (ADAS) for collision avoidance and warnings, require estimating the future automotive risk of a driving scene. We present a low-cost system that predicts the collision risk over an intermediate time horizon from a monocular video source, such as a dashboard-mounted camera. The modular system includes components for object detection, object tracking, and state estimation. We introduce solutions to the object tracking and distance estimation problems. Advanced approaches to the other tasks are used to produce real-time predictions of the automotive risk for the next 10 s at over 5 Hz. The system is designed such that alternative components can be substituted with minimal effort. It is demonstrated on common physical hardware, specifically an off-the-shelf gaming laptop and a webcam. We extend the framework to support absolute speed estimation and more advanced risk estimation techniques. |
| Tasks | Object Detection, Object Tracking |
| Published | 2019-02-04 |
| URL | http://arxiv.org/abs/1902.01293v1 |
| http://arxiv.org/pdf/1902.01293v1.pdf | |
| PWC | https://paperswithcode.com/paper/real-time-prediction-of-automotive-collision |
| Repo | |
| Framework | |
LabelSens: Enabling Real-time Sensor Data Labelling at the point of Collection on Edge Computing
| Title | LabelSens: Enabling Real-time Sensor Data Labelling at the point of Collection on Edge Computing |
| Authors | Kieran Woodward, Eiman Kanjo, Andreas Oikonomou |
| Abstract | In recent years, machine learning has made leaps and bounds enabling applications with high recognition accuracy for speech and images. However, other types of data to which these models can be applied have not yet been explored as thoroughly. In particular, it can be relatively challenging to accurately classify single or multi-model, real-time sensor data. Labelling is an indispensable stage of data pre-processing that can be even more challenging in real-time sensor data collection. Currently, real-time sensor data labelling is an unwieldly process with limited tools available and poor performance characteristics that can lead to the performance of the machine learning models being compromised. In this paper, we introduce new techniques for labelling at the point of collection coupled with a systematic performance comparison of two popular types of Deep Neural Networks running on five custom built edge devices. These state-of-the-art edge devices are designed to enable real-time labelling with various buttons, slide potentiometer and force sensors. This research provides results and insights that can help researchers utilising edge devices for real-time data collection select appropriate labelling techniques. We also identify common bottlenecks in each architecture and provide field tested guidelines to assist developers building adaptive, high performance edge solutions. |
| Tasks | |
| Published | 2019-10-03 |
| URL | https://arxiv.org/abs/1910.01400v2 |
| https://arxiv.org/pdf/1910.01400v2.pdf | |
| PWC | https://paperswithcode.com/paper/labelsens-enabling-real-time-sensor-data |
| Repo | |
| Framework | |
Assessing Sentiment of the Expressed Stance on Social Media
| Title | Assessing Sentiment of the Expressed Stance on Social Media |
| Authors | Abeer Aldayel, Walid Magdy |
| Abstract | Stance detection is the task of inferring viewpoint towards a given topic or entity either being supportive or opposing. One may express a viewpoint towards a topic by using positive or negative language. This paper examines how the stance is being expressed in social media according to the sentiment polarity. There has been a noticeable misconception of the similarity between the stance and sentiment when it comes to viewpoint discovery, where negative sentiment is assumed to mean against stance, and positive sentiment means in-favour stance. To analyze the relation between stance and sentiment, we construct a new dataset with four topics and examine how people express their viewpoint with regards these topics. We validate our results by carrying a further analysis of the popular stance benchmark SemEval stance dataset. Our analyses reveal that sentiment and stance are not highly aligned, and hence the simple sentiment polarity cannot be used solely to denote a stance toward a given topic. |
| Tasks | Stance Detection |
| Published | 2019-08-08 |
| URL | https://arxiv.org/abs/1908.03181v1 |
| https://arxiv.org/pdf/1908.03181v1.pdf | |
| PWC | https://paperswithcode.com/paper/assessing-sentiment-of-the-expressed-stance |
| Repo | |
| Framework | |
Mixed Pooling Multi-View Attention Autoencoder for Representation Learning in Healthcare
| Title | Mixed Pooling Multi-View Attention Autoencoder for Representation Learning in Healthcare |
| Authors | Shaika Chowdhury, Chenwei Zhang, Philip S. Yu, Yuan Luo |
| Abstract | Distributed representations have been used to support downstream tasks in healthcare recently. Healthcare data (e.g., electronic health records) contain multiple modalities of data from heterogeneous sources that can provide complementary information, alongside an added dimension to learning personalized patient representations. To this end, in this paper we propose a novel unsupervised encoder-decoder model, namely Mixed Pooling Multi-View Attention Autoencoder (MPVAA), that generates patient representations encapsulating a holistic view of their medical profile. Specifically, by first learning personalized graph embeddings pertaining to each patient’s heterogeneous healthcare data, it then integrates the non-linear relationships among them into a unified representation through multi-view attention mechanism. Additionally, a mixed pooling strategy is incorporated in the encoding step to learn diverse information specific to each data modality. Experiments conducted for multiple tasks demonstrate the effectiveness of the proposed model over the state-of-the-art representation learning methods in healthcare. |
| Tasks | Representation Learning |
| Published | 2019-10-14 |
| URL | https://arxiv.org/abs/1910.06456v1 |
| https://arxiv.org/pdf/1910.06456v1.pdf | |
| PWC | https://paperswithcode.com/paper/mixed-pooling-multi-view-attention |
| Repo | |
| Framework | |
