Paper Group ANR 705
Incorporating Priors with Feature Attribution on Text Classification. A Simple Recommender Engine for Matching Final-Year Project Student with Supervisor. Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting. Text2Node: a Cross-Domain System for Mapping Arbitrary Phrases to a Taxonomy. Deep Latent Space Lea …
Incorporating Priors with Feature Attribution on Text Classification
| Title | Incorporating Priors with Feature Attribution on Text Classification |
| Authors | Frederick Liu, Besim Avci |
| Abstract | Feature attribution methods, proposed recently, help users interpret the predictions of complex models. Our approach integrates feature attributions into the objective function to allow machine learning practitioners to incorporate priors in model building. To demonstrate the effectiveness our technique, we apply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in a scarce data setting by forcing the model to focus on toxic terms. Our approach adds an L2 distance loss between feature attributions and task-specific prior values to the objective. Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a trade off on the original task; ii) incorporating priors helps model performance in scarce data settings. |
| Tasks | Text Classification |
| Published | 2019-06-19 |
| URL | https://arxiv.org/abs/1906.08286v1 |
| https://arxiv.org/pdf/1906.08286v1.pdf | |
| PWC | https://paperswithcode.com/paper/incorporating-priors-with-feature-attribution |
| Repo | |
| Framework | |
A Simple Recommender Engine for Matching Final-Year Project Student with Supervisor
| Title | A Simple Recommender Engine for Matching Final-Year Project Student with Supervisor |
| Authors | Mohammad Hafiz Ismail, Tajul Rosli Razak, Muhamad Arif Hashim, Alif Faisal Ibrahim |
| Abstract | This paper discusses a simple recommender engine, which can match final year project student based on their interests with potential supervisors. The recommender engine is constructed based on Euclidean distance algorithm. The initial input data for the recommender system is obtained by distributing questionnaire to final year students and recording their response in CSV format. The recommender engine is implemented using Java class and application, and result of the initial tests has shown promises that the project is feasible to be pursued as it has the potential of solving the problem of final year students in finding their potential supervisors. |
| Tasks | Recommendation Systems |
| Published | 2019-08-08 |
| URL | https://arxiv.org/abs/1908.03475v1 |
| https://arxiv.org/pdf/1908.03475v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-simple-recommender-engine-for-matching |
| Repo | |
| Framework | |
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
| Title | Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting |
| Authors | Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, Caiming Xiong |
| Abstract | Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks. Despite recent remarkable progress in state-of-the-art deep learning, deep neural networks (DNNs) are still plagued with the catastrophic forgetting problem. This paper presents a conceptually simple yet general and effective framework for handling catastrophic forgetting in continual learning with DNNs. The proposed method consists of two components: a neural structure optimization component and a parameter learning and/or fine-tuning component. By separating the explicit neural structure learning and the parameter estimation, not only is the proposed method capable of evolving neural structures in an intuitively meaningful way, but also shows strong capabilities of alleviating catastrophic forgetting in experiments. Furthermore, the proposed method outperforms all other baselines on the permuted MNIST dataset, the split CIFAR100 dataset and the Visual Domain Decathlon dataset in continual learning setting. |
| Tasks | Continual Learning, Neural Architecture Search |
| Published | 2019-03-31 |
| URL | https://arxiv.org/abs/1904.00310v3 |
| https://arxiv.org/pdf/1904.00310v3.pdf | |
| PWC | https://paperswithcode.com/paper/learn-to-grow-a-continual-structure-learning |
| Repo | |
| Framework | |
Text2Node: a Cross-Domain System for Mapping Arbitrary Phrases to a Taxonomy
| Title | Text2Node: a Cross-Domain System for Mapping Arbitrary Phrases to a Taxonomy |
| Authors | Rohollah Soltani, Alexandre Tomberg |
| Abstract | Electronic health record (EHR) systems are used extensively throughout the healthcare domain. However, data interchangeability between EHR systems is limited due to the use of different coding standards across systems. Existing methods of mapping coding standards based on manual human experts mapping, dictionary mapping, symbolic NLP and classification are unscalable and cannot accommodate large scale EHR datasets. In this work, we present Text2Node, a cross-domain mapping system capable of mapping medical phrases to concepts in a large taxonomy (such as SNOMED CT). The system is designed to generalize from a limited set of training samples and map phrases to elements of the taxonomy that are not covered by training data. As a result, our system is scalable, robust to wording variants between coding systems and can output highly relevant concepts when no exact concept exists in the target taxonomy. Text2Node operates in three main stages: first, the lexicon is mapped to word embeddings; second, the taxonomy is vectorized using node embeddings; and finally, the mapping function is trained to connect the two embedding spaces. We compared multiple algorithms and architectures for each stage of the training, including GloVe and FastText word embeddings, CNN and Bi-LSTM mapping functions, and node2vec for node embeddings. We confirmed the robustness and generalisation properties of Text2Node by mapping ICD-9-CM Diagnosis phrases to SNOMED CT and by zero-shot training at comparable accuracy. This system is a novel methodological contribution to the task of normalizing and linking phrases to a taxonomy, advancing data interchangeability in healthcare. When applied, the system can use electronic health records to generate an embedding that incorporates taxonomical medical knowledge to improve clinical predictive models. |
| Tasks | Word Embeddings |
| Published | 2019-04-11 |
| URL | http://arxiv.org/abs/1905.01958v1 |
| http://arxiv.org/pdf/1905.01958v1.pdf | |
| PWC | https://paperswithcode.com/paper/190501958 |
| Repo | |
| Framework | |
Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals
| Title | Deep Latent Space Learning for Cross-modal Mapping of Audio and Visual Signals |
| Authors | Shah Nawaz, Muhammad Kamran Janjua, Ignazio Gallo, Arif Mahmood, Alessandro Calefati |
| Abstract | We propose a novel deep training algorithm for joint representation of audio and visual information which consists of a single stream network (SSNet) coupled with a novel loss function to learn a shared deep latent space representation of multimodal information. The proposed framework characterizes the shared latent space by leveraging the class centers which helps to eliminate the need for pairwise or triplet supervision. We quantitatively and qualitatively evaluate the proposed approach on VoxCeleb, a benchmarks audio-visual dataset on a multitude of tasks including cross-modal verification, cross-modal matching, and cross-modal retrieval. State-of-the-art performance is achieved on cross-modal verification and matching while comparable results are observed on the remaining applications. Our experiments demonstrate the effectiveness of the technique for cross-modal biometric applications. |
| Tasks | Cross-Modal Retrieval |
| Published | 2019-09-18 |
| URL | https://arxiv.org/abs/1909.08685v1 |
| https://arxiv.org/pdf/1909.08685v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-latent-space-learning-for-cross-modal |
| Repo | |
| Framework | |
Attention-based Deep Tropical Cyclone Rapid Intensification Prediction
| Title | Attention-based Deep Tropical Cyclone Rapid Intensification Prediction |
| Authors | Ching-Yuan Bai, Buo-Fu Chen, Hsuan-Tien Lin |
| Abstract | Rapid intensification (RI) is when a sudden and considerable increase in tropical cyclone (TC) intensity occurs. Accurate early prediction of RI from TC images is important for preventing the possible damages caused by TCs. The main difficulty of RI prediction is to extract important features that are effective for RI prediction, which is challenging even for experienced meteorologists. Inspired by the success of deep learning models for automatic feature extraction and strong predictive performance, we initiate this study that experiments with multiple domain-knowledge guided deep learning models. The goal is to evaluate the potential use of these models for RI prediction. Furthermore, we examine the internal states of the models to obtain visualizable insights for RI prediction. Our model is efficient in training while achieving state-of-the-art performance on the benchmark dataset on HSS metric. The results showcase the success of adapting deep learning to solve complex meteorology problems. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11616v1 |
| https://arxiv.org/pdf/1909.11616v1.pdf | |
| PWC | https://paperswithcode.com/paper/attention-based-deep-tropical-cyclone-rapid |
| Repo | |
| Framework | |
RecNets: Channel-wise Recurrent Convolutional Neural Networks
| Title | RecNets: Channel-wise Recurrent Convolutional Neural Networks |
| Authors | George Retsinas, Athena Elafrou, Georgios Goumas, Petros Maragos |
| Abstract | In this paper, we introduce Channel-wise recurrent convolutional neural networks (RecNets), a family of novel, compact neural network architectures for computer vision tasks inspired by recurrent neural networks (RNNs). RecNets build upon Channel-wise recurrent convolutional (CRC) layers, a novel type of convolutional layer that splits the input channels into disjoint segments and processes them in a recurrent fashion. In this way, we simulate wide, yet compact models, since the number of parameters is vastly reduced via the parameter sharing of the RNN formulation. Experimental results on the CIFAR-10 and CIFAR-100 image classification tasks demonstrate the superior size-accuracy trade-off of RecNets compared to other compact state-of-the-art architectures. |
| Tasks | Image Classification |
| Published | 2019-05-28 |
| URL | https://arxiv.org/abs/1905.11910v2 |
| https://arxiv.org/pdf/1905.11910v2.pdf | |
| PWC | https://paperswithcode.com/paper/recnets-channel-wise-recurrent-convolutional |
| Repo | |
| Framework | |
Mutual Information in Community Detection with Covariate Information and Correlated Networks
| Title | Mutual Information in Community Detection with Covariate Information and Correlated Networks |
| Authors | Vaishakhi Mayya, Galen Reeves |
| Abstract | We study the problem of community detection when there is covariate information about the node labels and one observes multiple correlated networks. We provide an asymptotic upper bound on the per-node mutual information as well as a heuristic analysis of a multivariate performance measure called the MMSE matrix. These results show that the combined effects of seemingly very different types of information can be characterized explicitly in terms of formulas involving low-dimensional estimation problems in additive Gaussian noise. Our analysis is supported by numerical simulations. |
| Tasks | Community Detection |
| Published | 2019-12-11 |
| URL | https://arxiv.org/abs/1912.05375v1 |
| https://arxiv.org/pdf/1912.05375v1.pdf | |
| PWC | https://paperswithcode.com/paper/mutual-information-in-community-detection |
| Repo | |
| Framework | |
Model Imitation for Model-Based Reinforcement Learning
| Title | Model Imitation for Model-Based Reinforcement Learning |
| Authors | Yueh-Hua Wu, Ting-Han Fan, Peter J. Ramadge, Hao Su |
| Abstract | Model-based reinforcement learning (MBRL) aims to learn a dynamic model to reduce the number of interactions with real-world environments. However, due to estimation error, rollouts in the learned model, especially those of long horizons, fail to match the ones in real-world environments. This mismatching has seriously impacted the sample complexity of MBRL. The phenomenon can be attributed to the fact that previous works employ supervised learning to learn the one-step transition models, which has inherent difficulty ensuring the matching of distributions from multi-step rollouts. Based on the claim, we propose to learn the transition model by matching the distributions of multi-step rollouts sampled from the transition model and the real ones via WGAN. We theoretically show that matching the two can minimize the difference of cumulative rewards between the real transition and the learned one. Our experiments also show that the proposed Model Imitation method can compete or outperform the state-of-the-art in terms of sample complexity and average return. |
| Tasks | |
| Published | 2019-09-25 |
| URL | https://arxiv.org/abs/1909.11821v3 |
| https://arxiv.org/pdf/1909.11821v3.pdf | |
| PWC | https://paperswithcode.com/paper/model-imitation-for-model-based-reinforcement |
| Repo | |
| Framework | |
Communication-Censored Distributed Stochastic Gradient Descent
| Title | Communication-Censored Distributed Stochastic Gradient Descent |
| Authors | Weiyu Li, Tianyi Chen, Liping Li, Zhaoxian Wu, Qing Ling |
| Abstract | This paper develops a communication-efficient algorithm to solve the stochastic optimization problem defined over a distributed network, aiming at reducing the burdensome communication in applications such as distributed machine learning.Different from the existing works based on quantization and sparsification, we introduce a communication-censoring technique to reduce the transmissions of variables, which leads to our communication-Censored distributed Stochastic Gradient Descent (CSGD) algorithm. Specifically, in CSGD, the latest mini-batch stochastic gradient at a worker will be transmitted to the server if and only if it is sufficiently informative. When the latest gradient is not available, the stale one will be reused at the server. To implement this communication-censoring strategy, the batch-size is increasing in order to alleviate the effect of stochastic gradient noise. Theoretically, CSGD enjoys the same order of convergence rate as that of SGD, but effectively reduces communication. Numerical experiments demonstrate the sizable communication saving of CSGD. |
| Tasks | Quantization, Stochastic Optimization |
| Published | 2019-09-09 |
| URL | https://arxiv.org/abs/1909.03631v2 |
| https://arxiv.org/pdf/1909.03631v2.pdf | |
| PWC | https://paperswithcode.com/paper/communication-censored-distributed-stochastic |
| Repo | |
| Framework | |
Personalizing Fast-Forward Videos Based on Visual and Textual Features from Social Network
| Title | Personalizing Fast-Forward Videos Based on Visual and Textual Features from Social Network |
| Authors | Washington L. S. Ramos, Michel M. Silva, Edson R. Araujo, Alan C. Neves, Erickson R. Nascimento |
| Abstract | The growth of Social Networks has fueled the habit of people logging their day-to-day activities, and long First-Person Videos (FPVs) are one of the main tools in this new habit. Semantic-aware fast-forward methods are able to decrease the watch time and select meaningful moments, which is key to increase the chances of these videos being watched. However, these methods can not handle semantics in terms of personalization. In this work, we present a new approach to automatically creating personalized fast-forward videos for FPVs. Our approach explores the availability of text-centric data from the user’s social networks such as status updates to infer her/his topics of interest and assigns scores to the input frames according to her/his preferences. Extensive experiments are conducted on three different datasets with simulated and real-world users as input, achieving an average F1 score of up to 12.8 percentage points higher than the best competitors. We also present a user study to demonstrate the effectiveness of our method. |
| Tasks | |
| Published | 2019-12-29 |
| URL | https://arxiv.org/abs/1912.12655v1 |
| https://arxiv.org/pdf/1912.12655v1.pdf | |
| PWC | https://paperswithcode.com/paper/personalizing-fast-forward-videos-based-on |
| Repo | |
| Framework | |
Application of Reinforcement Learning for 5G Scheduling Parameter Optimization
| Title | Application of Reinforcement Learning for 5G Scheduling Parameter Optimization |
| Authors | Ali Asgher Mansoor Habiby, Ahamed Thoppu |
| Abstract | RF Network parametric optimization requires a wealth of experience and knowledge to achieve the optimal balance between coverage, capacity, system efficiency and customer experience from the telecom sites serving the users. With 5G, the complications of Air interface scheduling have increased due to the usage of massive MIMO, beamforming and introduction of higher modulation schemes with varying numerologies. In this work, we tune a machine learning model to “learn” the best combination of parameters for a given traffic profile using Cross Entropy Method Reinforcement Learning and compare these with RF Subject Matter Expert “SME” recommendations. This work is aimed towards automatic parameter tuning and feature optimization by acting as a Self Organizing Network module |
| Tasks | |
| Published | 2019-10-21 |
| URL | https://arxiv.org/abs/1911.07608v1 |
| https://arxiv.org/pdf/1911.07608v1.pdf | |
| PWC | https://paperswithcode.com/paper/application-of-reinforcement-learning-for-5g |
| Repo | |
| Framework | |
Self-supervised Body Image Acquisition Using a Deep Neural Network for Sensorimotor Prediction
| Title | Self-supervised Body Image Acquisition Using a Deep Neural Network for Sensorimotor Prediction |
| Authors | Alban Laflaquière, Verena V. Hafner |
| Abstract | This work investigates how a naive agent can acquire its own body image in a self-supervised way, based on the predictability of its sensorimotor experience. Our working hypothesis is that, due to its temporal stability, an agent’s body produces more consistent sensory experiences than the environment, which exhibits a greater variability. Given its motor experience, an agent can thus reliably predict what appearance its body should have. This intrinsic predictability can be used to automatically isolate the body image from the rest of the environment. We propose a two-branches deconvolutional neural network to predict the visual sensory state associated with an input motor state, as well as the prediction error associated with this input. We train the network on a dataset of first-person images collected with a simulated Pepper robot, and show how the network outputs can be used to automatically isolate its visible arm from the rest of the environment. Finally, the quality of the body image produced by the network is evaluated. |
| Tasks | |
| Published | 2019-06-03 |
| URL | https://arxiv.org/abs/1906.00825v1 |
| https://arxiv.org/pdf/1906.00825v1.pdf | |
| PWC | https://paperswithcode.com/paper/190600825 |
| Repo | |
| Framework | |
TE141K: Artistic Text Benchmark for Text Effect Transfer
| Title | TE141K: Artistic Text Benchmark for Text Effect Transfer |
| Authors | Shuai Yang, Wenjing Wang, Jiaying Liu |
| Abstract | Text effects are combinations of visual elements such as outlines, colors and textures of text, which can dramatically improve its artistry. Although text effects are extensively utilized in the design industry, they are usually created by human experts due to their extreme complexity; this is laborious and not practical for normal users. In recent years, some efforts have been made toward automatic text effect transfer; however, the lack of data limits the capabilities of transfer models. To address this problem, we introduce a new text effects dataset, TE141K, with 141,081 text effect/glyph pairs in total. Our dataset consists of 152 professionally designed text effects rendered on glyphs, including English letters, Chinese characters, and Arabic numerals. To the best of our knowledge, this is the largest dataset for text effect transfer to date. Based on this dataset, we propose a baseline approach called text effect transfer GAN (TET-GAN), which supports the transfer of all 152 styles in one model and can efficiently extend to new styles. Finally, we conduct a comprehensive comparison in which 14 style transfer models are benchmarked. Experimental results demonstrate the superiority of TET-GAN both qualitatively and quantitatively and indicate that our dataset is effective and challenging. |
| Tasks | Style Transfer, Text Effects Transfer |
| Published | 2019-05-08 |
| URL | https://arxiv.org/abs/1905.03646v3 |
| https://arxiv.org/pdf/1905.03646v3.pdf | |
| PWC | https://paperswithcode.com/paper/190503646 |
| Repo | |
| Framework | |
No Press Diplomacy: Modeling Multi-Agent Gameplay
| Title | No Press Diplomacy: Modeling Multi-Agent Gameplay |
| Authors | Philip Paquette, Yuchen Lu, Steven Bocco, Max O. Smith, Satya Ortiz-Gagne, Jonathan K. Kummerfeld, Satinder Singh, Joelle Pineau, Aaron Courville |
| Abstract | Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. We present DipNet, a neural-network-based policy model for No Press Diplomacy. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play. Both the SL and RL agents demonstrate state-of-the-art No Press performance by beating popular rule-based bots. |
| Tasks | |
| Published | 2019-09-04 |
| URL | https://arxiv.org/abs/1909.02128v2 |
| https://arxiv.org/pdf/1909.02128v2.pdf | |
| PWC | https://paperswithcode.com/paper/no-press-diplomacy-modeling-multi-agent |
| Repo | |
| Framework | |
