Paper Group ANR 1002
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning. An Online Topic Modeling Framework with Topics Automatically Labeled. Detection of Classifier Inconsistencies in Image Steganalysis. Inferring Global Dynamics of a Black-Box System Using Machine Learning. Adversarial attacks on Copyright Detection Systems. A Corpus-free State2Seq …
Lipschitz Adaptivity with Multiple Learning Rates in Online Learning
| Title | Lipschitz Adaptivity with Multiple Learning Rates in Online Learning |
| Authors | Zakaria Mhammedi, Wouter M. Koolen, Tim van Erven |
| Abstract | We aim to design adaptive online learning algorithms that take advantage of any special structure that might be present in the learning task at hand, with as little manual tuning by the user as possible. A fundamental obstacle that comes up in the design of such adaptive algorithms is to calibrate a so-called step-size or learning rate hyperparameter depending on variance, gradient norms, etc. A recent technique promises to overcome this difficulty by maintaining multiple learning rates in parallel. This technique has been applied in the MetaGrad algorithm for online convex optimization and the Squint algorithm for prediction with expert advice. However, in both cases the user still has to provide in advance a Lipschitz hyperparameter that bounds the norm of the gradients. Although this hyperparameter is typically not available in advance, tuning it correctly is crucial: if it is set too small, the methods may fail completely; but if it is taken too large, performance deteriorates significantly. In the present work we remove this Lipschitz hyperparameter by designing new versions of MetaGrad and Squint that adapt to its optimal value automatically. We achieve this by dynamically updating the set of active learning rates. For MetaGrad, we further improve the computational efficiency of handling constraints on the domain of prediction, and we remove the need to specify the number of rounds in advance. |
| Tasks | Active Learning |
| Published | 2019-02-27 |
| URL | https://arxiv.org/abs/1902.10797v2 |
| https://arxiv.org/pdf/1902.10797v2.pdf | |
| PWC | https://paperswithcode.com/paper/lipschitz-adaptivity-with-multiple-learning |
| Repo | |
| Framework | |
An Online Topic Modeling Framework with Topics Automatically Labeled
| Title | An Online Topic Modeling Framework with Topics Automatically Labeled |
| Authors | Fenglei Jin, Cuiyun Gao, Michael R. Lyu |
| Abstract | In this paper, we propose a novel online topic tracking framework, named IEDL, for tracking the topic changes related to deep learning techniques on Stack Exchange and automatically interpreting each identified topic. The proposed framework combines the prior topic distributions in a time window during inferring the topics in current time slice, and introduces a new ranking scheme to select most representative phrases and sentences for the inferred topics in each time slice. Experiments on 7,076 Stack Exchange posts show the effectiveness of IEDL in tracking topic changes and labeling topics. |
| Tasks | |
| Published | 2019-06-22 |
| URL | https://arxiv.org/abs/1907.01638v1 |
| https://arxiv.org/pdf/1907.01638v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-online-topic-modeling-framework-with |
| Repo | |
| Framework | |
Detection of Classifier Inconsistencies in Image Steganalysis
| Title | Detection of Classifier Inconsistencies in Image Steganalysis |
| Authors | Daniel Lerch-Hostalot, David Megías |
| Abstract | In this paper, a methodology to detect inconsistencies in classification-based image steganalysis is presented. The proposed approach uses two classifiers: the usual one, trained with a set formed by cover and stego images, and a second classifier trained with the set obtained after embedding additional random messages into the original training set. When the decisions of these two classifiers are not consistent, we know that the prediction is not reliable. The number of inconsistencies in the predictions of a testing set may indicate that the classifier is not performing correctly in the testing scenario. This occurs, for example, in case of cover source mismatch, or when we are trying to detect a steganographic method that the classifier is no capable of modelling accurately. We also show how the number of inconsistencies can be used to predict the reliability of the classifier (classification errors). |
| Tasks | |
| Published | 2019-09-23 |
| URL | https://arxiv.org/abs/1909.10278v1 |
| https://arxiv.org/pdf/1909.10278v1.pdf | |
| PWC | https://paperswithcode.com/paper/190910278 |
| Repo | |
| Framework | |
Inferring Global Dynamics of a Black-Box System Using Machine Learning
| Title | Inferring Global Dynamics of a Black-Box System Using Machine Learning |
| Authors | Hong Zhao |
| Abstract | We present that, instead of establishing the equations of motion, one can model-freely reveal the dynamical properties of a black-box system using a learning machine. Trained only by a segment of time series of a state variable recorded at present parameters values, the dynamics of the learning machine at different training stages can be mapped to the dynamics of the target system along a particular path in its parameter space, following an appropriate training strategy that monotonously decreases the cost function. This path is important, because along that, the primary dynamical properties of the target system will subsequently emerge, in the simple-to-complex order, matching closely the evolution law of certain self-evolved systems in nature. Why such a path can be reproduced is attributed to our training strategy. This particular function of the learning machine opens up a novel way to probe the global dynamical properties of a black-box system without artificially establish the equations of motion, and as such it might have countless applications. As an example, this method is applied to infer what dynamical stages a variable star has experienced and how it will evolve in future, by using the light curve observed presently. |
| Tasks | Time Series |
| Published | 2019-05-10 |
| URL | https://arxiv.org/abs/1905.08313v2 |
| https://arxiv.org/pdf/1905.08313v2.pdf | |
| PWC | https://paperswithcode.com/paper/190508313 |
| Repo | |
| Framework | |
Adversarial attacks on Copyright Detection Systems
| Title | Adversarial attacks on Copyright Detection Systems |
| Authors | Parsa Saadatpanah, Ali Shafahi, Tom Goldstein |
| Abstract | It is well-known that many machine learning models are susceptible to adversarial attacks, in which an attacker evades a classifier by making small perturbations to inputs. This paper discusses how industrial copyright detection tools, which serve a central role on the web, are susceptible to adversarial attacks. We discuss a range of copyright detection systems, and why they are particularly vulnerable to attacks. These vulnerabilities are especially apparent for neural network based systems. As a proof of concept, we describe a well-known music identification method, and implement this system in the form of a neural net. We then attack this system using simple gradient methods. Adversarial music created this way successfully fools industrial systems, including the AudioTag copyright detector and YouTube’s Content ID system. Our goal is to raise awareness of the threats posed by adversarial examples in this space, and to highlight the importance of hardening copyright detection systems to attacks. |
| Tasks | |
| Published | 2019-06-17 |
| URL | https://arxiv.org/abs/1906.07153v2 |
| https://arxiv.org/pdf/1906.07153v2.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-attacks-on-copyright-detection |
| Repo | |
| Framework | |
A Corpus-free State2Seq User Simulator for Task-oriented Dialogue
| Title | A Corpus-free State2Seq User Simulator for Task-oriented Dialogue |
| Authors | Yutai Hou, Meng Fang, Wanxiang Che, Ting Liu |
| Abstract | Recent reinforcement learning algorithms for task-oriented dialogue system absorbs a lot of interest. However, an unavoidable obstacle for training such algorithms is that annotated dialogue corpora are often unavailable. One of the popular approaches addressing this is to train a dialogue agent with a user simulator. Traditional user simulators are built upon a set of dialogue rules and therefore lack response diversity. This severely limits the simulated cases for agent training. Later data-driven user models work better in diversity but suffer from data scarcity problem. To remedy this, we design a new corpus-free framework that taking advantage of their benefits. The framework builds a user simulator by first generating diverse dialogue data from templates and then build a new State2Seq user simulator on the data. To enhance the performance, we propose the State2Seq user simulator model to efficiently leverage dialogue state and history. Experiment results on an open dataset show that our user simulator helps agents achieve an improvement of 6.36% on success rate. State2Seq model outperforms the seq2seq baseline for 1.9 F-score. |
| Tasks | |
| Published | 2019-09-10 |
| URL | https://arxiv.org/abs/1909.04448v1 |
| https://arxiv.org/pdf/1909.04448v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-corpus-free-state2seq-user-simulator-for |
| Repo | |
| Framework | |
Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs
| Title | Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs |
| Authors | Rishabh Iyer, Jeff Bilmes |
| Abstract | In this paper, we investigate a class of submodular problems which in general are very hard. These include minimizing a submodular cost function under combinatorial constraints, which include cuts, matchings, paths, etc., optimizing a submodular function under submodular cover and submodular knapsack constraints, and minimizing a ratio of submodular functions. All these problems appear in several real world problems but have hardness factors of $\Omega(\sqrt{n})$ for general submodular cost functions. We show how we can achieve constant approximation factors when we restrict the cost functions to low rank sums of concave over modular functions. A wide variety of machine learning applications are very naturally modeled via this subclass of submodular functions. Our work therefore provides a tighter connection between theory and practice by enabling theoretically satisfying guarantees for a rich class of expressible, natural, and useful submodular cost models. We empirically demonstrate the utility of our models on real world problems of cooperative image matching and sensor placement with cooperative costs. |
| Tasks | |
| Published | 2019-02-26 |
| URL | http://arxiv.org/abs/1902.10172v1 |
| http://arxiv.org/pdf/1902.10172v1.pdf | |
| PWC | https://paperswithcode.com/paper/near-optimal-algorithms-for-hard-submodular |
| Repo | |
| Framework | |
Potential Flow Generator with $L_2$ Optimal Transport Regularity for Generative Models
| Title | Potential Flow Generator with $L_2$ Optimal Transport Regularity for Generative Models |
| Authors | Liu Yang, George Em Karniadakis |
| Abstract | We propose a potential flow generator with $L_2$ optimal transport regularity, which can be easily integrated into a wide range of generative models including different versions of GANs and flow-based models. We show the correctness and robustness of the potential flow generator in several 2D problems, and illustrate the concept of “proximity” due to the $L_2$ optimal transport regularity. Subsequently, we demonstrate the effectiveness of the potential flow generator in image translation tasks with unpaired training data from the MNIST dataset and the CelebA dataset. |
| Tasks | |
| Published | 2019-08-29 |
| URL | https://arxiv.org/abs/1908.11462v1 |
| https://arxiv.org/pdf/1908.11462v1.pdf | |
| PWC | https://paperswithcode.com/paper/potential-flow-generator-with-l_2-optimal |
| Repo | |
| Framework | |
Active Learning for Chinese Word Segmentation in Medical Text
| Title | Active Learning for Chinese Word Segmentation in Medical Text |
| Authors | Tingting Cai, Yangming Zhou, Zhiyuan Ma, Hong Zheng, Lingfei Zhang, Ping He, Ju Gao |
| Abstract | Electronic health records (EHRs) stored in hospital information systems completely reflect the patients’ diagnosis and treatment processes, which are essential to clinical data mining. Chinese word segmentation (CWS) is a fundamental and important task for Chinese natural language processing. Currently, most state-of-the-art CWS methods greatly depend on large-scale manually-annotated data, which is a very time-consuming and expensive work, specially for the annotation in medical field. In this paper, we present an active learning method for CWS in medical text. To effectively utilize complete segmentation history, a new scoring model in sampling strategy is proposed, which combines information entropy with neural network. Besides, to capture interactions between adjacent characters, K-means clustering features are additionally added in word segmenter. We experimentally evaluate our proposed CWS method in medical text, experimental results based on EHRs collected from the Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine show that our proposed method outperforms other reference methods, which can effectively save the cost of manual annotation. |
| Tasks | Active Learning, Chinese Word Segmentation |
| Published | 2019-08-22 |
| URL | https://arxiv.org/abs/1908.08419v1 |
| https://arxiv.org/pdf/1908.08419v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-learning-for-chinese-word-segmentation-1 |
| Repo | |
| Framework | |
Weakly Supervised Video Moment Retrieval From Text Queries
| Title | Weakly Supervised Video Moment Retrieval From Text Queries |
| Authors | Niluthpol Chowdhury Mithun, Sujoy Paul, Amit K. Roy-Chowdhury |
| Abstract | There have been a few recent methods proposed in text to video moment retrieval using natural language queries, but requiring full supervision during training. However, acquiring a large number of training videos with temporal boundary annotations for each text description is extremely time-consuming and often not scalable. In order to cope with this issue, in this work, we introduce the problem of learning from weak labels for the task of text to video moment retrieval. The weak nature of the supervision is because, during training, we only have access to the video-text pairs rather than the temporal extent of the video to which different text descriptions relate. We propose a joint visual-semantic embedding based framework that learns the notion of relevant segments from video using only video-level sentence descriptions. Specifically, our main idea is to utilize latent alignment between video frames and sentence descriptions using Text-Guided Attention (TGA). TGA is then used during the test phase to retrieve relevant moments. Experiments on two benchmark datasets demonstrate that our method achieves comparable performance to state-of-the-art fully supervised approaches. |
| Tasks | |
| Published | 2019-04-05 |
| URL | https://arxiv.org/abs/1904.03282v2 |
| https://arxiv.org/pdf/1904.03282v2.pdf | |
| PWC | https://paperswithcode.com/paper/weakly-supervised-video-moment-retrieval-from |
| Repo | |
| Framework | |
Random Projections and Sampling Algorithms for Clustering of High-Dimensional Polygonal Curves
| Title | Random Projections and Sampling Algorithms for Clustering of High-Dimensional Polygonal Curves |
| Authors | Stefan Meintrup, Alexander Munteanu, Dennis Rohde |
| Abstract | We study the $k$-median clustering problem for high-dimensional polygonal curves with finite but unbounded number of vertices. We tackle the computational issue that arises from the high number of dimensions by defining a Johnson-Lindenstrauss projection for polygonal curves. We analyze the resulting error in terms of the Fr'echet distance, which is a tractable and natural dissimilarity measure for curves. Our clustering algorithms achieve sublinear dependency on the number of input curves via subsampling. Also, we show that the Fr'echet distance can not be approximated within any factor of less than $\sqrt{2}$ by probabilistically reducing the dependency on the number of vertices of the curves. As a consequence we provide a fast, CUDA-parallelized version of the Alt and Godau algorithm for computing the Fr'echet distance and use it to evaluate our results empirically. |
| Tasks | |
| Published | 2019-07-16 |
| URL | https://arxiv.org/abs/1907.06969v2 |
| https://arxiv.org/pdf/1907.06969v2.pdf | |
| PWC | https://paperswithcode.com/paper/random-projections-and-sampling-algorithms |
| Repo | |
| Framework | |
Multi-Criteria Chinese Word Segmentation with Transformer
| Title | Multi-Criteria Chinese Word Segmentation with Transformer |
| Authors | Xipeng Qiu, Hengzhi Pei, Hang Yan, Xuanjing Huang |
| Abstract | Different linguistic perspectives cause many diverse segmentation criteria for Chinese word segmentation (CWS). Most existing methods focus on improving the performance of single-criterion CWS. However, it is interesting to exploit these heterogeneous segmentation criteria and mine their common underlying knowledge. In this paper, we propose a concise and effective model for multi-criteria CWS, which utilizes a shared fully-connected self-attention model to segment the sentence according to a criterion indicator. Experiments on eight datasets with heterogeneous segmentation criteria show that the performance of each corpus obtains a significant improvement, compared to single-criterion learning. |
| Tasks | Chinese Word Segmentation |
| Published | 2019-06-28 |
| URL | https://arxiv.org/abs/1906.12035v1 |
| https://arxiv.org/pdf/1906.12035v1.pdf | |
| PWC | https://paperswithcode.com/paper/multi-criteria-chinese-word-segmentation-with |
| Repo | |
| Framework | |
Adversarial Defense Through Network Profiling Based Path Extraction
| Title | Adversarial Defense Through Network Profiling Based Path Extraction |
| Authors | Yuxian Qiu, Jingwen Leng, Cong Guo, Quan Chen, Chao Li, Minyi Guo, Yuhao Zhu |
| Abstract | Recently, researchers have started decomposing deep neural network models according to their semantics or functions. Recent work has shown the effectiveness of decomposed functional blocks for defending adversarial attacks, which add small input perturbation to the input image to fool the DNN models. This work proposes a profiling-based method to decompose the DNN models to different functional blocks, which lead to the effective path as a new approach to exploring DNNs’ internal organization. Specifically, the per-image effective path can be aggregated to the class-level effective path, through which we observe that adversarial images activate effective path different from normal images. We propose an effective path similarity-based method to detect adversarial images with an interpretable model, which achieve better accuracy and broader applicability than the state-of-the-art technique. |
| Tasks | Adversarial Defense |
| Published | 2019-04-17 |
| URL | https://arxiv.org/abs/1904.08089v2 |
| https://arxiv.org/pdf/1904.08089v2.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-defense-through-network-profiling |
| Repo | |
| Framework | |
WideDTA: prediction of drug-target binding affinity
| Title | WideDTA: prediction of drug-target binding affinity |
| Authors | Hakime Öztürk, Elif Ozkirimli, Arzucan Özgür |
| Abstract | Motivation: Prediction of the interaction affinity between proteins and compounds is a major challenge in the drug discovery process. WideDTA is a deep-learning based prediction model that employs chemical and biological textual sequence information to predict binding affinity. Results: WideDTA uses four text-based information sources, namely the protein sequence, ligand SMILES, protein domains and motifs, and maximum common substructure words to predict binding affinity. WideDTA outperformed one of the state of the art deep learning methods for drug-target binding affinity prediction, DeepDTA on the KIBA dataset with a statistical significance. This indicates that the word-based sequence representation adapted by WideDTA is a promising alternative to the character-based sequence representation approach in deep learning models for binding affinity prediction, such as the one used in DeepDTA. In addition, the results showed that, given the protein sequence and ligand SMILES, the inclusion of protein domain and motif information as well as ligand maximum common substructure words do not provide additional useful information for the deep learning model. Interestingly, however, using only domain and motif information to represent proteins achieved similar performance to using the full protein sequence, suggesting that important binding relevant information is contained within the protein motifs and domains. |
| Tasks | Drug Discovery |
| Published | 2019-02-04 |
| URL | http://arxiv.org/abs/1902.04166v1 |
| http://arxiv.org/pdf/1902.04166v1.pdf | |
| PWC | https://paperswithcode.com/paper/widedta-prediction-of-drug-target-binding |
| Repo | |
| Framework | |
Unsupervised Anomalous Trajectory Detection for Crowded Scenes
| Title | Unsupervised Anomalous Trajectory Detection for Crowded Scenes |
| Authors | Deepan Das, Deepak Mishra |
| Abstract | We present an improved clustering based, unsupervised anomalous trajectory detection algorithm for crowded scenes. The proposed work is based on four major steps, namely, extraction of trajectories from crowded scene video, extraction of several features from these trajectories, independent mean-shift clustering and anomaly detection. First, the trajectories of all moving objects in a crowd are extracted using a multi feature video object tracker. These trajectories are then transformed into a set of feature spaces. Mean shift clustering is applied on these feature matrices to obtain distinct clusters, while a Shannon Entropy based anomaly detector identifies corresponding anomalies. In the final step, a voting mechanism identifies the trajectories that exhibit anomalous characteristics. The algorithm is tested on crowd scene videos from datasets. The videos represent various possible crowd scenes with different motion patterns and the method performs well to detect the expected anomalous trajectories from the scene. |
| Tasks | Anomaly Detection |
| Published | 2019-07-03 |
| URL | https://arxiv.org/abs/1907.01717v1 |
| https://arxiv.org/pdf/1907.01717v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-anomalous-trajectory-detection |
| Repo | |
| Framework | |
