January 31, 2020

3427 words 17 mins read

Paper Group ANR 15

Optimizing Norm-Bounded Weighted Ambiguity Sets for Robust MDPs. Beyond Explainability: Leveraging Interpretability for Improved Adversarial Learning. Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification. Adversarial Speaker Verification. Structural Pruning in Deep Neural Networks: A Small-World Approach. …

Optimizing Norm-Bounded Weighted Ambiguity Sets for Robust MDPs


Title	Optimizing Norm-Bounded Weighted Ambiguity Sets for Robust MDPs
Authors	Reazul Hasan Russel, Bahram Behzadian, Marek Petrik
Abstract	Optimal policies in Markov decision processes (MDPs) are very sensitive to model misspecification. This raises serious concerns about deploying them in high-stake domains. Robust MDPs (RMDP) provide a promising framework to mitigate vulnerabilities by computing policies with worst-case guarantees in reinforcement learning. The solution quality of an RMDP depends on the ambiguity set, which is a quantification of model uncertainties. In this paper, we propose a new approach for optimizing the shape of the ambiguity sets for RMDPs. Our method departs from the conventional idea of constructing a norm-bounded uniform and symmetric ambiguity set. We instead argue that the structure of a near-optimal ambiguity set is problem specific. Our proposed method computes a weight parameter from the value functions, and these weights then drive the shape of the ambiguity sets. Our theoretical analysis demonstrates the rationale of the proposed idea. We apply our method to several different problem domains, and the empirical results further furnish the practical promise of weighted near-optimal ambiguity sets.
Tasks
Published	2019-12-04
URL	https://arxiv.org/abs/1912.02696v1
PDF	https://arxiv.org/pdf/1912.02696v1.pdf
PWC	https://paperswithcode.com/paper/optimizing-norm-bounded-weighted-ambiguity
Repo
Framework

Beyond Explainability: Leveraging Interpretability for Improved Adversarial Learning


Title	Beyond Explainability: Leveraging Interpretability for Improved Adversarial Learning
Authors	Devinder Kumar, Ibrahim Ben-Daya, Kanav Vats, Jeffery Feng, Graham Taylor and, Alexander Wong
Abstract	In this study, we propose the leveraging of interpretability for tasks beyond purely the purpose of explainability. In particular, this study puts forward a novel strategy for leveraging gradient-based interpretability in the realm of adversarial examples, where we use insights gained to aid adversarial learning. More specifically, we introduce the concept of spatially constrained one-pixel adversarial perturbations, where we guide the learning of such adversarial perturbations towards more susceptible areas identified via gradient-based interpretability. Experimental results using different benchmark datasets show that such a spatially constrained one-pixel adversarial perturbation strategy can noticeably improve the speed of convergence as well as produce successful attacks that were also visually difficult to perceive, thus illustrating an effective use of interpretability methods for tasks outside of the purpose of purely explainability.
Tasks
Published	2019-04-21
URL	http://arxiv.org/abs/1904.09633v1
PDF	http://arxiv.org/pdf/1904.09633v1.pdf
PWC	https://paperswithcode.com/paper/beyond-explainability-leveraging
Repo
Framework

Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification


Title	Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification
Authors	Achintya kr. Sarkar, Zheng-Hua Tan, Hao Tang, Suwon Shon, James Glass
Abstract	There are a number of studies about extraction of bottleneck (BN) features from deep neural networks (DNNs)trained to discriminate speakers, pass-phrases and triphone states for improving the performance of text-dependent speaker verification (TD-SV). However, a moderate success has been achieved. A recent study [1] presented a time contrastive learning (TCL) concept to explore the non-stationarity of brain signals for classification of brain states. Speech signals have similar non-stationarity property, and TCL further has the advantage of having no need for labeled data. We therefore present a TCL based BN feature extraction method. The method uniformly partitions each speech utterance in a training dataset into a predefined number of multi-frame segments. Each segment in an utterance corresponds to one class, and class labels are shared across utterances. DNNs are then trained to discriminate all speech frames among the classes to exploit the temporal structure of speech. In addition, we propose a segment-based unsupervised clustering algorithm to re-assign class labels to the segments. TD-SV experiments were conducted on the RedDots challenge database. The TCL-DNNs were trained using speech data of fixed pass-phrases that were excluded from the TD-SV evaluation set, so the learned features can be considered phrase-independent. We compare the performance of the proposed TCL bottleneck (BN) feature with those of short-time cepstral features and BN features extracted from DNNs discriminating speakers, pass-phrases, speaker+pass-phrase, as well as monophones whose labels and boundaries are generated by three different automatic speech recognition (ASR) systems. Experimental results show that the proposed TCL-BN outperforms cepstral features and speaker+pass-phrase discriminant BN features, and its performance is on par with those of ASR derived BN features. Moreover,….
Tasks	Speaker Verification, Speech Recognition, Text-Dependent Speaker Verification
Published	2019-05-11
URL	https://arxiv.org/abs/1905.04554v1
PDF	https://arxiv.org/pdf/1905.04554v1.pdf
PWC	https://paperswithcode.com/paper/time-contrastive-learning-based-deep
Repo
Framework

Adversarial Speaker Verification


Title	Adversarial Speaker Verification
Authors	Zhong Meng, Yong Zhao, Jinyu Li, Yifan Gong
Abstract	The use of deep networks to extract embeddings for speaker recognition has proven successfully. However, such embeddings are susceptible to performance degradation due to the mismatches among the training, enrollment, and test conditions. In this work, we propose an adversarial speaker verification (ASV) scheme to learn the condition-invariant deep embedding via adversarial multi-task training. In ASV, a speaker classification network and a condition identification network are jointly optimized to minimize the speaker classification loss and simultaneously mini-maximize the condition loss. The target labels of the condition network can be categorical (environment types) and continuous (SNR values). We further propose multi-factorial ASV to simultaneously suppress multiple factors that constitute the condition variability. Evaluated on a Microsoft Cortana text-dependent speaker verification task, the ASV achieves 8.8% and 14.5% relative improvements in equal error rates (EER) for known and unknown conditions, respectively.
Tasks	Speaker Recognition, Speaker Verification, Text-Dependent Speaker Verification
Published	2019-04-29
URL	http://arxiv.org/abs/1904.12406v1
PDF	http://arxiv.org/pdf/1904.12406v1.pdf
PWC	https://paperswithcode.com/paper/adversarial-speaker-verification
Repo
Framework

Structural Pruning in Deep Neural Networks: A Small-World Approach


Title	Structural Pruning in Deep Neural Networks: A Small-World Approach
Authors	Gokul Krishnan, Xiaocong Du, Yu Cao
Abstract	Deep Neural Networks (DNNs) are usually over-parameterized, causing excessive memory and interconnection cost on the hardware platform. Existing pruning approaches remove secondary parameters at the end of training to reduce the model size; but without exploiting the intrinsic network property, they still require the full interconnection to prepare the network. Inspired by the observation that brain networks follow the Small-World model, we propose a novel structural pruning scheme, which includes (1) hierarchically trimming the network into a Small-World model before training, (2) training the network for a given dataset, and (3) optimizing the network for accuracy. The new scheme effectively reduces both the model size and the interconnection needed before training, achieving a locally clustered and globally sparse model. We demonstrate our approach on LeNet-5 for MNIST and VGG-16 for CIFAR-10, decreasing the number of parameters to 2.3% and 9.02% of the baseline model, respectively.
Tasks
Published	2019-11-11
URL	https://arxiv.org/abs/1911.04453v1
PDF	https://arxiv.org/pdf/1911.04453v1.pdf
PWC	https://paperswithcode.com/paper/structural-pruning-in-deep-neural-networks-a
Repo
Framework

Efficient Ridge Solutions for the Incremental Broad Learning System on Added Inputs by Updating the Inverse or the Inverse Cholesky Factor of the Hermitian matrix in the Ridge Inverse


Title	Efficient Ridge Solutions for the Incremental Broad Learning System on Added Inputs by Updating the Inverse or the Inverse Cholesky Factor of the Hermitian matrix in the Ridge Inverse
Authors	Hufei Zhu, Chenghao Wei
Abstract	This brief proposes two BLS algorithms to improve the existing BLS for new added inputs in [7]. The proposed BLS algorithms avoid computing the ridge inverse, by computing the ridge solution (i.e., the output weights) from the inverse or the inverse Cholesky factor of the Hermitian matrix in the ridge inverse. The proposed BLS algorithm 1 updates the inverse of the Hermitian matrix by the matrix inversion lemma [12]. To update the upper-triangular inverse Cholesky factor of the Hermitian matrix, the proposed BLS algorithm 2 multiplies the inverse Cholesky factor with an upper-triangular intermediate matrix, which is computed by a Cholesky factorization or an inverse Cholesky factorization. Assume that the newly added input matrix corresponding to the added inputs is p * k, where p and k are the number of added training samples and the total node number, respectively. When p > k, the inverse of a sum of matrices [11] is utilized to compute the intermediate variables by a smaller matrix inverse in the proposed algorithm 1, or by a smaller inverse Cholesky factorization in the proposed algorithm 2. Usually the Hermitian matrix in the ridge inverse is smaller than the ridge inverse. Thus the proposed algorithms 1 and 2 require less flops (floating-point operations) than the existing BLS algorithm, which is verified by the theoretical flops calculation. In numerical experiments, the speedups for the case of p > k in each additional training time of the proposed BLS algorithms 1 and 2 over the existing algorithm are 1.95 - 5.43 and 2.29 - 6.34, respectively, and the speedups for the case of p < k are 8.83 - 10.21 and 2.28 - 2.58, respectively.
Tasks
Published	2019-11-12
URL	https://arxiv.org/abs/1911.07292v1
PDF	https://arxiv.org/pdf/1911.07292v1.pdf
PWC	https://paperswithcode.com/paper/efficient-ridge-solutions-for-the-incremental
Repo
Framework

Synthesising a Database of Parameterised Linear and Non-Linear Invariants for Time-Series Constraints


Title	Synthesising a Database of Parameterised Linear and Non-Linear Invariants for Time-Series Constraints
Authors	Ekaterina Arafailova, Nicolas Beldiceanu, Helmut Simonis
Abstract	Many constraints restricting the result of some computations over an integer sequence can be compactly represented by register automata. We improve the propagation of the conjunction of such constraints on the same sequence by synthesising a database of linear and non-linear invariants using their register-automaton representation. The obtained invariants are formulae parameterised by a function of the sequence length and proven to be true for any long enough sequence. To assess the quality of such linear invariants, we developed a method to verify whether a generated linear invariant is a facet of the convex hull of the feasible points. This method, as well as the proof of non-linear invariants, are based on the systematic generation of constant-size deterministic finite automata that accept all integer sequences whose result verifies some simple condition. We apply such methodology to a set of 44 time-series constraints and obtain 1400 linear invariants from which 70% are facet defining, and 600 non-linear invariants, which were tested on short-term electricity production problems.
Tasks	Time Series
Published	2019-01-15
URL	http://arxiv.org/abs/1901.09793v1
PDF	http://arxiv.org/pdf/1901.09793v1.pdf
PWC	https://paperswithcode.com/paper/synthesising-a-database-of-parameterised
Repo
Framework

Optimization of the Area Under the ROC Curve using Neural Network Supervectors for Text-Dependent Speaker Verification


Title	Optimization of the Area Under the ROC Curve using Neural Network Supervectors for Text-Dependent Speaker Verification
Authors	Victoria Mingote, Antonio Miguel, Alfonso Ortega, Eduardo Lleida
Abstract	This paper explores two techniques to improve the performance of text-dependent speaker verification systems based on deep neural networks. Firstly, we propose a general alignment mechanism to keep the temporal structure of each phrase and obtain a supervector with the speaker and phrase information, since both are relevant for a text-dependent verification. As we show, it is possible to use different alignment techniques to replace the global average pooling providing significant gains in performance. Moreover, we also present a novel back-end approach to train a neural network for detection tasks by optimizing the Area Under the Curve (AUC) as an alternative to the usual triplet loss function, so the system is end-to-end, with a cost function close to our desired measure of performance. As we can see in the experimental section, this approach improves the system performance, since our triplet neural network based on an approximation of the AUC (aAUC) learns how to discriminate between pairs of examples from the same identity and pairs of different identities. The different alignment techniques to produce supervectors in addition to the new back-end approach were tested on the RSR2015-Part I database for text-dependent speaker verification, providing competitive results compared to similar size networks using the global average pooling to extract supervectors and using a simple back-end or triplet loss training.
Tasks	Speaker Verification, Text-Dependent Speaker Verification
Published	2019-01-31
URL	http://arxiv.org/abs/1901.11332v2
PDF	http://arxiv.org/pdf/1901.11332v2.pdf
PWC	https://paperswithcode.com/paper/optimization-of-the-area-under-the-roc-curve
Repo
Framework

Deep Learning for Interference Identification: Band, Training SNR, and Sample Selection


Title	Deep Learning for Interference Identification: Band, Training SNR, and Sample Selection
Authors	Xiwen Zhang, Tolunay Seyfi, Shengtai Ju, Sharan Ramjee, Aly El Gamal, Yonina C. Eldar
Abstract	We study the problem of interference source identification, through the lens of recognizing one of 15 different channels that belong to 3 different wireless technologies: Bluetooth, Zigbee, and WiFi. We employ deep learning algorithms trained on received samples taken from a 10 MHz band in the 2.4 GHz ISM Band. We obtain a classification accuracy of around 89.5% using any of four different deep neural network architectures: CNN, ResNet, CLDNN, and LSTM, which demonstrate the generality of the effectiveness of deep learning at the considered task. Interestingly, our proposed CNN architecture requires approximately 60% of the training time required by the state of the art while achieving slightly larger classification accuracy. We then focus on the CNN architecture and further optimize its training time while incurring minimal loss in classification accuracy using three different approaches: 1- Band Selection, where we only use samples belonging to the lower and uppermost 2 MHz bands, 2- SNR Selection, where we only use training samples belonging to a single SNR value, and 3- Sample Selection, where we try various sub-Nyquist sampling methods to select the subset of samples most relevant to the classification task. Our results confirm the feasibility of fast deep learning for wireless interference identification, by showing that the training time can be reduced by as much as 30x with minimal loss in accuracy.
Tasks
Published	2019-05-16
URL	https://arxiv.org/abs/1905.08054v1
PDF	https://arxiv.org/pdf/1905.08054v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-interference-identification
Repo
Framework

SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA


Title	SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA
Authors	Daniel Hershcovich, Zohar Aizenbud, Leshem Choshen, Elior Sulem, Ari Rappoport, Omri Abend
Abstract	We present the SemEval 2019 shared task on UCCA parsing in English, German and French, and discuss the participating systems and results. UCCA is a cross-linguistically applicable framework for semantic representation, which builds on extensive typological work and supports rapid annotation. UCCA poses a challenge for existing parsing techniques, as it exhibits reentrancy (resulting in DAG structures), discontinuous structures and non-terminal nodes corresponding to complex semantic units. The shared task has yielded improvements over the state-of-the-art baseline in all languages and settings. Full results can be found in the task’s website \url{https://competitions.codalab.org/competitions/19160}.
Tasks	Semantic Parsing
Published	2019-03-06
URL	http://arxiv.org/abs/1903.02953v2
PDF	http://arxiv.org/pdf/1903.02953v2.pdf
PWC	https://paperswithcode.com/paper/semeval-2019-task-1-cross-lingual-semantic
Repo
Framework

AMUSED: A Multi-Stream Vector Representation Method for Use in Natural Dialogue


Title	AMUSED: A Multi-Stream Vector Representation Method for Use in Natural Dialogue
Authors	Gaurav Kumar, Rishabh Joshi, Jaspreet Singh, Promod Yenigalla
Abstract	The problem of building a coherent and non-monotonous conversational agent with proper discourse and coverage is still an area of open research. Current architectures only take care of semantic and contextual information for a given query and fail to completely account for syntactic and external knowledge which are crucial for generating responses in a chit-chat system. To overcome this problem, we propose an end to end multi-stream deep learning architecture which learns unified embeddings for query-response pairs by leveraging contextual information from memory networks and syntactic information by incorporating Graph Convolution Networks (GCN) over their dependency parse. A stream of this network also utilizes transfer learning by pre-training a bidirectional transformer to extract semantic representation for each input sentence and incorporates external knowledge through the the neighborhood of the entities from a Knowledge Base (KB). We benchmark these embeddings on next sentence prediction task and significantly improve upon the existing techniques. Furthermore, we use AMUSED to represent query and responses along with its context to develop a retrieval based conversational agent which has been validated by expert linguists to have comprehensive engagement with humans.
Tasks	Transfer Learning
Published	2019-12-04
URL	https://arxiv.org/abs/1912.10160v1
PDF	https://arxiv.org/pdf/1912.10160v1.pdf
PWC	https://paperswithcode.com/paper/amused-a-multi-stream-vector-representation-1
Repo
Framework

No Free Lunch But A Cheaper Supper: A General Framework for Streaming Anomaly Detection


Title	No Free Lunch But A Cheaper Supper: A General Framework for Streaming Anomaly Detection
Authors	Ece Calikus, Slawomir Nowaczyk, Anita Sant’Anna, Onur Dikmen
Abstract	In recent years, there has been increased research interest in detecting anomalies in temporal streaming data. A variety of algorithms have been developed in the data mining community, which can be divided into two categories (i.e., general and ad hoc). In most cases, general approaches assume the one-size-fits-all solution model where a single anomaly detector can detect all anomalies in any domain. To date, there exists no single general method that has been shown to outperform the others across different anomaly types, use cases and datasets. In this paper, we propose SAFARI, a general framework formulated by abstracting and unifying the fundamental tasks in streaming anomaly detection, which provides a flexible and extensible anomaly detection procedure to overcome the limitations of one-size-fits-all solutions. SAFARI helps to facilitate more elaborate algorithm comparisons by allowing us to isolate the effects of shared and unique characteristics of different algorithms on detection performance. Using SAFARI, we have implemented various anomaly detectors and identified a research gap that motivates us to propose a novel learning strategy in this work. We conducted an extensive evaluation study of 20 detectors that are composed using SAFARI and compared their performances using real-world benchmark datasets with different properties. The results indicate that there is no single superior detector that works well for every case, proving our hypothesis that “there is no free lunch” in the streaming anomaly detection world. Finally, we discuss the benefits and drawbacks of each method in-depth and draw a set of conclusions to guide future users of SAFARI.
Tasks	Anomaly Detection
Published	2019-09-16
URL	https://arxiv.org/abs/1909.06927v3
PDF	https://arxiv.org/pdf/1909.06927v3.pdf
PWC	https://paperswithcode.com/paper/no-free-lunch-but-a-cheaper-supper-a-general
Repo
Framework

Tanbih: Get To Know What You Are Reading


Title	Tanbih: Get To Know What You Are Reading
Authors	Yifan Zhang, Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James Glass, Preslav Nakov
Abstract	We introduce Tanbih, a news aggregator with intelligent analysis tools to help readers understanding what’s behind a news story. Our system displays news grouped into events and generates media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting, and stance with respect to various claims and topics of a news outlet. In addition, we automatically analyse each article to detect whether it is propagandistic and to determine its stance with respect to a number of controversial topics.
Tasks
Published	2019-10-04
URL	https://arxiv.org/abs/1910.02028v1
PDF	https://arxiv.org/pdf/1910.02028v1.pdf
PWC	https://paperswithcode.com/paper/tanbih-get-to-know-what-you-are-reading
Repo
Framework

Single-Shot High Dynamic Range Imaging with Spatially Varying Exposures Considering Hue Distortion


Title	Single-Shot High Dynamic Range Imaging with Spatially Varying Exposures Considering Hue Distortion
Authors	Chihiro Go, Yuma Kinoshita, Sayaka Shiota, Hitoshi Kiya
Abstract	We proposes a novel single-shot high dynamic range imaging scheme with spatially varying exposures (SVE) considering hue distortion. Single-shot imaging with SVE enables us to capture multi-exposure images from a single-shot image, so high dynamic range images can be produced without ghost artifacts. However, SVE images have some pixels at which a range supported by camera sensors is exceeded. Therefore, generated images have some color distortion, so that conventional imaging with SVE has never considered the influence of this range limitation. To overcome this issue, we consider estimating the correct hue of a scene from raw images, and propose a method with the estimated hue information for correcting the hue of SVE images on the constant hue plain in the RGB color space.
Tasks
Published	2019-08-01
URL	https://arxiv.org/abs/1908.00186v1
PDF	https://arxiv.org/pdf/1908.00186v1.pdf
PWC	https://paperswithcode.com/paper/single-shot-high-dynamic-range-imaging-with
Repo
Framework

Are Query-Based Ontology Debuggers Really Helping Knowledge Engineers?


Title	Are Query-Based Ontology Debuggers Really Helping Knowledge Engineers?
Authors	Patrick Rodler, Dietmar Jannach, Konstantin Schekotihin, Philipp Fleiss
Abstract	Real-world semantic or knowledge-based systems, e.g., in the biomedical domain, can become large and complex. Tool support for the localization and repair of faults within knowledge bases of such systems can therefore be essential for their practical success. Correspondingly, a number of knowledge base debugging approaches, in particular for ontology-based systems, were proposed throughout recent years. Query-based debugging is a comparably recent interactive approach that localizes the true cause of an observed problem by asking knowledge engineers a series of questions. Concrete implementations of this approach exist, such as the OntoDebug plug-in for the ontology editor Prot'eg'e. To validate that a newly proposed method is favorable over an existing one, researchers often rely on simulation-based comparisons. Such an evaluation approach however has certain limitations and often cannot fully inform us about a method’s true usefulness. We therefore conducted different user studies to assess the practical value of query-based ontology debugging. One main insight from the studies is that the considered interactive approach is indeed more efficient than an alternative algorithmic debugging based on test cases. We also observed that users frequently made errors in the process, which highlights the importance of a careful design of the queries that users need to answer.
Tasks
Published	2019-04-02
URL	http://arxiv.org/abs/1904.01484v1
PDF	http://arxiv.org/pdf/1904.01484v1.pdf
PWC	https://paperswithcode.com/paper/are-query-based-ontology-debuggers-really
Repo
Framework