Paper Group ANR 287
THUEE system description for NIST 2019 SRE CTS Challenge. Certifiable Robustness to Graph Perturbations. Evolving Neural Networks in Reinforcement Learning by means of UMDAc. Practical applicability of deep neural networks for overlapping speaker separation. Auditory Separation of a Conversation from Background via Attentional Gating. Optimal query …
THUEE system description for NIST 2019 SRE CTS Challenge
| Title | THUEE system description for NIST 2019 SRE CTS Challenge |
| Authors | Yi Liu, Tianyu Liang, Can Xu, Xianwei Zhang, Xianhong Chen, Wei-Qiang Zhang, Liang He, Dandan song, Ruyun Li, Yangcheng Wu, Peng Ouyang, Shouyi Yin |
| Abstract | This paper describes the systems submitted by the department of electronic engineering, institute of microelectronics of Tsinghua university and TsingMicro Co. Ltd. (THUEE) to the NIST 2019 speaker recognition evaluation CTS challenge. Six subsystems, including etdnn/ams, ftdnn/as, eftdnn/ams, resnet, multitask and c-vector are developed in this evaluation. |
| Tasks | Speaker Recognition |
| Published | 2019-12-25 |
| URL | https://arxiv.org/abs/1912.11585v1 |
| https://arxiv.org/pdf/1912.11585v1.pdf | |
| PWC | https://paperswithcode.com/paper/thuee-system-description-for-nist-2019-sre |
| Repo | |
| Framework | |
Certifiable Robustness to Graph Perturbations
| Title | Certifiable Robustness to Graph Perturbations |
| Authors | Aleksandar Bojchevski, Stephan Günnemann |
| Abstract | Despite the exploding interest in graph neural networks there has been little effort to verify and improve their robustness. This is even more alarming given recent findings showing that they are extremely vulnerable to adversarial attacks on both the graph structure and the node attributes. We propose the first method for verifying certifiable (non-)robustness to graph perturbations for a general class of models that includes graph neural networks and label/feature propagation. By exploiting connections to PageRank and Markov decision processes our certificates can be efficiently (and under many threat models exactly) computed. Furthermore, we investigate robust training procedures that increase the number of certifiably robust nodes while maintaining or improving the clean predictive accuracy. |
| Tasks | |
| Published | 2019-10-31 |
| URL | https://arxiv.org/abs/1910.14356v2 |
| https://arxiv.org/pdf/1910.14356v2.pdf | |
| PWC | https://paperswithcode.com/paper/certifiable-robustness-to-graph-perturbations |
| Repo | |
| Framework | |
Evolving Neural Networks in Reinforcement Learning by means of UMDAc
| Title | Evolving Neural Networks in Reinforcement Learning by means of UMDAc |
| Authors | Mikel Malagon, Josu Ceberio |
| Abstract | Neural networks are gaining popularity in the reinforcement learning field due to the vast number of successfully solved complex benchmark problems. In fact, artificial intelligence algorithms are, in some cases, able to overcome human professionals. Usually, neural networks have more than a couple of hidden layers, and thus, they involve a large quantity of parameters that need to be optimized. Commonly, numeric approaches are used to optimize the inner parameters of neural networks, such as the stochastic gradient descent. However, these techniques tend to be computationally very expensive, and for some tasks, where effectiveness is crucial, high computational costs are not acceptable. Along these research lines, in this paper we propose to optimize the parameters of neural networks by means of estimation of distribution algorithms. More precisely, the univariate marginal distribution algorithm is used for evolving neural networks in various reinforcement learning tasks. For the sake of validating our idea, we run the proposed algorithm on four OpenAI Gym benchmark problems. In addition, the obtained results were compared with a standard genetic algorithm. Revealing, that optimizing with UMDAc provides better results than the genetic algorithm in most of the cases. |
| Tasks | |
| Published | 2019-04-24 |
| URL | http://arxiv.org/abs/1904.10932v1 |
| http://arxiv.org/pdf/1904.10932v1.pdf | |
| PWC | https://paperswithcode.com/paper/evolving-neural-networks-in-reinforcement |
| Repo | |
| Framework | |
Practical applicability of deep neural networks for overlapping speaker separation
| Title | Practical applicability of deep neural networks for overlapping speaker separation |
| Authors | Pieter Appeltans, Jeroen Zegers, Hugo Van hamme |
| Abstract | This paper examines the applicability in realistic scenarios of two deep learning based solutions to the overlapping speaker separation problem. Firstly, we present experiments that show that these methods are applicable for a broad range of languages. Further experimentation indicates limited performance loss for untrained languages, when these have common features with the trained language(s). Secondly, it investigates how the methods deal with realistic background noise and proposes some modifications to better cope with these disturbances. The deep learning methods that will be examined are deep clustering and deep attractor networks. |
| Tasks | Speaker Separation |
| Published | 2019-12-19 |
| URL | https://arxiv.org/abs/1912.09261v1 |
| https://arxiv.org/pdf/1912.09261v1.pdf | |
| PWC | https://paperswithcode.com/paper/practical-applicability-of-deep-neural |
| Repo | |
| Framework | |
Auditory Separation of a Conversation from Background via Attentional Gating
| Title | Auditory Separation of a Conversation from Background via Attentional Gating |
| Authors | Shariq Mobin, Bruno Olshausen |
| Abstract | We present a model for separating a set of voices out of a sound mixture containing an unknown number of sources. Our Attentional Gating Network (AGN) uses a variable attentional context to specify which speakers in the mixture are of interest. The attentional context is specified by an embedding vector which modifies the processing of a neural network through an additive bias. Individual speaker embeddings are learned to separate a single speaker while superpositions of the individual speaker embeddings are used to separate sets of speakers. We first evaluate AGN on a traditional single speaker separation task and show an improvement of 9% with respect to comparable models. Then, we introduce a new task to separate an arbitrary subset of voices from a mixture of an unknown-sized set of voices, inspired by the human ability to separate a conversation of interest from background chatter at a cafeteria. We show that AGN is the only model capable of solving this task, performing only 7% worse than on the single speaker separation task. |
| Tasks | Speaker Separation |
| Published | 2019-05-26 |
| URL | https://arxiv.org/abs/1905.10751v1 |
| https://arxiv.org/pdf/1905.10751v1.pdf | |
| PWC | https://paperswithcode.com/paper/auditory-separation-of-a-conversation-from |
| Repo | |
| Framework | |
Optimal query complexity for private sequential learning
| Title | Optimal query complexity for private sequential learning |
| Authors | Jiaming Xu, Dana Yang |
| Abstract | Motivated by privacy concerns in many practical applications such as Federated Learning, we study a stylized private sequential learning problem: a learner tries to estimate an unknown scalar value, by sequentially querying an external database and receiving binary responses; meanwhile, a third-party adversary observes the learner’s queries but not the responses. The learner’s goal is to design a querying strategy with the minimum number of queries (optimal query complexity) so that she can accurately estimate the true value, while the adversary even with the complete knowledge of her querying strategy cannot. Prior work has obtained both upper and lower bounds on the optimal query complexity, however, these upper and lower bounds have a large gap in general. In this paper, we construct new querying strategies and prove almost matching upper and lower bounds, providing a complete characterization of the optimal query complexity as a function of the estimation accuracy and the desired levels of privacy. |
| Tasks | |
| Published | 2019-09-21 |
| URL | https://arxiv.org/abs/1909.09836v1 |
| https://arxiv.org/pdf/1909.09836v1.pdf | |
| PWC | https://paperswithcode.com/paper/190909836 |
| Repo | |
| Framework | |
An End-to-end Approach for Lexical Stress Detection based on Transformer
| Title | An End-to-end Approach for Lexical Stress Detection based on Transformer |
| Authors | Yong Ruan, Xiangdong Wang, Hong Liu, Zhigang Ou, Yun Gao, Jianfeng Cheng, Yueliang Qian |
| Abstract | The dominant automatic lexical stress detection method is to split the utterance into syllable segments using phoneme sequence and their time-aligned boundaries. Then we extract features from syllable to use classification method to classify the lexical stress. However, we can’t get very accurate time boundaries of each phoneme and we have to design some features in the syllable segments to classify the lexical stress. Therefore, we propose a end-to-end approach using sequence to sequence model of transformer to estimate lexical stress. For this, we train transformer model using feature sequence of audio and their phoneme sequence with lexical stress marks. During the recognition process, the recognized phoneme sequence is restricted according to the original standard phoneme sequence without lexical stress marks, but the lexical stress mark of each phoneme is not limited. We train the model in different subset of Librispeech and do lexical stress recognition in TIMIT and L2-ARCTIC dataset. For all subsets, the end-to-end model will perform better than the syllable segments classification method. Our method can achieve a 6.36% phoneme error rate on the TIMIT dataset, which exceeds the 7.2% error rate in other studies. |
| Tasks | |
| Published | 2019-11-06 |
| URL | https://arxiv.org/abs/1911.04862v1 |
| https://arxiv.org/pdf/1911.04862v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-end-to-end-approach-for-lexical-stress |
| Repo | |
| Framework | |
Farkas layers: don’t shift the data, fix the geometry
| Title | Farkas layers: don’t shift the data, fix the geometry |
| Authors | Aram-Alexandre Pooladian, Chris Finlay, Adam M Oberman |
| Abstract | Successfully training deep neural networks often requires either batch normalization, appropriate weight initialization, both of which come with their own challenges. We propose an alternative, geometrically motivated method for training. Using elementary results from linear programming, we introduce Farkas layers: a method that ensures at least one neuron is active at a given layer. Focusing on residual networks with ReLU activation, we empirically demonstrate a significant improvement in training capacity in the absence of batch normalization or methods of initialization across a broad range of network sizes on benchmark datasets. |
| Tasks | |
| Published | 2019-10-04 |
| URL | https://arxiv.org/abs/1910.02840v1 |
| https://arxiv.org/pdf/1910.02840v1.pdf | |
| PWC | https://paperswithcode.com/paper/farkas-layers-dont-shift-the-data-fix-the |
| Repo | |
| Framework | |
Jointly Sparse Convolutional Neural Networks in Dual Spatial-Winograd Domains
| Title | Jointly Sparse Convolutional Neural Networks in Dual Spatial-Winograd Domains |
| Authors | Yoojin Choi, Mostafa El-Khamy, Jungwon Lee |
| Abstract | We consider the optimization of deep convolutional neural networks (CNNs) such that they provide good performance while having reduced complexity if deployed on either conventional systems with spatial-domain convolution or lower-complexity systems designed for Winograd convolution. The proposed framework produces one compressed model whose convolutional filters can be made sparse either in the spatial domain or in the Winograd domain. Hence, the compressed model can be deployed universally on any platform, without need for re-training on the deployed platform. To get a better compression ratio, the sparse model is compressed in the spatial domain that has a fewer number of parameters. From our experiments, we obtain $24.2\times$ and $47.7\times$ compressed models for ResNet-18 and AlexNet trained on the ImageNet dataset, while their computational cost is also reduced by $4.5\times$ and $5.1\times$, respectively. |
| Tasks | |
| Published | 2019-02-21 |
| URL | http://arxiv.org/abs/1902.08192v1 |
| http://arxiv.org/pdf/1902.08192v1.pdf | |
| PWC | https://paperswithcode.com/paper/jointly-sparse-convolutional-neural-networks |
| Repo | |
| Framework | |
A Neural Attention Model for Adaptive Learning of Social Friends’ Preferences
| Title | A Neural Attention Model for Adaptive Learning of Social Friends’ Preferences |
| Authors | Dimitrios Rafailidis, Gerhard Weiss |
| Abstract | Social-based recommendation systems exploit the selections of friends to combat the data sparsity on user preferences, and improve the recommendation accuracy of the collaborative filtering strategy. The main challenge is to capture and weigh friends’ preferences, as in practice they do necessarily match. In this paper, we propose a Neural Attention mechanism for Social collaborative filtering, namely NAS. We design a neural architecture, to carefully compute the non-linearity in friends’ preferences by taking into account the social latent effects of friends on user behavior. In addition, we introduce a social behavioral attention mechanism to adaptively weigh the influence of friends on user preferences and consequently generate accurate recommendations. Our experiments on publicly available datasets demonstrate the effectiveness of the proposed NAS model over other state-of-the-art methods. Furthermore, we study the effect of the proposed social behavioral attention mechanism and show that it is a key factor to our model’s performance. |
| Tasks | Recommendation Systems |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.01644v1 |
| https://arxiv.org/pdf/1907.01644v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-neural-attention-model-for-adaptive |
| Repo | |
| Framework | |
Error-Correcting Neural Network
| Title | Error-Correcting Neural Network |
| Authors | Yang Song, Qiyu Kang, Wee Peng Tay |
| Abstract | Error-correcting output codes (ECOC) is an ensemble method combining a set of binary classifiers for multi-class learning problems. However, in traditional ECOC framework, the binary classifiers are trained independently. To explore the interaction between the binary classifiers, we construct an error correction network (ECN) that jointly trains all binary classifiers while maximizing the ensemble diversity to improve its robustness against adversarial attacks. An ECN is built based on a code matrix which is generated by maximizing the error tolerance, i.e., the minimum Hamming distance between any two rows, as well as the ensemble diversity, i.e., the variation of information between any two columns. Though ECN inherently promotes the diversity between the binary classifiers as each ensemble member solves a different classification problem (specified by the corresponding column of the code matrix), we empirically show that the ensemble diversity can be further improved by forcing the weight matrices learned by ensemble members to be orthogonal. The ECN is trained in end-to-end fashion and can be complementary to other defense approaches including adversarial training. We show empirically that ECN is effective against the state-of-the-art while-box attacks while maintaining good accuracy on normal examples. |
| Tasks | |
| Published | 2019-11-30 |
| URL | https://arxiv.org/abs/1912.00181v1 |
| https://arxiv.org/pdf/1912.00181v1.pdf | |
| PWC | https://paperswithcode.com/paper/error-correcting-neural-network |
| Repo | |
| Framework | |
Stochastic gradient descent for hybrid quantum-classical optimization
| Title | Stochastic gradient descent for hybrid quantum-classical optimization |
| Authors | Ryan Sweke, Frederik Wilde, Johannes Meyer, Maria Schuld, Paul K. Fährmann, Barthélémy Meynard-Piganeau, Jens Eisert |
| Abstract | Within the context of hybrid quantum-classical optimization, gradient descent based optimizers typically require the evaluation of expectation values with respect to the outcome of parameterized quantum circuits. In this work, we explore the significant consequences of the simple observation that the estimation of these quantities on quantum hardware is a form of stochastic gradient descent optimization. We formalize this notion, which allows us to show that in many relevant cases, including VQE, QAOA and certain quantum classifiers, estimating expectation values with $k$ measurement outcomes results in optimization algorithms whose convergence properties can be rigorously well understood, for any value of $k$. In fact, even using single measurement outcomes for the estimation of expectation values is sufficient. Moreover, in many settings the required gradients can be expressed as linear combinations of expectation values – originating, e.g., from a sum over local terms of a Hamiltonian, a parameter shift rule, or a sum over data-set instances – and we show that in these cases $k$-shot expectation value estimation can be combined with sampling over terms of the linear combination, to obtain “doubly stochastic” gradient descent optimizers. For all algorithms we prove convergence guarantees, providing a framework for the derivation of rigorous optimization results in the context of near-term quantum devices. Additionally, we explore numerically these methods on benchmark VQE, QAOA and quantum-enhanced machine learning tasks and show that treating the stochastic settings as hyper-parameters allows for state-of-the-art results with significantly fewer circuit executions and measurements. |
| Tasks | |
| Published | 2019-10-02 |
| URL | https://arxiv.org/abs/1910.01155v1 |
| https://arxiv.org/pdf/1910.01155v1.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-gradient-descent-for-hybrid |
| Repo | |
| Framework | |
RTFM: Generalising to Novel Environment Dynamics via Reading
| Title | RTFM: Generalising to Novel Environment Dynamics via Reading |
| Authors | Victor Zhong, Tim Rocktäschel, Edward Grefenstette |
| Abstract | Obtaining policies that can generalise to new environments in reinforcement learning is challenging. In this work, we demonstrate that language understanding via a reading policy learner is a promising vehicle for generalisation to new environments. We propose a grounded policy learning problem, Read to Fight Monsters (RTFM), in which the agent must jointly reason over a language goal, relevant dynamics described in a document, and environment observations. We procedurally generate environment dynamics and corresponding language descriptions of the dynamics, such that agents must read to understand new environment dynamics instead of memorising any particular information. In addition, we propose txt2$\pi$, a model that captures three-way interactions between the goal, document, and observations. On RTFM, txt2$\pi$ generalises to new environments with dynamics not seen during training via reading. Furthermore, our model outperforms baselines such as FiLM and language-conditioned CNNs on RTFM. Through curriculum learning, txt2$\pi$ produces policies that excel on complex RTFM tasks requiring several reasoning and coreference steps. |
| Tasks | |
| Published | 2019-10-18 |
| URL | https://arxiv.org/abs/1910.08210v5 |
| https://arxiv.org/pdf/1910.08210v5.pdf | |
| PWC | https://paperswithcode.com/paper/rtfm-generalising-to-novel-environment |
| Repo | |
| Framework | |
Challenges in Time-Stamp Aware Anomaly Detection in Traffic Videos
| Title | Challenges in Time-Stamp Aware Anomaly Detection in Traffic Videos |
| Authors | Kuldeep Marotirao Biradar, Ayushi Gupta, Murari Mandal, Santosh Kumar Vipparthi |
| Abstract | Time-stamp aware anomaly detection in traffic videos is an essential task for the advancement of the intelligent transportation system. Anomaly detection in videos is a challenging problem due to sparse occurrence of anomalous events, inconsistent behavior of a different type of anomalies and imbalanced available data for normal and abnormal scenarios. In this paper, we present a three-stage pipeline to learn the motion patterns in videos to detect a visual anomaly. First, the background is estimated from recent history frames to identify the motionless objects. This background image is used to localize the normal/abnormal behavior within the frame. Further, we detect an object of interest in the estimated background and categorize it into anomaly based on a time-stamp aware anomaly detection algorithm. We also discuss the challenges faced in improving performance over the unseen test data for traffic anomaly detection. Experiments are conducted over Track 3 of NVIDIA AI city challenge 2019. The results show the effectiveness of the proposed method in detecting time-stamp aware anomalies in traffic/road videos. |
| Tasks | Anomaly Detection |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.04574v1 |
| https://arxiv.org/pdf/1906.04574v1.pdf | |
| PWC | https://paperswithcode.com/paper/challenges-in-time-stamp-aware-anomaly |
| Repo | |
| Framework | |
Stochastic Approximation Algorithms for Principal Component Analysis
| Title | Stochastic Approximation Algorithms for Principal Component Analysis |
| Authors | Jian Vora |
| Abstract | Principal Component Analysis is a novel way of of dimensionality reduction. This problem essentially boils down to finding the top k eigen vectors of the data covariance matrix. A considerable amount of literature is found on algorithms meant to do so such as an online method be Warmuth and Kuzmin, Matrix Stochastic Gradient by Arora, Oja’s method and many others. In this paper we see some of these stochastic approaches to the PCA optimization problem and comment on their convergence and runtime to obtain an epsilon sub-optimal solution. We revisit convex relaxation based methods for stochastic optimization of principal component analysis. While methods that directly solve the non convex problem have been shown to be optimal in terms of statistical and computational efficiency, the methods based on convex relaxation have been shown to enjoy comparable, or even superior, empirical performance. This motivates the need for a deeper formal understanding of the latter. |
| Tasks | Dimensionality Reduction, Stochastic Optimization |
| Published | 2019-01-07 |
| URL | http://arxiv.org/abs/1901.01798v1 |
| http://arxiv.org/pdf/1901.01798v1.pdf | |
| PWC | https://paperswithcode.com/paper/stochastic-approximation-algorithms-for |
| Repo | |
| Framework | |
