Paper Group ANR 458
On the regularity and conditioning of low rank semidefinite programs. Finiteness of fibers in matrix completion via Plücker coordinates. Optimal Adaptive Matrix Completion. Convolutional Hierarchical Attention Network for Query-Focused Video Summarization. High Performance Sequence-to-Sequence Model for Streaming Speech Recognition. Style Transfer …
On the regularity and conditioning of low rank semidefinite programs
| Title | On the regularity and conditioning of low rank semidefinite programs |
| Authors | Lijun Ding, Madeleine Udell |
| Abstract | Low rank matrix recovery problems appear widely in statistics, combinatorics, and imaging. One celebrated method for solving these problems is to formulate and solve a semidefinite program (SDP). It is often known that the exact solution to the SDP with perfect data recovers the solution to the original low rank matrix recovery problem. It is more challenging to show that an approximate solution to the SDP formulated with noisy problem data acceptably solves the original problem; arguments are usually ad hoc for each problem setting, and can be complex. In this note, we identify a set of conditions that we call regularity that limit the error due to noisy problem data or incomplete convergence. In this sense, regular SDPs are robust: regular SDPs can be (approximately) solved efficiently at scale; and the resulting approximate solutions, even with noisy data, can be trusted. Moreover, we show that regularity holds generically, and also for many structured low rank matrix recovery problems, including the stochastic block model, $\mathbb{Z}_2$ synchronization, and matrix completion. Formally, we call an SDP regular if it has a surjective constraint map, admits a unique primal and dual solution pair, and satisfies strong duality and strict complementarity. However, regularity is not a panacea: we show the Burer-Monteiro formulation of the SDP may have spurious second-order critical points, even for a regular SDP with a rank 1 solution. |
| Tasks | Matrix Completion |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.10673v1 |
| https://arxiv.org/pdf/2002.10673v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-regularity-and-conditioning-of-low |
| Repo | |
| Framework | |
Finiteness of fibers in matrix completion via Plücker coordinates
| Title | Finiteness of fibers in matrix completion via Plücker coordinates |
| Authors | Manolis C. Tsakiris |
| Abstract | Let $\Omega \subseteq {1,\dots,m} \times {1,\dots,n}$. We consider fibers of coordinate projections $\pi_\Omega : \mathscr{M}_k(r,m \times n) \rightarrow k^{# \Omega}$ from the algebraic variety of $m \times n$ matrices of rank at most $r$ over an infinite field $k$. For $#\Omega = \dim \mathscr{M}_k(r,m \times n)$ we describe a class of $\Omega$'s for which there exist non-empty Zariski open sets $\mathscr{U}_\Omega \subset \mathscr{M}_k(r,m \times n)$ such that $\pi_\Omega^{-1}\big(\pi_\Omega(X)\big) \cap \mathscr{U}_\Omega$ is a finite set $\forall X \in \mathscr{U}_\Omega$. For this we interpret matrix completion from a point of view of hyperplane sections on the Grassmannian $\operatorname{Gr}(r,m)$. Crucial is a description by Sturmfels $&$ Zelevinsky of classes of local coordinates on $\operatorname{Gr}(r,m)$ induced by vertices of the Newton polytope of the product of maximal minors of an $m \times (m-r)$ matrix of variables. |
| Tasks | Matrix Completion |
| Published | 2020-02-12 |
| URL | https://arxiv.org/abs/2002.05082v2 |
| https://arxiv.org/pdf/2002.05082v2.pdf | |
| PWC | https://paperswithcode.com/paper/finiteness-of-fibers-in-matrix-completion-via |
| Repo | |
| Framework | |
Optimal Adaptive Matrix Completion
| Title | Optimal Adaptive Matrix Completion |
| Authors | Ilqar ramazanli, Barnabas Poczos |
| Abstract | We study the problem of exact completion for $m \times n$ sized matrix of rank r with the adaptive sampling method. We introduce a relation of the exact completion problem with the sparsest vector of column and row spaces (which we call sparsity-number here). Using this relation, we propose matrix completion algorithms that exactly recovers the target matrix. These algorithms are superior to previous works in two important ways. First, our algorithms exactly recover $\mu_0$-coherent column space matrices by probability at least $1-\epsilon$ using much smaller observations complexity than - $\mathcal{O}(\mu_0 rn \mathrm{log}\frac{r}{\epsilon})$ - the state of art. Specifically, many of the previous adaptive sampling methods require to observe the entire matrix when the column space is highly coherent. However, we show that our method is still able to recover this type of matrices by observing a small fraction of entries under many scenarios. Second, we propose an exact completion algorithm, which requires minimal pre-information as either row or column space is not being highly coherent. We provide an extension of these algorithms that is robust to sparse random noise. Besides, we propose an additional low-rank estimation algorithm that is robust to any small noise by adaptively studying the shape of column space. At the end of the paper, we provide experimental results that illustrate the strength of the algorithms proposed here. |
| Tasks | Matrix Completion |
| Published | 2020-02-06 |
| URL | https://arxiv.org/abs/2002.02431v1 |
| https://arxiv.org/pdf/2002.02431v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-adaptive-matrix-completion |
| Repo | |
| Framework | |
Convolutional Hierarchical Attention Network for Query-Focused Video Summarization
| Title | Convolutional Hierarchical Attention Network for Query-Focused Video Summarization |
| Authors | Shuwen Xiao, Zhou Zhao, Zijian Zhang, Xiaohui Yan, Min Yang |
| Abstract | Previous approaches for video summarization mainly concentrate on finding the most diverse and representative visual contents as video summary without considering the user’s preference. This paper addresses the task of query-focused video summarization, which takes user’s query and a long video as inputs and aims to generate a query-focused video summary. In this paper, we consider the task as a problem of computing similarity between video shots and query. To this end, we propose a method, named Convolutional Hierarchical Attention Network (CHAN), which consists of two parts: feature encoding network and query-relevance computing module. In the encoding network, we employ a convolutional network with local self-attention mechanism and query-aware global attention mechanism to learns visual information of each shot. The encoded features will be sent to query-relevance computing module to generate queryfocused video summary. Extensive experiments on the benchmark dataset demonstrate the competitive performance and show the effectiveness of our approach. |
| Tasks | Video Summarization |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2002.03740v3 |
| https://arxiv.org/pdf/2002.03740v3.pdf | |
| PWC | https://paperswithcode.com/paper/convolutional-hierarchical-attention-network |
| Repo | |
| Framework | |
High Performance Sequence-to-Sequence Model for Streaming Speech Recognition
| Title | High Performance Sequence-to-Sequence Model for Streaming Speech Recognition |
| Authors | Thai-Son Nguyen, Ngoc-Quan Pham, Sebastian Stueker, Alex Waibel |
| Abstract | Recently sequence-to-sequence models have started to achieve state-of-the art performance on standard speech recognition tasks when processing audio data in batch mode, i.e., the complete audio data is available when starting processing. However, when it comes to perform run-on recognition on an input stream of audio data while producing recognition results in real-time and with a low word-based latency, these models face several challenges. For many techniques, the whole audio sequence to be decoded needs to be available at the start of the processing, e.g., for the attention mechanism or for the bidirectional LSTM (BLSTM). In this paper we propose several techniques to mitigate these problems. We introduce an additional loss function controlling the uncertainty of the attention mechanism, a modified beam search identifying partial, stable hypotheses, ways of working with BLSTM in the encoder, and the use of chunked BLSTM. Our experiments show that with the right combination of these techniques it is possible to perform run-on speech recognition with a low word-based latency without sacrificing performance in terms of word error rate. |
| Tasks | Speech Recognition |
| Published | 2020-03-22 |
| URL | https://arxiv.org/abs/2003.10022v1 |
| https://arxiv.org/pdf/2003.10022v1.pdf | |
| PWC | https://paperswithcode.com/paper/high-performance-sequence-to-sequence-model |
| Repo | |
| Framework | |
Style Transfer for Light Field Photography
| Title | Style Transfer for Light Field Photography |
| Authors | David Hart, Jessica Greenland, Bryan Morse |
| Abstract | As light field images continue to increase in use and application, it becomes necessary to adapt existing image processing methods to this unique form of photography. In this paper we explore methods for applying neural style transfer to light field images. Feed-forward style transfer networks provide fast, high-quality results for monocular images, but no such networks exist for full light field images. Because of the size of these images, current light field data sets are small and are insufficient for training purely feed-forward style-transfer networks from scratch. Thus, it is necessary to adapt existing monocular style transfer networks in a way that allows for the stylization of each view of the light field while maintaining visual consistencies between views. Instead, the proposed method backpropagates the loss through the network, and the process is iterated to optimize (essentially overfit) the resulting stylization for a single light field image alone. The network architecture allows for the incorporation of pre-trained fast monocular stylization networks while avoiding the need for a large light field training set. |
| Tasks | Style Transfer |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.11220v1 |
| https://arxiv.org/pdf/2002.11220v1.pdf | |
| PWC | https://paperswithcode.com/paper/style-transfer-for-light-field-photography |
| Repo | |
| Framework | |
Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning
| Title | Parallel Knowledge Transfer in Multi-Agent Reinforcement Learning |
| Authors | Yongyuan Liang, Bangwei Li |
| Abstract | Multi-agent reinforcement learning is a standard framework for modeling multi-agent interactions applied in real-world scenarios. Inspired by experience sharing in human groups, learning knowledge parallel reusing between agents can potentially promote team learning performance, especially in multi-task environments. When all agents interact with the environment and learn simultaneously, how each independent agent selectively learns from other agents’ behavior knowledge is a problem that we need to solve. This paper proposes a novel knowledge transfer framework in MARL, PAT (Parallel Attentional Transfer). We design two acting modes in PAT, student mode and self-learning mode. Each agent in our approach trains a decentralized student actor-critic to determine its acting mode at each time step. When agents are unfamiliar with the environment, the shared attention mechanism in student mode effectively selects learning knowledge from other agents to decide agents’ actions. PAT outperforms state-of-the-art empirical evaluation results against the prior advising approaches. Our approach not only significantly improves team learning rate and global performance, but also is flexible and transferable to be applied in various multi-agent systems. |
| Tasks | Multi-agent Reinforcement Learning, Transfer Learning |
| Published | 2020-03-29 |
| URL | https://arxiv.org/abs/2003.13085v1 |
| https://arxiv.org/pdf/2003.13085v1.pdf | |
| PWC | https://paperswithcode.com/paper/parallel-knowledge-transfer-in-multi-agent |
| Repo | |
| Framework | |
Unsupervised Adaptive Neural Network Regularization for Accelerated Radial Cine MRI
| Title | Unsupervised Adaptive Neural Network Regularization for Accelerated Radial Cine MRI |
| Authors | Andreas Kofler, Marc Dewey, Tobias Schaeffter, Christoph Kolbitsch, Markus Haltmeier |
| Abstract | In this work, we propose an iterative reconstruction scheme (ALONE - Adaptive Learning Of NEtworks) for 2D radial cine MRI based on ground truth-free unsupervised learning of shallow convolutional neural networks. The network is trained to approximate patches of the current estimate of the solution during the reconstruction. By imposing a shallow network topology and constraining the $L_2$-norm of the learned filters, the network’s representation power is limited in order not to be able to recover noise. Therefore, the network can be interpreted to perform a low dimensional approximation of the patches for stabilizing the inversion process. We compare the proposed reconstruction scheme to two ground truth-free reconstruction methods, namely a well known Total Variation (TV) minimization and an unsupervised adaptive Dictionary Learning (DIC) method. The proposed method outperforms both methods with respect to all reported quantitative measures. Further, in contrast to DIC, where the sparse approximation of the patches involves the solution of a complex optimization problem, ALONE only requires a forward pass of all patches through the shallow network and therefore significantly accelerates the reconstruction. |
| Tasks | Dictionary Learning |
| Published | 2020-02-10 |
| URL | https://arxiv.org/abs/2002.03820v1 |
| https://arxiv.org/pdf/2002.03820v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-adaptive-neural-network |
| Repo | |
| Framework | |
Learning efficient structured dictionary for image classification
| Title | Learning efficient structured dictionary for image classification |
| Authors | Zi-Qi Li, Jun Sun, Xiao-Jun Wu, He-Feng Yin |
| Abstract | Recent years have witnessed the success of dictionary learning (DL) based approaches in the domain of pattern classification. In this paper, we present an efficient structured dictionary learning (ESDL) method which takes both the diversity and label information of training samples into account. Specifically, ESDL introduces alternative training samples into the process of dictionary learning. To increase the discriminative capability of representation coefficients for classification, an ideal regularization term is incorporated into the objective function of ESDL. Moreover, in contrast with conventional DL approaches which impose computationally expensive L1-norm constraint on the coefficient matrix, ESDL employs L2-norm regularization term. Experimental results on benchmark databases (including four face databases and one scene dataset) demonstrate that ESDL outperforms previous DL approaches. More importantly, ESDL can be applied in a wide range of pattern classification tasks. The demo code of our proposed ESDL will be available at https://github.com/li-zi-qi/ESDL. |
| Tasks | Dictionary Learning, Image Classification |
| Published | 2020-02-09 |
| URL | https://arxiv.org/abs/2002.03271v1 |
| https://arxiv.org/pdf/2002.03271v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-efficient-structured-dictionary-for |
| Repo | |
| Framework | |
Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge
| Title | Detecting Asks in SE attacks: Impact of Linguistic and Structural Knowledge |
| Authors | Bonnie J. Dorr, Archna Bhatia, Adam Dalton, Brodie Mather, Bryanna Hebenstreit, Sashank Santhanam, Zhuo Cheng, Samira Shaikh, Alan Zemel, Tomek Strzalkowski |
| Abstract | Social engineers attempt to manipulate users into undertaking actions such as downloading malware by clicking links or providing access to money or sensitive information. Natural language processing, computational sociolinguistics, and media-specific structural clues provide a means for detecting both the ask (e.g., buy gift card) and the risk/reward implied by the ask, which we call framing (e.g., lose your job, get a raise). We apply linguistic resources such as Lexical Conceptual Structure to tackle ask detection and also leverage structural clues such as links and their proximity to identified asks to improve confidence in our results. Our experiments indicate that the performance of ask detection, framing detection, and identification of the top ask is improved by linguistically motivated classes coupled with structural clues such as links. Our approach is implemented in a system that informs users about social engineering risk situations. |
| Tasks | |
| Published | 2020-02-25 |
| URL | https://arxiv.org/abs/2002.10931v1 |
| https://arxiv.org/pdf/2002.10931v1.pdf | |
| PWC | https://paperswithcode.com/paper/detecting-asks-in-se-attacks-impact-of |
| Repo | |
| Framework | |
Towards a Framework for Certification of Reliable Autonomous Systems
| Title | Towards a Framework for Certification of Reliable Autonomous Systems |
| Authors | Michael Fisher, Viviana Mascardi, Kristin Yvonne Rozier, Bernd-Holger Schlingloff, Michael Winikoff, Neil Yorke-Smith |
| Abstract | A computational system is called autonomous if it is able to make its own decisions, or take its own actions, without human supervision or control. The capability and spread of such systems have reached the point where they are beginning to touch much of everyday life. However, regulators grapple with how to deal with autonomous systems, for example how could we certify an Unmanned Aerial System for autonomous use in civilian airspace? We here analyse what is needed in order to provide verified reliable behaviour of an autonomous system, analyse what can be done as the state-of-the-art in automated verification, and propose a roadmap towards developing regulatory guidelines, including articulating challenges to researchers, to engineers, and to regulators. Case studies in seven distinct domains illustrate the article. |
| Tasks | |
| Published | 2020-01-24 |
| URL | https://arxiv.org/abs/2001.09124v1 |
| https://arxiv.org/pdf/2001.09124v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-a-framework-for-certification-of |
| Repo | |
| Framework | |
Cluster-based Zero-shot learning for multivariate data
| Title | Cluster-based Zero-shot learning for multivariate data |
| Authors | Toshitaka Hayashi, Hamido Fujita |
| Abstract | Supervised learning requires a sufficient training dataset which includes all label. However, there are cases that some class is not in the training data. Zero-Shot Learning (ZSL) is the task of predicting class that is not in the training data(target class). The existing ZSL method is done for image data. However, the zero-shot problem should happen to every data type. Hence, considering ZSL for other data types is required. In this paper, we propose the cluster-based ZSL method, which is a baseline method for multivariate binary classification problems. The proposed method is based on the assumption that if data is far from training data, the data is considered as target class. In training, clustering is done for training data. In prediction, the data is determined belonging to a cluster or not. If data does not belong to a cluster, the data is predicted as target class. The proposed method is evaluated and demonstrated using the KEEL dataset. |
| Tasks | Zero-Shot Learning |
| Published | 2020-01-16 |
| URL | https://arxiv.org/abs/2001.05624v2 |
| https://arxiv.org/pdf/2001.05624v2.pdf | |
| PWC | https://paperswithcode.com/paper/cluster-based-zero-shot-learning-for |
| Repo | |
| Framework | |
Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation
| Title | Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation |
| Authors | Zhenda Xie, Zheng Zhang, Xizhou Zhu, Gao Huang, Stephen Lin |
| Abstract | In the feature maps of CNNs, there commonly exists considerable spatial redundancy that leads to much repetitive processing. Towards reducing this superfluous computation, we propose to compute features only at sparsely sampled locations, which are probabilistically chosen according to activation responses, and then densely reconstruct the feature map with an efficient interpolation procedure. With this sampling-interpolation scheme, our network avoids expending computation on spatial locations that can be effectively interpolated, while being robust to activation prediction errors through broadly distributed sampling. A technical challenge of this sampling-based approach is that the binary decision variables for representing discrete sampling locations are non-differentiable, making them incompatible with backpropagation. To circumvent this issue, we make use of a reparameterization trick based on the Gumbel-Softmax distribution, with which backpropagation can iterate these variables towards binary values. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks. |
| Tasks | |
| Published | 2020-03-19 |
| URL | https://arxiv.org/abs/2003.08866v1 |
| https://arxiv.org/pdf/2003.08866v1.pdf | |
| PWC | https://paperswithcode.com/paper/spatially-adaptive-inference-with-stochastic |
| Repo | |
| Framework | |
RNN-based Online Learning: An Efficient First-Order Optimization Algorithm with a Convergence Guarantee
| Title | RNN-based Online Learning: An Efficient First-Order Optimization Algorithm with a Convergence Guarantee |
| Authors | N. Mert Vural, Selim F. Yilmaz, Fatih Ilhan, Suleyman S. Kozat |
| Abstract | We investigate online nonlinear regression with continually running recurrent neural network networks (RNNs), i.e., RNN-based online learning. For RNN-based online learning, we introduce an efficient first-order training algorithm that theoretically guarantees to converge to the optimum network parameters. Our algorithm is truly online such that it does not make any assumption on the learning environment to guarantee convergence. Through numerical simulations, we verify our theoretical results and illustrate significant performance improvements achieved by our algorithm with respect to the state-of-the-art RNN training methods. |
| Tasks | |
| Published | 2020-03-07 |
| URL | https://arxiv.org/abs/2003.03601v1 |
| https://arxiv.org/pdf/2003.03601v1.pdf | |
| PWC | https://paperswithcode.com/paper/rnn-based-online-learning-an-efficient-first |
| Repo | |
| Framework | |
Documentation of Machine Learning Software
| Title | Documentation of Machine Learning Software |
| Authors | Yalda Hashemi, Maleknaz Nayebi, Giuliano Antoniol |
| Abstract | Machine Learning software documentation is different from most of the documentations that were studied in software engineering research. Often, the users of these documentations are not software experts. The increasing interest in using data science and in particular, machine learning in different fields attracted scientists and engineers with various levels of knowledge about programming and software engineering. Our ultimate goal is automated generation and adaptation of machine learning software documents for users with different levels of expertise. We are interested in understanding the nature and triggers of the problems and the impact of the users’ levels of expertise in the process of documentation evolution. We will investigate the Stack Overflow Q/As and classify the documentation related Q/As within the machine learning domain to understand the types and triggers of the problems as well as the potential change requests to the documentation. We intend to use the results for building on top of the state of the art techniques for automatic documentation generation and extending on the adoption, summarization, and explanation of software functionalities. |
| Tasks | |
| Published | 2020-01-30 |
| URL | https://arxiv.org/abs/2001.11956v1 |
| https://arxiv.org/pdf/2001.11956v1.pdf | |
| PWC | https://paperswithcode.com/paper/documentation-of-machine-learning-software |
| Repo | |
| Framework | |
