Paper Group ANR 381
The Sylvester Graphical Lasso (SyGlasso). Structured Linear Contextual Bandits: A Sharp and Geometric Smoothed Analysis. Classification of High-Dimensional Motor Imagery Tasks based on An End-to-end role assigned convolutional neural network. Simultaneous Skull Conductivity and Focal Source Imaging from EEG Recordings with the help of Bayesian Unce …
The Sylvester Graphical Lasso (SyGlasso)
| Title | The Sylvester Graphical Lasso (SyGlasso) |
| Authors | Yu Wang, Byoungwook Jang, Alfred Hero |
| Abstract | This paper introduces the Sylvester graphical lasso (SyGlasso) that captures multiway dependencies present in tensor-valued data. The model is based on the Sylvester equation that defines a generative model. The proposed model complements the tensor graphical lasso (Greenewald et al., 2019) that imposes a Kronecker sum model for the inverse covariance matrix by providing an alternative Kronecker sum model that is generative and interpretable. A nodewise regression approach is adopted for estimating the conditional independence relationships among variables. The statistical convergence of the method is established, and empirical studies are provided to demonstrate the recovery of meaningful conditional dependency graphs. We apply the SyGlasso to an electroencephalography (EEG) study to compare the brain connectivity of alcoholic and nonalcoholic subjects. We demonstrate that our model can simultaneously estimate both the brain connectivity and its temporal dependencies. |
| Tasks | EEG |
| Published | 2020-02-01 |
| URL | https://arxiv.org/abs/2002.00288v1 |
| https://arxiv.org/pdf/2002.00288v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-sylvester-graphical-lasso-syglasso |
| Repo | |
| Framework | |
Structured Linear Contextual Bandits: A Sharp and Geometric Smoothed Analysis
| Title | Structured Linear Contextual Bandits: A Sharp and Geometric Smoothed Analysis |
| Authors | Vidyashankar Sivakumar, Zhiwei Steven Wu, Arindam Banerjee |
| Abstract | Bandit learning algorithms typically involve the balance of exploration and exploitation. However, in many practical applications, worst-case scenarios needing systematic exploration are seldom encountered. In this work, we consider a smoothed setting for structured linear contextual bandits where the adversarial contexts are perturbed by Gaussian noise and the unknown parameter $\theta^$ has structure, e.g., sparsity, group sparsity, low rank, etc. We propose simple greedy algorithms for both the single- and multi-parameter (i.e., different parameter for each context) settings and provide a unified regret analysis for $\theta^$ with any assumed structure. The regret bounds are expressed in terms of geometric quantities such as Gaussian widths associated with the structure of $\theta^$. We also obtain sharper regret bounds compared to earlier work for the unstructured $\theta^$ setting as a consequence of our improved analysis. We show there is implicit exploration in the smoothed setting where a simple greedy algorithm works. |
| Tasks | Multi-Armed Bandits |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11332v1 |
| https://arxiv.org/pdf/2002.11332v1.pdf | |
| PWC | https://paperswithcode.com/paper/structured-linear-contextual-bandits-a-sharp |
| Repo | |
| Framework | |
Classification of High-Dimensional Motor Imagery Tasks based on An End-to-end role assigned convolutional neural network
| Title | Classification of High-Dimensional Motor Imagery Tasks based on An End-to-end role assigned convolutional neural network |
| Authors | Byeong-Hoo Lee, Ji-Hoon Jeong, Kyung-Hwan Shim, Seong-Whan Lee |
| Abstract | A brain-computer interface (BCI) provides a direct communication pathway between user and external devices. Electroencephalogram (EEG) motor imagery (MI) paradigm is widely used in non-invasive BCI to obtain encoded signals contained user intention of movement execution. However, EEG has intricate and non-stationary properties resulting in insufficient decoding performance. By imagining numerous movements of a single-arm, decoding performance can be improved without artificial command matching. In this study, we collected intuitive EEG data contained the nine different types of movements of a single-arm from 9 subjects. We propose an end-to-end role assigned convolutional neural network (ERA-CNN) which considers discriminative features of each upper limb region by adopting the principle of a hierarchical CNN architecture. The proposed model outperforms previous methods on 3-class, 5-class and two different types of 7-class classification tasks. Hence, we demonstrate the possibility of decoding user intention by using only EEG signals with robust performance using an ERA-CNN. |
| Tasks | EEG |
| Published | 2020-02-01 |
| URL | https://arxiv.org/abs/2002.00210v2 |
| https://arxiv.org/pdf/2002.00210v2.pdf | |
| PWC | https://paperswithcode.com/paper/classification-of-high-dimensional-motor |
| Repo | |
| Framework | |
Simultaneous Skull Conductivity and Focal Source Imaging from EEG Recordings with the help of Bayesian Uncertainty Modelling
| Title | Simultaneous Skull Conductivity and Focal Source Imaging from EEG Recordings with the help of Bayesian Uncertainty Modelling |
| Authors | Alexandra Koulouri, Ville Rimpilainen |
| Abstract | The electroencephalography (EEG) source imaging problem is very sensitive to the electrical modelling of the skull of the patient under examination. Unfortunately, the currently available EEG devices and their embedded software do not take this into account; instead, it is common to use a literature-based skull conductivity parameter. In this paper, we propose a statistical method based on the Bayesian approximation error approach to compensate for source imaging errors due to the unknown skull conductivity and, simultaneously, to compute a low-order estimate for the actual skull conductivity value. By using simulated EEG data that corresponds to focal source activity, we demonstrate the potential of the method to reconstruct the underlying focal sources and low-order errors induced by the unknown skull conductivity. Subsequently, the estimated errors are used to approximate the skull conductivity. The results indicate clear improvements in the source localization accuracy and feasible skull conductivity estimates. |
| Tasks | EEG |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2002.00066v1 |
| https://arxiv.org/pdf/2002.00066v1.pdf | |
| PWC | https://paperswithcode.com/paper/simultaneous-skull-conductivity-and-focal |
| Repo | |
| Framework | |
Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution
| Title | Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution |
| Authors | Eunju Cha, Hyungjin Chung, Eung Yeop Kim, Jong Chul Ye |
| Abstract | Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k-space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-off. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k-space reference data for supervised training, which is not suitable for tMRA. This is because high spatio-temporal resolution ground-truth images are not available for tMRA. To address this problem, here we propose a novel unsupervised deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler and improves the performance. Reconstruction results using in vivo tMRA data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography. |
| Tasks | |
| Published | 2020-03-29 |
| URL | https://arxiv.org/abs/2003.13096v1 |
| https://arxiv.org/pdf/2003.13096v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-deep-learning-for-mr-angiography |
| Repo | |
| Framework | |
How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
| Title | How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS |
| Authors | Kaicheng Yu, Rene Ranftl, Mathieu Salzmann |
| Abstract | Weight sharing promises to make neural architecture search (NAS) tractable even on commodity hardware. Existing methods in this space rely on a diverse set of heuristics to design and train the shared-weight backbone network, a.k.a. the super-net. Since heuristics and hyperparameters substantially vary across different methods, a fair comparison between them can only be achieved by systematically analyzing the influence of these factors. In this paper, we therefore provide a systematic evaluation of the heuristics and hyperparameters that are frequently employed by weight-sharing NAS algorithms. Our analysis uncovers that some commonly-used heuristics for super-net training negatively impact the correlation between super-net and stand-alone performance, and evidences the strong influence of certain hyperparameters and architectural choices. Our code and experiments set a strong and reproducible baseline that future works can build on. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-09 |
| URL | https://arxiv.org/abs/2003.04276v1 |
| https://arxiv.org/pdf/2003.04276v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-to-train-your-super-net-an-analysis-of |
| Repo | |
| Framework | |
Fuzzy k-Nearest Neighbors with monotonicity constraints: Moving towards the robustness of monotonic noise
| Title | Fuzzy k-Nearest Neighbors with monotonicity constraints: Moving towards the robustness of monotonic noise |
| Authors | Sergio González, Salvador García, Sheng-Tun Li, Robert John, Francisco Herrera |
| Abstract | This paper proposes a new model based on Fuzzy k-Nearest Neighbors for classification with monotonic constraints, Monotonic Fuzzy k-NN (MonFkNN). Real-life data-sets often do not comply with monotonic constraints due to class noise. MonFkNN incorporates a new calculation of fuzzy memberships, which increases robustness against monotonic noise without the need for relabeling. Our proposal has been designed to be adaptable to the different needs of the problem being tackled. In several experimental studies, we show significant improvements in accuracy while matching the best degree of monotonicity obtained by comparable methods. We also show that MonFkNN empirically achieves improved performance compared with Monotonic k-NN in the presence of large amounts of class noise. |
| Tasks | |
| Published | 2020-03-05 |
| URL | https://arxiv.org/abs/2003.02601v1 |
| https://arxiv.org/pdf/2003.02601v1.pdf | |
| PWC | https://paperswithcode.com/paper/fuzzy-k-nearest-neighbors-with-monotonicity |
| Repo | |
| Framework | |
Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices
| Title | Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices |
| Authors | Byung Hoon Ahn, Jinwon Lee, Jamie Menjay Lin, Hsin-Pai Cheng, Jilei Hou, Hadi Esmaeilzadeh |
| Abstract | Recent advances demonstrate that irregularly wired neural networks from Neural Architecture Search (NAS) and Random Wiring can not only automate the design of deep neural networks but also emit models that outperform previous manual designs. These designs are especially effective while designing neural architectures under hard resource constraints (memory, MACs, . . . ) which highlights the importance of this class of designing neural networks. However, such a move creates complication in the previously streamlined pattern of execution. In fact one of the main challenges is that the order of such nodes in the neural network significantly effects the memory footprint of the intermediate activations. Current compilers do not schedule with regard to activation memory footprint that it significantly increases its peak compared to the optimum, rendering it not applicable for edge devices. To address this standing issue, we present a memory-aware compiler, dubbed SERENITY, that utilizes dynamic programming to find a sequence that finds a schedule with optimal memory footprint. Our solution also comprises of graph rewriting technique that allows further reduction beyond the optimum. As such, SERENITY achieves optimal peak memory, and the graph rewriting technique further improves this resulting in 1.68x improvement with dynamic programming-based scheduler and 1.86x with graph rewriting, against TensorFlow Lite with less than one minute overhead. |
| Tasks | Neural Architecture Search |
| Published | 2020-03-04 |
| URL | https://arxiv.org/abs/2003.02369v1 |
| https://arxiv.org/pdf/2003.02369v1.pdf | |
| PWC | https://paperswithcode.com/paper/ordering-chaos-memory-aware-scheduling-of |
| Repo | |
| Framework | |
Handling Missing Annotations in Supervised Learning Data
| Title | Handling Missing Annotations in Supervised Learning Data |
| Authors | Alaa E. Abdel-Hakim, Wael Deabes |
| Abstract | Data annotation is an essential stage in supervised learning. However, the annotation process is exhaustive and time consuming, specially for large datasets. Activities of Daily Living (ADL) recognition is an example of systems that exploit very large raw sensor data readings. In such systems, sensor readings are collected from activity-monitoring sensors in a 24/7 manner. The size of the generated dataset is so huge that it is almost impossible for a human annotator to give a certain label to every single instance in the dataset. This results in annotation gaps in the input data to the adopting supervised learning system. The performance of the recognition system is negatively affected by these gaps. In this work, we propose and investigate three different paradigms to handle these gaps. In the first paradigm, the gaps are taken out by dropping all unlabeled readings. A single “Unknown” or “Do-Nothing” label is given to the unlabeled readings within the operation of the second paradigm. The last paradigm handles these gaps by giving every one of them a unique label identifying the encapsulating deterministic labels. Also, we propose a semantic preprocessing method of annotation gaps by constructing a hybrid combination of some of these paradigms for further performance improvement. The performance of the proposed three paradigms and their hybrid combination is evaluated using an ADL benchmark dataset containing more than $2.5\times 10^6$ sensor readings that had been collected over more than nine months. The evaluation results emphasize the performance contrast under the operation of each paradigm and support a specific gap handling approach for better performance. |
| Tasks | |
| Published | 2020-02-17 |
| URL | https://arxiv.org/abs/2002.07113v1 |
| https://arxiv.org/pdf/2002.07113v1.pdf | |
| PWC | https://paperswithcode.com/paper/handling-missing-annotations-in-supervised |
| Repo | |
| Framework | |
Static and Dynamic Values of Computation in MCTS
| Title | Static and Dynamic Values of Computation in MCTS |
| Authors | Eren Sezener, Peter Dayan |
| Abstract | Monte-Carlo Tree Search (MCTS) is one of the most-widely used methods for planning, and has powered many recent advances in artificial intelligence. In MCTS, one typically performs computations (i.e., simulations) to collect statistics about the possible future consequences of actions, and then chooses accordingly. Many popular MCTS methods such as UCT and its variants decide which computations to perform by trading-off exploration and exploitation. In this work, we take a more direct approach, and explicitly quantify the value of a computation based on its expected impact on the quality of the action eventually chosen. Our approach goes beyond the “myopic” limitations of existing computation-value-based methods in two senses: (I) we are able to account for the impact of non-immediate (ie, future) computations (II) on non-immediate actions. We show that policies that greedily optimize computation values are optimal under certain assumptions and obtain results that are competitive with the state-of-the-art. |
| Tasks | |
| Published | 2020-02-11 |
| URL | https://arxiv.org/abs/2002.04335v1 |
| https://arxiv.org/pdf/2002.04335v1.pdf | |
| PWC | https://paperswithcode.com/paper/static-and-dynamic-values-of-computation-in |
| Repo | |
| Framework | |
Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects
| Title | Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects |
| Authors | Fredrik D. Johansson, Uri Shalit, Nathan Kallus, David Sontag |
| Abstract | Practitioners in diverse fields such as healthcare, economics and education are eager to apply machine learning to improve decision making. The cost and impracticality of performing experiments and a recent monumental increase in electronic record keeping has brought attention to the problem of evaluating decisions based on non-experimental observational data. This is the setting of this work. In particular, we study estimation of individual-level causal effects, such as a single patient’s response to alternative medication, from recorded contexts, decisions and outcomes. We give generalization bounds on the error in estimated effects based on distance measures between groups receiving different treatments, allowing for sample re-weighting. We provide conditions under which our bound is tight and show how it relates to results for unsupervised domain adaptation. Led by our theoretical results, we devise representation learning algorithms that minimize our bound, by regularizing the representation’s induced treatment group distance, and encourage sharing of information between treatment groups. We extend these algorithms to simultaneously learn a weighted representation to further reduce treatment group distances. Finally, an experimental evaluation on real and synthetic data shows the value of our proposed representation architecture and regularization scheme. |
| Tasks | Decision Making, Domain Adaptation, Representation Learning, Unsupervised Domain Adaptation |
| Published | 2020-01-21 |
| URL | https://arxiv.org/abs/2001.07426v1 |
| https://arxiv.org/pdf/2001.07426v1.pdf | |
| PWC | https://paperswithcode.com/paper/generalization-bounds-and-representation |
| Repo | |
| Framework | |
Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications
| Title | Finding the Sparsest Vectors in a Subspace: Theory, Algorithms, and Applications |
| Authors | Qing Qu, Zhihui Zhu, Xiao Li, Manolis C. Tsakiris, John Wright, René Vidal |
| Abstract | The problem of finding the sparsest vector (direction) in a low dimensional subspace can be considered as a homogeneous variant of the sparse recovery problem, which finds applications in robust subspace recovery, dictionary learning, sparse blind deconvolution, and many other problems in signal processing and machine learning. However, in contrast to the classical sparse recovery problem, the most natural formulation for finding the sparsest vector in a subspace is usually nonconvex. In this paper, we overview recent advances on global nonconvex optimization theory for solving this problem, ranging from geometric analysis of its optimization landscapes, to efficient optimization algorithms for solving the associated nonconvex optimization problem, to applications in machine intelligence, representation learning, and imaging sciences. Finally, we conclude this review by pointing out several interesting open problems for future research. |
| Tasks | Dictionary Learning, Representation Learning |
| Published | 2020-01-20 |
| URL | https://arxiv.org/abs/2001.06970v1 |
| https://arxiv.org/pdf/2001.06970v1.pdf | |
| PWC | https://paperswithcode.com/paper/finding-the-sparsest-vectors-in-a-subspace |
| Repo | |
| Framework | |
Physics-informed Neural Networks for Solving Nonlinear Diffusivity and Biot’s equations
| Title | Physics-informed Neural Networks for Solving Nonlinear Diffusivity and Biot’s equations |
| Authors | Teeratorn Kadeethum, Thomas M Jorgensen, Hamidreza M Nick |
| Abstract | This paper presents the potential of applying physics-informed neural networks for solving nonlinear multiphysics problems, which are essential to many fields such as biomedical engineering, earthquake prediction, and underground energy harvesting. Specifically, we investigate how to extend the methodology of physics-informed neural networks to solve both the forward and inverse problems in relation to the nonlinear diffusivity and Biot’s equations. We explore the accuracy of the physics-informed neural networks with different training example sizes and choices of hyperparameters. The impacts of the stochastic variations between various training realizations are also investigated. In the inverse case, we also study the effects of noisy measurements. Furthermore, we address the challenge of selecting the hyperparameters of the inverse model and illustrate how this challenge is linked to the hyperparameters selection performed for the forward one. |
| Tasks | |
| Published | 2020-02-19 |
| URL | https://arxiv.org/abs/2002.08235v1 |
| https://arxiv.org/pdf/2002.08235v1.pdf | |
| PWC | https://paperswithcode.com/paper/physics-informed-neural-networks-for-solving |
| Repo | |
| Framework | |
Machine learning enables completely automatic tuning of a quantum device faster than human experts
| Title | Machine learning enables completely automatic tuning of a quantum device faster than human experts |
| Authors | H. Moon, D. T. Lennon, J. Kirkpatrick, N. M. van Esbroeck, L. C. Camenzind, Liuqi Yu, F. Vigneau, D. M. Zumbühl, G. A. D. Briggs, M. A Osborne, D. Sejdinovic, E. A. Laird, N. Ares |
| Abstract | Device variability is a bottleneck for the scalability of semiconductor quantum devices. Increasing device control comes at the cost of a large parameter space that has to be explored in order to find the optimal operating conditions. We demonstrate a statistical tuning algorithm that navigates this entire parameter space, using just a few modelling assumptions, in the search for specific electron transport features. We focused on gate-defined quantum dot devices, demonstrating fully automated tuning of two different devices to double quantum dot regimes in an up to eight-dimensional gate voltage space. We considered a parameter space defined by the maximum range of each gate voltage in these devices, demonstrating expected tuning in under 70 minutes. This performance exceeded a human benchmark, although we recognise that there is room for improvement in the performance of both humans and machines. Our approach is approximately 180 times faster than a pure random search of the parameter space, and it is readily applicable to different material systems and device architectures. With an efficient navigation of the gate voltage space we are able to give a quantitative measurement of device variability, from one device to another and after a thermal cycle of a device. This is a key demonstration of the use of machine learning techniques to explore and optimise the parameter space of quantum devices and overcome the challenge of device variability. |
| Tasks | |
| Published | 2020-01-08 |
| URL | https://arxiv.org/abs/2001.02589v1 |
| https://arxiv.org/pdf/2001.02589v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-enables-completely-automatic |
| Repo | |
| Framework | |
Examining the Benefits of Capsule Neural Networks
| Title | Examining the Benefits of Capsule Neural Networks |
| Authors | Arjun Punjabi, Jonas Schmid, Aggelos K. Katsaggelos |
| Abstract | Capsule networks are a recently developed class of neural networks that potentially address some of the deficiencies with traditional convolutional neural networks. By replacing the standard scalar activations with vectors, and by connecting the artificial neurons in a new way, capsule networks aim to be the next great development for computer vision applications. However, in order to determine whether these networks truly operate differently than traditional networks, one must look at the differences in the capsule features. To this end, we perform several analyses with the purpose of elucidating capsule features and determining whether they perform as described in the initial publication. First, we perform a deep visualization analysis to visually compare capsule features and convolutional neural network features. Then, we look at the ability for capsule features to encode information across the vector components and address what changes in the capsule architecture provides the most benefit. Finally, we look at how well the capsule features are able to encode instantiation parameters of class objects via visual transformations. |
| Tasks | |
| Published | 2020-01-29 |
| URL | https://arxiv.org/abs/2001.10964v1 |
| https://arxiv.org/pdf/2001.10964v1.pdf | |
| PWC | https://paperswithcode.com/paper/examining-the-benefits-of-capsule-neural |
| Repo | |
| Framework | |
