January 30, 2020

3209 words 16 mins read

Paper Group ANR 394

5 Parallel Prism: A topology for pipelined implementations of convolutional neural networks using computational memory. A Novel Self-Organizing PID Approach for Controlling Mobile Robot Locomotion. Mirovia: A Benchmarking Suite for Modern Heterogeneous Computing. Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsuper …

5 Parallel Prism: A topology for pipelined implementations of convolutional neural networks using computational memory


Title	5 Parallel Prism: A topology for pipelined implementations of convolutional neural networks using computational memory
Authors	Martino Dazzi, Abu Sebastian, Pier Andrea Francese, Thomas Parnell, Luca Benini, Evangelos Eleftheriou
Abstract	In-memory computing is an emerging computing paradigm that could enable deeplearning inference at significantly higher energy efficiency and reduced latency. The essential idea is to map the synaptic weights corresponding to each layer to one or more computational memory (CM) cores. During inference, these cores perform the associated matrix-vector multiply operations in place with O(1) time complexity, thus obviating the need to move the synaptic weights to an additional processing unit. Moreover, this architecture could enable the execution of these networks in a highly pipelined fashion. However, a key challenge is to design an efficient communication fabric for the CM cores. Here, we present one such communication fabric based on a graph topology that is well suited for the widely successful convolutional neural networks (CNNs). We show that this communication fabric facilitates the pipelined execution of all state of-the-art CNNs by proving the existence of a homomorphism between one graph representation of these networks and the proposed graph topology. We then present a quantitative comparison with established communication topologies and show that our proposed topology achieves the lowest bandwidth requirements per communication channel. Finally, we present a concrete example of mapping ResNet-32 onto an array of CM cores.
Tasks
Published	2019-06-08
URL	https://arxiv.org/abs/1906.03474v1
PDF	https://arxiv.org/pdf/1906.03474v1.pdf
PWC	https://paperswithcode.com/paper/5-parallel-prism-a-topology-for-pipelined
Repo
Framework

A Novel Self-Organizing PID Approach for Controlling Mobile Robot Locomotion


Title	A Novel Self-Organizing PID Approach for Controlling Mobile Robot Locomotion
Authors	Xiaowei Gu, Muhammad Aurangzeb Khan, Plamen Angelov, Bikash Tiwary, Elnaz Shafipour Yourdshah, Zhao-Xu Yang
Abstract	A novel self-organizing fuzzy proportional-integral-derivative (SOF-PID) control system is proposed in this paper. The proposed system consists of a pair of control and reference models, both of which are implemented by a first-order autonomous learning multiple model (ALMMo) neuro-fuzzy system. The SOF-PID controller self-organizes and self-updates the structures and meta-parameters of both the control and reference models during the control process “on the fly”. This gives the SOF-PID control system the capability of quickly adapting to entirely new operating environments without a full re-training. Moreover, the SOF-PID control system is free from user- and problem-specific parameters, and the uniform stability of the SOF-PID control system is theoretically guaranteed. Simulations and real-world experiments with mobile robots demonstrate the effectiveness and validity of the proposed SOF-PID control system.
Tasks
Published	2019-12-17
URL	https://arxiv.org/abs/1912.08057v1
PDF	https://arxiv.org/pdf/1912.08057v1.pdf
PWC	https://paperswithcode.com/paper/a-novel-self-organizing-pid-approach-for
Repo
Framework

Mirovia: A Benchmarking Suite for Modern Heterogeneous Computing


Title	Mirovia: A Benchmarking Suite for Modern Heterogeneous Computing
Authors	Bodun Hu, Christopher J. Rossbach
Abstract	This paper presents Mirovia, a benchmark suite developed for modern day heterogeneous computing. Previous benchmark suites such as Rodinia and SHOC are well written and have many desirable features. However, these tools were developed years ago when hardware was less powerful and software had fewer features. For example, unified memory was introduced in CUDA 6 as a new programming model and wasn’t available when Rodinia was released. Meanwhile, the increasing demand for graphics processing units (GPUs) due to the recent rise in popularity of deep neural networks (DNNs) has opened doors for many new research problems. It is essential to consider DNNs as first-class citizens in a comprehensive benchmark suite. However, the main focus is usually limited to inference and model performance evaluation, which is not desirable for hardware architects studying for emerging platforms. Drawing inspiration from Rodinia and SHOC, Mirovia is a benchmark suite that is designed to take advantage of modern GPU architectures, while also representing a diverse set of application domains. By adopting applications from Rodinia and SHOC, and including newly written applications with special focus on DNNs, Mirovia better characterizes modern heterogeneous systems.
Tasks
Published	2019-06-25
URL	https://arxiv.org/abs/1906.10347v1
PDF	https://arxiv.org/pdf/1906.10347v1.pdf
PWC	https://paperswithcode.com/paper/mirovia-a-benchmarking-suite-for-modern
Repo
Framework

Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsupervised Learning


Title	Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsupervised Learning
Authors	Peter Y. Lu, Samuel Kim, Marin Soljačić
Abstract	Experimental data is often affected by uncontrolled variables that make analysis and interpretation difficult. For spatiotemporal systems, this problem is further exacerbated by their intricate dynamics. Modern machine learning methods are particularly well-suited for analyzing and modeling complex datasets, but to be effective in science, the result needs to be interpretable. We demonstrate an unsupervised learning technique for extracting interpretable physical parameters from noisy spatiotemporal data and for building a transferable model of the system. In particular, we implement a physics-informed architecture based on variational autoencoders that is designed for analyzing systems governed by partial differential equations (PDEs). The architecture is trained end-to-end and extracts latent parameters that parameterize the dynamics of a learned predictive model for the system. To test our method, we train our model on simulated data from a variety of PDEs with varying dynamical parameters that act as uncontrolled variables. Numerical experiments show that our method can accurately identify relevant parameters and extract them from raw and even noisy spatiotemporal data (tested with roughly 10% added noise). These extracted parameters correlate well (linearly with $R^2 > 0.95$) with the ground truth physical parameters used to generate the datasets. Our method for discovering interpretable latent parameters in spatiotemporal systems will allow us to better analyze and understand real-world phenomena and datasets, which often have unknown and uncontrolled variables that alter the system dynamics and cause varying behaviors that are difficult to disentangle.
Tasks
Published	2019-07-13
URL	https://arxiv.org/abs/1907.06011v2
PDF	https://arxiv.org/pdf/1907.06011v2.pdf
PWC	https://paperswithcode.com/paper/extracting-interpretable-physical-parameters
Repo
Framework

Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control


Title	Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
Authors	Jörg K. H. Franke, Gregor Köhler, Noor Awad, Frank Hutter
Abstract	Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neuroevolution techniques with off-policy training and propose a novel architecture mutation operator. Experiments on five continuous control benchmarks show that the proposed Actor-Critic Neuroevolution algorithm often outperforms the strong Actor-Critic baseline and is capable of automatically finding topologies in a sample-efficient manner which would otherwise have to be found by expensive architecture search.
Tasks	Continuous Control
Published	2019-10-28
URL	https://arxiv.org/abs/1910.12824v3
PDF	https://arxiv.org/pdf/1910.12824v3.pdf
PWC	https://paperswithcode.com/paper/neural-architecture-evolution-in-deep
Repo
Framework

SurReal: Fréchet Mean and Distance Transform for Complex-Valued Deep Learning


Title	SurReal: Fréchet Mean and Distance Transform for Complex-Valued Deep Learning
Authors	Rudrasis Chakraborty, Jiayun Wang, Stella X. Yu
Abstract	We develop a novel deep learning architecture for naturally complex-valued data, which is often subject to complex scaling ambiguity. We treat each sample as a field in the space of complex numbers. With the polar form of a complex-valued number, the general group that acts in this space is the product of planar rotation and non-zero scaling. This perspective allows us to develop not only a novel convolution operator using weighted Fr'echet mean (wFM) on a Riemannian manifold, but also a novel fully connected layer operator using the distance to the wFM, with natural equivariant properties to non-zero scaling and planar rotation for the former and invariance properties for the latter. Compared to the baseline approach of learning real-valued neural network models on the two-channel real-valued representation of complex-valued data, our method achieves surreal performance on two publicly available complex-valued datasets: MSTAR on SAR images and RadioML on radio frequency signals. On MSTAR, at 8% of the baseline model size and with fewer than 45,000 parameters, our model improves the target classification accuracy from 94% to 98% on this highly imbalanced dataset. On RadioML, our model achieves comparable RF modulation classification accuracy at 10% of the baseline model size.
Tasks
Published	2019-06-24
URL	https://arxiv.org/abs/1906.10048v1
PDF	https://arxiv.org/pdf/1906.10048v1.pdf
PWC	https://paperswithcode.com/paper/surreal-frechet-mean-and-distance-transform
Repo
Framework

Deep Learning in Medical Image Registration: A Survey


Title	Deep Learning in Medical Image Registration: A Survey
Authors	Grant Haskins, Uwe Kruger, Pingkun Yan
Abstract	The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring, and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.
Tasks	Image Registration, Medical Image Registration
Published	2019-03-05
URL	https://arxiv.org/abs/1903.02026v2
PDF	https://arxiv.org/pdf/1903.02026v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-in-medical-image-registration-a
Repo
Framework

Financial Time Series Data Processing for Machine Learning


Title	Financial Time Series Data Processing for Machine Learning
Authors	Fabrice Daniel
Abstract	This article studies the financial time series data processing for machine learning. It introduces the most frequent scaling methods, then compares the resulting stationarity and preservation of useful information for trend forecasting. It proposes an empirical test based on the capability to learn simple data relationship with simple models. It also speaks about the data split method specific to time series, avoiding unwanted overfitting and proposes various labelling for classification and regression.
Tasks	Time Series
Published	2019-07-03
URL	https://arxiv.org/abs/1907.03010v1
PDF	https://arxiv.org/pdf/1907.03010v1.pdf
PWC	https://paperswithcode.com/paper/financial-time-series-data-processing-for
Repo
Framework

Distinguishing Individual Red Pandas from Their Faces


Title	Distinguishing Individual Red Pandas from Their Faces
Authors	Qi He, Qijun Zhao, Ning Liu, Peng Chen, Zhihe Zhang, Rong Hou
Abstract	Individual identification is essential to animal behavior and ecology research and is of significant importance for protecting endangered species. Red pandas, among the world’s rarest animals, are currently identified mainly by visual inspection and microelectronic chips, which are costly and inefficient. Motivated by recent advancement in computer-vision-based animal identification, in this paper, we propose an automatic framework for identifying individual red pandas based on their face images. We implement the framework by exploring well-established deep learning models with necessary adaptation for effectively dealing with red panda images. Based on a database of red panda images constructed by ourselves, we evaluate the effectiveness of the proposed automatic individual red panda identification method. The evaluation results show the promising potential of automatically recognizing individual red pandas from their faces. We are going to release our database and model in the public domain to promote the research on automatic animal identification and particularly on the technique for protecting red pandas.
Tasks
Published	2019-08-09
URL	https://arxiv.org/abs/1908.03391v1
PDF	https://arxiv.org/pdf/1908.03391v1.pdf
PWC	https://paperswithcode.com/paper/distinguishing-individual-red-pandas-from
Repo
Framework

Few-Shot Video Classification via Temporal Alignment


Title	Few-Shot Video Classification via Temporal Alignment
Authors	Kaidi Cao, Jingwei Ji, Zhangjie Cao, Chien-Yi Chang, Juan Carlos Niebles
Abstract	There is a growing interest in learning a model which could recognize novel classes with only a few labeled examples. In this paper, we propose Temporal Alignment Module (TAM), a novel few-shot learning framework that can learn to classify a previous unseen video. While most previous works neglect long-term temporal ordering information, our proposed model explicitly leverages the temporal ordering information in video data through temporal alignment. This leads to strong data-efficiency for few-shot learning. In concrete, TAM calculates the distance value of query video with respect to novel class proxies by averaging the per frame distances along its alignment path. We introduce continuous relaxation to TAM so the model can be learned in an end-to-end fashion to directly optimize the few-shot learning objective. We evaluate TAM on two challenging real-world datasets, Kinetics and Something-Something-V2, and show that our model leads to significant improvement of few-shot video classification over a wide range of competitive baselines.
Tasks	Action Recognition In Videos, Few-Shot Learning, Video Classification
Published	2019-06-27
URL	https://arxiv.org/abs/1906.11415v1
PDF	https://arxiv.org/pdf/1906.11415v1.pdf
PWC	https://paperswithcode.com/paper/few-shot-video-classification-via-temporal
Repo
Framework

Auto-encoding graph-valued data with applications to brain connectomes


Title	Auto-encoding graph-valued data with applications to brain connectomes
Authors	Meimei Liu, Zhengwu Zhang, David B. Dunson
Abstract	Our interest focuses on developing statistical methods for analysis of brain structural connectomes. Nodes in the brain connectome graph correspond to different regions of interest (ROIs) while edges correspond to white matter fiber connections between these ROIs. Due to the high-dimensionality and non-Euclidean nature of the data, it becomes challenging to conduct analyses of the population distribution of brain connectomes and relate connectomes to other factors, such as cognition. Current approaches focus on summarizing the graph using either pre-specified topological features or principal components analysis (PCA). In this article, we instead develop a nonlinear latent factor model for summarizing the brain graph in both unsupervised and supervised settings. The proposed approach builds on methods for hierarchical modeling of replicated graph data, as well as variational auto-encoders that use neural networks for dimensionality reduction. We refer to our method as Graph AuTo-Encoding (GATE). We compare GATE with tensor PCA and other competitors through simulations and applications to data from the Human Connectome Project (HCP).
Tasks	Dimensionality Reduction
Published	2019-11-07
URL	https://arxiv.org/abs/1911.02728v1
PDF	https://arxiv.org/pdf/1911.02728v1.pdf
PWC	https://paperswithcode.com/paper/auto-encoding-graph-valued-data-with
Repo
Framework

Kernel Conditional Density Operators


Title	Kernel Conditional Density Operators
Authors	Ingmar Schuster, Mattes Mollenhauer, Stefan Klus, Krikamol Muandet
Abstract	We introduce a novel conditional density estimation model termed the conditional density operator (CDO). It naturally captures multivariate, multimodal output densities and shows performance that is competitive with recent neural conditional density models and Gaussian processes. The proposed model is based on a novel approach to the reconstruction of probability densities from their kernel mean embeddings by drawing connections to estimation of Radon-Nikodym derivatives in the reproducing kernel Hilbert space (RKHS). We prove finite sample bounds for the estimation error in a standard density reconstruction scenario, independent of problem dimensionality. Interestingly, when a kernel is used that is also a probability density, the CDO allows us to both evaluate and sample the output density efficiently. We demonstrate the versatility and performance of the proposed model on both synthetic and real-world data.
Tasks	Density Estimation, Gaussian Processes
Published	2019-05-27
URL	https://arxiv.org/abs/1905.11255v2
PDF	https://arxiv.org/pdf/1905.11255v2.pdf
PWC	https://paperswithcode.com/paper/kernel-conditional-density-operators
Repo
Framework

On Dimension-free Tail Inequalities for Sums of Random Matrices and Applications


Title	On Dimension-free Tail Inequalities for Sums of Random Matrices and Applications
Authors	Chao Zhang, Min-Hsiu Hsieh, Dacheng Tao
Abstract	In this paper, we present a new framework to obtain tail inequalities for sums of random matrices. Compared with existing works, our tail inequalities have the following characteristics: 1) high feasibility–they can be used to study the tail behavior of various matrix functions, e.g., arbitrary matrix norms, the absolute value of the sum of the sum of the $j$ largest singular values (resp. eigenvalues) of complex matrices (resp. Hermitian matrices); and 2) independence of matrix dimension — they do not have the matrix-dimension term as a product factor, and thus are suitable to the scenario of high-dimensional or infinite-dimensional random matrices. The price we pay to obtain these advantages is that the convergence rate of the resulting inequalities will become slow when the number of summand random matrices is large. We also develop the tail inequalities for matrix random series and matrix martingale difference sequence. We also demonstrate usefulness of our tail bounds in several fields. In compressed sensing, we employ the resulted tail inequalities to achieve a proof of the restricted isometry property when the measurement matrix is the sum of random matrices without any assumption on the distributions of matrix entries. In probability theory, we derive a new upper bound to the supreme of stochastic processes. In machine learning, we prove new expectation bounds of sums of random matrices matrix and obtain matrix approximation schemes via random sampling. In quantum information, we show a new analysis relating to the fractional cover number of quantum hypergraphs. In theoretical computer science, we obtain randomness-efficient samplers using matrix expander graphs that can be efficiently implemented in time without dependence on matrix dimensions.
Tasks
Published	2019-10-08
URL	https://arxiv.org/abs/1910.03718v1
PDF	https://arxiv.org/pdf/1910.03718v1.pdf
PWC	https://paperswithcode.com/paper/on-dimension-free-tail-inequalities-for-sums
Repo
Framework

Semi-supervised Breast Lesion Detection in Ultrasound Video Based on Temporal Coherence


Title	Semi-supervised Breast Lesion Detection in Ultrasound Video Based on Temporal Coherence
Authors	Sihong Chen, Weiping Yu, Kai Ma, Xinlong Sun, Xiaona Lin, Desheng Sun, Yefeng Zheng
Abstract	Breast lesion detection in ultrasound video is critical for computer-aided diagnosis. However, detecting lesion in video is quite challenging due to the blurred lesion boundary, high similarity to soft tissue and lack of video annotations. In this paper, we propose a semi-supervised breast lesion detection method based on temporal coherence which can detect the lesion more accurately. We aggregate features extracted from the historical key frames with adaptive key-frame scheduling strategy. Our proposed method accomplishes the unlabeled videos detection task by leveraging the supervision information from a different set of labeled images. In addition, a new WarpNet is designed to replace both the traditional spatial warping and feature aggregation operation, leading to a tremendous increase in speed. Experiments on 1,060 2D ultrasound sequences demonstrate that our proposed method achieves state-of-the-art video detection result as 91.3% in mean average precision and 19 ms per frame on GPU, compared to a RetinaNet based detection method in 86.6% and 32 ms.
Tasks
Published	2019-07-16
URL	https://arxiv.org/abs/1907.06941v1
PDF	https://arxiv.org/pdf/1907.06941v1.pdf
PWC	https://paperswithcode.com/paper/semi-supervised-breast-lesion-detection-in
Repo
Framework

Multi-year Long-term Load Forecast for Area Distribution Feeders based on Selective Sequence Learning


Title	Multi-year Long-term Load Forecast for Area Distribution Feeders based on Selective Sequence Learning
Authors	Ming Dong, Kaigui Xie, QingXin Shi
Abstract	Long-term load forecast (LTLF) for area distribution feeders is one of the most critical tasks frequently performed in electric distribution utility companies. For a specific planning area, cost-effective system upgrades can only be planned out based on accurate feeder LTLF results. In our previous research, we established a unique sequence prediction method which has the tremendous advantage of combining area top-down, feeder bottom-up and multi-year historical data all together for forecast and achieved a superior performance over various traditional methods by real-world tests. However, the previous method only focused on the forecast of the next one-year. In our current work, we significantly improved this method: the forecast can now be extended to a multi-year forecast window in the future; unsupervised learning techniques are used to group feeders by their load composition features to improve accuracy; we also propose a novel selective sequence learning mechanism which uses Gated Recurrent Unit network to not only learn how to predict sequence values but also learn to select the best-performing sequential configuration for each individual feeder. The proposed method was tested on an actual urban distribution system in West Canada. It was compared with traditional methods and our previous sequence prediction method. It demonstrates the best forecasting performance as well as the possibility of using sequence prediction models for multi-year component-level load forecast.
Tasks
Published	2019-07-18
URL	https://arxiv.org/abs/1907.07836v2
PDF	https://arxiv.org/pdf/1907.07836v2.pdf
PWC	https://paperswithcode.com/paper/multi-year-long-term-load-forecast-for-area
Repo
Framework