Paper Group ANR 411
NeuroQuery: comprehensive meta-analysis of human brain mapping. Adaptive Temporal Difference Learning with Linear Function Approximation. On Leveraging Pretrained GANs for Limited-Data Generation. Rethinking Randomized Smoothing for Adversarial Robustness. Automatic Machine Learning Derived from Scholarly Big Data. Bit Allocation for Multi-Task Col …
NeuroQuery: comprehensive meta-analysis of human brain mapping
| Title | NeuroQuery: comprehensive meta-analysis of human brain mapping |
| Authors | Jérôme Dockès, Russell Poldrack, Romain Primet, Hande Gözükan, Tal Yarkoni, Fabian Suchanek, Bertrand Thirion, Gaël Varoquaux |
| Abstract | Reaching a global view of brain organization requires assembling evidence on widely different mental processes and mechanisms. The variety of human neuroscience concepts and terminology poses a fundamental challenge to relating brain imaging results across the scientific literature. Existing meta-analysis methods perform statistical tests on sets of publications associated with a particular concept. Thus, large-scale meta-analyses only tackle single terms that occur frequently. We propose a new paradigm, focusing on prediction rather than inference. Our multivariate model predicts the spatial distribution of neurological observations, given text describing an experiment, cognitive process, or disease. This approach handles text of arbitrary length and terms that are too rare for standard meta-analysis. We capture the relationships and neural correlates of 7 547 neuroscience terms across 13 459 neuroimaging publications. The resulting meta-analytic tool, neuroquery.org, can ground hypothesis generation and data-analysis priors on a comprehensive view of published findings on the brain. |
| Tasks | |
| Published | 2020-02-21 |
| URL | https://arxiv.org/abs/2002.09261v1 |
| https://arxiv.org/pdf/2002.09261v1.pdf | |
| PWC | https://paperswithcode.com/paper/neuroquery-comprehensive-meta-analysis-of |
| Repo | |
| Framework | |
Adaptive Temporal Difference Learning with Linear Function Approximation
| Title | Adaptive Temporal Difference Learning with Linear Function Approximation |
| Authors | Tao Sun, Han Shen, Tianyi Chen, Dongsheng Li |
| Abstract | This paper revisits the celebrated temporal difference (TD) learning algorithm for the policy evaluation in reinforcement learning. Typically, the performance of the plain-vanilla TD algorithm is sensitive to the choice of stepsizes. Oftentimes, TD suffers from slow convergence. Motivated by the tight connection between the TD learning algorithm and the stochastic gradient methods, we develop the first adaptive variant of the TD learning algorithm with linear function approximation that we term AdaTD. In contrast to the original TD, AdaTD is robust or less sensitive to the choice of stepsizes. Analytically, we establish that to reach an $\epsilon$ accuracy, the number of iterations needed is $\tilde{O}(\epsilon^2\ln^4\frac{1}{\epsilon}/\ln^4\frac{1}{\rho})$, where $\rho$ represents the speed of the underlying Markov chain converges to the stationary distribution. This implies that the iteration complexity of AdaTD is no worse than that of TD in the worst case. Going beyond TD, we further develop an adaptive variant of TD($\lambda$), which is referred to as AdaTD($\lambda$). We evaluate the empirical performance of AdaTD and AdaTD($\lambda$) on several standard reinforcement learning tasks in OpenAI Gym on both linear and nonlinear function approximation, which demonstrate the effectiveness of our new approaches over existing ones. |
| Tasks | |
| Published | 2020-02-20 |
| URL | https://arxiv.org/abs/2002.08537v1 |
| https://arxiv.org/pdf/2002.08537v1.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-temporal-difference-learning-with |
| Repo | |
| Framework | |
On Leveraging Pretrained GANs for Limited-Data Generation
| Title | On Leveraging Pretrained GANs for Limited-Data Generation |
| Authors | Miaoyun Zhao, Yulai Cong, Lawrence Carin |
| Abstract | Recent work has shown GANs can generate highly realistic images that are indistinguishable by human. Of particular interest here is the empirical observation that most generated images are not contained in training datasets, indicating potential generalization with GANs. That generalizability makes it appealing to exploit GANs to help applications with limited available data, e.g., augment training data to alleviate overfitting. To better facilitate training a GAN on limited data, we propose to leverage already-available GAN models pretrained on large-scale datasets (like ImageNet) to introduce additional common knowledge (which may not exist within the limited data) following the transfer learning idea. Specifically, exampled by natural image generation tasks, we reveal the fact that low-level filters (those close to observations) of both the generator and discriminator of pretrained GANs can be transferred to help the target limited-data generation. For better adaption of the transferred filters to the target domain, we introduce a new technique named adaptive filter modulation (AdaFM), which provides boosted performance over baseline methods. Unifying the transferred filters and the introduced techniques, we present our method and conduct extensive experiments to demonstrate its training efficiency and better performance on limited-data generation. |
| Tasks | Image Generation, Transfer Learning |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11810v1 |
| https://arxiv.org/pdf/2002.11810v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-leveraging-pretrained-gans-for-limited |
| Repo | |
| Framework | |
Rethinking Randomized Smoothing for Adversarial Robustness
| Title | Rethinking Randomized Smoothing for Adversarial Robustness |
| Authors | Jeet Mohapatra, Ching-Yun Ko, Tsui-Wei, Weng, Sijia Liu, Pin-Yu Chen, Luca Daniel |
| Abstract | The fragility of modern machine learning models has drawn a considerable amount of attention from both academia and the public. While immense interests were in either crafting adversarial attacks as a way to measure the robustness of neural networks or devising worst-case analytical robustness verification with guarantees, few methods could enjoy both scalability and robustness guarantees at the same time. As an alternative to these attempts, randomized smoothing adopts a different prediction rule that enables statistical robustness arguments and can scale to large networks. However, in this paper, we point out for the first time the side effects of current randomized smoothing workflows. Specifically, we articulate and prove two major points: 1) the decision boundaries shrink with the adoption of randomized smoothing prediction rule; 2) noise augmentation does not necessarily resolve the shrinking issue and can even create additional issues. |
| Tasks | |
| Published | 2020-03-02 |
| URL | https://arxiv.org/abs/2003.01249v1 |
| https://arxiv.org/pdf/2003.01249v1.pdf | |
| PWC | https://paperswithcode.com/paper/rethinking-randomized-smoothing-for |
| Repo | |
| Framework | |
Automatic Machine Learning Derived from Scholarly Big Data
| Title | Automatic Machine Learning Derived from Scholarly Big Data |
| Authors | Asnat Greenstein-Messica, Roman Vainshtein, Gilad Katz, Bracha Shapira, Lior Rokach |
| Abstract | One of the challenging aspects of applying machine learning is the need to identify the algorithms that will perform best for a given dataset. This process can be difficult, time consuming and often requires a great deal of domain knowledge. We present Sommelier, an expert system for recommending the machine learning algorithms that should be applied on a previously unseen dataset. Sommelier is based on word embedding representations of the domain knowledge extracted from a large corpus of academic publications. When presented with a new dataset and its problem description, Sommelier leverages a recommendation model trained on the word embedding representation to provide a ranked list of the most relevant algorithms to be used on the dataset. We demonstrate Sommelier’s effectiveness by conducting an extensive evaluation on 121 publicly available datasets and 53 classification algorithms. The top algorithms recommended for each dataset by Sommelier were able to achieve on average 97.7% of the optimal accuracy of all surveyed algorithms. |
| Tasks | |
| Published | 2020-03-06 |
| URL | https://arxiv.org/abs/2003.03470v1 |
| https://arxiv.org/pdf/2003.03470v1.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-machine-learning-derived-from |
| Repo | |
| Framework | |
Bit Allocation for Multi-Task Collaborative Intelligence
| Title | Bit Allocation for Multi-Task Collaborative Intelligence |
| Authors | Saeed Ranjbar Alvar, Ivan V. Bajić |
| Abstract | Recent studies have shown that collaborative intelligence (CI) is a promising framework for deployment of Artificial Intelligence (AI)-based services on mobile devices. In CI, a deep neural network is split between the mobile device and the cloud. Deep features obtained at the mobile are compressed and transferred to the cloud to complete the inference. So far, the methods in the literature focused on transferring a single deep feature tensor from the mobile to the cloud. Such methods are not applicable to some recent, high-performance networks with multiple branches and skip connections. In this paper, we propose the first bit allocation method for multi-stream, multi-task CI. We first establish a model for the joint distortion of the multiple tasks as a function of the bit rates assigned to different deep feature tensors. Then, using the proposed model, we solve the rate-distortion optimization problem under a total rate constraint to obtain the best rate allocation among the tensors to be transferred. Experimental results illustrate the efficacy of the proposed scheme compared to several alternative bit allocation methods. |
| Tasks | |
| Published | 2020-02-14 |
| URL | https://arxiv.org/abs/2002.07048v1 |
| https://arxiv.org/pdf/2002.07048v1.pdf | |
| PWC | https://paperswithcode.com/paper/bit-allocation-for-multi-task-collaborative |
| Repo | |
| Framework | |
Switchable Precision Neural Networks
| Title | Switchable Precision Neural Networks |
| Authors | Luis Guerra, Bohan Zhuang, Ian Reid, Tom Drummond |
| Abstract | Instantaneous and on demand accuracy-efficiency trade-off has been recently explored in the context of neural networks slimming. In this paper, we propose a flexible quantization strategy, termed Switchable Precision neural Networks (SP-Nets), to train a shared network capable of operating at multiple quantization levels. At runtime, the network can adjust its precision on the fly according to instant memory, latency, power consumption and accuracy demands. For example, by constraining the network weights to 1-bit with switchable precision activations, our shared network spans from BinaryConnect to Binarized Neural Network, allowing to perform dot-products using only summations or bit operations. In addition, a self-distillation scheme is proposed to increase the performance of the quantized switches. We tested our approach with three different quantizers and demonstrate the performance of SP-Nets against independently trained quantized models in classification accuracy for Tiny ImageNet and ImageNet datasets using ResNet-18 and MobileNet architectures. |
| Tasks | Quantization |
| Published | 2020-02-07 |
| URL | https://arxiv.org/abs/2002.02815v1 |
| https://arxiv.org/pdf/2002.02815v1.pdf | |
| PWC | https://paperswithcode.com/paper/switchable-precision-neural-networks |
| Repo | |
| Framework | |
FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Semi-supervised Optical Flow Estimation
| Title | FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Semi-supervised Optical Flow Estimation |
| Authors | Xiaolin Song, Jingyu Yang, Cuiling Lan, Wenjun Zeng |
| Abstract | Optical flow estimation is an important yet challenging problem in the field of video analytics. The features of different semantics levels/layers of a convolutional neural network can provide information of different granularity. To exploit such flexible and comprehensive information, we propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. The pyramid correlation mapping module takes advantage of the multi-scale correlations of global/local patches by aggregating features of different scales to form a multi-level cost volume. The residual reconstruction module aims to reconstruct the sub-band high-frequency residuals of finer optical flow in each stage. Based on the pyramid correlation mapping, we further propose a correlation-warping-normalization (CWN) module to efficiently exploit the correlation dependency. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods - FlowNet2, LiteFlowNet and PWC-Net on the Final pass of Sintel dataset, respectively. |
| Tasks | Optical Flow Estimation |
| Published | 2020-01-17 |
| URL | https://arxiv.org/abs/2001.06171v1 |
| https://arxiv.org/pdf/2001.06171v1.pdf | |
| PWC | https://paperswithcode.com/paper/fpcr-net-feature-pyramidal-correlation-and |
| Repo | |
| Framework | |
Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
| Title | Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs |
| Authors | Matthew L. Leavitt, Ari Morcos |
| Abstract | Class selectivity, typically defined as how different a neuron’s responses are across different classes of stimuli or data samples, is a common metric used to interpret the function of individual neurons in biological and artificial neural networks. However, it remains an open question whether it is necessary and/or sufficient for deep neural networks (DNNs) to learn class selectivity in individual units. In order to investigate the causal impact of class selectivity on network function, we directly regularize for or against class selectivity. Using this regularizer, we were able to reduce mean class selectivity across units in convolutional neural networks by a factor of 2.5 with no impact on test accuracy, and reduce it nearly to zero with only a small ($\sim$2%) change in test accuracy. In contrast, increasing class selectivity beyond the levels naturally learned during training had rapid and disastrous effects on test accuracy. These results indicate that class selectivity in individual units is neither neither sufficient nor strictly necessary for DNN performance, and more generally encourage caution when focusing on the properties of single units as representative of the mechanisms by which DNNs function. |
| Tasks | |
| Published | 2020-03-03 |
| URL | https://arxiv.org/abs/2003.01262v1 |
| https://arxiv.org/pdf/2003.01262v1.pdf | |
| PWC | https://paperswithcode.com/paper/selectivity-considered-harmful-evaluating-the |
| Repo | |
| Framework | |
ANN-Based Detection in MIMO-OFDM Systems with Low-Resolution ADCs
| Title | ANN-Based Detection in MIMO-OFDM Systems with Low-Resolution ADCs |
| Authors | Shabnam Rezaei, Sofiene Affes |
| Abstract | In this paper, we propose a multi-layer artificial neural network (ANN) that is trained with the Levenberg-Marquardt algorithm for use in signal detection over multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) systems, particularly those with low-resolution analog-to-digital converters (LR-ADCs). We consider a blind detection scheme where data symbol estimation is carried out without knowing the channel state information at the receiver (CSIR)—in contrast to classical algorithms. The main power of the proposed ANN-based detector (ANND) lies in its versatile use with any modulation scheme, blindly, yet without a change in its structure. We compare by simulations this new receiver with conventional ones, namely, the maximum likelihood (ML), minimum mean square error (MMSE), and zero-forcing (ZF), in terms of symbol error rate (SER) performance. Results suggest that ANND approaches ML at much lower complexity, outperforms ZF over the entire range of assessed signal-to-noise ratio (SNR) values, and so does it also, though, with the MMSE over different SNR ranges. |
| Tasks | |
| Published | 2020-01-31 |
| URL | https://arxiv.org/abs/2001.11643v1 |
| https://arxiv.org/pdf/2001.11643v1.pdf | |
| PWC | https://paperswithcode.com/paper/ann-based-detection-in-mimo-ofdm-systems-with |
| Repo | |
| Framework | |
Rethinking Motion Representation: Residual Frames with 3D ConvNets for Better Action Recognition
| Title | Rethinking Motion Representation: Residual Frames with 3D ConvNets for Better Action Recognition |
| Authors | Li Tao, Xueting Wang, Toshihiko Yamasaki |
| Abstract | Recently, 3D convolutional networks yield good performance in action recognition. However, optical flow stream is still needed to ensure better performance, the cost of which is very high. In this paper, we propose a fast but effective way to extract motion features from videos utilizing residual frames as the input data in 3D ConvNets. By replacing traditional stacked RGB frames with residual ones, 20.5% and 12.5% points improvements over top-1 accuracy can be achieved on the UCF101 and HMDB51 datasets when trained from scratch. Because residual frames contain little information of object appearance, we further use a 2D convolutional network to extract appearance features and combine them with the results from residual frames to form a two-path solution. In three benchmark datasets, our two-path solution achieved better or comparable performances than those using additional optical flow methods, especially outperformed the state-of-the-art models on Mini-kinetics dataset. Further analysis indicates that better motion features can be extracted using residual frames with 3D ConvNets, and our residual-frame-input path is a good supplement for existing RGB-frame-input models. |
| Tasks | Optical Flow Estimation |
| Published | 2020-01-16 |
| URL | https://arxiv.org/abs/2001.05661v1 |
| https://arxiv.org/pdf/2001.05661v1.pdf | |
| PWC | https://paperswithcode.com/paper/rethinking-motion-representation-residual |
| Repo | |
| Framework | |
Aggressive Perception-Aware Navigation using Deep Optical Flow Dynamics and PixelMPC
| Title | Aggressive Perception-Aware Navigation using Deep Optical Flow Dynamics and PixelMPC |
| Authors | Keuntaek Lee, Jason Gibson, Evangelos A. Theodorou |
| Abstract | Recently, vision-based control has gained traction by leveraging the power of machine learning. In this work, we couple a model predictive control (MPC) framework to a visual pipeline. We introduce deep optical flow (DOF) dynamics, which is a combination of optical flow and robot dynamics. Using the DOF dynamics, MPC explicitly incorporates the predicted movement of relevant pixels into the planned trajectory of a robot. Our implementation of DOF is memory-efficient, data-efficient, and computationally cheap so that it can be computed in real-time for use in an MPC framework. The suggested Pixel Model Predictive Control (PixelMPC) algorithm controls the robot to accomplish a high-speed racing task while maintaining visibility of the important features (gates). This improves the reliability of vision-based estimators for localization and can eventually lead to safe autonomous flight. The proposed algorithm is tested in a photorealistic simulation with a high-speed drone racing task. |
| Tasks | Optical Flow Estimation |
| Published | 2020-01-07 |
| URL | https://arxiv.org/abs/2001.02307v1 |
| https://arxiv.org/pdf/2001.02307v1.pdf | |
| PWC | https://paperswithcode.com/paper/aggressive-perception-aware-navigation-using |
| Repo | |
| Framework | |
First image then video: A two-stage network for spatiotemporal video denoising
| Title | First image then video: A two-stage network for spatiotemporal video denoising |
| Authors | Ce Wang, S. Kevin Zhou, Zhiwei Cheng |
| Abstract | Video denoising is to remove noise from noise-corrupted data, thus recovering true signals via spatiotemporal processing. Existing approaches for spatiotemporal video denoising tend to suffer from motion blur artifacts, that is, the boundary of a moving object tends to appear blurry especially when the object undergoes a fast motion, causing optical flow calculation to break down. In this paper, we address this challenge by designing a first-image-then-video two-stage denoising neural network, consisting of an image denoising module for spatially reducing intra-frame noise followed by a regular spatiotemporal video denoising module. The intuition is simple yet powerful and effective: the first stage of image denoising effectively reduces the noise level and, therefore, allows the second stage of spatiotemporal denoising for better modeling and learning everywhere, including along the moving object boundaries. This two-stage network, when trained in an end-to-end fashion, yields the state-of-the-art performances on the video denoising benchmark Vimeo90K dataset in terms of both denoising quality and computation. It also enables an unsupervised approach that achieves comparable performance to existing supervised approaches. |
| Tasks | Denoising, Image Denoising, Optical Flow Estimation, Video Denoising |
| Published | 2020-01-02 |
| URL | https://arxiv.org/abs/2001.00346v2 |
| https://arxiv.org/pdf/2001.00346v2.pdf | |
| PWC | https://paperswithcode.com/paper/first-image-then-video-a-two-stage-network |
| Repo | |
| Framework | |
Optimization of Passive Chip Components Placement with Self-Alignment Effect for Advanced Surface Mounting Technology
| Title | Optimization of Passive Chip Components Placement with Self-Alignment Effect for Advanced Surface Mounting Technology |
| Authors | Irandokht Parviziomran, Shun Cao, Haeyong Yang, Seungbae Park, Daehan Won |
| Abstract | Surface mount technology (SMT) is an enhanced method in electronic packaging in which electronic components are placed directly on soldered printing circuit board (PCB) and are permanently attached on PCB with the aim of reflow soldering process. During reflow process, once deposited solder pastes start melting, electronic components move in a direction that achieve their highest symmetry. This motion is known as self-alignment since can correct potential mounting misalignment. In this study, two noticeable machine learning algorithms, including support vector regression (SVR) and random forest regression (RFR) are proposed as a prediction technique to (1) diagnose the relation among component self-alignment, deposited solder paste status and placement machining parameters, (2) predict the final component position on PCB in x, y, and rotational directions before entering in the reflow process. Based on the prediction result, a non-linear optimization model (NLP) is developed to optimize placement parameters at initial stage. Resultantly, RFR outperforms in terms of prediction model fitness and error. The optimization model is run for 6 samples in which the minimum Euclidean distance from component position after reflow process from ideal position (i.e., the center of pads) is outlined as 25.57 ({\mu}m) regarding defined boundaries in model. |
| Tasks | |
| Published | 2020-01-27 |
| URL | https://arxiv.org/abs/2001.09612v1 |
| https://arxiv.org/pdf/2001.09612v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimization-of-passive-chip-components |
| Repo | |
| Framework | |
An $O(s^r)$-Resolution ODE Framework for Discrete-Time Optimization Algorithms and Applications to Convex-Concave Saddle-Point Problems
| Title | An $O(s^r)$-Resolution ODE Framework for Discrete-Time Optimization Algorithms and Applications to Convex-Concave Saddle-Point Problems |
| Authors | Haihao Lu |
| Abstract | There has been a long history of using Ordinary Differential Equations (ODEs) to understand the dynamic of discrete-time optimization algorithms. However, one major difficulty of applying this approach is that there can be multiple ODEs that correspond to the same discrete-time algorithm, depending on how to take the continuous limit, which makes it unclear how to obtain the suitable ODE from a discrete-time optimization algorithm. Inspired by the recent paper \cite{shi2018understanding}, we propose the $r$-th degree ODE expansion of a discrete-time optimization algorithm, which provides a principal approach to construct the unique $O(s^r)$-resolution ODE systems for a given discrete-time algorithm, where $s$ is the step-size of the algorithm. We utilize this machinery to study three classic algorithms – gradient method (GM), proximal point method (PPM) and extra-gradient method (EGM) – for finding the solution to the unconstrained convex-concave saddle-point problem $\min_{x\in\RR^n} \max_{y\in \RR^m} L(x,y)$, which explains their puzzling convergent/divergent behaviors when $L(x,y)$ is a bilinear function. Moreover, their $O(s)$-resolution ODEs inspire us to define the $O(s)$-linear-convergence condition on $L(x,y)$, under which PPM and EGM exhabit linear convergence. This condition not only unifies the known linear convergence rate of PPM and EGM, but also showcases that these two algorithms exhibit linear convergence in broader contexts. |
| Tasks | |
| Published | 2020-01-23 |
| URL | https://arxiv.org/abs/2001.08826v1 |
| https://arxiv.org/pdf/2001.08826v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-osr-resolution-ode-framework-for-discrete |
| Repo | |
| Framework | |
