Paper Group ANR 535
Random Bias Initialization Improving Binary Neural Network Training. Performance Advantages of Deep Neural Networks for Angle of Arrival Estimation. Improved Generalization Bound of Group Invariant / Equivariant Deep Networks via Quotient Feature Space. Rapid online learning and robust recall in a neuromorphic olfactory circuit. Dynamic Origin-Dest …
Random Bias Initialization Improving Binary Neural Network Training
| Title | Random Bias Initialization Improving Binary Neural Network Training |
| Authors | Xinlin Li, Vahid Partovi Nia |
| Abstract | Edge intelligence especially binary neural network (BNN) has attracted considerable attention of the artificial intelligence community recently. BNNs significantly reduce the computational cost, model size, and memory footprint. However, there is still a performance gap between the successful full-precision neural network with ReLU activation and BNNs. We argue that the accuracy drop of BNNs is due to their geometry. We analyze the behaviour of the full-precision neural network with ReLU activation and compare it with its binarized counterpart. This comparison suggests random bias initialization as a remedy to activation saturation in full-precision networks and leads us towards an improved BNN training. Our numerical experiments confirm our geometric intuition. |
| Tasks | |
| Published | 2019-09-30 |
| URL | https://arxiv.org/abs/1909.13446v1 |
| https://arxiv.org/pdf/1909.13446v1.pdf | |
| PWC | https://paperswithcode.com/paper/random-bias-initialization-improving-binary-1 |
| Repo | |
| Framework | |
Performance Advantages of Deep Neural Networks for Angle of Arrival Estimation
| Title | Performance Advantages of Deep Neural Networks for Angle of Arrival Estimation |
| Authors | Oded Bialer, Noa Garnett, Tom Tirer |
| Abstract | The problem of estimating the number of sources and their angles of arrival from a single antenna array observation has been an active area of research in the signal processing community for the last few decades. When the number of sources is large, the maximum likelihood estimator is intractable due to its very high complexity, and therefore alternative signal processing methods have been developed with some performance loss. In this paper, we apply a deep neural network (DNN) approach to the problem and analyze its advantages with respect to signal processing algorithms. We show that an appropriate designed network can attain the maximum likelihood performance with feasible complexity and outperform other feasible signal processing estimation methods over various signal to noise ratios and array response inaccuracies. |
| Tasks | |
| Published | 2019-02-10 |
| URL | http://arxiv.org/abs/1902.03569v2 |
| http://arxiv.org/pdf/1902.03569v2.pdf | |
| PWC | https://paperswithcode.com/paper/performance-advantages-of-deep-neural |
| Repo | |
| Framework | |
Improved Generalization Bound of Group Invariant / Equivariant Deep Networks via Quotient Feature Space
| Title | Improved Generalization Bound of Group Invariant / Equivariant Deep Networks via Quotient Feature Space |
| Authors | Akiyoshi Sannai, Masaaki Imaizumi |
| Abstract | A large number of group invariant (or equivariant) networks have succeeded in handling invariant data such as point clouds and graphs. However, generalization theory for the networks has not been well developed, because several essential factors for generalization theory, such as size and margin distribution, are not very suitable to explain invariance and equivariance. In this paper, we develop a generalization error bound for invariant and equivariant deep neural networks. To describe the effect of the properties on generalization, we develop a quotient feature space, which measures the effect of group action for invariance or equivariance. Our main theorem proves that the volume of quotient feature spaces largely improves the main term of the developed bound. We apply our result to a specific invariant and equivariant networks, such as DeepSets (Zaheer et al. (2017)), then show that their generalization bound is drastically improved by $\sqrt{n!}$ where $n$ is a number of permuting coordinates of data. Moreover, we additionally discuss the representation power of invariant DNNs, and show that they can achieve an optimal approximation rate. This paper is the first study to provide a general and tight generalization bound for a broad class of group invariant and equivariant deep neural networks. |
| Tasks | |
| Published | 2019-10-15 |
| URL | https://arxiv.org/abs/1910.06552v2 |
| https://arxiv.org/pdf/1910.06552v2.pdf | |
| PWC | https://paperswithcode.com/paper/improved-generalization-bound-of-permutation |
| Repo | |
| Framework | |
Rapid online learning and robust recall in a neuromorphic olfactory circuit
| Title | Rapid online learning and robust recall in a neuromorphic olfactory circuit |
| Authors | Nabil Imam, Thomas A. Cleland |
| Abstract | We present a neural algorithm for the rapid online learning and identification of odorant samples under noise, based on the architecture of the mammalian olfactory bulb and implemented on the Intel Loihi neuromorphic system. As with biological olfaction, the spike timing-based algorithm utilizes distributed, event-driven computations and rapid (one-shot) online learning. Spike timing-dependent plasticity rules operate iteratively over sequential gamma-frequency packets to construct odor representations from the activity of chemosensor arrays mounted in a wind tunnel. Learned odorants then are reliably identified despite strong destructive interference. Noise resistance is further enhanced by neuromodulation and contextual priming. Lifelong learning capabilities are enabled by adult neurogenesis. The algorithm is applicable to any signal identification problem in which high-dimensional signals are embedded in unknown backgrounds. |
| Tasks | |
| Published | 2019-06-17 |
| URL | https://arxiv.org/abs/1906.07067v3 |
| https://arxiv.org/pdf/1906.07067v3.pdf | |
| PWC | https://paperswithcode.com/paper/rapid-online-learning-and-robust-recall-in-a |
| Repo | |
| Framework | |
Dynamic Origin-Destination Matrix Prediction with Line Graph Neural Networks and Kalman Filter
| Title | Dynamic Origin-Destination Matrix Prediction with Line Graph Neural Networks and Kalman Filter |
| Authors | Xi Xiong, Kaan Ozbay, Li Jin, Chen Feng |
| Abstract | Modern intelligent transportation systems provide data that allow real-time dynamic demand prediction, which is essential for planning and operations. The main challenge of prediction of dynamic Origin-Destination (O-D) demand matrices is that demands cannot be directly measured by traffic sensors; instead, they have to be inferred from aggregate traffic flow data on traffic links. Specifically, spatial correlation, congestion and time dependent factors need to be considered in general transportation networks. In this paper we propose a novel O-D prediction framework combining heterogeneous prediction in graph neural networks and Kalman filter to recognize spatial and temporal patterns simultaneously. The underlying road network topology is converted into a corresponding line graph in the newly designed Fusion Line Graph Convolutional Networks (FL-GCNs), which provide a general framework of predicting spatial-temporal O-D flows from link information. Data from New Jersey Turnpike network are used to evaluate the proposed model. The results show that our proposed approach yields the best performance under various prediction scenarios. In addition, the advantage of combining deep neural networks and Kalman filter is demonstrated. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://arxiv.org/abs/1905.00406v3 |
| https://arxiv.org/pdf/1905.00406v3.pdf | |
| PWC | https://paperswithcode.com/paper/dynamic-prediction-of-origin-destination |
| Repo | |
| Framework | |
Artificial Intelligence for Pediatric Ophthalmology
| Title | Artificial Intelligence for Pediatric Ophthalmology |
| Authors | Julia E. Reid, Eric Eaton |
| Abstract | PURPOSE OF REVIEW: Despite the impressive results of recent artificial intelligence (AI) applications to general ophthalmology, comparatively less progress has been made toward solving problems in pediatric ophthalmology using similar techniques. This article discusses the unique needs of pediatric ophthalmology patients and how AI techniques can address these challenges, surveys recent applications of AI to pediatric ophthalmology, and discusses future directions in the field. RECENT FINDINGS: The most significant advances involve the automated detection of retinopathy of prematurity (ROP), yielding results that rival experts. Machine learning (ML) has also been successfully applied to the classification of pediatric cataracts, prediction of post-operative complications following cataract surgery, detection of strabismus and refractive error, prediction of future high myopia, and diagnosis of reading disability via eye tracking. In addition, ML techniques have been used for the study of visual development, vessel segmentation in pediatric fundus images, and ophthalmic image synthesis. SUMMARY: AI applications could significantly benefit clinical care for pediatric ophthalmology patients by optimizing disease detection and grading, broadening access to care, furthering scientific discovery, and improving clinical efficiency. These methods need to match or surpass physician performance in clinical trials before deployment with patients. Due to widespread use of closed-access data sets and software implementations, it is difficult to directly compare the performance of these approaches, and reproducibility is poor. Open-access data sets and software implementations could alleviate these issues, and encourage further AI applications to pediatric ophthalmology. KEYWORDS: pediatric ophthalmology, machine learning, artificial intelligence, deep learning |
| Tasks | Eye Tracking, Image Generation |
| Published | 2019-04-06 |
| URL | http://arxiv.org/abs/1904.08796v1 |
| http://arxiv.org/pdf/1904.08796v1.pdf | |
| PWC | https://paperswithcode.com/paper/190408796 |
| Repo | |
| Framework | |
Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset
| Title | Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset |
| Authors | Ruohan Zhang, Calen Walshe, Zhuode Liu, Lin Guan, Karl S. Muller, Jake A. Whritner, Luxin Zhang, Mary M. Hayhoe, Dana H. Ballard |
| Abstract | Large-scale public datasets have been shown to benefit research in multiple areas of modern artificial intelligence. For decision-making research that requires human data, high-quality datasets serve as important benchmarks to facilitate the development of new methods by providing a common reproducible standard. Many human decision-making tasks require visual attention to obtain high levels of performance. Therefore, measuring eye movements can provide a rich source of information about the strategies that humans use to solve decision-making tasks. Here, we provide a large-scale, high-quality dataset of human actions with simultaneously recorded eye movements while humans play Atari video games. The dataset consists of 117 hours of gameplay data from a diverse set of 20 games, with 8 million action demonstrations and 328 million gaze samples. We introduce a novel form of gameplay, in which the human plays in a semi-frame-by-frame manner. This leads to near-optimal game decisions and game scores that are comparable or better than known human records. We demonstrate the usefulness of the dataset through two simple applications: predicting human gaze and imitating human demonstrated actions. The quality of the data leads to promising results in both tasks. Moreover, using a learned human gaze model to inform imitation learning leads to an 115% increase in game performance. We interpret these results as highlighting the importance of incorporating human visual attention in models of decision making and demonstrating the value of the current dataset to the research community. We hope that the scale and quality of this dataset can provide more opportunities to researchers in the areas of visual attention, imitation learning, and reinforcement learning. |
| Tasks | Decision Making, Eye Tracking, Imitation Learning |
| Published | 2019-03-15 |
| URL | https://arxiv.org/abs/1903.06754v2 |
| https://arxiv.org/pdf/1903.06754v2.pdf | |
| PWC | https://paperswithcode.com/paper/atari-head-atari-human-eye-tracking-and |
| Repo | |
| Framework | |
Multiparametric Deep Learning Tissue Signatures for Muscular Dystrophy: Preliminary Results
| Title | Multiparametric Deep Learning Tissue Signatures for Muscular Dystrophy: Preliminary Results |
| Authors | Alex E. Bocchieri, Vishwa S. Parekh, Kathryn R. Wagner. Shivani Ahlawat, Vladimir Braverman, Doris G. Leung, Michael A. Jacobs |
| Abstract | A current clinical challenge is identifying limb girdle muscular dystrophy 2I(LGMD2I)tissue changes in the thighs, in particular, separating fat, fat-infiltrated muscle, and muscle tissue. Deep learning algorithms have the ability to learn different features by using the inherent tissue contrasts from multiparametric magnetic resonance imaging (mpMRI). To that end, we developed a novel multiparametric deep learning network (MPDL) tissue signature model based on mpMRI and applied it to LGMD2I. We demonstrate a new tissue signature model of muscular dystrophy with the MPDL algorithm segments different tissue types with excellent results. |
| Tasks | |
| Published | 2019-08-01 |
| URL | https://arxiv.org/abs/1908.00175v1 |
| https://arxiv.org/pdf/1908.00175v1.pdf | |
| PWC | https://paperswithcode.com/paper/multiparametric-deep-learning-tissue-1 |
| Repo | |
| Framework | |
Rectifying Classifier Chains for Multi-Label Classification
| Title | Rectifying Classifier Chains for Multi-Label Classification |
| Authors | Robin Senge, Juan José del Coz, Eyke Hüllermeier |
| Abstract | Classifier chains have recently been proposed as an appealing method for tackling the multi-label classification task. In addition to several empirical studies showing its state-of-the-art performance, especially when being used in its ensemble variant, there are also some first results on theoretical properties of classifier chains. Continuing along this line, we analyze the influence of a potential pitfall of the learning process, namely the discrepancy between the feature spaces used in training and testing: While true class labels are used as supplementary attributes for training the binary models along the chain, the same models need to rely on estimations of these labels at prediction time. We elucidate under which circumstances the attribute noise thus created can affect the overall prediction performance. As a result of our findings, we propose two modifications of classifier chains that are meant to overcome this problem. Experimentally, we show that our variants are indeed able to produce better results in cases where the original chaining process is likely to fail. |
| Tasks | Multi-Label Classification |
| Published | 2019-06-07 |
| URL | https://arxiv.org/abs/1906.02915v1 |
| https://arxiv.org/pdf/1906.02915v1.pdf | |
| PWC | https://paperswithcode.com/paper/rectifying-classifier-chains-for-multi-label |
| Repo | |
| Framework | |
Unsupervised Learning for Neural Network-based Polar Decoder via Syndrome Loss
| Title | Unsupervised Learning for Neural Network-based Polar Decoder via Syndrome Loss |
| Authors | Chieh-Fang Teng, An-Yeu Wu |
| Abstract | With the rapid growth of deep learning in many fields, machine learning-assisted communication systems had attracted lots of researches with many eye-catching initial results. At the present stage, most of the methods still have great demand of massive labeled data for supervised learning. However, obtaining labeled data in the practical applications is not feasible, which may result in severe performance degradation due to channel variations. To overcome such a constraint, syndrome loss has been proposed to penalize non-valid decoded codewords and achieve unsupervised learning for neural network-based decoder. However, it cannot be applied to polar decoder directly. In this work, by exploiting the nature of polar codes, we propose a modified syndrome loss. From simulation results, the proposed method demonstrates that domain-specific knowledge and know-how in code structure can enable unsupervised learning for neural network-based polar decoder. |
| Tasks | |
| Published | 2019-11-05 |
| URL | https://arxiv.org/abs/1911.01710v1 |
| https://arxiv.org/pdf/1911.01710v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-learning-for-neural-network |
| Repo | |
| Framework | |
Deep Local Trajectory Replanning and Control for Robot Navigation
| Title | Deep Local Trajectory Replanning and Control for Robot Navigation |
| Authors | Ashwini Pokle, Roberto Martín-Martín, Patrick Goebel, Vincent Chow, Hans M. Ewald, Junwei Yang, Zhenkai Wang, Amir Sadeghian, Dorsa Sadigh, Silvio Savarese, Marynel Vázquez |
| Abstract | We present a navigation system that combines ideas from hierarchical planning and machine learning. The system uses a traditional global planner to compute optimal paths towards a goal, and a deep local trajectory planner and velocity controller to compute motion commands. The latter components of the system adjust the behavior of the robot through attention mechanisms such that it moves towards the goal, avoids obstacles, and respects the space of nearby pedestrians. Both the structure of the proposed deep models and the use of attention mechanisms make the system’s execution interpretable. Our simulation experiments suggest that the proposed architecture outperforms baselines that try to map global plan information and sensor data directly to velocity commands. In comparison to a hand-designed traditional navigation system, the proposed approach showed more consistent performance. |
| Tasks | Robot Navigation |
| Published | 2019-05-13 |
| URL | https://arxiv.org/abs/1905.05279v1 |
| https://arxiv.org/pdf/1905.05279v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-local-trajectory-replanning-and-control |
| Repo | |
| Framework | |
Self-Supervised Visual Place Recognition Learning in Mobile Robots
| Title | Self-Supervised Visual Place Recognition Learning in Mobile Robots |
| Authors | Sudeep Pillai, John Leonard |
| Abstract | Place recognition is a critical component in robot navigation that enables it to re-establish previously visited locations, and simultaneously use this information to correct the drift incurred in its dead-reckoned estimate. In this work, we develop a self-supervised approach to place recognition in robots. The task of visual loop-closure identification is cast as a metric learning problem, where the labels for positive and negative examples of loop-closures can be bootstrapped using a GPS-aided navigation solution that the robot already uses. By leveraging the synchronization between sensors, we show that we are able to learn an appropriate distance metric for arbitrary real-valued image descriptors (including state-of-the-art CNN models), that is specifically geared for visual place recognition in mobile robots. Furthermore, we show that the newly learned embedding can be particularly powerful in disambiguating visual scenes for the task of vision-based loop-closure identification in mobile robots. |
| Tasks | Metric Learning, Robot Navigation, Visual Place Recognition |
| Published | 2019-05-11 |
| URL | https://arxiv.org/abs/1905.04453v1 |
| https://arxiv.org/pdf/1905.04453v1.pdf | |
| PWC | https://paperswithcode.com/paper/self-supervised-visual-place-recognition |
| Repo | |
| Framework | |
Macro-action Multi-time scale Dynamic Programming for Energy Management in Buildings with Phase Change Materials
| Title | Macro-action Multi-time scale Dynamic Programming for Energy Management in Buildings with Phase Change Materials |
| Authors | Zahra Rahimpour, Gregor Verbic, Archie C. Chapman |
| Abstract | This paper focuses on energy management in buildings with phase change material (PCM), which is primarily used to improve thermal performance, but can also serve as an energy storage system. In this setting, optimal scheduling of an HVAC system is challenging because of the nonlinear and non-convex characteristics of the PCM, which makes solving the corresponding optimization problem using conventional optimization techniques impractical. Instead, we use dynamic programming (DP) to deal with the nonlinear nature of the PCM. To overcome DP’s curse of dimensionality, this paper proposes a novel methodology to reduce the computational burden, while maintaining the quality of the solution. Specifically, the method incorporates approaches from sequential decision making in artificial intelligence, including macro actions and multi-time scale Markov decision processes, coupled with an underlying state-space approximation to reduce the state-space and action-space size. The performance of the method is demonstrated on an energy management problem for a typical residential building located in Sydney, Australia. The results demonstrate that the proposed method performs well with a computational speed-up of up to 12,900 times compared to the direct application of DP. |
| Tasks | Decision Making |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.05200v2 |
| https://arxiv.org/pdf/1906.05200v2.pdf | |
| PWC | https://paperswithcode.com/paper/macro-action-multi-timescale-dynamic |
| Repo | |
| Framework | |
Table-Based Neural Units: Fully Quantizing Networks for Multiply-Free Inference
| Title | Table-Based Neural Units: Fully Quantizing Networks for Multiply-Free Inference |
| Authors | Michele Covell, David Marwood, Shumeet Baluja, Nick Johnston |
| Abstract | In this work, we propose to quantize all parts of standard classification networks and replace the activation-weight–multiply step with a simple table-based lookup. This approach results in networks that are free of floating-point operations and free of multiplications, suitable for direct FPGA and ASIC implementations. It also provides us with two simple measures of per-layer and network-wide compactness as well as insight into the distribution characteristics of activationoutput and weight values. We run controlled studies across different quantization schemes, both fixed and adaptive and, within the set of adaptive approaches, both parametric and model-free. We implement our approach to quantization with minimal, localized changes to the training process, allowing us to benefit from advances in training continuous-valued network architectures. We apply our approach successfully to AlexNet, ResNet, and MobileNet. We show results that are within 1.6% of the reported, non-quantized performance on MobileNet using only 40 entries in our table. This performance gap narrows to zero when we allow tables with 320 entries. Our results give the best accuracies among multiply-free networks. |
| Tasks | Quantization |
| Published | 2019-06-11 |
| URL | https://arxiv.org/abs/1906.04798v1 |
| https://arxiv.org/pdf/1906.04798v1.pdf | |
| PWC | https://paperswithcode.com/paper/table-based-neural-units-fully-quantizing |
| Repo | |
| Framework | |
Combining Optimal Control and Learning for Visual Navigation in Novel Environments
| Title | Combining Optimal Control and Learning for Visual Navigation in Novel Environments |
| Authors | Somil Bansal, Varun Tolani, Saurabh Gupta, Jitendra Malik, Claire Tomlin |
| Abstract | Model-based control is a popular paradigm for robot navigation because it can leverage a known dynamics model to efficiently plan robust robot trajectories. However, it is challenging to use model-based methods in settings where the environment is a priori unknown and can only be observed partially through on-board sensors on the robot. In this work, we address this short-coming by coupling model-based control with learning-based perception. The learning-based perception module produces a series of waypoints that guide the robot to the goal via a collision-free path. These waypoints are used by a model-based planner to generate a smooth and dynamically feasible trajectory that is executed on the physical system using feedback control. Our experiments in simulated real-world cluttered environments and on an actual ground vehicle demonstrate that the proposed approach can reach goal locations more reliably and efficiently in novel environments as compared to purely geometric mapping-based or end-to-end learning-based alternatives. Our approach does not rely on detailed explicit 3D maps of the environment, works well with low frame rates, and generalizes well from simulation to the real world. Videos describing our approach and experiments are available on the project website. |
| Tasks | Robot Navigation, Visual Navigation |
| Published | 2019-03-06 |
| URL | https://arxiv.org/abs/1903.02531v2 |
| https://arxiv.org/pdf/1903.02531v2.pdf | |
| PWC | https://paperswithcode.com/paper/combining-optimal-control-and-learning-for |
| Repo | |
| Framework | |
