Paper Group AWR 362
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours. Autoencoders and Generative Adversarial Networks for Imbalanced Sequence Classification. Real or Fake? Spoofing State-Of-The-Art Face Synthesis Detection Systems. MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams. Planning Beyond the Sensing Horizon Using a L …
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours
| Title | Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours |
| Authors | Dimitrios Stamoulis, Ruizhou Ding, Di Wang, Dimitrios Lymberopoulos, Bodhi Priyantha, Jie Liu, Diana Marculescu |
| Abstract | Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one single-path over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar constraints (<80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPU-hours), which is up to 5,000x faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas. |
| Tasks | Image Classification, Neural Architecture Search |
| Published | 2019-04-05 |
| URL | http://arxiv.org/abs/1904.02877v1 |
| http://arxiv.org/pdf/1904.02877v1.pdf | |
| PWC | https://paperswithcode.com/paper/single-path-nas-designing-hardware-efficient |
| Repo | https://github.com/osmr/imgclsmob |
| Framework | mxnet |
Autoencoders and Generative Adversarial Networks for Imbalanced Sequence Classification
| Title | Autoencoders and Generative Adversarial Networks for Imbalanced Sequence Classification |
| Authors | Stephanie Ger, Diego Klabjan |
| Abstract | We introduce a novel synthetic oversampling method for variable length, multi-feature sequence datasetsbased on autoencoders and generative adversarial networks. We show that this method improves classification accuracy for highly imbalanced sequence classification tasks. We show that this method outperformsstandard oversampling techniques that use techniques such as SMOTE and autoencoders. We also use generative adversarial networks on the majority class as an outlier detection method for novelty detection, with limited classification improvement. We show that the use of generative adversarial network based synthetic data improves classification model performance on a variety of sequence data sets. |
| Tasks | Anomaly Detection, Outlier Detection |
| Published | 2019-01-08 |
| URL | https://arxiv.org/abs/1901.02514v4 |
| https://arxiv.org/pdf/1901.02514v4.pdf | |
| PWC | https://paperswithcode.com/paper/autoencoders-and-generative-adversarial |
| Repo | https://github.com/stephanieger/sequence-anomaly-detection |
| Framework | none |
Real or Fake? Spoofing State-Of-The-Art Face Synthesis Detection Systems
| Title | Real or Fake? Spoofing State-Of-The-Art Face Synthesis Detection Systems |
| Authors | João C. Neves, Ruben Tolosana, Ruben Vera-Rodriguez, Vasco Lopes, Hugo Proença |
| Abstract | The availability of large-scale facial databases, together with the remarkable progresses of deep learning technologies, in particular Generative Adversarial Networks (GANs), have led to the generation of extremely realistic fake facial content, which raises obvious concerns about the potential for misuse. These concerns have fostered the research of manipulation detection methods that, contrary to humans, have already achieved astonishing results in some scenarios. In this study, we focus on the entire face synthesis, which is one specific type of facial manipulation. The main contributions of this study are: i) a novel strategy to remove GAN “fingerprints” from synthetic fake images in order to spoof facial manipulation detection systems, while keeping the visual quality of the resulting images, ii) an in-depth analysis of state-of-the-art detection approaches for the entire face synthesis manipulation, iii) a complete experimental assessment of this type of facial manipulation considering state-of-the-art detection systems, remarking how challenging is this task in unconstrained scenarios, and finally iv) a novel public database named FSRemovalDB produced after applying our proposed GAN-fingerprint removal approach to original synthetic fake images. The observed results led us to conclude that more efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques such as the one proposed in this study. |
| Tasks | Face Generation |
| Published | 2019-11-13 |
| URL | https://arxiv.org/abs/1911.05351v2 |
| https://arxiv.org/pdf/1911.05351v2.pdf | |
| PWC | https://paperswithcode.com/paper/real-or-fake-spoofing-state-of-the-art-face |
| Repo | https://github.com/joaocneves/gan_fingerprint_removal |
| Framework | pytorch |
MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams
| Title | MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams |
| Authors | Siddharth Bhatia, Bryan Hooi, Minji Yoon, Kijung Shin, Christos Faloutsos |
| Abstract | Given a stream of graph edges from a dynamic graph, how can we assign anomaly scores to edges in an online manner, for the purpose of detecting unusual behavior, using constant time and memory? Existing approaches aim to detect individually surprising edges. In this work, we propose MIDAS, which focuses on detecting microcluster anomalies, or suddenly arriving groups of suspiciously similar edges, such as lockstep behavior, including denial of service attacks in network traffic data. MIDAS has the following properties: (a) it detects microcluster anomalies while providing theoretical guarantees about its false positive probability; (b) it is online, thus processing each edge in constant time and constant memory, and also processes the data 162-644 times faster than state-of-the-art approaches; (c) it provides 42%-48% higher accuracy (in terms of AUC) than state-of-the-art approaches. |
| Tasks | Anomaly Detection in Edge Streams |
| Published | 2019-11-11 |
| URL | https://arxiv.org/abs/1911.04464v3 |
| https://arxiv.org/pdf/1911.04464v3.pdf | |
| PWC | https://paperswithcode.com/paper/midas-microcluster-based-detector-of |
| Repo | https://github.com/bhatiasiddharth/MIDAS |
| Framework | none |
Planning Beyond the Sensing Horizon Using a Learned Context
| Title | Planning Beyond the Sensing Horizon Using a Learned Context |
| Authors | Michael Everett, Justin Miller, Jonathan P. How |
| Abstract | Last-mile delivery systems commonly propose the use of autonomous robotic vehicles to increase scalability and efficiency. The economic inefficiency of collecting accurate prior maps for navigation motivates the use of planning algorithms that operate in unmapped environments. However, these algorithms typically waste time exploring regions that are unlikely to contain the delivery destination. Context is key information about structured environments that could guide exploration toward the unknown goal location, but the abstract idea is difficult to quantify for use in a planning algorithm. Some approaches specifically consider contextual relationships between objects, but would perform poorly in object-sparse environments like outdoors. Recent deep learning-based approaches consider context too generally, making training/transferability difficult. Therefore, this work proposes a novel formulation of utilizing context for planning as an image-to-image translation problem, which is shown to extract terrain context from semantic gridmaps, into a metric that an exploration-based planner can use. The proposed framework has the benefit of training on a static dataset instead of requiring a time-consuming simulator. Across 42 test houses with layouts from satellite images, the trained algorithm enables a robot to reach its goal 189% faster than with a context-unaware planner, and within 63% of the optimal path computed with a prior map. The proposed algorithm is also implemented on a vehicle with a forward-facing camera in a high-fidelity, Unreal simulation of neighborhood houses. |
| Tasks | Image-to-Image Translation |
| Published | 2019-08-24 |
| URL | https://arxiv.org/abs/1908.09171v2 |
| https://arxiv.org/pdf/1908.09171v2.pdf | |
| PWC | https://paperswithcode.com/paper/planning-beyond-the-sensing-horizon-using-a |
| Repo | https://github.com/maximecb/gym-minigrid |
| Framework | pytorch |
VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation
| Title | VUSFA:Variational Universal Successor Features Approximator to Improve Transfer DRL for Target Driven Visual Navigation |
| Authors | Shamane Siriwardhana, Rivindu Weerasakera, Denys J. C. Matthies, Suranga Nanayakkara |
| Abstract | In this paper, we show how novel transfer reinforcement learning techniques can be applied to the complex task of target driven navigation using the photorealistic AI2THOR simulator. Specifically, we build on the concept of Universal Successor Features with an A3C agent. We introduce the novel architectural contribution of a Successor Feature Dependant Policy (SFDP) and adopt the concept of Variational Information Bottlenecks to achieve state of the art performance. VUSFA, our final architecture, is a straightforward approach that can be implemented using our open source repository. Our approach is generalizable, showed greater stability in training, and outperformed recent approaches in terms of transfer learning ability. |
| Tasks | Transfer Learning, Transfer Reinforcement Learning, Visual Navigation |
| Published | 2019-08-18 |
| URL | https://arxiv.org/abs/1908.06376v1 |
| https://arxiv.org/pdf/1908.06376v1.pdf | |
| PWC | https://paperswithcode.com/paper/vusfavariational-universal-successor-features |
| Repo | https://github.com/shamanez/VUSFA-Variational-Universal-Successor-Features-Approximator |
| Framework | tf |
TableBank: Table Benchmark for Image-based Table Detection and Recognition
| Title | TableBank: Table Benchmark for Image-based Table Detection and Recognition |
| Authors | Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, Zhoujun Li |
| Abstract | We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet. Existing research for image-based table detection and recognition usually fine-tunes pre-trained models on out-of-domain data with a few thousands human labeled examples, which is difficult to generalize on real world applications. With TableBank that contains 417K high-quality labeled tables, we build several strong baselines using state-of-the-art models with deep neural networks. We make TableBank publicly available (https://github.com/doc-analysis/TableBank) and hope it will empower more deep learning approaches in the table detection and recognition task. |
| Tasks | Table Detection |
| Published | 2019-03-05 |
| URL | http://arxiv.org/abs/1903.01949v1 |
| http://arxiv.org/pdf/1903.01949v1.pdf | |
| PWC | https://paperswithcode.com/paper/tablebank-table-benchmark-for-image-based |
| Repo | https://github.com/doc-analysis/TableBank |
| Framework | none |
Materials property prediction using symmetry-labeled graphs as atomic-position independent descriptors
| Title | Materials property prediction using symmetry-labeled graphs as atomic-position independent descriptors |
| Authors | Peter Bjørn Jørgensen, Estefanía Garijo del Río, Mikkel N. Schmidt, Karsten Wedel Jacobsen |
| Abstract | Computational materials screening studies require fast calculation of the properties of thousands of materials. The calculations are often performed with Density Functional Theory (DFT), but the necessary computer time sets limitations for the investigated material space. Therefore, the development of machine learning models for prediction of DFT calculated properties are currently of interest. A particular challenge for \emph{new} materials is that the atomic positions are generally not known. We present a machine learning model for the prediction of DFT-calculated formation energies based on Voronoi quotient graphs and local symmetry classification without the need for detailed information about atomic positions. The model is implemented as a message passing neural network and tested on the Open Quantum Materials Database (OQMD) and the Materials Project database. The test mean absolute error is 20 meV on the OQMD database and 40 meV on Materials Project Database. The possibilities for prediction in a realistic computational screening setting is investigated on a dataset of 5976 ABSe$_3$ selenides with very limited overlap with the OQMD training set. Pretraining on OQMD and subsequent training on 100 selenides result in a mean absolute error below 0.1 eV for the formation energy of the selenides. |
| Tasks | Formation Energy, Materials Screening |
| Published | 2019-05-15 |
| URL | https://arxiv.org/abs/1905.06048v3 |
| https://arxiv.org/pdf/1905.06048v3.pdf | |
| PWC | https://paperswithcode.com/paper/materials-property-prediction-using-symmetry |
| Repo | https://github.com/peterbjorgensen/vorosym |
| Framework | none |
An Efficient Approximate kNN Graph Method for Diffusion on Image Retrieval
| Title | An Efficient Approximate kNN Graph Method for Diffusion on Image Retrieval |
| Authors | Federico Magliani, Kevin McGuinness, Eva Mohedano, Andrea Prati |
| Abstract | The application of the diffusion in many computer vision and artificial intelligence projects has been shown to give excellent improvements in performance. One of the main bottlenecks of this technique is the quadratic growth of the kNN graph size due to the high-quantity of new connections between nodes in the graph, resulting in long computation times. Several strategies have been proposed to address this, but none are effective and efficient. Our novel technique, based on LSH projections, obtains the same performance as the exact kNN graph after diffusion, but in less time (approximately 18 times faster on a dataset of a hundred thousand images). The proposed method was validated and compared with other state-of-the-art on several public image datasets, including Oxford5k, Paris6k, and Oxford105k. |
| Tasks | Image Retrieval |
| Published | 2019-04-18 |
| URL | http://arxiv.org/abs/1904.08668v1 |
| http://arxiv.org/pdf/1904.08668v1.pdf | |
| PWC | https://paperswithcode.com/paper/190408668 |
| Repo | https://github.com/fmaglia/LSH_kNN_graph |
| Framework | none |
Unconstrained Monotonic Neural Networks
| Title | Unconstrained Monotonic Neural Networks |
| Authors | Antoine Wehenkel, Gilles Louppe |
| Abstract | Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous probability distributions. Architectures that ensure monotonicity typically enforce constraints on weights and activation functions, which enables invertibility but leads to a cap on the expressiveness of the resulting transformations. In this work, we propose the Unconstrained Monotonic Neural Network (UMNN) architecture based on the insight that a function is monotonic as long as its derivative is strictly positive. In particular, this latter condition can be enforced with a free-form neural network whose only constraint is the positiveness of its output. We evaluate our new invertible building block within a new autoregressive flow (UMNN-MAF) and demonstrate its effectiveness on density estimation experiments. We also illustrate the ability of UMNNs to improve variational inference. |
| Tasks | Density Estimation |
| Published | 2019-08-14 |
| URL | https://arxiv.org/abs/1908.05164v2 |
| https://arxiv.org/pdf/1908.05164v2.pdf | |
| PWC | https://paperswithcode.com/paper/unconstrained-monotonic-neural-networks |
| Repo | https://github.com/AWehenkel/UMNN |
| Framework | pytorch |
Deep neural network approximations for Monte Carlo algorithms
| Title | Deep neural network approximations for Monte Carlo algorithms |
| Authors | Philipp Grohs, Arnulf Jentzen, Diyora Salimova |
| Abstract | Recently, it has been proposed in the literature to employ deep neural networks (DNNs) together with stochastic gradient descent methods to approximate solutions of PDEs. There are also a few results in the literature which prove that DNNs can approximate solutions of certain PDEs without the curse of dimensionality in the sense that the number of real parameters used to describe the DNN grows at most polynomially both in the PDE dimension and the reciprocal of the prescribed approximation accuracy. One key argument in most of these results is, first, to use a Monte Carlo approximation scheme which can approximate the solution of the PDE under consideration at a fixed space-time point without the curse of dimensionality and, thereafter, to prove that DNNs are flexible enough to mimic the behaviour of the used approximation scheme. Having this in mind, one could aim for a general abstract result which shows under suitable assumptions that if a certain function can be approximated by any kind of (Monte Carlo) approximation scheme without the curse of dimensionality, then this function can also be approximated with DNNs without the curse of dimensionality. It is a key contribution of this article to make a first step towards this direction. In particular, the main result of this paper, essentially, shows that if a function can be approximated by means of some suitable discrete approximation scheme without the curse of dimensionality and if there exist DNNs which satisfy certain regularity properties and which approximate this discrete approximation scheme without the curse of dimensionality, then the function itself can also be approximated with DNNs without the curse of dimensionality. As an application of this result we establish that solutions of suitable Kolmogorov PDEs can be approximated with DNNs without the curse of dimensionality. |
| Tasks | |
| Published | 2019-08-28 |
| URL | https://arxiv.org/abs/1908.10828v1 |
| https://arxiv.org/pdf/1908.10828v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-neural-network-approximations-for-monte |
| Repo | https://github.com/SmartAppUnipi/HowToBeGraded30 |
| Framework | none |
A Discussion on Solving Partial Differential Equations using Neural Networks
| Title | A Discussion on Solving Partial Differential Equations using Neural Networks |
| Authors | Tim Dockhorn |
| Abstract | Can neural networks learn to solve partial differential equations (PDEs)? We investigate this question for two (systems of) PDEs, namely, the Poisson equation and the steady Navier–Stokes equations. The contributions of this paper are five-fold. (1) Numerical experiments show that small neural networks (< 500 learnable parameters) are able to accurately learn complex solutions for systems of partial differential equations. (2) It investigates the influence of random weight initialization on the quality of the neural network approximate solution and demonstrates how one can take advantage of this non-determinism using ensemble learning. (3) It investigates the suitability of the loss function used in this work. (4) It studies the benefits and drawbacks of solving (systems of) PDEs with neural networks compared to classical numerical methods. (5) It proposes an exhaustive list of possible directions of future work. |
| Tasks | |
| Published | 2019-04-15 |
| URL | http://arxiv.org/abs/1904.07200v1 |
| http://arxiv.org/pdf/1904.07200v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-discussion-on-solving-partial-differential |
| Repo | https://github.com/timudk/SPDENN |
| Framework | tf |
RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completion
| Title | RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completion |
| Authors | Muhammad Sarmad, Hyunjoo Jenny Lee, Young Min Kim |
| Abstract | We present RL-GAN-Net, where a reinforcement learning (RL) agent provides fast and robust control of a generative adversarial network (GAN). Our framework is applied to point cloud shape completion that converts noisy, partial point cloud data into a high-fidelity completed shape by controlling the GAN. While a GAN is unstable and hard to train, we circumvent the problem by (1) training the GAN on the latent space representation whose dimension is reduced compared to the raw point cloud input and (2) using an RL agent to find the correct input to the GAN to generate the latent space representation of the shape that best fits the current input of incomplete point cloud. The suggested pipeline robustly completes point cloud with large missing regions. To the best of our knowledge, this is the first attempt to train an RL agent to control the GAN, which effectively learns the highly nonlinear mapping from the input noise of the GAN to the latent space of point cloud. The RL agent replaces the need for complex optimization and consequently makes our technique real time. Additionally, we demonstrate that our pipelines can be used to enhance the classification accuracy of point cloud with missing data. |
| Tasks | |
| Published | 2019-04-28 |
| URL | http://arxiv.org/abs/1904.12304v1 |
| http://arxiv.org/pdf/1904.12304v1.pdf | |
| PWC | https://paperswithcode.com/paper/rl-gan-net-a-reinforcement-learning-agent |
| Repo | https://github.com/iSarmad/RL-GAN-Net |
| Framework | pytorch |
Machine Discovery of Partial Differential Equations from Spatiotemporal Data
| Title | Machine Discovery of Partial Differential Equations from Spatiotemporal Data |
| Authors | Ye Yuan, Junlin Li, Liang Li, Frank Jiang, Xiuchuan Tang, Fumin Zhang, Sheng Liu, Jorge Goncalves, Henning U. Voss, Xiuting Li, Jürgen Kurths, Han Ding |
| Abstract | The study presents a general framework for discovering underlying Partial Differential Equations (PDEs) using measured spatiotemporal data. The method, called Sparse Spatiotemporal System Discovery ($\text{S}^3\text{d}$), decides which physical terms are necessary and which can be removed (because they are physically negligible in the sense that they do not affect the dynamics too much) from a pool of candidate functions. The method is built on the recent development of Sparse Bayesian Learning; which enforces the sparsity in the to-be-identified PDEs, and therefore can balance the model complexity and fitting error with theoretical guarantees. Without leveraging prior knowledge or assumptions in the discovery process, we use an automated approach to discover ten types of PDEs, including the famous Navier-Stokes and sine-Gordon equations, from simulation data alone. Moreover, we demonstrate our data-driven discovery process with the Complex Ginzburg-Landau Equation (CGLE) using data measured from a traveling-wave convection experiment. Our machine discovery approach presents solutions that has the potential to inspire, support and assist physicists for the establishment of physical laws from measured spatiotemporal data, especially in notorious fields that are often too complex to allow a straightforward establishment of physical law, such as biophysics, fluid dynamics, neuroscience or nonlinear optics. |
| Tasks | |
| Published | 2019-09-15 |
| URL | https://arxiv.org/abs/1909.06730v1 |
| https://arxiv.org/pdf/1909.06730v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-discovery-of-partial-differential |
| Repo | https://github.com/HAIRLAB/S3d |
| Framework | none |
Copy and Paste: A Simple But Effective Initialization Method for Black-Box Adversarial Attacks
| Title | Copy and Paste: A Simple But Effective Initialization Method for Black-Box Adversarial Attacks |
| Authors | Thomas Brunner, Frederik Diehl, Alois Knoll |
| Abstract | Many optimization methods for generating black-box adversarial examples have been proposed, but the aspect of initializing said optimizers has not been considered in much detail. We show that the choice of starting points is indeed crucial, and that the performance of state-of-the-art attacks depends on it. First, we discuss desirable properties of starting points for attacking image classifiers, and how they can be chosen to increase query efficiency. Notably, we find that simply copying small patches from other images is a valid strategy. We then present an evaluation on ImageNet that clearly demonstrates the effectiveness of this method: Our initialization scheme reduces the number of queries required for a state-of-the-art Boundary Attack by 81%, significantly outperforming previous results reported for targeted black-box adversarial examples. |
| Tasks | |
| Published | 2019-06-14 |
| URL | https://arxiv.org/abs/1906.06086v2 |
| https://arxiv.org/pdf/1906.06086v2.pdf | |
| PWC | https://paperswithcode.com/paper/copy-and-paste-a-simple-but-effective |
| Repo | https://github.com/ttbrunner/blackbox_starting_points |
| Framework | tf |
