October 18, 2019

3172 words 15 mins read

Paper Group ANR 583

DeepLaser: Practical Fault Attack on Deep Neural Networks. A Blended Deep Learning Approach for Predicting User Intended Actions. Finger-GAN: Generating Realistic Fingerprint Images Using Connectivity Imposed GAN. A Main/Subsidiary Network Framework for Simplifying Binary Neural Network. Channel-wise pruning of neural networks with tapering resourc …

DeepLaser: Practical Fault Attack on Deep Neural Networks


Title	DeepLaser: Practical Fault Attack on Deep Neural Networks
Authors	Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
Abstract	As deep learning systems are widely adopted in safety- and security-critical applications, such as autonomous vehicles, banking systems, etc., malicious faults and attacks become a tremendous concern, which potentially could lead to catastrophic consequences. In this paper, we initiate the first study of leveraging physical fault injection attacks on Deep Neural Networks (DNNs), by using laser injection technique on embedded systems. In particular, our exploratory study targets four widely used activation functions in DNNs development, that are the general main building block of DNNs that creates non-linear behaviors – ReLu, softmax, sigmoid, and tanh. Our results show that by targeting these functions, it is possible to achieve a misclassification by injecting faults into the hidden layer of the network. Such result can have practical implications for real-world applications, where faults can be introduced by simpler means (such as altering the supply voltage).
Tasks	Autonomous Vehicles
Published	2018-06-15
URL	http://arxiv.org/abs/1806.05859v2
PDF	http://arxiv.org/pdf/1806.05859v2.pdf
PWC	https://paperswithcode.com/paper/deeplaser-practical-fault-attack-on-deep
Repo
Framework

A Blended Deep Learning Approach for Predicting User Intended Actions


Title	A Blended Deep Learning Approach for Predicting User Intended Actions
Authors	Fei Tan, Zhi Wei, Jun He, Xiang Wu, Bo Peng, Haoran Liu, Zhenyu Yan
Abstract	User intended actions are widely seen in many areas. Forecasting these actions and taking proactive measures to optimize business outcome is a crucial step towards sustaining the steady business growth. In this work, we focus on pre- dicting attrition, which is one of typical user intended actions. Conventional attrition predictive modeling strategies suffer a few inherent drawbacks. To overcome these limitations, we propose a novel end-to-end learning scheme to keep track of the evolution of attrition patterns for the predictive modeling. It integrates user activity logs, dynamic and static user profiles based on multi-path learning. It exploits historical user records by establishing a decaying multi-snapshot technique. And finally it employs the precedent user intentions via guiding them to the subsequent learning procedure. As a result, it addresses all disadvantages of conventional methods. We evaluate our methodology on two public data repositories and one private user usage dataset provided by Adobe Creative Cloud. The extensive experiments demonstrate that it can offer the appealing performance in comparison with several existing approaches as rated by different popular metrics. Furthermore, we introduce an advanced interpretation and visualization strategy to effectively characterize the periodicity of user activity logs. It can help to pinpoint important factors that are critical to user attrition and retention and thus suggests actionable improvement targets for business practice. Our work will provide useful insights into the prediction and elucidation of other user intended actions as well.
Tasks
Published	2018-10-11
URL	http://arxiv.org/abs/1810.04824v1
PDF	http://arxiv.org/pdf/1810.04824v1.pdf
PWC	https://paperswithcode.com/paper/a-blended-deep-learning-approach-for
Repo
Framework

Finger-GAN: Generating Realistic Fingerprint Images Using Connectivity Imposed GAN


Title	Finger-GAN: Generating Realistic Fingerprint Images Using Connectivity Imposed GAN
Authors	Shervin Minaee, Amirali Abdolrashidi
Abstract	Generating realistic biometric images has been an interesting and, at the same time, challenging problem. Classical statistical models fail to generate realistic-looking fingerprint images, as they are not powerful enough to capture the complicated texture representation in fingerprint images. In this work, we present a machine learning framework based on generative adversarial networks (GAN), which is able to generate fingerprint images sampled from a prior distribution (learned from a set of training images). We also add a suitable regularization term to the loss function, to impose the connectivity of generated fingerprint images. This is highly desirable for fingerprints, as the lines in each finger are usually connected. We apply this framework to two popular fingerprint databases, and generate images which look very realistic, and similar to the samples in those databases. Through experimental results, we show that the generated fingerprint images have a good diversity, and are able to capture different parts of the prior distribution. We also evaluate the Frechet Inception distance (FID) of our proposed model, and show that our model is able to achieve good quantitative performance in terms of this score.
Tasks
Published	2018-12-25
URL	http://arxiv.org/abs/1812.10482v1
PDF	http://arxiv.org/pdf/1812.10482v1.pdf
PWC	https://paperswithcode.com/paper/finger-gan-generating-realistic-fingerprint
Repo
Framework

A Main/Subsidiary Network Framework for Simplifying Binary Neural Network


Title	A Main/Subsidiary Network Framework for Simplifying Binary Neural Network
Authors	Yinghao Xu, Xin Dong, Yudian Li, Hao Su
Abstract	To reduce memory footprint and run-time latency, techniques such as neural network pruning and binarization have been explored separately. However, it is unclear how to combine the best of the two worlds to get extremely small and efficient models. In this paper, we, for the first time, define the filter-level pruning problem for binary neural networks, which cannot be solved by simply migrating existing structural pruning methods for full-precision models. A novel learning-based approach is proposed to prune filters in our main/subsidiary network framework, where the main network is responsible for learning representative features to optimize the prediction performance, and the subsidiary component works as a filter selector on the main network. To avoid gradient mismatch when training the subsidiary component, we propose a layer-wise and bottom-up scheme. We also provide the theoretical and experimental comparison between our learning-based and greedy rule-based methods. Finally, we empirically demonstrate the effectiveness of our approach applied on several binary models, including binarized NIN, VGG-11, and ResNet-18, on various image classification datasets.
Tasks	Image Classification, Network Pruning
Published	2018-12-11
URL	http://arxiv.org/abs/1812.04210v1
PDF	http://arxiv.org/pdf/1812.04210v1.pdf
PWC	https://paperswithcode.com/paper/a-mainsubsidiary-network-framework-for
Repo
Framework

Channel-wise pruning of neural networks with tapering resource constraint


Title	Channel-wise pruning of neural networks with tapering resource constraint
Authors	Alexey Kruglov
Abstract	Neural network pruning is an important step in design process of efficient neural networks for edge devices with limited computational power. Pruning is a form of knowledge transfer from the weights of the original network to a smaller target subnetwork. We propose a new method for compute-constrained structured channel-wise pruning of convolutional neural networks. The method iteratively fine-tunes the network, while gradually tapering the computation resources available to the pruned network via a holonomic constraint in the method of Lagrangian multipliers framework. An explicit and adaptive automatic control over the rate of tapering is provided. The trainable parameters of our pruning method are separate from the weights of the neural network, which allows us to avoid the interference with the neural network solver (e.g. avoid the direct dependence of pruning speed on neural network learning rates). Our method combines the `rigoristic’ approach by the direct application of constrained optimization, avoiding the pitfalls of ADMM-based methods, like their need to define the target amount of resources for each pruning run, and direct dependence of pruning speed and priority of pruning on the relative scale of weights between layers. For VGG-16 @ ILSVRC-2012, we achieve reduction of 15.47 -> 3.87 GMAC with only 1% top-1 accuracy reduction (68.4% -> 67.4%). For AlexNet @ ILSVRC-2012, we achieve 0.724 -> 0.411 GMAC with 1% top-1 accuracy reduction (56.8% -> 55.8%). \|
Tasks	Network Pruning, Transfer Learning
Published	2018-12-04
URL	https://arxiv.org/abs/1812.07060v1
PDF	https://arxiv.org/pdf/1812.07060v1.pdf
PWC	https://paperswithcode.com/paper/channel-wise-pruning-of-neural-networks-with
Repo
Framework

Target-Independent Active Learning via Distribution-Splitting


Title	Target-Independent Active Learning via Distribution-Splitting
Authors	Xiaofeng Cao, Ivor W. Tsang, Xiaofeng Xu, Guandong Xu
Abstract	To reduce the label complexity in Agnostic Active Learning (A^2 algorithm), volume-splitting splits the hypothesis edges to reduce the Vapnik-Chervonenkis (VC) dimension in version space. However, the effectiveness of volume-splitting critically depends on the initial hypothesis and this problem is also known as target-dependent label complexity gap. This paper attempts to minimize this gap by introducing a novel notion of number density which provides a more natural and direct way to describe the hypothesis distribution than volume. By discovering the connections between hypothesis and input distribution, we map the volume of version space into the number density and propose a target-independent distribution-splitting strategy with the following advantages: 1) provide theoretical guarantees on reducing label complexity and error rate as volume-splitting; 2) break the curse of initial hypothesis; 3) provide model guidance for a target-independent AL algorithm in real AL tasks. With these guarantees, for AL application, we then split the input distribution into more near-optimal spheres and develop an application algorithm called Distribution-based A^2 (DA^2). Experiments further verify the effectiveness of the halving and querying abilities of DA^2. Contributions of this paper are as follows.
Tasks	Active Learning
Published	2018-09-28
URL	http://arxiv.org/abs/1809.10962v1
PDF	http://arxiv.org/pdf/1809.10962v1.pdf
PWC	https://paperswithcode.com/paper/target-independent-active-learning-via
Repo
Framework

Synaptic partner prediction from point annotations in insect brains


Title	Synaptic partner prediction from point annotations in insect brains
Authors	Julia Buhmann, Renate Krause, Rodrigo Ceballos Lentini, Nils Eckstein, Matthew Cook, Srinivas Turaga, Jan Funke
Abstract	High-throughput electron microscopy allows recording of lar- ge stacks of neural tissue with sufficient resolution to extract the wiring diagram of the underlying neural network. Current efforts to automate this process focus mainly on the segmentation of neurons. However, in order to recover a wiring diagram, synaptic partners need to be identi- fied as well. This is especially challenging in insect brains like Drosophila melanogaster, where one presynaptic site is associated with multiple post- synaptic elements. Here we propose a 3D U-Net architecture to directly identify pairs of voxels that are pre- and postsynaptic to each other. To that end, we formulate the problem of synaptic partner identification as a classification problem on long-range edges between voxels to encode both the presence of a synaptic pair and its direction. This formulation allows us to directly learn from synaptic point annotations instead of more ex- pensive voxel-based synaptic cleft or vesicle annotations. We evaluate our method on the MICCAI 2016 CREMI challenge and improve over the current state of the art, producing 3% fewer errors than the next best method.
Tasks
Published	2018-06-21
URL	http://arxiv.org/abs/1806.08205v2
PDF	http://arxiv.org/pdf/1806.08205v2.pdf
PWC	https://paperswithcode.com/paper/synaptic-partner-prediction-from-point
Repo
Framework

Query-Efficient Black-Box Attack by Active Learning


Title	Query-Efficient Black-Box Attack by Active Learning
Authors	Pengcheng Li, Jinfeng Yi, Lijun Zhang
Abstract	Deep neural network (DNN) as a popular machine learning model is found to be vulnerable to adversarial attack. This attack constructs adversarial examples by adding small perturbations to the raw input, while appearing unmodified to human eyes but will be misclassified by a well-trained classifier. In this paper, we focus on the black-box attack setting where attackers have almost no access to the underlying models. To conduct black-box attack, a popular approach aims to train a substitute model based on the information queried from the target DNN. The substitute model can then be attacked using existing white-box attack approaches, and the generated adversarial examples will be used to attack the target DNN. Despite its encouraging results, this approach suffers from poor query efficiency, i.e., attackers usually needs to query a huge amount of times to collect enough information for training an accurate substitute model. To this end, we first utilize state-of-the-art white-box attack methods to generate samples for querying, and then introduce an active learning strategy to significantly reduce the number of queries needed. Besides, we also propose a diversity criterion to avoid the sampling bias. Our extensive experimental results on MNIST and CIFAR-10 show that the proposed method can reduce more than $90%$ of queries while preserve attacking success rates and obtain an accurate substitute model which is more than $85%$ similar with the target oracle.
Tasks	Active Learning, Adversarial Attack
Published	2018-09-13
URL	http://arxiv.org/abs/1809.04913v1
PDF	http://arxiv.org/pdf/1809.04913v1.pdf
PWC	https://paperswithcode.com/paper/query-efficient-black-box-attack-by-active
Repo
Framework

Three Dimensional Convolutional Neural Network Pruning with Regularization-Based Method


Title	Three Dimensional Convolutional Neural Network Pruning with Regularization-Based Method
Authors	Yuxin Zhang, Huan Wang, Yang Luo, Lu Yu, Haoji Hu, Hangguan Shan, Tony Q. S. Quek
Abstract	Despite enjoying extensive applications in video analysis, three-dimensional convolutional neural networks (3D CNNs)are restricted by their massive computation and storage consumption. To solve this problem, we propose a threedimensional regularization-based neural network pruning method to assign different regularization parameters to different weight groups based on their importance to the network. Further we analyze the redundancy and computation cost for each layer to determine the different pruning ratios. Experiments show that pruning based on our method can lead to 2x theoretical speedup with only 0.41% accuracy loss for 3DResNet18 and 3.28% accuracy loss for C3D. The proposed method performs favorably against other popular methods for model compression and acceleration.
Tasks	Model Compression, Network Pruning
Published	2018-11-19
URL	https://arxiv.org/abs/1811.07555v2
PDF	https://arxiv.org/pdf/1811.07555v2.pdf
PWC	https://paperswithcode.com/paper/three-dimensional-convolutional-neural
Repo
Framework

Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings


Title	Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings
Authors	Shao-Yen Tseng, Brian Baucom, Panayiotis Georgiou
Abstract	Unsupervised learning has been an attractive method for easily deriving meaningful data representations from vast amounts of unlabeled data. These representations, or embeddings, often yield superior results in many tasks, whether used directly or as features in subsequent training stages. However, the quality of the embeddings is highly dependent on the assumed knowledge in the unlabeled data and how the system extracts information without supervision. Domain portability is also very limited in unsupervised learning, often requiring re-training on other in-domain corpora to achieve robustness. In this work we present a multitask paradigm for unsupervised contextual learning of behavioral interactions which addresses unsupervised domain adaption. We introduce an online multitask objective into unsupervised learning and show that sentence embeddings generated through this process increases performance of affective tasks.
Tasks	Domain Adaptation, Sentence Embeddings
Published	2018-07-18
URL	http://arxiv.org/abs/1807.06792v2
PDF	http://arxiv.org/pdf/1807.06792v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-online-multitask-learning-of
Repo
Framework

Convolutional Neural Network Pruning to Accelerate Membrane Segmentation in Electron Microscopy


Title	Convolutional Neural Network Pruning to Accelerate Membrane Segmentation in Electron Microscopy
Authors	Joris Roels, Jonas De Vylder, Jan Aelterman, Yvan Saeys, Wilfried Philips
Abstract	Biological membranes are one of the most basic structures and regions of interest in cell biology. In the study of membranes, segment extraction is a well-known and difficult problem because of impeding noise, directional and thickness variability, etc. Recent advances in electron microscopy membrane segmentation are able to cope with such difficulties by training convolutional neural networks. However, because of the massive amount of features that have to be extracted while propagating forward, the practical usability diminishes, even with state-of-the-art GPU’s. A significant part of these network features typically contains redundancy through correlation and sparsity. In this work, we propose a pruning method for convolutional neural networks that ensures the training loss increase is minimized. We show that the pruned networks, after retraining, are more efficient in terms of time and memory, without significantly affecting the network accuracy. This way, we manage to obtain real-time membrane segmentation performance, for our specific electron microscopy setup.
Tasks	Network Pruning
Published	2018-10-23
URL	http://arxiv.org/abs/1810.09735v1
PDF	http://arxiv.org/pdf/1810.09735v1.pdf
PWC	https://paperswithcode.com/paper/convolutional-neural-network-pruning-to
Repo
Framework

Light Field Neural Network


Title	Light Field Neural Network
Authors	Yuchi Huo, Sung-Eui Yoon
Abstract	We introduce an optical neural network system made by off-the-shelf components. In order to test the evaluate the physical property of the proposed system, we are making a prototype. After further discussions with our cooperators, we are agreed that the prototype implementation may take longer time than we expected earlier. Therefore we reach a consensus on withdrawing the paper until the physical data is available.
Tasks
Published	2018-09-19
URL	http://arxiv.org/abs/1809.07009v2
PDF	http://arxiv.org/pdf/1809.07009v2.pdf
PWC	https://paperswithcode.com/paper/light-field-neural-network
Repo
Framework

Coordinating users of shared facilities via data-driven predictive assistants and game theory


Title	Coordinating users of shared facilities via data-driven predictive assistants and game theory
Authors	Philipp Geiger, Michel Besserve, Justus Winkelmann, Claudius Proissl, Bernhard Schölkopf
Abstract	We study data-driven assistants that provide congestion forecasts to users of shared facilities (roads, cafeterias, etc.), to support coordination between them, and increase efficiency of such collective systems. Key questions are: (1) when and how much can (accurate) predictions help for coordination, and (2) which assistant algorithms reach optimal predictions? First we lay conceptual ground for this setting where user preferences are a priori unknown and predictions influence outcomes. Addressing (1), we establish conditions under which self-fulfilling prophecies, i.e., “perfect” (probabilistic) predictions of what will happen, solve the coordination problem in the game-theoretic sense of selecting a Bayesian Nash equilibrium (BNE). Next we prove that such prophecies exist even in large-scale settings where only aggregated statistics about users are available. This entails a new (nonatomic) BNE existence result. Addressing (2), we propose two assistant algorithms that sequentially learn from users’ reactions, together with optimality/convergence guarantees. We validate one of them in a large real-world experiment.
Tasks	Time Series
Published	2018-03-16
URL	https://arxiv.org/abs/1803.06247v5
PDF	https://arxiv.org/pdf/1803.06247v5.pdf
PWC	https://paperswithcode.com/paper/coordination-via-predictive-assistants-time
Repo
Framework

Distributed Data Vending on Blockchain


Title	Distributed Data Vending on Blockchain
Authors	Jiayu Zhou, Fengyi Tang, He Zhu, Ning Nan, Ziheng Zhou
Abstract	Recent advances in blockchain technologies have provided exciting opportunities for decentralized applications. Specifically, blockchain-based smart contracts enable credible transactions without authorized third parties. The attractive properties of smart contracts facilitate distributed data vending, allowing for proprietary data to be securely exchanged on a blockchain. Distributed data vending can transform domains such as healthcare by encouraging data distribution from owners and enabling large-scale data aggregation. However, one key challenge in distributed data vending is the trade-off dilemma between the effectiveness of data retrieval, and the leakage risk from indexing the data. In this paper, we propose a framework for distributed data vending through a combination of data embedding and similarity learning. We illustrate our framework through a practical scenario of distributing and aggregating electronic medical records on a blockchain. Extensive empirical results demonstrate the effectiveness of our framework.
Tasks
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05871v2
PDF	http://arxiv.org/pdf/1803.05871v2.pdf
PWC	https://paperswithcode.com/paper/distributed-data-vending-on-blockchain
Repo
Framework

Two-Layer Lossless HDR Coding using Histogram Packing Technique with Backward Compatibility to JPEG


Title	Two-Layer Lossless HDR Coding using Histogram Packing Technique with Backward Compatibility to JPEG
Authors	Osamu Watanabe, Hiroyuki Kobayashi, Hitoshi Kiya
Abstract	An efficient two-layer coding method using the histogram packing technique with the backward compatibility to the legacy JPEG is proposed in this paper. The JPEG XT, which is the international standard to compress HDR images, adopts two-layer coding scheme for backward compatibility to the legacy JPEG. However, this two-layer coding structure does not give better lossless performance than the other existing methods for HDR image compression with single-layer structure. Moreover, the lossless compression of the JPEG XT has a problem on determination of the coding parameters; The lossless performance is affected by the input images and/or the parameter values. That is, finding appropriate combination of the values is necessary to achieve good lossless performance. It is firstly pointed out that the histogram packing technique considering the histogram sparseness of HDR images is able to improve the performance of lossless compression. Then, a novel two-layer coding with the histogram packing technique and an additional lossless encoder is proposed. The experimental results demonstrate that not only the proposed method has a better lossless compression performance than that of the JPEG XT, but also there is no need to determine image-dependent parameter values for good compression performance without losing the backward compatibility to the well known legacy JPEG standard.
Tasks	Image Compression
Published	2018-08-02
URL	http://arxiv.org/abs/1808.00956v1
PDF	http://arxiv.org/pdf/1808.00956v1.pdf
PWC	https://paperswithcode.com/paper/two-layer-lossless-hdr-coding-using-histogram
Repo
Framework