Paper Group AWR 187
Cross-layer Optimization for High Speed Adders: A Pareto Driven Machine Learning Approach. B-DCGAN:Evaluation of Binarized DCGAN for FPGA. Automatic Classification of Music Genre using Masked Conditional Neural Networks. Deep Interest Evolution Network for Click-Through Rate Prediction. Gaussian Process Classification for Variable Fidelity Data. Mo …
Cross-layer Optimization for High Speed Adders: A Pareto Driven Machine Learning Approach
| Title | Cross-layer Optimization for High Speed Adders: A Pareto Driven Machine Learning Approach |
| Authors | Yuzhe Ma, Subhendu Roy, Jin Miao, Jiamin Chen, Bei Yu |
| Abstract | In spite of maturity to the modern electronic design automation (EDA) tools, optimized designs at architectural stage may become sub-optimal after going through physical design flow. Adder design has been such a long studied fundamental problem in VLSI industry yet designers cannot achieve optimal solutions by running EDA tools on the set of available prefix adder architectures. In this paper, we enhance a state-of-the-art prefix adder synthesis algorithm to obtain a much wider solution space in architectural domain. On top of that, a machine learning-based design space exploration methodology is applied to predict the Pareto frontier of the adders in physical domain, which is infeasible by exhaustively running EDA tools for innumerable architectural solutions. Considering the high cost of obtaining the true values for learning, an active learning algorithm is utilized to select the representative data during learning process, which uses less labeled data while achieving better quality of Pareto frontier. Experimental results demonstrate that our framework can achieve Pareto frontier of high quality over a wide design space, bridging the gap between architectural and physical designs. |
| Tasks | Active Learning |
| Published | 2018-07-18 |
| URL | http://arxiv.org/abs/1807.07023v2 |
| http://arxiv.org/pdf/1807.07023v2.pdf | |
| PWC | https://paperswithcode.com/paper/cross-layer-optimization-for-high-speed |
| Repo | https://github.com/yuzhe630/adder-DSE |
| Framework | none |
B-DCGAN:Evaluation of Binarized DCGAN for FPGA
| Title | B-DCGAN:Evaluation of Binarized DCGAN for FPGA |
| Authors | Hideo Terada, Hayaru Shouno |
| Abstract | We are trying to implement deep neural networks in the edge computing environment for real-world applications such as the IoT(Internet of Things), the FinTech etc., for the purpose of utilizing the significant achievement of Deep Learning in recent years. Especially, we now focus algorithm implementation on FPGA, because FPGA is one of the promising devices for low-cost and low-power implementation of the edge computer. In this work, we introduce Binary-DCGAN(B-DCGAN) - Deep Convolutional GAN model with binary weights and activations, and with using integer-valued operations in forward pass(train-time and run-time). And we show how to implement B-DCGAN on FPGA(Xilinx Zynq). Using the B-DCGAN, we do feasibility study of FPGA’s characteristic and performance for Deep Learning. Because the binarization and using integer-valued operation reduce the memory capacity and the number of the circuit gates, it is very effective for FPGA implementation. On the other hand, the quality of generated data from the model will be decreased by these reductions. So we investigate the influence of these reductions. |
| Tasks | |
| Published | 2018-03-29 |
| URL | https://arxiv.org/abs/1803.10930v2 |
| https://arxiv.org/pdf/1803.10930v2.pdf | |
| PWC | https://paperswithcode.com/paper/b-dcganevaluation-of-binarized-dcgan-for-fpga |
| Repo | https://github.com/hterada/b-dcgan |
| Framework | none |
Automatic Classification of Music Genre using Masked Conditional Neural Networks
| Title | Automatic Classification of Music Genre using Masked Conditional Neural Networks |
| Authors | Fady Medhat, David Chesmore, John Robinson |
| Abstract | Neural network based architectures used for sound recognition are usually adapted from other application domains such as image recognition, which may not harness the time-frequency representation of a signal. The ConditionaL Neural Networks (CLNN) and its extension the Masked ConditionaL Neural Networks (MCLNN) are designed for multidimensional temporal signal recognition. The CLNN is trained over a window of frames to preserve the inter-frame relation, and the MCLNN enforces a systematic sparseness over the network’s links that mimics a filterbank-like behavior. The masking operation induces the network to learn in frequency bands, which decreases the network susceptibility to frequency-shifts in time-frequency representations. Additionally, the mask allows an exploration of a range of feature combinations concurrently analogous to the manual handcrafting of the optimum collection of features for a recognition task. MCLNN have achieved competitive performance on the Ballroom music dataset compared to several hand-crafted attempts and outperformed models based on state-of-the-art Convolutional Neural Networks. |
| Tasks | |
| Published | 2018-01-16 |
| URL | http://arxiv.org/abs/1801.05504v2 |
| http://arxiv.org/pdf/1801.05504v2.pdf | |
| PWC | https://paperswithcode.com/paper/automatic-classification-of-music-genre-using |
| Repo | https://github.com/fadymedhat/MCLNN |
| Framework | tf |
Deep Interest Evolution Network for Click-Through Rate Prediction
| Title | Deep Interest Evolution Network for Click-Through Rate Prediction |
| Authors | Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, Kun Gai |
| Abstract | Easy-to-use,Modular and Extendible package of deep-learning based CTR models.DeepFM,DeepInterestNetwork(DIN),DeepInterestEvolutionNetwork(DIEN),DeepCrossNetwork(DCN),AttentionalFactorizationMachine(AFM),Neural Factorization Machine(NFM),AutoInt |
| Tasks | Click-Through Rate Prediction |
| Published | 2018-09-11 |
| URL | http://arxiv.org/abs/1809.03672v5 |
| http://arxiv.org/pdf/1809.03672v5.pdf | |
| PWC | https://paperswithcode.com/paper/deep-interest-evolution-network-for-click |
| Repo | https://github.com/mouna99/dien |
| Framework | tf |
Gaussian Process Classification for Variable Fidelity Data
| Title | Gaussian Process Classification for Variable Fidelity Data |
| Authors | Nikita Klyuchnikov, Evgeny Burnaev |
| Abstract | In this paper we address a classification problem where two sources of labels with different levels of fidelity are available. Our approach is to combine data from both sources by applying a co-kriging schema on latent functions, which allows the model to account item-dependent labeling discrepancy. We provide an extension of Laplace inference for Gaussian process classification, that takes into account multi-fidelity data. We evaluate the proposed method on real and synthetic datasets and show that it is more resistant to different levels of discrepancy between sources than other approaches for data fusion. Our method can provide accuracy/cost trade-off for a number of practical tasks such as crowd-sourced data annotation and feasibility regions construction in engineering design. |
| Tasks | |
| Published | 2018-09-13 |
| URL | https://arxiv.org/abs/1809.05143v3 |
| https://arxiv.org/pdf/1809.05143v3.pdf | |
| PWC | https://paperswithcode.com/paper/gaussian-process-classification-for-variable |
| Repo | https://github.com/user525/mfgpc |
| Framework | none |
Moments in Time Dataset: one million videos for event understanding
| Title | Moments in Time Dataset: one million videos for event understanding |
| Authors | Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl Vondrick, Aude Oliva |
| Abstract | We present the Moments in Time Dataset, a large-scale human-annotated collection of one million short videos corresponding to dynamic events unfolding within three seconds. Modeling the spatial-audio-temporal dynamics even for actions occurring in 3 second videos poses many challenges: meaningful events do not include only people, but also objects, animals, and natural phenomena; visual and auditory events can be symmetrical in time (“opening” is “closing” in reverse), and either transient or sustained. We describe the annotation process of our dataset (each video is tagged with one action or activity label among 339 different classes), analyze its scale and diversity in comparison to other large-scale video datasets for action recognition, and report results of several baseline models addressing separately, and jointly, three modalities: spatial, temporal and auditory. The Moments in Time dataset, designed to have a large coverage and diversity of events in both visual and auditory modalities, can serve as a new challenge to develop models that scale to the level of complexity and abstract reasoning that a human processes on a daily basis. |
| Tasks | Action Recognition In Videos, Multimodal Activity Recognition, Temporal Action Localization |
| Published | 2018-01-09 |
| URL | http://arxiv.org/abs/1801.03150v3 |
| http://arxiv.org/pdf/1801.03150v3.pdf | |
| PWC | https://paperswithcode.com/paper/moments-in-time-dataset-one-million-videos |
| Repo | https://github.com/shubhambitsg/activity-recognition |
| Framework | pytorch |
Competitive Inner-Imaging Squeeze and Excitation for Residual Network
| Title | Competitive Inner-Imaging Squeeze and Excitation for Residual Network |
| Authors | Yang Hu, Guihua Wen, Mingnan Luo, Dan Dai, Jiajiong Ma, Zhiwen Yu |
| Abstract | Residual networks, which use a residual unit to supplement the identity mappings, enable very deep convolutional architecture to operate well, however, the residual architecture has been proved to be diverse and redundant, which may leads to low-efficient modeling. In this work, we propose a competitive squeeze-excitation (SE) mechanism for the residual network. Re-scaling the value for each channel in this structure will be determined by the residual and identity mappings jointly, and this design enables us to expand the meaning of channel relationship modeling in residual blocks. Modeling of the competition between residual and identity mappings cause the identity flow to control the complement of the residual feature maps for itself. Furthermore, we design a novel inner-imaging competitive SE block to shrink the consumption and re-image the global features of intermediate network structure, by using the inner-imaging mechanism, we can model the channel-wise relations with convolution in spatial. We carry out experiments on the CIFAR, SVHN, and ImageNet datasets, and the proposed method can challenge state-of-the-art results. |
| Tasks | |
| Published | 2018-07-24 |
| URL | http://arxiv.org/abs/1807.08920v4 |
| http://arxiv.org/pdf/1807.08920v4.pdf | |
| PWC | https://paperswithcode.com/paper/competitive-inner-imaging-squeeze-and |
| Repo | https://github.com/scut-aitcm/Competitive-Inner-Imaging-SENet |
| Framework | tf |
Factorized Machine Self-Confidence for Decision-Making Agents
| Title | Factorized Machine Self-Confidence for Decision-Making Agents |
| Authors | Brett W Israelsen, Nisar R Ahmed, Eric Frew, Dale Lawrence, Brian Argrow |
| Abstract | Algorithmic assurances from advanced autonomous systems assist human users in understanding, trusting, and using such systems appropriately. Designing these systems with the capacity of assessing their own capabilities is one approach to creating an algorithmic assurance. The idea of machine self-confidence' is introduced for autonomous systems. Using a factorization based framework for self-confidence assessment, one component of self-confidence, called solver-quality’, is discussed in the context of Markov decision processes for autonomous systems. Markov decision processes underlie much of the theory of reinforcement learning, and are commonly used for planning and decision making under uncertainty in robotics and autonomous systems. A `solver quality’ metric is formally defined in the context of decision making algorithms based on Markov decision processes. A method for assessing solver quality is then derived, drawing inspiration from empirical hardness models. Finally, numerical experiments for an unmanned autonomous vehicle navigation problem under different solver, parameter, and environment conditions indicate that the self-confidence metric exhibits the desired properties. Discussion of results, and avenues for future investigation are included. | |
| Tasks | Decision Making, Decision Making Under Uncertainty |
| Published | 2018-10-15 |
| URL | http://arxiv.org/abs/1810.06519v2 |
| http://arxiv.org/pdf/1810.06519v2.pdf | |
| PWC | https://paperswithcode.com/paper/factorized-machine-self-confidence-for |
| Repo | https://github.com/COHRINT/FaMSeC |
| Framework | mxnet |
A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks
| Title | A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks |
| Authors | Jinghui Chen, Dongruo Zhou, Jinfeng Yi, Quanquan Gu |
| Abstract | Depending on how much information an adversary can access to, adversarial attacks can be classified as white-box attack and black-box attack. For white-box attack, optimization-based attack algorithms such as projected gradient descent (PGD) can achieve relatively high attack success rates within moderate iterates. However, they tend to generate adversarial examples near or upon the boundary of the perturbation set, resulting in large distortion. Furthermore, their corresponding black-box attack algorithms also suffer from high query complexities, thereby limiting their practical usefulness. In this paper, we focus on the problem of developing efficient and effective optimization-based adversarial attack algorithms. In particular, we propose a novel adversarial attack framework for both white-box and black-box settings based on a variant of Frank-Wolfe algorithm. We show in theory that the proposed attack algorithms are efficient with an $O(1/\sqrt{T})$ convergence rate. The empirical results of attacking the ImageNet and MNIST datasets also verify the efficiency and effectiveness of the proposed algorithms. More specifically, our proposed algorithms attain the best attack performances in both white-box and black-box attacks among all baselines, and are more time and query efficient than the state-of-the-art. |
| Tasks | Adversarial Attack |
| Published | 2018-11-27 |
| URL | https://arxiv.org/abs/1811.10828v2 |
| https://arxiv.org/pdf/1811.10828v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-frank-wolfe-framework-for-efficient-and |
| Repo | https://github.com/sipka/fw-adversarial-imagenet |
| Framework | none |
High-Accuracy Low-Precision Training
| Title | High-Accuracy Low-Precision Training |
| Authors | Christopher De Sa, Megan Leszczynski, Jian Zhang, Alana Marzoev, Christopher R. Aberger, Kunle Olukotun, Christopher Ré |
| Abstract | Low-precision computation is often used to lower the time and energy cost of machine learning, and recently hardware accelerators have been developed to support it. Still, it has been used primarily for inference - not training. Previous low-precision training algorithms suffered from a fundamental tradeoff: as the number of bits of precision is lowered, quantization noise is added to the model, which limits statistical accuracy. To address this issue, we describe a simple low-precision stochastic gradient descent variant called HALP. HALP converges at the same theoretical rate as full-precision algorithms despite the noise introduced by using low precision throughout execution. The key idea is to use SVRG to reduce gradient variance, and to combine this with a novel technique called bit centering to reduce quantization error. We show that on the CPU, HALP can run up to $4 \times$ faster than full-precision SVRG and can match its convergence trajectory. We implemented HALP in TensorQuant, and show that it exceeds the validation performance of plain low-precision SGD on two deep learning tasks. |
| Tasks | Quantization |
| Published | 2018-03-09 |
| URL | http://arxiv.org/abs/1803.03383v1 |
| http://arxiv.org/pdf/1803.03383v1.pdf | |
| PWC | https://paperswithcode.com/paper/high-accuracy-low-precision-training |
| Repo | https://github.com/HazyResearch/lp_rffs |
| Framework | pytorch |
Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts
| Title | Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstracts |
| Authors | Di Jin, Peter Szolovits |
| Abstract | Prevalent models based on artificial neural network (ANN) for sentence classification often classify sentences in isolation without considering the context in which sentences appear. This hampers the traditional sentence classification approaches to the problem of sequential sentence classification, where structured prediction is needed for better overall classification performance. In this work, we present a hierarchical sequential labeling network to make use of the contextual information within surrounding sentences to help classify the current sentence. Our model outperforms the state-of-the-art results by 2%-3% on two benchmarking datasets for sequential sentence classification in medical scientific abstracts. |
| Tasks | Sentence Classification, Structured Prediction |
| Published | 2018-08-19 |
| URL | http://arxiv.org/abs/1808.06161v1 |
| http://arxiv.org/pdf/1808.06161v1.pdf | |
| PWC | https://paperswithcode.com/paper/hierarchical-neural-networks-for-sequential |
| Repo | https://github.com/jind11/HSLN-Joint-Sentence-Classification |
| Framework | tf |
Adversarial Attacks and Defences Competition
| Title | Adversarial Attacks and Defences Competition |
| Authors | Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, Motoki Abe |
| Abstract | To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them. In this chapter, we describe the structure and organization of the competition and the solutions developed by several of the top-placing teams. |
| Tasks | |
| Published | 2018-03-31 |
| URL | http://arxiv.org/abs/1804.00097v1 |
| http://arxiv.org/pdf/1804.00097v1.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-attacks-and-defences-competition |
| Repo | https://github.com/pfnet-research/nips17-adversarial-attack |
| Framework | tf |
Sym-parameterized Dynamic Inference for Mixed-Domain Image Translation
| Title | Sym-parameterized Dynamic Inference for Mixed-Domain Image Translation |
| Authors | Simyung Chang, SeongUk Park, John Yang, Nojun Kwak |
| Abstract | Recent advances in image-to-image translation have led to some ways to generate multiple domain images through a single network. However, there is still a limit in creating an image of a target domain without a dataset on it. We propose a method that expands the concept of `multi-domain’ from data to the loss area and learns the combined characteristics of each domain to dynamically infer translations of images in mixed domains. First, we introduce Sym-parameter and its learning method for variously mixed losses while synchronizing them with input conditions. Then, we propose Sym-parameterized Generative Network (SGN) which is empirically confirmed of learning mixed characteristics of various data and losses, and translating images to any mixed-domain without ground truths, such as 30% Van Gogh and 20% Monet and 40% snowy. | |
| Tasks | Image-to-Image Translation |
| Published | 2018-11-29 |
| URL | https://arxiv.org/abs/1811.12362v3 |
| https://arxiv.org/pdf/1811.12362v3.pdf | |
| PWC | https://paperswithcode.com/paper/image-translation-to-mixed-domain-using-sym |
| Repo | https://github.com/TimeLighter/pytorch-sym-parameter |
| Framework | pytorch |
Deep EHR: Chronic Disease Prediction Using Medical Notes
| Title | Deep EHR: Chronic Disease Prediction Using Medical Notes |
| Authors | Jingshu Liu, Zachariah Zhang, Narges Razavian |
| Abstract | Early detection of preventable diseases is important for better disease management, improved inter-ventions, and more efficient health-care resource allocation. Various machine learning approacheshave been developed to utilize information in Electronic Health Record (EHR) for this task. Majorityof previous attempts, however, focus on structured fields and lose the vast amount of information inthe unstructured notes. In this work we propose a general multi-task framework for disease onsetprediction that combines both free-text medical notes and structured information. We compareperformance of different deep learning architectures including CNN, LSTM and hierarchical models.In contrast to traditional text-based prediction models, our approach does not require disease specificfeature engineering, and can handle negations and numerical values that exist in the text. Ourresults on a cohort of about 1 million patients show that models using text outperform modelsusing just structured data, and that models capable of using numerical values and negations in thetext, in addition to the raw text, further improve performance. Additionally, we compare differentvisualization methods for medical professionals to interpret model predictions. |
| Tasks | Disease Prediction |
| Published | 2018-08-15 |
| URL | http://arxiv.org/abs/1808.04928v1 |
| http://arxiv.org/pdf/1808.04928v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-ehr-chronic-disease-prediction-using |
| Repo | https://github.com/NYUMedML/DeepEHR |
| Framework | pytorch |
TOM-Net: Learning Transparent Object Matting from a Single Image
| Title | TOM-Net: Learning Transparent Object Matting from a Single Image |
| Authors | Guanying Chen, Kai Han, Kwan-Yee K. Wong |
| Abstract | This paper addresses the problem of transparent object matting. Existing image matting approaches for transparent objects often require tedious capturing procedures and long processing time, which limit their practical use. In this paper, we first formulate transparent object matting as a refractive flow estimation problem. We then propose a deep learning framework, called TOM-Net, for learning the refractive flow. Our framework comprises two parts, namely a multi-scale encoder-decoder network for producing a coarse prediction, and a residual network for refinement. At test time, TOM-Net takes a single image as input, and outputs a matte (consisting of an object mask, an attenuation mask and a refractive flow field) in a fast feed-forward pass. As no off-the-shelf dataset is available for transparent object matting, we create a large-scale synthetic dataset consisting of 158K images of transparent objects rendered in front of images sampled from the Microsoft COCO dataset. We also collect a real dataset consisting of 876 samples using 14 transparent objects and 60 background images. Promising experimental results have been achieved on both synthetic and real data, which clearly demonstrate the effectiveness of our approach. |
| Tasks | Image Matting |
| Published | 2018-03-13 |
| URL | http://arxiv.org/abs/1803.04636v3 |
| http://arxiv.org/pdf/1803.04636v3.pdf | |
| PWC | https://paperswithcode.com/paper/tom-net-learning-transparent-object-matting |
| Repo | https://github.com/guanyingc/TOM-Net |
| Framework | pytorch |
