October 21, 2019

3027 words 15 mins read

Paper Group AWR 93

Capturing Structure Implicitly from Time-Series having Limited Data. SPSA-FSR: Simultaneous Perturbation Stochastic Approximation for Feature Selection and Ranking. Efficient Sparse-Winograd Convolutional Neural Networks. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. ChainGAN: A sequential approach t …

Capturing Structure Implicitly from Time-Series having Limited Data


Title	Capturing Structure Implicitly from Time-Series having Limited Data
Authors	Daniel Emaasit, Matthew Johnson
Abstract	Scientific fields such as insider-threat detection and highway-safety planning often lack sufficient amounts of time-series data to estimate statistical models for the purpose of scientific discovery. Moreover, the available limited data are quite noisy. This presents a major challenge when estimating time-series models that are robust to overfitting and have well-calibrated uncertainty estimates. Most of the current literature in these fields involve visualizing the time-series for noticeable structure and hard coding them into pre-specified parametric functions. This approach is associated with two limitations. First, given that such trends may not be easily noticeable in small data, it is difficult to explicitly incorporate expressive structure into the models during formulation. Second, it is difficult to know $\textit{a priori}$ the most appropriate functional form to use. To address these limitations, a nonparametric Bayesian approach was proposed to implicitly capture hidden structure from time series having limited data. The proposed model, a Gaussian process with a spectral mixture kernel, precludes the need to pre-specify a functional form and hard code trends, is robust to overfitting and has well-calibrated uncertainty estimates.
Tasks	Time Series
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05867v1
PDF	http://arxiv.org/pdf/1803.05867v1.pdf
PWC	https://paperswithcode.com/paper/capturing-structure-implicitly-from-time
Repo	https://github.com/emaasit/long-range-extrapolation
Framework	none

SPSA-FSR: Simultaneous Perturbation Stochastic Approximation for Feature Selection and Ranking


Title	SPSA-FSR: Simultaneous Perturbation Stochastic Approximation for Feature Selection and Ranking
Authors	Zeren D. Yenice, Niranjan Adhikari, Yong Kai Wong, Vural Aksakalli, Alev Taskin Gumus, Babak Abbasi
Abstract	This manuscript presents the following: (1) an improved version of the Binary Simultaneous Perturbation Stochastic Approximation (SPSA) Method for feature selection in machine learning (Aksakalli and Malekipirbazari, Pattern Recognition Letters, Vol. 75, 2016) based on non-monotone iteration gains computed via the Barzilai and Borwein (BB) method, (2) its adaptation for feature ranking, and (3) comparison against popular methods on public benchmark datasets. The improved method, which we call SPSA-FSR, dramatically reduces the number of iterations required for convergence without impacting solution quality. SPSA-FSR can be used for feature ranking and feature selection both for classification and regression problems. After a review of the current state-of-the-art, we discuss our improvements in detail and present three sets of computational experiments: (1) comparison of SPSA-FS as a (wrapper) feature selection method against sequential methods as well as genetic algorithms, (2) comparison of SPSA-FS as a feature ranking method in a classification setting against random forest importance, chi-squared, and information main methods, and (3) comparison of SPSA-FS as a feature ranking method in a regression setting against minimum redundancy maximum relevance (MRMR), RELIEF, and linear correlation methods. The number of features in the datasets we use range from a few dozens to a few thousands. Our results indicate that SPSA-FS converges to a good feature set in no more than 100 iterations and therefore it is quite fast for a wrapper method. SPSA-FS also outperforms popular feature selection as well as feature ranking methods in majority of test cases, sometimes by a large margin, and it stands as a promising new feature selection and ranking method.
Tasks	Feature Selection
Published	2018-04-16
URL	http://arxiv.org/abs/1804.05589v1
PDF	http://arxiv.org/pdf/1804.05589v1.pdf
PWC	https://paperswithcode.com/paper/spsa-fsr-simultaneous-perturbation-stochastic
Repo	https://github.com/vaksakalli/spsaml_py
Framework	none

Efficient Sparse-Winograd Convolutional Neural Networks


Title	Efficient Sparse-Winograd Convolutional Neural Networks
Authors	Xingyu Liu, Jeff Pool, Song Han, William J. Dally
Abstract	Convolutional Neural Networks (CNNs) are computationally intensive, which limits their application on mobile devices. Their energy is dominated by the number of multiplies needed to perform the convolutions. Winograd’s minimal filtering algorithm (Lavin, 2015) and network pruning (Han et al., 2015) can reduce the operation count, but these two methods cannot be directly combined $-$ applying the Winograd transform fills in the sparsity in both the weights and the activations. We propose two modifications to Winograd-based CNNs to enable these methods to exploit sparsity. First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations. Second, we prune the weights in the Winograd domain to exploit static weight sparsity. For models on CIFAR-10, CIFAR-100 and ImageNet datasets, our method reduces the number of multiplications by $10.4\times$, $6.8\times$ and $10.8\times$ respectively with loss of accuracy less than $0.1%$, outperforming previous baselines by $2.0\times$-$3.0\times$. We also show that moving ReLU to the Winograd domain allows more aggressive pruning.
Tasks	Network Pruning
Published	2018-02-18
URL	http://arxiv.org/abs/1802.06367v1
PDF	http://arxiv.org/pdf/1802.06367v1.pdf
PWC	https://paperswithcode.com/paper/efficient-sparse-winograd-convolutional
Repo	https://github.com/xingyul/Sparse-Winograd-CNN
Framework	tf

Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization


Title	Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Authors	Nicolas Y. Masse, Gregory D. Grant, David J. Freedman
Abstract	Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically cause it to forget previously learned tasks. This phenomenon is the result of “catastrophic forgetting”, in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks when combined with weight stabilization. This work provides another example of how neuroscience-inspired algorithms can benefit ANN design and capability.
Tasks
Published	2018-02-02
URL	http://arxiv.org/abs/1802.01569v2
PDF	http://arxiv.org/pdf/1802.01569v2.pdf
PWC	https://paperswithcode.com/paper/alleviating-catastrophic-forgetting-using
Repo	https://github.com/gdgrant/Context-Controller-RNN
Framework	tf

ChainGAN: A sequential approach to GANs


Title	ChainGAN: A sequential approach to GANs
Authors	Safwan Hossain, Kiarash Jamali, Yuchen Li, Frank Rudzicz
Abstract	We propose a new architecture and training methodology for generative adversarial networks. Current approaches attempt to learn the transformation from a noise sample to a generated data sample in one shot. Our proposed generator architecture, called $\textit{ChainGAN}$, uses a two-step process. It first attempts to transform a noise vector into a crude sample, similar to a traditional generator. Next, a chain of networks, called $\textit{editors}$, attempt to sequentially enhance this sample. We train each of these units independently, instead of with end-to-end backpropagation on the entire chain. Our model is robust, efficient, and flexible as we can apply it to various network architectures. We provide rationale for our choices and experimentally evaluate our model, achieving competitive results on several datasets.
Tasks
Published	2018-11-20
URL	http://arxiv.org/abs/1811.08081v2
PDF	http://arxiv.org/pdf/1811.08081v2.pdf
PWC	https://paperswithcode.com/paper/chaingan-a-sequential-approach-to-gans
Repo	https://github.com/safwanhossain/chainGAN_repo
Framework	pytorch

Efficient Low-rank Multimodal Fusion with Modality-Specific Factors


Title	Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
Authors	Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency
Abstract	Multimodal research is an emerging field of artificial intelligence, and one of the main research problems in this field is multimodal fusion. The fusion of multimodal data is the process of integrating multiple unimodal representations into one compact multimodal representation. Previous research in this field has exploited the expressiveness of tensors for multimodal representation. However, these methods often suffer from exponential increase in dimensions and in computational complexity introduced by transformation of input into tensor. In this paper, we propose the Low-rank Multimodal Fusion method, which performs multimodal fusion using low-rank tensors to improve efficiency. We evaluate our model on three different tasks: multimodal sentiment analysis, speaker trait analysis, and emotion recognition. Our model achieves competitive results on all these tasks while drastically reducing computational complexity. Additional experiments also show that our model can perform robustly for a wide range of low-rank settings, and is indeed much more efficient in both training and inference compared to other methods that utilize tensor representations.
Tasks	Emotion Recognition, Multimodal Sentiment Analysis, Sentiment Analysis
Published	2018-05-31
URL	http://arxiv.org/abs/1806.00064v1
PDF	http://arxiv.org/pdf/1806.00064v1.pdf
PWC	https://paperswithcode.com/paper/efficient-low-rank-multimodal-fusion-with
Repo	https://github.com/Justin1904/Low-rank-Multimodal-Fusion
Framework	pytorch

Word2Bits - Quantized Word Vectors


Title	Word2Bits - Quantized Word Vectors
Authors	Maximilian Lam
Abstract	Word vectors require significant amounts of memory and storage, posing issues to resource limited devices like mobile phones and GPUs. We show that high quality quantized word vectors using 1-2 bits per parameter can be learned by introducing a quantization function into Word2Vec. We furthermore show that training with the quantization function acts as a regularizer. We train word vectors on English Wikipedia (2017) and evaluate them on standard word similarity and analogy tasks and on question answering (SQuAD). Our quantized word vectors not only take 8-16x less space than full precision (32 bit) word vectors but also outperform them on word similarity tasks and question answering.
Tasks	Quantization, Question Answering
Published	2018-03-15
URL	http://arxiv.org/abs/1803.05651v3
PDF	http://arxiv.org/pdf/1803.05651v3.pdf
PWC	https://paperswithcode.com/paper/word2bits-quantized-word-vectors
Repo	https://github.com/agnusmaximus/Word2Bits
Framework	none

Fully Convolutional Siamese Networks for Change Detection


Title	Fully Convolutional Siamese Networks for Change Detection
Authors	Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch
Abstract	This paper presents three fully convolutional neural network architectures which perform change detection using a pair of coregistered images. Most notably, we propose two Siamese extensions of fully convolutional networks which use heuristics about the current problem to achieve the best results in our tests on two open change detection datasets, using both RGB and multispectral images. We show that our system is able to learn from scratch using annotated change detection images. Our architectures achieve better performance than previously proposed methods, while being at least 500 times faster than related systems. This work is a step towards efficient processing of data from large scale Earth observation systems such as Copernicus or Landsat.
Tasks
Published	2018-10-19
URL	http://arxiv.org/abs/1810.08462v1
PDF	http://arxiv.org/pdf/1810.08462v1.pdf
PWC	https://paperswithcode.com/paper/fully-convolutional-siamese-networks-for
Repo	https://github.com/kyoukuntaro/FCSN_for_ChangeDetection_IGARSS2018
Framework	pytorch

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware


Title	Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Authors	Florian Tramèr, Dan Boneh
Abstract	As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations. A pragmatic solution comes from Trusted Execution Environments (TEEs), which use hardware and software protections to isolate sensitive computations from the untrusted software stack. However, these isolation guarantees come at a price in performance, compared to untrusted alternatives. This paper initiates the study of high performance execution of Deep Neural Networks (DNNs) in TEEs by efficiently partitioning DNN computations between trusted and untrusted devices. Building upon an efficient outsourcing scheme for matrix multiplication, we propose Slalom, a framework that securely delegates execution of all linear layers in a DNN from a TEE (e.g., Intel SGX or Sanctum) to a faster, yet untrusted, co-located processor. We evaluate Slalom by running DNNs in an Intel SGX enclave, which selectively delegates work to an untrusted GPU. For canonical DNNs (VGG16, MobileNet and ResNet variants) we obtain 6x to 20x increases in throughput for verifiable inference, and 4x to 11x for verifiable and private inference.
Tasks
Published	2018-06-08
URL	http://arxiv.org/abs/1806.03287v2
PDF	http://arxiv.org/pdf/1806.03287v2.pdf
PWC	https://paperswithcode.com/paper/slalom-fast-verifiable-and-private-execution
Repo	https://github.com/ftramer/slalom
Framework	tf

Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization


Title	Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization
Authors	Shashi Narayan, Shay B. Cohen, Mirella Lapata
Abstract	We introduce extreme summarization, a new single-document summarization task which does not favor extractive strategies and calls for an abstractive modeling approach. The idea is to create a short, one-sentence news summary answering the question “What is the article about?". We collect a real-world, large-scale dataset for this task by harvesting online articles from the British Broadcasting Corporation (BBC). We propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. We demonstrate experimentally that this architecture captures long-range dependencies in a document and recognizes pertinent content, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans.
Tasks	Document Summarization
Published	2018-08-27
URL	http://arxiv.org/abs/1808.08745v1
PDF	http://arxiv.org/pdf/1808.08745v1.pdf
PWC	https://paperswithcode.com/paper/dont-give-me-the-details-just-the-summary
Repo	https://github.com/shashiongithub/XSum
Framework	pytorch

Mapper on Graphs for Network Visualization


Title	Mapper on Graphs for Network Visualization
Authors	Mustafa Hajij, Paul Rosen, Bei Wang
Abstract	Networks are an exceedingly popular type of data for representing relationships between individuals, businesses, proteins, brain regions, telecommunication endpoints, etc. Network or graph visualization provides an intuitive way to explore the node-link structures of network data for instant sense-making. However, naive node-link diagrams can fail to convey insights regarding network structures, even for moderately sized data of a few hundred nodes. We propose to apply the mapper construction–a popular tool in topological data analysis–to graph visualization, which provides a strong theoretical basis for summarizing network data while preserving their core structures. We develop a variation of the mapper construction targeting weighted, undirected graphs, called mapper on graphs, which generates property-preserving summaries of graphs. We provide a software tool that enables interactive explorations of such summaries and demonstrates the effectiveness of our method for synthetic and real-world data. The mapper on graphs approach we propose represents a new class of techniques that leverages tools from topological data analysis in addressing challenges in graph visualization.
Tasks	Topological Data Analysis
Published	2018-04-03
URL	https://arxiv.org/abs/1804.11242v4
PDF	https://arxiv.org/pdf/1804.11242v4.pdf
PWC	https://paperswithcode.com/paper/mog-mapper-on-graphs-for-relationship
Repo	https://github.com/USFDataVisualization/MapperOnGraphs
Framework	none

Layer-structured 3D Scene Inference via View Synthesis


Title	Layer-structured 3D Scene Inference via View Synthesis
Authors	Shubham Tulsiani, Richard Tucker, Noah Snavely
Abstract	We present an approach to infer a layer-structured 3D representation of a scene from a single input image. This allows us to infer not only the depth of the visible pixels, but also to capture the texture and depth for content in the scene that is not directly visible. We overcome the challenge posed by the lack of direct supervision by instead leveraging a more naturally available multi-view supervisory signal. Our insight is to use view synthesis as a proxy task: we enforce that our representation (inferred from a single image), when rendered from a novel perspective, matches the true observed image. We present a learning framework that operationalizes this insight using a new, differentiable novel view renderer. We provide qualitative and quantitative validation of our approach in two different settings, and demonstrate that we can learn to capture the hidden aspects of a scene.
Tasks
Published	2018-07-26
URL	http://arxiv.org/abs/1807.10264v1
PDF	http://arxiv.org/pdf/1807.10264v1.pdf
PWC	https://paperswithcode.com/paper/layer-structured-3d-scene-inference-via-view
Repo	https://github.com/VCL3D/SphericalViewSynthesis
Framework	pytorch

Adversarial Deep Learning for Robust Detection of Binary Encoded Malware


Title	Adversarial Deep Learning for Robust Detection of Binary Encoded Malware
Authors	Abdullah Al-Dujaili, Alex Huang, Erik Hemberg, Una-May O’Reilly
Abstract	Malware is constantly adapting in order to avoid detection. Model based malware detectors, such as SVM and neural networks, are vulnerable to so-called adversarial examples which are modest changes to detectable malware that allows the resulting malware to evade detection. Continuous-valued methods that are robust to adversarial examples of images have been developed using saddle-point optimization formulations. We are inspired by them to develop similar methods for the discrete, e.g. binary, domain which characterizes the features of malware. A specific extra challenge of malware is that the adversarial examples must be generated in a way that preserves their malicious functionality. We introduce methods capable of generating functionally preserved adversarial malware examples in the binary domain. Using the saddle-point formulation, we incorporate the adversarial examples into the training of models that are robust to them. We evaluate the effectiveness of the methods and others in the literature on a set of Portable Execution~(PE) files. Comparison prompts our introduction of an online measure computed during training to assess general expectation of robustness.
Tasks
Published	2018-01-09
URL	http://arxiv.org/abs/1801.02950v3
PDF	http://arxiv.org/pdf/1801.02950v3.pdf
PWC	https://paperswithcode.com/paper/adversarial-deep-learning-for-robust
Repo	https://github.com/ALFA-group/robust-adv-malware-detection
Framework	pytorch

Robust Gyroscope-Aided Camera Self-Calibration


Title	Robust Gyroscope-Aided Camera Self-Calibration
Authors	Santiago Cortés Reina, Arno Solin, Juho Kannala
Abstract	Camera calibration for estimating the intrinsic parameters and lens distortion is a prerequisite for various monocular vision applications including feature tracking and video stabilization. This application paper proposes a model for estimating the parameters on the fly by fusing gyroscope and camera data, both readily available in modern day smartphones. The model is based on joint estimation of visual feature positions, camera parameters, and the camera pose, the movement of which is assumed to follow the movement predicted by the gyroscope. Our model assumes the camera movement to be free, but continuous and differentiable, and individual features are assumed to stay stationary. The estimation is performed online using an extended Kalman filter, and it is shown to outperform existing methods in robustness and insensitivity to initialization. We demonstrate the method using simulated data and empirical data from an iPad.
Tasks	Calibration
Published	2018-05-31
URL	http://arxiv.org/abs/1805.12506v1
PDF	http://arxiv.org/pdf/1805.12506v1.pdf
PWC	https://paperswithcode.com/paper/robust-gyroscope-aided-camera-self
Repo	https://github.com/AaltoVision/camera-gyro-calibration
Framework	none

Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations


Title	Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations
Authors	Ju-chieh Chou, Cheng-chieh Yeh, Hung-yi Lee, Lin-shan Lee
Abstract	Recently, cycle-consistent adversarial network (Cycle-GAN) has been successfully applied to voice conversion to a different speaker without parallel data, although in those approaches an individual model is needed for each target speaker. In this paper, we propose an adversarial learning framework for voice conversion, with which a single model can be trained to convert the voice to many different speakers, all without parallel data, by separating the speaker characteristics from the linguistic content in speech signals. An autoencoder is first trained to extract speaker-independent latent representations and speaker embedding separately using another auxiliary speaker classifier to regularize the latent representation. The decoder then takes the speaker-independent latent representation and the target speaker embedding as the input to generate the voice of the target speaker with the linguistic content of the source utterance. The quality of decoder output is further improved by patching with the residual signal produced by another pair of generator and discriminator. A target speaker set size of 20 was tested in the preliminary experiments, and very good voice quality was obtained. Conventional voice conversion metrics are reported. We also show that the speaker information has been properly reduced from the latent representations.
Tasks	Voice Conversion
Published	2018-04-09
URL	http://arxiv.org/abs/1804.02812v2
PDF	http://arxiv.org/pdf/1804.02812v2.pdf
PWC	https://paperswithcode.com/paper/multi-target-voice-conversion-without
Repo	https://github.com/arshd91/multitarget-voice-conversion-vctk
Framework	pytorch