Paper Group ANR 551
Memory-augmented Chinese-Uyghur Neural Machine Translation. An Epistemic Foundation for Authentication Logics (Extended Abstract). Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks. SamBaTen: Sampling-based Batch Incremental Tensor Decomposition. Automated Low-cost Terrestrial Laser Scanner for Measuring Diamet …
Memory-augmented Chinese-Uyghur Neural Machine Translation
| Title | Memory-augmented Chinese-Uyghur Neural Machine Translation |
| Authors | Shiyue Zhang, Gulnigar Mahmut, Dong Wang, Askar Hamdulla |
| Abstract | Neural machine translation (NMT) has achieved notable performance recently. However, this approach has not been widely applied to the translation task between Chinese and Uyghur, partly due to the limited parallel data resource and the large proportion of rare words caused by the agglutinative nature of Uyghur. In this paper, we collect ~200,000 sentence pairs and show that with this middle-scale database, an attention-based NMT can perform very well on Chinese-Uyghur/Uyghur-Chinese translation. To tackle rare words, we propose a novel memory structure to assist the NMT inference. Our experiments demonstrated that the memory-augmented NMT (M-NMT) outperforms both the vanilla NMT and the phrase-based statistical machine translation (SMT). Interestingly, the memory structure provides an elegant way for dealing with words that are out of vocabulary. |
| Tasks | Machine Translation |
| Published | 2017-06-27 |
| URL | http://arxiv.org/abs/1706.08683v1 |
| http://arxiv.org/pdf/1706.08683v1.pdf | |
| PWC | https://paperswithcode.com/paper/memory-augmented-chinese-uyghur-neural |
| Repo | |
| Framework | |
An Epistemic Foundation for Authentication Logics (Extended Abstract)
| Title | An Epistemic Foundation for Authentication Logics (Extended Abstract) |
| Authors | Joseph Y. Halpern, Ron van der Meyden, Riccardo Pucella |
| Abstract | While there have been many attempts, going back to BAN logic, to base reasoning about security protocols on epistemic notions, they have not been all that successful. Arguably, this has been due to the particular logics chosen. We present a simple logic based on the well-understood modal operators of knowledge, time, and probability, and show that it is able to handle issues that have often been swept under the rug by other approaches, while being flexible enough to capture all the higher- level security notions that appear in BAN logic. Moreover, while still assuming that the knowledge operator allows for unbounded computation, it can handle the fact that a computationally bounded agent cannot decrypt messages in a natural way, by distinguishing strings and message terms. We demonstrate that our logic can capture BAN logic notions by providing a translation of the BAN operators into our logic, capturing belief by a form of probabilistic knowledge. |
| Tasks | |
| Published | 2017-07-27 |
| URL | http://arxiv.org/abs/1707.08750v1 |
| http://arxiv.org/pdf/1707.08750v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-epistemic-foundation-for-authentication |
| Repo | |
| Framework | |
Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
| Title | Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks |
| Authors | Chunjie Luo, Jianfeng Zhan, Lei Wang, Qiang Yang |
| Abstract | Traditionally, multi-layer neural networks use dot product between the output vector of previous layer and the incoming weight vector as the input to activation function. The result of dot product is unbounded, thus increases the risk of large variance. Large variance of neuron makes the model sensitive to the change of input distribution, thus results in poor generalization, and aggravates the internal covariate shift which slows down the training. To bound dot product and decrease the variance, we propose to use cosine similarity or centered cosine similarity (Pearson Correlation Coefficient) instead of dot product in neural networks, which we call cosine normalization. We compare cosine normalization with batch, weight and layer normalization in fully-connected neural networks as well as convolutional networks on the data sets of MNIST, 20NEWS GROUP, CIFAR-10/100 and SVHN. Experiments show that cosine normalization achieves better performance than other normalization techniques. |
| Tasks | |
| Published | 2017-02-20 |
| URL | http://arxiv.org/abs/1702.05870v5 |
| http://arxiv.org/pdf/1702.05870v5.pdf | |
| PWC | https://paperswithcode.com/paper/cosine-normalization-using-cosine-similarity |
| Repo | |
| Framework | |
SamBaTen: Sampling-based Batch Incremental Tensor Decomposition
| Title | SamBaTen: Sampling-based Batch Incremental Tensor Decomposition |
| Authors | Ekta Gujral, Ravdeep Pasricha, Evangelos E. Papalexakis |
| Abstract | Tensor decompositions are invaluable tools in analyzing multimodal datasets. In many real-world scenarios, such datasets are far from being static, to the contrary they tend to grow over time. For instance, in an online social network setting, as we observe new interactions over time, our dataset gets updated in its “time” mode. How can we maintain a valid and accurate tensor decomposition of such a dynamically evolving multimodal dataset, without having to re-compute the entire decomposition after every single update? In this paper we introduce SaMbaTen, a Sampling-based Batch Incremental Tensor Decomposition algorithm, which incrementally maintains the decomposition given new updates to the tensor dataset. SaMbaTen is able to scale to datasets that the state-of-the-art in incremental tensor decomposition is unable to operate on, due to its ability to effectively summarize the existing tensor and the incoming updates, and perform all computations in the reduced summary space. We extensively evaluate SaMbaTen using synthetic and real datasets. Indicatively, SaMbaTen achieves comparable accuracy to state-of-the-art incremental and non-incremental techniques, while being 25-30 times faster. Furthermore, SaMbaTen scales to very large sparse and dense dynamically evolving tensors of dimensions up to 100K x 100K x 100K where state-of-the-art incremental approaches were not able to operate. |
| Tasks | |
| Published | 2017-09-03 |
| URL | http://arxiv.org/abs/1709.00668v2 |
| http://arxiv.org/pdf/1709.00668v2.pdf | |
| PWC | https://paperswithcode.com/paper/sambaten-sampling-based-batch-incremental |
| Repo | |
| Framework | |
Automated Low-cost Terrestrial Laser Scanner for Measuring Diameters at Breast Height and Heights of Forest Trees
| Title | Automated Low-cost Terrestrial Laser Scanner for Measuring Diameters at Breast Height and Heights of Forest Trees |
| Authors | Pei Wang, Guochao Bu, Ronghao Li, Rui Zhao |
| Abstract | Terrestrial laser scanner is a kind of fast, high-precision data acquisition device, which had been more and more applied to the research areas of forest inventory. In this study, a kind of automated low-cost terrestrial laser scanner was designed and implemented based on a two-dimensional laser radar sensor SICK LMS-511 and a stepper motor. The new scanner was named as BEE, which can scan the forest trees in three dimension. The BEE scanner and its supporting software are specifically designed for forest inventory. The experiments have been performed by using the BEE scanner in an artificial ginkgo forest which was located in Haidian district of Beijing. Four square plots were selected to do the experiments. The BEE scanner scanned in the four plots and acquired the single scan data respectively. The DBH, tree height and tree position of trees in the four plots were estimated and analyzed. For comparison, the manual measured data was also collected in the four plots. The tree stem detection rate for all four plots was 92.75%; the root mean square error of the DBH estimation was 1.27cm; the root mean square error of the tree height estimation was 0.24m; the tree position estimation was in line with the actual position. Experimental results show that the BEE scanner can efficiently estimate the structure parameters of forest trees and has a good potential in practical application of forest inventory. |
| Tasks | |
| Published | 2017-02-08 |
| URL | http://arxiv.org/abs/1702.02235v2 |
| http://arxiv.org/pdf/1702.02235v2.pdf | |
| PWC | https://paperswithcode.com/paper/automated-low-cost-terrestrial-laser-scanner |
| Repo | |
| Framework | |
ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks
| Title | ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks |
| Authors | Priyadarshini Panda, Jason M. Allred, Shriram Ramanathan, Kaushik Roy |
| Abstract | A fundamental feature of learning in animals is the “ability to forget” that allows an organism to perceive, model and make decisions from disparate streams of information and adapt to changing environments. Against this backdrop, we present a novel unsupervised learning mechanism ASP (Adaptive Synaptic Plasticity) for improved recognition with Spiking Neural Networks (SNNs) for real time on-line learning in a dynamic environment. We incorporate an adaptive weight decay mechanism with the traditional Spike Timing Dependent Plasticity (STDP) learning to model adaptivity in SNNs. The leak rate of the synaptic weights is modulated based on the temporal correlation between the spiking patterns of the pre- and post-synaptic neurons. This mechanism helps in gradual forgetting of insignificant data while retaining significant, yet old, information. ASP, thus, maintains a balance between forgetting and immediate learning to construct a stable-plastic self-adaptive SNN for continuously changing inputs. We demonstrate that the proposed learning methodology addresses catastrophic forgetting while yielding significantly improved accuracy over the conventional STDP learning method for digit recognition applications. Additionally, we observe that the proposed learning model automatically encodes selective attention towards relevant features in the input data while eliminating the influence of background noise (or denoising) further improving the robustness of the ASP learning. |
| Tasks | Denoising |
| Published | 2017-03-22 |
| URL | http://arxiv.org/abs/1703.07655v2 |
| http://arxiv.org/pdf/1703.07655v2.pdf | |
| PWC | https://paperswithcode.com/paper/asp-learning-to-forget-with-adaptive-synaptic |
| Repo | |
| Framework | |
Smart Mining for Deep Metric Learning
| Title | Smart Mining for Deep Metric Learning |
| Authors | Ben Harwood, Vijay Kumar B G, Gustavo Carneiro, Ian Reid, Tom Drummond |
| Abstract | To solve deep metric learning problems and producing feature embeddings, current methodologies will commonly use a triplet model to minimise the relative distance between samples from the same class and maximise the relative distance between samples from different classes. Though successful, the training convergence of this triplet model can be compromised by the fact that the vast majority of the training samples will produce gradients with magnitudes that are close to zero. This issue has motivated the development of methods that explore the global structure of the embedding and other methods that explore hard negative/positive mining. The effectiveness of such mining methods is often associated with intractable computational requirements. In this paper, we propose a novel deep metric learning method that combines the triplet model and the global structure of the embedding space. We rely on a smart mining procedure that produces effective training samples for a low computational cost. In addition, we propose an adaptive controller that automatically adjusts the smart mining hyper-parameters and speeds up the convergence of the training process. We show empirically that our proposed method allows for fast and more accurate training of triplet ConvNets than other competing mining methods. Additionally, we show that our method achieves new state-of-the-art embedding results for CUB-200-2011 and Cars196 datasets. |
| Tasks | Metric Learning |
| Published | 2017-04-05 |
| URL | http://arxiv.org/abs/1704.01285v3 |
| http://arxiv.org/pdf/1704.01285v3.pdf | |
| PWC | https://paperswithcode.com/paper/smart-mining-for-deep-metric-learning |
| Repo | |
| Framework | |
Incremental Eigenpair Computation for Graph Laplacian Matrices: Theory and Applications
| Title | Incremental Eigenpair Computation for Graph Laplacian Matrices: Theory and Applications |
| Authors | Pin-Yu Chen, Baichuan Zhang, Mohammad Al Hasan |
| Abstract | The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used in spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of the existing methods either choose $K$ heuristically or they repeat the clustering method with different choices of $K$ and accept the best clustering result. The first option, more often, yields suboptimal result, while the second option is computationally expensive. In this work, we propose an incremental method for constructing the eigenspectrum of the graph Laplacian matrix. This method leverages the eigenstructure of graph Laplacian matrix to obtain the $K$-th smallest eigenpair of the Laplacian matrix given a collection of all previously computed $K-1$ smallest eigenpairs. Our proposed method adapts the Laplacian matrix such that the batch eigenvalue decomposition problem transforms into an efficient sequential leading eigenpair computation problem. As a practical application, we consider user-guided spectral clustering. Specifically, we demonstrate that users can utilize the proposed incremental method for effective eigenpair computation and for determining the desired number of clusters based on multiple clustering metrics. |
| Tasks | Community Detection |
| Published | 2017-12-13 |
| URL | http://arxiv.org/abs/1801.08196v1 |
| http://arxiv.org/pdf/1801.08196v1.pdf | |
| PWC | https://paperswithcode.com/paper/incremental-eigenpair-computation-for-graph |
| Repo | |
| Framework | |
Optimal Shrinkage of Singular Values Under Random Data Contamination
| Title | Optimal Shrinkage of Singular Values Under Random Data Contamination |
| Authors | Danny Barash, Matan Gavish |
| Abstract | A low rank matrix X has been contaminated by uniformly distributed noise, missing values, outliers and corrupt entries. Reconstruction of X from the singular values and singular vectors of the contaminated matrix Y is a key problem in machine learning, computer vision and data science. In this paper we show that common contamination models (including arbitrary combinations of uniform noise,missing values, outliers and corrupt entries) can be described efficiently using a single framework. We develop an asymptotically optimal algorithm that estimates X by manipulation of the singular values of Y , which applies to any of the contamination models considered. Finally, we find an explicit signal-to-noise cutoff, below which estimation of X from the singular value decomposition of Y must fail, in a well-defined sense. |
| Tasks | |
| Published | 2017-10-26 |
| URL | http://arxiv.org/abs/1710.09787v2 |
| http://arxiv.org/pdf/1710.09787v2.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-shrinkage-of-singular-values-under |
| Repo | |
| Framework | |
Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments
| Title | Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments |
| Authors | Dan Barnes, Will Maddern, Geoffrey Pascoe, Ingmar Posner |
| Abstract | We present a self-supervised approach to ignoring “distractors” in camera images for the purposes of robustly estimating vehicle motion in cluttered urban environments. We leverage offline multi-session mapping approaches to automatically generate a per-pixel ephemerality mask and depth map for each input image, which we use to train a deep convolutional network. At run-time we use the predicted ephemerality and depth as an input to a monocular visual odometry (VO) pipeline, using either sparse features or dense photometric matching. Our approach yields metric-scale VO using only a single camera and can recover the correct egomotion even when 90% of the image is obscured by dynamic, independently moving objects. We evaluate our robust VO methods on more than 400km of driving from the Oxford RobotCar Dataset and demonstrate reduced odometry drift and significantly improved egomotion estimation in the presence of large moving vehicles in urban traffic. |
| Tasks | Monocular Visual Odometry, Visual Odometry |
| Published | 2017-11-17 |
| URL | http://arxiv.org/abs/1711.06623v2 |
| http://arxiv.org/pdf/1711.06623v2.pdf | |
| PWC | https://paperswithcode.com/paper/driven-to-distraction-self-supervised |
| Repo | |
| Framework | |
Transfer Learning for Neural Semantic Parsing
| Title | Transfer Learning for Neural Semantic Parsing |
| Authors | Xing Fan, Emilio Monti, Lambert Mathias, Markus Dreyer |
| Abstract | The goal of semantic parsing is to map natural language to a machine interpretable meaning representation language (MRL). One of the constraints that limits full exploration of deep learning technologies for semantic parsing is the lack of sufficient annotation training data. In this paper, we propose using sequence-to-sequence in a multi-task setup for semantic parsing with a focus on transfer learning. We explore three multi-task architectures for sequence-to-sequence modeling and compare their performance with an independently trained model. Our experiments show that the multi-task setup aids transfer learning from an auxiliary task with large labeled data to a target task with smaller labeled data. We see absolute accuracy gains ranging from 1.0% to 4.4% in our in- house data set, and we also see good gains ranging from 2.5% to 7.0% on the ATIS semantic parsing tasks with syntactic and semantic auxiliary tasks. |
| Tasks | Semantic Parsing, Transfer Learning |
| Published | 2017-06-14 |
| URL | http://arxiv.org/abs/1706.04326v1 |
| http://arxiv.org/pdf/1706.04326v1.pdf | |
| PWC | https://paperswithcode.com/paper/transfer-learning-for-neural-semantic-parsing |
| Repo | |
| Framework | |
When are epsilon-nets small?
| Title | When are epsilon-nets small? |
| Authors | Andrey Kupavskii, Nikita Zhivotovskiy |
| Abstract | In many interesting situations the size of epsilon-nets depends only on $\epsilon$ together with different complexity measures. The aim of this paper is to give a systematic treatment of such complexity measures arising in Discrete and Computational Geometry and Statistical Learning, and to bridge the gap between the results appearing in these two fields. As a byproduct, we obtain several new upper bounds on the sizes of epsilon-nets that generalize/improve the best known general guarantees. In particular, our results work with regimes when small epsilon-nets of size $o(\frac{1}{\epsilon})$ exist, which are not usually covered by standard upper bounds. Inspired by results in Statistical Learning we also give a short proof of the Haussler’s upper bound on packing numbers. |
| Tasks | |
| Published | 2017-11-28 |
| URL | https://arxiv.org/abs/1711.10414v3 |
| https://arxiv.org/pdf/1711.10414v3.pdf | |
| PWC | https://paperswithcode.com/paper/when-are-epsilon-nets-small |
| Repo | |
| Framework | |
Speeding-up ProbLog’s Parameter Learning
| Title | Speeding-up ProbLog’s Parameter Learning |
| Authors | Francisco H. O. V. de Faria, Arthur C. Gusmão, Fabio G. Cozman, Denis D. Mauá |
| Abstract | ProbLog is a state-of-art combination of logic programming and probabilities; in particular ProbLog offers parameter learning through a variant of the EM algorithm. However, the resulting learning algorithm is rather slow, even when the data are complete. In this short paper we offer some insights that lead to orders of magnitude improvements in ProbLog’s parameter learning speed with complete data. |
| Tasks | |
| Published | 2017-07-25 |
| URL | http://arxiv.org/abs/1707.08151v2 |
| http://arxiv.org/pdf/1707.08151v2.pdf | |
| PWC | https://paperswithcode.com/paper/speeding-up-problogs-parameter-learning |
| Repo | |
| Framework | |
Rényi Differential Privacy Mechanisms for Posterior Sampling
| Title | Rényi Differential Privacy Mechanisms for Posterior Sampling |
| Authors | Joseph Geumlek, Shuang Song, Kamalika Chaudhuri |
| Abstract | Using a recently proposed privacy definition of R'enyi Differential Privacy (RDP), we re-examine the inherent privacy of releasing a single sample from a posterior distribution. We exploit the impact of the prior distribution in mitigating the influence of individual data points. In particular, we focus on sampling from an exponential family and specific generalized linear models, such as logistic regression. We propose novel RDP mechanisms as well as offering a new RDP analysis for an existing method in order to add value to the RDP framework. Each method is capable of achieving arbitrary RDP privacy guarantees, and we offer experimental results of their efficacy. |
| Tasks | |
| Published | 2017-10-02 |
| URL | http://arxiv.org/abs/1710.00892v1 |
| http://arxiv.org/pdf/1710.00892v1.pdf | |
| PWC | https://paperswithcode.com/paper/renyi-differential-privacy-mechanisms-for |
| Repo | |
| Framework | |
CSSTag: Optical Nanoscale Radar and Particle Tracking for In-Body and Microfluidic Systems with Vibrating Graphene and Resonance Energy Transfer
| Title | CSSTag: Optical Nanoscale Radar and Particle Tracking for In-Body and Microfluidic Systems with Vibrating Graphene and Resonance Energy Transfer |
| Authors | Burhan Gulbahar, Gorkem Memisoglu |
| Abstract | Single particle tracking systems monitor cellular processes with great accuracy in nano-biological systems. The emissions of the fluorescent molecules are detected with cameras or photodetectors. However, state-of-the-art imaging systems have challenges in the detection capability, collection and analysis of imaging data, penetration depth and complicated set-ups. In this article, a \textit{signaling based nanoscale acousto-optic radar and microfluidic particle tracking system} is proposed based on the theoretical design providing nanoscale optical modulator with vibrating F{"{o}}rster resonance energy transfer (VFRET) and vibrating CdSe/ZnS quantum dots (QDs) on graphene resonators. The modulator structure combines the significant advantages of graphene membranes having wideband resonance frequencies with QDs having broad absorption spectrum and tunable properties. The solution denoted by chirp spread spectrum (CSS) Tag (\textit{CSSTag}) utilizes classical radar target tracking approaches in nanoscale environments based on the capability to generate CSS sequences to identify different bio-particles. Numerical and Monte-Carlo simulations are realized showing the significant performance for multiple particle tracking (MPT) with a modulator of $10 , \mu$m $\times$ $10 , \mu$m $\times$ $10 , \mu$m dimension and several picograms of weight, signal to noise ratio (SNR) in the range $-7$ dB to $10$ dB and high speed tracking capability for microfluidic and in-body environments. |
| Tasks | |
| Published | 2017-09-04 |
| URL | http://arxiv.org/abs/1709.00907v1 |
| http://arxiv.org/pdf/1709.00907v1.pdf | |
| PWC | https://paperswithcode.com/paper/csstag-optical-nanoscale-radar-and-particle |
| Repo | |
| Framework | |
