January 28, 2020

3308 words 16 mins read

Paper Group ANR 961

Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space. Ocular Recognition Databases and Competitions: A Survey. Optimization-Guided Binary Diversification to Mislead Neural Networks for Malware Detection. Degenerative Adversarial NeuroImage Nets for 4D Simulations: Application in Longitudinal MRI. A Multiple …

Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space


Title	Rotate King to get Queen: Word Relationships as Orthogonal Transformations in Embedding Space
Authors	Kawin Ethayarajh
Abstract	A notable property of word embeddings is that word relationships can exist as linear substructures in the embedding space. For example, $\textit{gender}$ corresponds to $\vec{\textit{woman}} - \vec{\textit{man}}$ and $\vec{\textit{queen}} - \vec{\textit{king}}$. This, in turn, allows word analogies to be solved arithmetically: $\vec{\textit{king}} - \vec{\textit{man}} + \vec{\textit{woman}} \approx \vec{\textit{queen}}$. This property is notable because it suggests that models trained on word embeddings can easily learn such relationships as geometric translations. However, there is no evidence that models $\textit{exclusively}$ represent relationships in this manner. We document an alternative way in which downstream models might learn these relationships: orthogonal and linear transformations. For example, given a translation vector for $\textit{gender}$, we can find an orthogonal matrix $R$, representing a rotation and reflection, such that $R(\vec{\textit{king}}) \approx \vec{\textit{queen}}$ and $R(\vec{\textit{man}}) \approx \vec{\textit{woman}}$. Analogical reasoning using orthogonal transformations is almost as accurate as using vector arithmetic; using linear transformations is more accurate than both. Our findings suggest that these transformations can be as good a representation of word relationships as translation vectors.
Tasks	Word Embeddings
Published	2019-09-02
URL	https://arxiv.org/abs/1909.00504v2
PDF	https://arxiv.org/pdf/1909.00504v2.pdf
PWC	https://paperswithcode.com/paper/rotate-textitking-to-get-textitqueen-word
Repo
Framework

Ocular Recognition Databases and Competitions: A Survey


Title	Ocular Recognition Databases and Competitions: A Survey
Authors	Luiz A. Zanlorensi, Rayson Laroca, Eduardo Luz, Alceu S. Britto Jr., Luiz S. Oliveira, David Menotti
Abstract	The use of the iris and periocular region as biometric traits has been extensively investigated, mainly due to the singularity of the iris features and the use of the periocular region when the image resolution is not sufficient to extract iris information. In addition to providing information about an individual’s identity, features extracted from these traits can also be explored to obtain other information such as the individual’s gender, the influence of drug use, the use of contact lenses, spoofing, among others. This work presents a survey of the databases created for ocular recognition, detailing their protocols and how their images were acquired. We also describe and discuss the most popular ocular recognition competitions (contests), highlighting the submitted algorithms that achieved the best results using only iris trait and also fusing iris and periocular region information. Finally, we describe some relevant works applying deep learning techniques to ocular recognition and point out new challenges and future directions. Considering that there are a large number of ocular databases, and each one is usually designed for a specific problem, we believe this survey can provide a broad overview of the challenges in ocular biometrics.
Tasks
Published	2019-11-21
URL	https://arxiv.org/abs/1911.09646v1
PDF	https://arxiv.org/pdf/1911.09646v1.pdf
PWC	https://paperswithcode.com/paper/ocular-recognition-databases-and-competitions
Repo
Framework

Optimization-Guided Binary Diversification to Mislead Neural Networks for Malware Detection


Title	Optimization-Guided Binary Diversification to Mislead Neural Networks for Malware Detection
Authors	Mahmood Sharif, Keane Lucas, Lujo Bauer, Michael K. Reiter, Saurabh Shintre
Abstract	Motivated by the transformative impact of deep neural networks (DNNs) on different areas (e.g., image and speech recognition), researchers and anti-virus vendors are proposing end-to-end DNNs for malware detection from raw bytes that do not require manual feature engineering. Given the security sensitivity of the task that these DNNs aim to solve, it is important to assess their susceptibility to evasion. In this work, we propose an attack that guides binary-diversification tools via optimization to mislead DNNs for malware detection while preserving the functionality of binaries. Unlike previous attacks on such DNNs, ours manipulates instructions that are a functional part of the binary, which makes it particularly challenging to defend against. We evaluated our attack against three DNNs in white-box and black-box settings, and found that it can often achieve success rates near 100%. Moreover, we found that our attack can fool some commercial anti-viruses, in certain cases with a success rate of 85%. We explored several defenses, both new and old, and identified some that can successfully prevent over 80% of our evasion attempts. However, these defenses may still be susceptible to evasion by adaptive attackers, and so we advocate for augmenting malware-detection systems with methods that do not rely on machine learning.
Tasks	Feature Engineering, Malware Detection, Speech Recognition
Published	2019-12-19
URL	https://arxiv.org/abs/1912.09064v1
PDF	https://arxiv.org/pdf/1912.09064v1.pdf
PWC	https://paperswithcode.com/paper/optimization-guided-binary-diversification-to
Repo
Framework

Degenerative Adversarial NeuroImage Nets for 4D Simulations: Application in Longitudinal MRI


Title	Degenerative Adversarial NeuroImage Nets for 4D Simulations: Application in Longitudinal MRI
Authors	Daniele Ravi, Stefano B. Blumberg, Kyriaki Mengoudi, Moucheng Xu, Daniel C. Alexander, Neil P. Oxtoby
Abstract	Accurate and realistic simulation of medical images is a growing area of research relevant to many healthcare applications. However, current image simulators have been unsuccessful when deployed on longitudinal clinical data — for example, disease progression modelling designed to generate 3D MRI sequences (4D). Failures are typically due to inability to produce subject-specific simulation, and inefficient implementations incapable of synthesizing spatiotemporal images in high resolution. Memory limitations preclude training of the full-4D model, necessitating techniques that discard spatiotemporal information, such as 2D slice-by-slice implementations or patch-based approaches. Here we introduce a novel technique to address this challenge, called Profile Weight Functions (PWF). We demonstrate the power of PWFs by extending a recent framework for neuroimage simulation from 2D (plus time) to 3D (plus time), which is not currently available. To our knowledge, we are the first to implement a disease progression simulator able to predict accurate sequences of realistic, high-resolution, 3D medical images. We demonstrate our framework by training a model using 9652 T1-weighted MRI from the Alzheimer’s Disease Neuroimaging Initiative dataset. We validate our results on a separate test set of 1216 MRI, demonstrating the capability to synthesize a personalized time-series of images given a single-time point and other metadata.
Tasks	Time Series
Published	2019-12-03
URL	https://arxiv.org/abs/1912.01526v2
PDF	https://arxiv.org/pdf/1912.01526v2.pdf
PWC	https://paperswithcode.com/paper/degenerative-adversarial-neuroimage-nets-for
Repo
Framework

A Multiple Source Hourglass Deep Network for Multi-Focus Image Fusion


Title	A Multiple Source Hourglass Deep Network for Multi-Focus Image Fusion
Authors	Fidel Alejandro Guerrero Peña, Pedro Diamel Marrero Fernández, Tsang Ing Ren, Germano Crispim Vasconcelos, Alexandre Cunha
Abstract	Multi-Focus Image Fusion seeks to improve the quality of an acquired burst of images with different focus planes. For solving the task, an activity level measurement and a fusion rule are typically established to select and fuse the most relevant information from the sources. However, the design of this kind of method by hand is really hard and sometimes restricted to solution spaces where the optimal all-in-focus images are not contained. Then, we propose here two fast and straightforward approaches for image fusion based on deep neural networks. Our solution uses a multiple source Hourglass architecture trained in an end-to-end fashion. Models are data-driven and can be easily generalized for other kinds of fusion problems. A segmentation approach is used for recognition of the focus map, while the weighted average rule is used for fusion. We designed a training loss function for our regression-based fusion function, which allows the network to learn both the activity level measurement and the fusion rule. Experimental results show our approach has comparable results to the state-of-the-art methods with a 60X increase of computational efficiency for 520X520 resolution images.
Tasks
Published	2019-08-28
URL	https://arxiv.org/abs/1908.10945v1
PDF	https://arxiv.org/pdf/1908.10945v1.pdf
PWC	https://paperswithcode.com/paper/a-multiple-source-hourglass-deep-network-for
Repo
Framework

An Integrated Multi-Time-Scale Modeling for Solar Irradiance Forecasting Using Deep Learning


Title	An Integrated Multi-Time-Scale Modeling for Solar Irradiance Forecasting Using Deep Learning
Authors	Sakshi Mishra, Praveen Palanisamy
Abstract	For short-term solar irradiance forecasting, the traditional point forecasting methods are rendered less useful due to the non-stationary characteristic of solar power. The amount of operating reserves required to maintain reliable operation of the electric grid rises due to the variability of solar energy. The higher the uncertainty in the generation, the greater the operating-reserve requirements, which translates to an increased cost of operation. In this research work, we propose a unified architecture for multi-time-scale predictions for intra-day solar irradiance forecasting using recurrent neural networks (RNN) and long-short-term memory networks (LSTMs). This paper also lays out a framework for extending this modeling approach to intra-hour forecasting horizons thus, making it a multi-time-horizon forecasting approach, capable of predicting intra-hour as well as intra-day solar irradiance. We develop an end-to-end pipeline to effectuate the proposed architecture. The performance of the prediction model is tested and validated by the methodical implementation. The robustness of the approach is demonstrated with case studies conducted for geographically scattered sites across the United States. The predictions demonstrate that our proposed unified architecture-based approach is effective for multi-time-scale solar forecasts and achieves a lower root-mean-square prediction error when benchmarked against the best-performing methods documented in the literature that use separate models for each time-scale during the day. Our proposed method results in a 71.5% reduction in the mean RMSE averaged across all the test sites compared to the ML-based best-performing method reported in the literature. Additionally, the proposed method enables multi-time-horizon forecasts with real-time inputs, which have a significant potential for practical industry applications in the evolving grid.
Tasks
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02616v2
PDF	https://arxiv.org/pdf/1905.02616v2.pdf
PWC	https://paperswithcode.com/paper/an-integrated-multi-time-scale-modeling-for
Repo
Framework

IPC-Net: 3D point-cloud segmentation using deep inter-point convolutional layers


Title	IPC-Net: 3D point-cloud segmentation using deep inter-point convolutional layers
Authors	Felipe Gomez Marulanda, Pieter Libin, Timothy Verstraeten, Ann Nowé
Abstract	Over the last decade, the demand for better segmentation and classification algorithms in 3D spaces has significantly grown due to the popularity of new 3D sensor technologies and advancements in the field of robotics. Point-clouds are one of the most popular representations to store a digital description of 3D shapes. However, point-clouds are stored in irregular and unordered structures, which limits the direct use of segmentation algorithms such as Convolutional Neural Networks. The objective of our work is twofold: First, we aim to provide a full analysis of the PointNet architecture to illustrate which features are being extracted from the point-clouds. Second, to propose a new network architecture called IPC-Net to improve the state-of-the-art point cloud architectures. We show that IPC-Net extracts a larger set of unique features allowing the model to produce more accurate segmentations compared to the PointNet architecture. In general, our approach outperforms PointNet on every family of 3D geometries on which the models were tested. A high generalisation improvement was observed on every 3D shape, especially on the rockets dataset. Our experiments demonstrate that our main contribution, inter-point activation on the network’s layers, is essential to accurately segment 3D point-clouds.
Tasks
Published	2019-09-30
URL	https://arxiv.org/abs/1909.13726v1
PDF	https://arxiv.org/pdf/1909.13726v1.pdf
PWC	https://paperswithcode.com/paper/ipc-net-3d-point-cloud-segmentation-using
Repo
Framework

Automatic Extraction of Personality from Text: Challenges and Opportunities


Title	Automatic Extraction of Personality from Text: Challenges and Opportunities
Authors	Nazar Akrami, Johan Fernquist, Tim Isbister, Lisa Kaati, Björn Pelzer
Abstract	In this study, we examined the possibility to extract personality traits from a text. We created an extensive dataset by having experts annotate personality traits in a large number of texts from multiple online sources. From these annotated texts, we selected a sample and made further annotations ending up in a large low-reliability dataset and a small high-reliability dataset. We then used the two datasets to train and test several machine learning models to extract personality from text, including a language model. Finally, we evaluated our best models in the wild, on datasets from different domains. Our results show that the models based on the small high-reliability dataset performed better (in terms of $\textrm{R}^2$) than models based on large low-reliability dataset. Also, language model based on small high-reliability dataset performed better than the random baseline. Finally, and more importantly, the results showed our best model did not perform better than the random baseline when tested in the wild. Taken together, our results show that determining personality traits from a text remains a challenge and that no firm conclusions can be made on model performance before testing in the wild.
Tasks	Language Modelling
Published	2019-10-22
URL	https://arxiv.org/abs/1910.09916v1
PDF	https://arxiv.org/pdf/1910.09916v1.pdf
PWC	https://paperswithcode.com/paper/automatic-extraction-of-personality-from-text
Repo
Framework

Efficient Search for Diverse Coherent Explanations


Title	Efficient Search for Diverse Coherent Explanations
Authors	Chris Russell
Abstract	This paper proposes new search algorithms for counterfactual explanations based upon mixed integer programming. We are concerned with complex data in which variables may take any value from a contiguous range or an additional set of discrete states. We propose a novel set of constraints that we refer to as a “mixed polytope” and show how this can be used with an integer programming solver to efficiently find coherent counterfactual explanations i.e. solutions that are guaranteed to map back onto the underlying data structure, while avoiding the need for brute-force enumeration. We also look at the problem of diverse explanations and show how these can be generated within our framework.
Tasks
Published	2019-01-02
URL	https://arxiv.org/abs/1901.04909v1
PDF	https://arxiv.org/pdf/1901.04909v1.pdf
PWC	https://paperswithcode.com/paper/efficient-search-for-diverse-coherent
Repo
Framework

RelExt: Relation Extraction using Deep Learning approaches for Cybersecurity Knowledge Graph Improvement


Title	RelExt: Relation Extraction using Deep Learning approaches for Cybersecurity Knowledge Graph Improvement
Authors	Aditya Pingle, Aritran Piplai, Sudip Mittal, Anupam Joshi, James Holt, Richard Zak
Abstract	Security Analysts that work in a `Security Operations Center’ (SoC) play a major role in ensuring the security of the organization. The amount of background knowledge they have about the evolving and new attacks makes a significant difference in their ability to detect attacks. Open source threat intelligence sources, like text descriptions about cyber-attacks, can be stored in a structured fashion in a cybersecurity knowledge graph. A cybersecurity knowledge graph can be paramount in aiding a security analyst to detect cyber threats because it stores a vast range of cyber threat information in the form of semantic triples which can be queried. A semantic triple contains two cybersecurity entities with a relationship between them. In this work, we propose a system to create semantic triples over cybersecurity text, using deep learning approaches to extract possible relationships. We use the set of semantic triples generated through our system to assert in a cybersecurity knowledge graph. Security Analysts can retrieve this data from the knowledge graph, and use this information to form a decision about a cyber-attack. \|
Tasks	Relation Extraction
Published	2019-05-07
URL	https://arxiv.org/abs/1905.02497v2
PDF	https://arxiv.org/pdf/1905.02497v2.pdf
PWC	https://paperswithcode.com/paper/relext-relation-extraction-using-deep
Repo
Framework

Dissecting Ethereum Blockchain Analytics: What We Learn from Topology and Geometry of Ethereum Graph


Title	Dissecting Ethereum Blockchain Analytics: What We Learn from Topology and Geometry of Ethereum Graph
Authors	Yitao Li, Umar Islambekov, Cuneyt Akcora, Ekaterina Smirnova, Yulia R. Gel, Murat Kantarcioglu
Abstract	Blockchain technology and, in particular, blockchain-based cryptocurrencies offer us information that has never been seen before in the financial world. In contrast to fiat currencies, all transactions of crypto-currencies and crypto-tokens are permanently recorded on distributed ledgers and are publicly available. As a result, this allows us to construct a transaction graph and to assess not only its organization but to glean relationships between transaction graph properties and crypto price dynamics. The ultimate goal of this paper is to facilitate our understanding on horizons and limitations of what can be learned on crypto-tokens from local topology and geometry of the Ethereum transaction network whose even global network properties remain scarcely explored. By introducing novel tools based on topological data analysis and functional data depth into Blockchain Data Analytics, we show that Ethereum network (one of the most popular blockchains for creating new crypto-tokens) can provide critical insights on price strikes of crypto-tokens that are otherwise largely inaccessible with conventional data sources and traditional analytic methods.
Tasks	Topological Data Analysis
Published	2019-12-20
URL	https://arxiv.org/abs/1912.10105v1
PDF	https://arxiv.org/pdf/1912.10105v1.pdf
PWC	https://paperswithcode.com/paper/dissecting-ethereum-blockchain-analytics-what
Repo
Framework

Auto-Rotating Perceptrons


Title	Auto-Rotating Perceptrons
Authors	Daniel Saromo, Elizabeth Villota, Edwin Villanueva
Abstract	This paper proposes an improved design of the perceptron unit to mitigate the vanishing gradient problem. This nuisance appears when training deep multilayer perceptron networks with bounded activation functions. The new neuron design, named auto-rotating perceptron (ARP), has a mechanism to ensure that the node always operates in the dynamic region of the activation function, by avoiding saturation of the perceptron. The proposed method does not change the inference structure learned at each neuron. We test the effect of using ARP units in some network architectures which use the sigmoid activation function. The results support our hypothesis that neural networks with ARP units can achieve better learning performance than equivalent models with classic perceptrons.
Tasks
Published	2019-10-06
URL	https://arxiv.org/abs/1910.02483v2
PDF	https://arxiv.org/pdf/1910.02483v2.pdf
PWC	https://paperswithcode.com/paper/auto-rotating-perceptrons
Repo
Framework

Automatic Parallel Corpus Creation for Hindi-English News Translation Task


Title	Automatic Parallel Corpus Creation for Hindi-English News Translation Task
Authors	Aditya Kumar Pathak, Priyankit Acharya, Dilpreet Kaur, Rakesh Chandra Balabantaray
Abstract	The parallel corpus for multilingual NLP tasks, deep learning applications like Statistical Machine Translation Systems is very important. The parallel corpus of Hindi-English language pair available for news translation task till date is of very limited size as per the requirement of the systems are concerned. In this work we have developed an automatic parallel corpus generation system prototype, which creates Hindi-English parallel corpus for news translation task. Further to verify the quality of generated parallel corpus we have experimented by taking various performance metrics and the results are quite interesting.
Tasks	Machine Translation
Published	2019-01-24
URL	http://arxiv.org/abs/1901.08625v1
PDF	http://arxiv.org/pdf/1901.08625v1.pdf
PWC	https://paperswithcode.com/paper/automatic-parallel-corpus-creation-for-hindi
Repo
Framework

Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding


Title	Contextualized Word Embeddings Enhanced Event Temporal Relation Extraction for Story Understanding
Authors	Rujun Han, Mengyue Liang, Bashar Alhafni, Nanyun Peng
Abstract	Learning causal and temporal relationships between events is an important step towards deeper story and commonsense understanding. Though there are abundant datasets annotated with event relations for story comprehension, many have no empirical results associated with them. In this work, we establish strong baselines for event temporal relation extraction on two under-explored story narrative datasets: Richer Event Description (RED) and Causal and Temporal Relation Scheme (CaTeRS). To the best of our knowledge, these are the first results reported on these two datasets. We demonstrate that neural network-based models can outperform some strong traditional linguistic feature-based models. We also conduct comparative studies to show the contribution of adopting contextualized word embeddings (BERT) for event temporal relation extraction from stories. Detailed analyses are offered to better understand the results.
Tasks	Relation Extraction, Word Embeddings
Published	2019-04-26
URL	http://arxiv.org/abs/1904.11942v1
PDF	http://arxiv.org/pdf/1904.11942v1.pdf
PWC	https://paperswithcode.com/paper/contextualized-word-embeddings-enhanced-event
Repo
Framework

Fragment Graphical Variational AutoEncoding for Screening Molecules with Small Data


Title	Fragment Graphical Variational AutoEncoding for Screening Molecules with Small Data
Authors	John Armitage, Leszek J. Spalek, Malgorzata Nguyen, Mark Nikolka, Ian E. Jacobs, Lorena Marañón, Iyad Nasrallah, Guillaume Schweicher, Ivan Dimov, Dimitrios Simatos, Iain McCulloch, Christian B. Nielsen, Gareth Conduit, Henning Sirringhaus
Abstract	In the majority of molecular optimization tasks, predictive machine learning (ML) models are limited due to the unavailability and cost of generating big experimental datasets on the specific task. To circumvent this limitation, ML models are trained on big theoretical datasets or experimental indicators of molecular suitability that are either publicly available or inexpensive to acquire. These approaches produce a set of candidate molecules which have to be ranked using limited experimental data or expert knowledge. Under the assumption that structure is related to functionality, here we use a molecular fragment-based graphical autoencoder to generate unique structural fingerprints to efficiently search through the candidate set. We demonstrate that fragment-based graphical autoencoding reduces the error in predicting physical characteristics such as the solubility and partition coefficient in the small data regime compared to other extended circular fingerprints and string based approaches. We further demonstrate that this approach is capable of providing insight into real world molecular optimization problems, such as searching for stabilization additives in organic semiconductors by accurately predicting 92% of test molecules given 69 training examples. This task is a model example of black box molecular optimization as there is minimal theoretical and experimental knowledge to accurately predict the suitability of the additives.
Tasks
Published	2019-10-21
URL	https://arxiv.org/abs/1910.13325v2
PDF	https://arxiv.org/pdf/1910.13325v2.pdf
PWC	https://paperswithcode.com/paper/fragment-graphical-variational-autoencoding
Repo
Framework