January 27, 2020

3120 words 15 mins read

Paper Group ANR 1339

On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network. Learning to Project in Multi-Objective Binary Linear Programming. Knowledge-aware Pronoun Coreference Resolution. A Neural Network Based Method to Solve Boundary Value Problems. SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks. D …

On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network


Title	On-chip learning in a conventional silicon MOSFET based Analog Hardware Neural Network
Authors	Nilabjo Dey, Janak Sharda, Utkarsh Saxena, Divya Kaushik, Utkarsh Singh, Debanjan Bhowmik
Abstract	On-chip learning in a crossbar array based analog hardware Neural Network (NN) has been shown to have major advantages in terms of speed and energy compared to training NN on a traditional computer. However analog hardware NN proposals and implementations thus far have mostly involved Non Volatile Memory (NVM) devices like Resistive Random Access Memory (RRAM), Phase Change Memory (PCM), spintronic devices or floating gate transistors as synapses. Fabricating systems based on RRAM, PCM or spintronic devices need in-house laboratory facilities and cannot be done through merchant foundries, unlike conventional silicon based CMOS chips. Floating gate transistors need large voltage pulses for weight update, making on-chip learning in such systems energy inefficient. This paper proposes and implements through SPICE simulations on-chip learning in analog hardware NN using only conventional silicon based MOSFETs (without any floating gate) as synapses since they are easy to fabricate. We first model the synaptic characteristic of our single transistor synapse using SPICE circuit simulator and benchmark it against experimentally obtained current-voltage characteristics of a transistor. Next we design a Fully Connected Neural Network (FCNN) crossbar array using such transistor synapses. We also design analog peripheral circuits for neuron and synaptic weight update calculation, needed for on-chip learning, again using conventional transistors. Simulating the entire system on SPICE simulator, we obtain high training and test accuracy on the standard Fisher’s Iris dataset, widely used in machine learning. We also compare the speed and energy performance of our transistor based implementation of analog hardware NN with some previous implementations of NN with NVM devices and show comparable performance with respect to on-chip learning.
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00625v1
PDF	https://arxiv.org/pdf/1907.00625v1.pdf
PWC	https://paperswithcode.com/paper/on-chip-learning-in-a-conventional-silicon
Repo
Framework

Learning to Project in Multi-Objective Binary Linear Programming


Title	Learning to Project in Multi-Objective Binary Linear Programming
Authors	Alvaro Sierra-Altamiranda, Hadi Charkhgard, Iman Dayarian, Ali Eshragh, Sorna Javadi
Abstract	In this paper, we investigate the possibility of improving the performance of multi-objective optimization solution approaches using machine learning techniques. Specifically, we focus on multi-objective binary linear programs and employ one of the most effective and recently developed criterion space search algorithms, the so-called KSA, during our study. This algorithm computes all nondominated points of a problem with p objectives by searching on a projected criterion space, i.e., a (p-1)-dimensional criterion apace. We present an effective and fast learning approach to identify on which projected space the KSA should work. We also present several generic features/variables that can be used in machine learning techniques for identifying the best projected space. Finally, we present an effective bi-objective optimization based heuristic for selecting the best subset of the features to overcome the issue of overfitting in learning. Through an extensive computational study over 2000 instances of tri-objective Knapsack and Assignment problems, we demonstrate that an improvement of up to 12% in time can be achieved by the proposed learning method compared to a random selection of the projected space.
Tasks
Published	2019-01-30
URL	http://arxiv.org/abs/1901.10868v1
PDF	http://arxiv.org/pdf/1901.10868v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-project-in-multi-objective-binary
Repo
Framework

Knowledge-aware Pronoun Coreference Resolution


Title	Knowledge-aware Pronoun Coreference Resolution
Authors	Hongming Zhang, Yan Song, Yangqiu Song, Dong Yu
Abstract	Resolving pronoun coreference requires knowledge support, especially for particular domains (e.g., medicine). In this paper, we explore how to leverage different types of knowledge to better resolve pronoun coreference with a neural model. To ensure the generalization ability of our model, we directly incorporate knowledge in the format of triplets, which is the most common format of modern knowledge graphs, instead of encoding it with features or rules as that in conventional approaches. Moreover, since not all knowledge is helpful in certain contexts, to selectively use them, we propose a knowledge attention module, which learns to select and use informative knowledge based on contexts, to enhance our model. Experimental results on two datasets from different domains prove the validity and effectiveness of our model, where it outperforms state-of-the-art baselines by a large margin. Moreover, since our model learns to use external knowledge rather than only fitting the training data, it also demonstrates superior performance to baselines in the cross-domain setting.
Tasks	Coreference Resolution, Knowledge Graphs
Published	2019-07-08
URL	https://arxiv.org/abs/1907.03663v1
PDF	https://arxiv.org/pdf/1907.03663v1.pdf
PWC	https://paperswithcode.com/paper/knowledge-aware-pronoun-coreference
Repo
Framework

A Neural Network Based Method to Solve Boundary Value Problems


Title	A Neural Network Based Method to Solve Boundary Value Problems
Authors	Sethu Hareesh Kolluru
Abstract	A Neural Network (NN) based numerical method is formulated and implemented for solving Boundary Value Problems (BVPs) and numerical results are presented to validate this method by solving Laplace equation with Dirichlet boundary condition and Poisson’s equation with mixed boundary conditions. The principal advantage of NN based numerical method is the discrete data points where the field is computed, can be unstructured and do not suffer from issues of meshing like traditional numerical methods such as Finite Difference Time Domain or Finite Element Method. Numerical investigations are carried out for both uniform and non-uniform training grid distributions to understand the efficacy and limitations of this method and to provide qualitative understanding of various parameters involved.
Tasks
Published	2019-09-24
URL	https://arxiv.org/abs/1909.11082v1
PDF	https://arxiv.org/pdf/1909.11082v1.pdf
PWC	https://paperswithcode.com/paper/a-neural-network-based-method-to-solve
Repo
Framework

SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks


Title	SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
Authors	Cenk Baykal, Lucas Liebenwein, Igor Gilitschenski, Dan Feldman, Daniela Rus
Abstract	We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model’s predictive accuracy. Our algorithm uses a small batch of input points to construct a data-informed importance sampling distribution over the network’s parameters, and adaptively mixes a sampling-based and deterministic pruning procedure to discard redundant weights. Our pruning method is simultaneously computationally efficient, provably accurate, and broadly applicable to various network architectures and data distributions. Our empirical comparisons show that our algorithm reliably generates highly compressed networks that incur minimal loss in performance relative to that of the original network. We present experimental results that demonstrate our algorithm’s potential to unearth essential network connections that can be trained successfully in isolation, which may be of independent interest.
Tasks
Published	2019-10-11
URL	https://arxiv.org/abs/1910.05422v1
PDF	https://arxiv.org/pdf/1910.05422v1.pdf
PWC	https://paperswithcode.com/paper/sipping-neural-networks-sensitivity-informed
Repo
Framework

Deep Context-Aware Recommender System Utilizing Sequential Latent Context


Title	Deep Context-Aware Recommender System Utilizing Sequential Latent Context
Authors	Amit Livne, Moshe Unger, Bracha Shapira, Lior Rokach
Abstract	Context-aware recommender systems (CARSs) apply sensing and analysis of user context in order to provide personalized services. Adding context to a recommendation model is challenging, since the addition of context may increases both the dimensionality and sparsity of the model. Recent research has shown that modeling contextual information as a latent vector may address the sparsity and dimensionality challenges. We suggest a new latent modeling of sequential context by generating sequences of contextual information and reducing their contextual space to a compressed latent space.We train a long short-term memory (LSTM) encoder-decoder network on sequences of contextual information and extract sequential latent context from the hidden layer of the network in order to represent a compressed representation of sequential data. We propose new context-aware recommendation models that extend the neural collaborative filtering approach and learn nonlinear interactions between latent features of users, items, and contexts which take into account the sequential latent context representation as part of the recommendation process. We deployed our approach using two context-aware datasets with different context dimensions. Empirical analysis of our results validates that our proposed sequential latent context-aware model (SLCM), surpasses state of the art CARS models.
Tasks	Recommendation Systems
Published	2019-09-09
URL	https://arxiv.org/abs/1909.03999v1
PDF	https://arxiv.org/pdf/1909.03999v1.pdf
PWC	https://paperswithcode.com/paper/deep-context-aware-recommender-system
Repo
Framework

WASA: A Web Application for Sequence Annotation


Title	WASA: A Web Application for Sequence Annotation
Authors	Fahad AlGhamdi, Mona Diab
Abstract	Data annotation is an important and necessary task for all NLP applications. Designing and implementing a web-based application that enables many annotators to annotate and enter their input into one central database is not a trivial task. These kinds of web-based applications require a consistent and robust backup for the underlying database and support to enhance the efficiency and speed of the annotation. Also, they need to ensure that the annotations are stored with a minimal amount of redundancy in order to take advantage of the available resources(e.g, storage space). In this paper, we introduce WASA, a web-based annotation system for managing large-scale multilingual Code Switching (CS) data annotation. Although WASA has the ability to perform the annotation for any token sequence with arbitrary tag sets, we will focus on how WASA is used for CS annotation. The system supports concurrent annotation, handles multiple encodings, allows for several levels of management control, and enables quality control measures while seamlessly reporting annotation statistics from various perspectives and at different levels of granularity. Moreover, the system is integrated with a robust language specific date prepossessing tool to enhance the speed and efficiency of the annotation. We describe the annotation and the administration interfaces as well as the backend engine.
Tasks
Published	2019-09-28
URL	https://arxiv.org/abs/1909.13008v1
PDF	https://arxiv.org/pdf/1909.13008v1.pdf
PWC	https://paperswithcode.com/paper/wasa-a-web-application-for-sequence-1
Repo
Framework

Gradient Descent Maximizes the Margin of Homogeneous Neural Networks


Title	Gradient Descent Maximizes the Margin of Homogeneous Neural Networks
Authors	Kaifeng Lyu, Jian Li
Abstract	In this paper, we study the implicit regularization of the gradient descent algorithm in homogeneous neural networks, including fully-connected and convolutional neural networks with ReLU or LeakyReLU activations. In particular, we study the gradient descent or gradient flow (i.e., gradient descent with infinitesimal step size) optimizing the logistic loss or cross-entropy loss of any homogeneous model (possibly non-smooth), and show that if the training loss decreases below a certain threshold, then we can define a smoothed version of the normalized margin which increases over time. We also formulate a natural constrained optimization problem related to margin maximization, and prove that both the normalized margin and its smoothed version converge to the objective value at a KKT point of the optimization problem. Our results generalize the previous results for logistic regression with one-layer or multi-layer linear networks, and provide more quantitative convergence results with weaker assumptions than previous results for homogeneous smooth neural networks. We conduct several experiments to justify our theoretical finding on MNIST and CIFAR-10 datasets. Finally, as margin is closely related to robustness, we discuss potential benefits of training longer for improving the robustness of the model.
Tasks
Published	2019-06-13
URL	https://arxiv.org/abs/1906.05890v3
PDF	https://arxiv.org/pdf/1906.05890v3.pdf
PWC	https://paperswithcode.com/paper/gradient-descent-maximizes-the-margin-of
Repo
Framework

Control What You Can: Intrinsically Motivated Task-Planning Agent


Title	Control What You Can: Intrinsically Motivated Task-Planning Agent
Authors	Sebastian Blaes, Marin Vlastelica Pogančić, Jia-Jie Zhu, Georg Martius
Abstract	We present a novel intrinsically motivated agent that learns how to control the environment in the fastest possible manner by optimizing learning progress. It learns what can be controlled, how to allocate time and attention, and the relations between objects using surprise based motivation. The effectiveness of our method is demonstrated in a synthetic as well as a robotic manipulation environment yielding considerably improved performance and smaller sample complexity. In a nutshell, our work combines several task-level planning agent structures (backtracking search on task graph, probabilistic road-maps, allocation of search efforts) with intrinsic motivation to achieve learning from scratch.
Tasks
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08190v2
PDF	https://arxiv.org/pdf/1906.08190v2.pdf
PWC	https://paperswithcode.com/paper/control-what-you-can-intrinsically-motivated
Repo
Framework

Thinking Outside the Pool: Active Training Image Creation for Relative Attributes


Title	Thinking Outside the Pool: Active Training Image Creation for Relative Attributes
Authors	Aron Yu, Kristen Grauman
Abstract	Current wisdom suggests more labeled image data is always better, and obtaining labels is the bottleneck. Yet curating a pool of sufficiently diverse and informative images is itself a challenge. In particular, training image curation is problematic for fine-grained attributes, where the subtle visual differences of interest may be rare within traditional image sources. We propose an active image generation approach to address this issue. The main idea is to jointly learn the attribute ranking task while also learning to generate novel realistic image samples that will benefit that task. We introduce an end-to-end framework that dynamically “imagines” image pairs that would confuse the current model, presents them to human annotators for labeling, then improves the predictive model with the new examples. With results on two datasets, we show that by thinking outside the pool of real images, our approach gains generalization accuracy for challenging fine-grained attribute comparisons.
Tasks	Image Generation
Published	2019-01-08
URL	http://arxiv.org/abs/1901.02551v1
PDF	http://arxiv.org/pdf/1901.02551v1.pdf
PWC	https://paperswithcode.com/paper/thinking-outside-the-pool-active-training
Repo
Framework

Deep Morphological Neural Networks


Title	Deep Morphological Neural Networks
Authors	Yucong Shen, Xin Zhong, Frank Y. Shih
Abstract	Mathematical morphology is a theory and technique to collect features like geometric and topological structures in digital images. Given a target image, determining suitable morphological operations and structuring elements is a cumbersome and time-consuming task. In this paper, a morphological neural network is proposed to address this problem. Serving as a nonlinear feature extracting layer in deep learning frameworks, the efficiency of the proposed morphological layer is confirmed analytically and empirically. With a known target, a single-filter morphological layer learns the structuring element correctly, and an adaptive layer can automatically select appropriate morphological operations. For practical applications, the proposed morphological neural networks are tested on several classification datasets related to shape or geometric image features, and the experimental results have confirmed the high computational efficiency and high accuracy.
Tasks
Published	2019-09-04
URL	https://arxiv.org/abs/1909.01532v1
PDF	https://arxiv.org/pdf/1909.01532v1.pdf
PWC	https://paperswithcode.com/paper/deep-morphological-neural-networks
Repo
Framework

An Approach to Super-Resolution of Sentinel-2 Images Based on Generative Adversarial Networks


Title	An Approach to Super-Resolution of Sentinel-2 Images Based on Generative Adversarial Networks
Authors	Kexin Zhang, Gencer Sumbul, Begüm Demir
Abstract	This paper presents a generative adversarial network based super-resolution (SR) approach (which is called as S2GAN) to enhance the spatial resolution of Sentinel-2 spectral bands. The proposed approach consists of two main steps. The first step aims to increase the spatial resolution of the bands with 20m and 60m spatial resolutions by the scaling factors of 2 and 6, respectively. To this end, we introduce a generator network that performs SR on the lower resolution bands with the guidance of the bands associated to 10m spatial resolution by utilizing the convolutional layers with residual connections and a long skip-connection between inputs and outputs. The second step aims to distinguish SR bands from their ground truth bands. This is achieved by the proposed discriminator network, which alternately characterizes the high level features of the two sets of bands and applying binary classification on the extracted features. Then, we formulate the adversarial learning of the generator and discriminator networks as a min-max game. In this learning procedure, the generator aims to produce realistic SR bands as much as possible so that the discriminator incorrectly classifies SR bands. Experimental results obtained on different Sentinel-2 images show the effectiveness of the proposed approach compared to both conventional and deep learning based SR approaches.
Tasks	Super-Resolution
Published	2019-12-12
URL	https://arxiv.org/abs/1912.06013v2
PDF	https://arxiv.org/pdf/1912.06013v2.pdf
PWC	https://paperswithcode.com/paper/an-approach-to-super-resolution-of-sentinel-2
Repo
Framework

Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets


Title	Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets
Authors	Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Karan Singh
Abstract	We present a learning method for predicting animation skeletons for input 3D models of articulated characters. In contrast to previous approaches that fit pre-defined skeleton templates or predict fixed sets of joints, our method produces an animation skeleton tailored for the structure and geometry of the input 3D model. Our architecture is based on a stack of hourglass modules trained on a large dataset of 3D rigged characters mined from the web. It operates on the volumetric representation of the input 3D shapes augmented with geometric shape features that provide additional cues for joint and bone locations. Our method also enables intuitive user control of the level-of-detail for the output skeleton. Our evaluation demonstrates that our approach predicts animation skeletons that are much more similar to the ones created by humans compared to several alternatives and baselines.
Tasks
Published	2019-08-22
URL	https://arxiv.org/abs/1908.08506v1
PDF	https://arxiv.org/pdf/1908.08506v1.pdf
PWC	https://paperswithcode.com/paper/predicting-animation-skeletons-for-3d
Repo
Framework

Image Denoising with Graph-Convolutional Neural Networks


Title	Image Denoising with Graph-Convolutional Neural Networks
Authors	Diego Valsesia, Giulia Fracastoro, Enrico Magli
Abstract	Recovering an image from a noisy observation is a key problem in signal processing. Recently, it has been shown that data-driven approaches employing convolutional neural networks can outperform classical model-based techniques, because they can capture more powerful and discriminative features. However, since these methods are based on convolutional operations, they are only capable of exploiting local similarities without taking into account non-local self-similarities. In this paper we propose a convolutional neural network that employs graph-convolutional layers in order to exploit both local and non-local similarities. The graph-convolutional layers dynamically construct neighborhoods in the feature space to detect latent correlations in the feature maps produced by the hidden layers. The experimental results show that the proposed architecture outperforms classical convolutional neural networks for the denoising task.
Tasks	Denoising, Image Denoising
Published	2019-05-29
URL	https://arxiv.org/abs/1905.12281v1
PDF	https://arxiv.org/pdf/1905.12281v1.pdf
PWC	https://paperswithcode.com/paper/image-denoising-with-graph-convolutional
Repo
Framework

Using Deep Learning to Predict Plant Growth and Yield in Greenhouse Environments


Title	Using Deep Learning to Predict Plant Growth and Yield in Greenhouse Environments
Authors	Bashar Alhnaity, Simon Pearson, Georgios Leontidis, Stefanos Kollias
Abstract	Effective plant growth and yield prediction is an essential task for greenhouse growers and for agriculture in general. Developing models which can effectively model growth and yield can help growers improve the environmental control for better production, match supply and market demand and lower costs. Recent developments in Machine Learning (ML) and, in particular, Deep Learning (DL) can provide powerful new analytical tools. The proposed study utilises ML and DL techniques to predict yield and plant growth variation across two different scenarios, tomato yield forecasting and Ficus benjamina stem growth, in controlled greenhouse environments. We deploy a new deep recurrent neural network (RNN), using the Long Short-Term Memory (LSTM) neuron model, in the prediction formulations. Both the former yield, growth and stem diameter values, as well as the microclimate conditions, are used by the RNN architecture to model the targeted growth parameters. A comparative study is presented, using ML methods, such as support vector regression and random forest regression, utilising the mean square error criterion, in order to evaluate the performance achieved by the different methods. Very promising results, based on data that have been obtained from two greenhouses, in Belgium and the UK, in the framework of the EU Interreg SMARTGREEN project (2017-2021), are presented.
Tasks
Published	2019-07-01
URL	https://arxiv.org/abs/1907.00624v1
PDF	https://arxiv.org/pdf/1907.00624v1.pdf
PWC	https://paperswithcode.com/paper/using-deep-learning-to-predict-plant-growth
Repo
Framework