July 27, 2019

3265 words 16 mins read

Paper Group ANR 690

Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopy. Ontologies in System Engineering: a Field Report. Unsupervised robust nonparametric learning of hidden community properties. Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression. Deep Reservoir Computing Using Cellular Automat …

Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopy


Title	Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopy
Authors	Larissa Heinrich, John A. Bogovic, Stephan Saalfeld
Abstract	The most sophisticated existing methods to generate 3D isotropic super-resolution (SR) from non-isotropic electron microscopy (EM) are based on learned dictionaries. Unfortunately, none of the existing methods generate practically satisfying results. For 2D natural images, recently developed super-resolution methods that use deep learning have been shown to significantly outperform the previous state of the art. We have adapted one of the most successful architectures (FSRCNN) for 3D super-resolution, and compared its performance to a 3D U-Net architecture that has not been used previously to generate super-resolution. We trained both architectures on artificially downscaled isotropic ground truth from focused ion beam milling scanning EM (FIB-SEM) and tested the performance for various hyperparameter settings. Our results indicate that both architectures can successfully generate 3D isotropic super-resolution from non-isotropic EM, with the U-Net performing consistently better. We propose several promising directions for practical application.
Tasks	Super-Resolution
Published	2017-06-09
URL	http://arxiv.org/abs/1706.03142v1
PDF	http://arxiv.org/pdf/1706.03142v1.pdf
PWC	https://paperswithcode.com/paper/deep-learning-for-isotropic-super-resolution
Repo
Framework

Ontologies in System Engineering: a Field Report


Title	Ontologies in System Engineering: a Field Report
Authors	Marco Menapace, Armando Tacchella
Abstract	In recent years ontologies enjoyed a growing popularity outside specialized AI communities. System engineering is no exception to this trend, with ontologies being proposed as a basis for several tasks in complex industrial implements, including system design, monitoring and diagnosis. In this paper, we consider four different contributions to system engineering wherein ontologies are instrumental to provide enhancements over traditional ad-hoc techniques. For each application, we briefly report the methodologies, the tools and the results obtained with the goal to provide an assessment of merits and limits of ontologies in such domains.
Tasks
Published	2017-02-23
URL	http://arxiv.org/abs/1702.07193v1
PDF	http://arxiv.org/pdf/1702.07193v1.pdf
PWC	https://paperswithcode.com/paper/ontologies-in-system-engineering-a-field
Repo
Framework

Unsupervised robust nonparametric learning of hidden community properties


Title	Unsupervised robust nonparametric learning of hidden community properties
Authors	Mikhail A. Langovoy, Akhilesh Gotmare, Martin Jaggi
Abstract	We consider learning of fundamental properties of communities in large noisy networks, in the prototypical situation where the nodes or users are split into two classes according to a binary property, e.g., according to their opinions or preferences on a topic. For learning these properties, we propose a nonparametric, unsupervised, and scalable graph scan procedure that is, in addition, robust against a class of powerful adversaries. In our setup, one of the communities can fall under the influence of a knowledgeable adversarial leader, who knows the full network structure, has unlimited computational resources and can completely foresee our planned actions on the network. We prove strong consistency of our results in this setup with minimal assumptions. In particular, the learning procedure estimates the baseline activity of normal users asymptotically correctly with probability 1; the only assumption being the existence of a single implicit community of asymptotically negligible logarithmic size. We provide experiments on real and synthetic data to illustrate the performance of our method, including examples with adversaries.
Tasks
Published	2017-07-11
URL	http://arxiv.org/abs/1707.03494v2
PDF	http://arxiv.org/pdf/1707.03494v2.pdf
PWC	https://paperswithcode.com/paper/unsupervised-robust-nonparametric-learning-of
Repo
Framework

Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression


Title	Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression
Authors	Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, Duen Horng Chau
Abstract	Deep neural networks (DNNs) have achieved great success in solving a variety of machine learning (ML) problems, especially in the domain of image recognition. However, recent research showed that DNNs can be highly vulnerable to adversarially generated instances, which look seemingly normal to human observers, but completely confuse DNNs. These adversarial samples are crafted by adding small perturbations to normal, benign images. Such perturbations, while imperceptible to the human eye, are picked up by DNNs and cause them to misclassify the manipulated instances with high confidence. In this work, we explore and demonstrate how systematic JPEG compression can work as an effective pre-processing step in the classification pipeline to counter adversarial attacks and dramatically reduce their effects (e.g., Fast Gradient Sign Method, DeepFool). An important component of JPEG compression is its ability to remove high frequency signal components, inside square blocks of an image. Such an operation is equivalent to selective blurring of the image, helping remove additive perturbations. Further, we propose an ensemble-based technique that can be constructed quickly from a given well-performing DNN, and empirically show how such an ensemble that leverages JPEG compression can protect a model from multiple types of adversarial attacks, without requiring knowledge about the model.
Tasks
Published	2017-05-08
URL	http://arxiv.org/abs/1705.02900v1
PDF	http://arxiv.org/pdf/1705.02900v1.pdf
PWC	https://paperswithcode.com/paper/keeping-the-bad-guys-out-protecting-and
Repo
Framework

Deep Reservoir Computing Using Cellular Automata


Title	Deep Reservoir Computing Using Cellular Automata
Authors	Stefano Nichele, Andreas Molund
Abstract	Recurrent Neural Networks (RNNs) have been a prominent concept within artificial intelligence. They are inspired by Biological Neural Networks (BNNs) and provide an intuitive and abstract representation of how BNNs work. Derived from the more generic Artificial Neural Networks (ANNs), the recurrent ones are meant to be used for temporal tasks, such as speech recognition, because they are capable of memorizing historic input. However, such networks are very time consuming to train as a result of their inherent nature. Recently, Echo State Networks and Liquid State Machines have been proposed as possible RNN alternatives, under the name of Reservoir Computing (RC). RCs are far more easy to train. In this paper, Cellular Automata are used as reservoir, and are tested on the 5-bit memory task (a well known benchmark within the RC community). The work herein provides a method of mapping binary inputs from the task onto the automata, and a recurrent architecture for handling the sequential aspects of it. Furthermore, a layered (deep) reservoir architecture is proposed. Performances are compared towards earlier work, in addition to its single-layer version. Results show that the single CA reservoir system yields similar results to state-of-the-art work. The system comprised of two layered reservoirs do show a noticeable improvement compared to a single CA reservoir. This indicates potential for further research and provides valuable insight on how to design CA reservoir systems.
Tasks	Speech Recognition
Published	2017-03-08
URL	http://arxiv.org/abs/1703.02806v1
PDF	http://arxiv.org/pdf/1703.02806v1.pdf
PWC	https://paperswithcode.com/paper/deep-reservoir-computing-using-cellular
Repo
Framework

Pruned non-local means


Title	Pruned non-local means
Authors	Sanjay Ghosh, Amit K. Mandal, Kunal N. Chaudhury
Abstract	In Non-Local Means (NLM), each pixel is denoised by performing a weighted averaging of its neighboring pixels, where the weights are computed using image patches. We demonstrate that the denoising performance of NLM can be improved by pruning the neighboring pixels, namely, by rejecting neighboring pixels whose weights are below a certain threshold $\lambda$. While pruning can potentially reduce pixel averaging in uniform-intensity regions, we demonstrate that there is generally an overall improvement in the denoising performance. In particular, the improvement comes from pixels situated close to edges and corners. The success of the proposed method strongly depends on the choice of the global threshold $\lambda$, which in turn depends on the noise level and the image characteristics. We show how Stein’s unbiased estimator of the mean-squared error can be used to optimally tune $\lambda$, at a marginal computational overhead. We present some representative denoising results to demonstrate the superior performance of the proposed method over NLM and its variants.
Tasks	Denoising
Published	2017-01-28
URL	http://arxiv.org/abs/1701.08280v2
PDF	http://arxiv.org/pdf/1701.08280v2.pdf
PWC	https://paperswithcode.com/paper/pruned-non-local-means
Repo
Framework

Convergence of Langevin MCMC in KL-divergence


Title	Convergence of Langevin MCMC in KL-divergence
Authors	Xiang Cheng, Peter Bartlett
Abstract	Langevin diffusion is a commonly used tool for sampling from a given distribution. In this work, we establish that when the target density $p^$ is such that $\log p^$ is $L$ smooth and $m$ strongly convex, discrete Langevin diffusion produces a distribution $p$ with $KL(pp^*)\leq \epsilon$ in $\tilde{O}(\frac{d}{\epsilon})$ steps, where $d$ is the dimension of the sample space. We also study the convergence rate when the strong-convexity assumption is absent. By considering the Langevin diffusion as a gradient flow in the space of probability distributions, we obtain an elegant analysis that applies to the stronger property of convergence in KL-divergence and gives a conceptually simpler proof of the best-known convergence results in weaker metrics.
Tasks
Published	2017-05-25
URL	http://arxiv.org/abs/1705.09048v2
PDF	http://arxiv.org/pdf/1705.09048v2.pdf
PWC	https://paperswithcode.com/paper/convergence-of-langevin-mcmc-in-kl-divergence
Repo
Framework

Situationally Aware Options


Title	Situationally Aware Options
Authors	Daniel J. Mankowitz, Aviv Tamar, Shie Mannor
Abstract	Hierarchical abstractions, also known as options – a type of temporally extended action (Sutton et. al. 1999) that enables a reinforcement learning agent to plan at a higher level, abstracting away from the lower-level details. In this work, we learn reusable options whose parameters can vary, encouraging different behaviors, based on the current situation. In principle, these behaviors can include vigor, defence or even risk-averseness. These are some examples of what we refer to in the broader context as Situational Awareness (SA). We incorporate SA, in the form of vigor, into hierarchical RL by defining and learning situationally aware options in a Probabilistic Goal Semi-Markov Decision Process (PG-SMDP). This is achieved using our Situationally Aware oPtions (SAP) policy gradient algorithm which comes with a theoretical convergence guarantee. We learn reusable options in different scenarios in a RoboCup soccer domain (i.e., winning/losing). These options learn to execute with different levels of vigor resulting in human-like behaviours such as `time-wasting’ in the winning scenario. We show the potential of the agent to exit bad local optima using reusable options in RoboCup. Finally, using SAP, the agent mitigates feature-based model misspecification in a Bottomless Pit of Death domain. \|
Tasks
Published	2017-11-20
URL	http://arxiv.org/abs/1711.07832v1
PDF	http://arxiv.org/pdf/1711.07832v1.pdf
PWC	https://paperswithcode.com/paper/situationally-aware-options
Repo
Framework

Labelled network subgraphs reveal stylistic subtleties in written texts


Title	Labelled network subgraphs reveal stylistic subtleties in written texts
Authors	Vanessa Q. Marinho, Graeme Hirst, Diego R. Amancio
Abstract	The vast amount of data and increase of computational capacity have allowed the analysis of texts from several perspectives, including the representation of texts as complex networks. Nodes of the network represent the words, and edges represent some relationship, usually word co-occurrence. Even though networked representations have been applied to study some tasks, such approaches are not usually combined with traditional models relying upon statistical paradigms. Because networked models are able to grasp textual patterns, we devised a hybrid classifier, called labelled subgraphs, that combines the frequency of common words with small structures found in the topology of the network, known as motifs. Our approach is illustrated in two contexts, authorship attribution and translationese identification. In the former, a set of novels written by different authors is analyzed. To identify translationese, texts from the Canadian Hansard and the European parliament were classified as to original and translated instances. Our results suggest that labelled subgraphs are able to represent texts and it should be further explored in other tasks, such as the analysis of text complexity, language proficiency, and machine translation.
Tasks	Machine Translation
Published	2017-05-01
URL	http://arxiv.org/abs/1705.00545v3
PDF	http://arxiv.org/pdf/1705.00545v3.pdf
PWC	https://paperswithcode.com/paper/labelled-network-subgraphs-reveal-stylistic
Repo
Framework

Exploring and Exploiting Diversity for Image Segmentation


Title	Exploring and Exploiting Diversity for Image Segmentation
Authors	Payman Yadollahpour
Abstract	Semantic image segmentation is an important computer vision task that is difficult because it consists of both recognition and segmentation. The task is often cast as a structured output problem on an exponentially large output-space, which is typically modeled by a discrete probabilistic model. The best segmentation is found by inferring the Maximum a-Posteriori (MAP) solution over the output distribution defined by the model. Due to limitations in optimization, the model cannot be arbitrarily complex. This leads to a trade-off: devise a more accurate model that incorporates rich high-order interactions between image elements at the cost of inaccurate and possibly intractable optimization OR leverage a tractable model which produces less accurate MAP solutions but may contain high quality solutions as other modes of its output distribution. This thesis investigates the latter and presents a two stage approach to semantic segmentation. In the first stage a tractable segmentation model outputs a set of high probability segmentations from the underlying distribution that are not just minor perturbations of each other. Critically the output of this stage is a diverse set of plausible solutions and not just a single one. In the second stage, a discriminatively trained re-ranking model selects the best segmentation from this set. The re-ranking stage can use much more complex features than what could be tractably used in the segmentation model, allowing a better exploration of the solution space than simply returning the MAP solution. The formulation is agnostic to the underlying segmentation model (e.g. CRF, CNN, etc.) and optimization algorithm, which makes it applicable to a wide range of models and inference methods. Evaluation of the approach on a number of semantic image segmentation benchmark datasets highlight its superiority over inferring the MAP solution.
Tasks	Semantic Segmentation
Published	2017-09-05
URL	http://arxiv.org/abs/1709.01625v1
PDF	http://arxiv.org/pdf/1709.01625v1.pdf
PWC	https://paperswithcode.com/paper/exploring-and-exploiting-diversity-for-image
Repo
Framework

Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention


Title	Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention
Authors	Gedas Bertasius, Jianbo Shi
Abstract	We present a first-person method for cooperative basketball intention prediction: we predict with whom the camera wearer will cooperate in the near future from unlabeled first-person images. This is a challenging task that requires inferring the camera wearer’s visual attention, and decoding the social cues of other players. Our key observation is that a first-person view provides strong cues to infer the camera wearer’s momentary visual attention, and his/her intentions. We exploit this observation by proposing a new cross-model EgoSupervision learning scheme that allows us to predict with whom the camera wearer will cooperate in the near future, without using manually labeled intention labels. Our cross-model EgoSupervision operates by transforming the outputs of a pretrained pose-estimation network, into pseudo ground truth labels, which are then used as a supervisory signal to train a new network for a cooperative intention task. We evaluate our method, and show that it achieves similar or even better accuracy than the fully supervised methods do.
Tasks	Pose Estimation
Published	2017-09-05
URL	http://arxiv.org/abs/1709.01630v1
PDF	http://arxiv.org/pdf/1709.01630v1.pdf
PWC	https://paperswithcode.com/paper/using-cross-model-egosupervision-to-learn
Repo
Framework

Structured Sparse Ternary Weight Coding of Deep Neural Networks for Efficient Hardware Implementations


Title	Structured Sparse Ternary Weight Coding of Deep Neural Networks for Efficient Hardware Implementations
Authors	Yoonho Boo, Wonyong Sung
Abstract	Deep neural networks (DNNs) usually demand a large amount of operations for real-time inference. Especially, fully-connected layers contain a large number of weights, thus they usually need many off-chip memory accesses for inference. We propose a weight compression method for deep neural networks, which allows values of +1 or -1 only at predetermined positions of the weights so that decoding using a table can be conducted easily. For example, the structured sparse (8,2) coding allows at most two non-zero values among eight weights. This method not only enables multiplication-free DNN implementations but also compresses the weight storage by up to x32 compared to floating-point networks. Weight distribution normalization and gradual pruning techniques are applied to mitigate the performance degradation. The experiments are conducted with fully-connected deep neural networks and convolutional neural networks.
Tasks
Published	2017-07-01
URL	http://arxiv.org/abs/1707.03684v1
PDF	http://arxiv.org/pdf/1707.03684v1.pdf
PWC	https://paperswithcode.com/paper/structured-sparse-ternary-weight-coding-of
Repo
Framework

Corpus-level Fine-grained Entity Typing


Title	Corpus-level Fine-grained Entity Typing
Authors	Yadollah Yaghoobzadeh, Heike Adel, Hinrich Schütze
Abstract	This paper addresses the problem of corpus-level entity typing, i.e., inferring from a large corpus that an entity is a member of a class such as “food” or “artist”. The application of entity typing we are interested in is knowledge base completion, specifically, to learn which classes an entity is a member of. We propose FIGMENT to tackle this problem. FIGMENT is embedding- based and combines (i) a global model that scores based on aggregated contextual information of an entity and (ii) a context model that first scores the individual occurrences of an entity and then aggregates the scores. Each of the two proposed models has some specific properties. For the global model, learning high quality entity representations is crucial because it is the only source used for the predictions. Therefore, we introduce representations using name and contexts of entities on the three levels of entity, word, and character. We show each has complementary information and a multi-level representation is the best. For the context model, we need to use distant supervision since the context-level labels are not available for entities. Distant supervised labels are noisy and this harms the performance of models. Therefore, we introduce and apply new algorithms for noise mitigation using multi-instance learning. We show the effectiveness of our models in a large entity typing dataset, built from Freebase.
Tasks	Entity Typing, Knowledge Base Completion
Published	2017-08-07
URL	http://arxiv.org/abs/1708.02275v2
PDF	http://arxiv.org/pdf/1708.02275v2.pdf
PWC	https://paperswithcode.com/paper/corpus-level-fine-grained-entity-typing
Repo
Framework

Deep Embedding Convolutional Neural Network for Synthesizing CT Image from T1-Weighted MR Image


Title	Deep Embedding Convolutional Neural Network for Synthesizing CT Image from T1-Weighted MR Image
Authors	Lei Xiang, Qian Wang, Xiyao Jin, Dong Nie, Yu Qiao, Dinggang Shen
Abstract	Recently, more and more attention is drawn to the field of medical image synthesis across modalities. Among them, the synthesis of computed tomography (CT) image from T1-weighted magnetic resonance (MR) image is of great importance, although the mapping between them is highly complex due to large gaps of appearances of the two modalities. In this work, we aim to tackle this MR-to-CT synthesis by a novel deep embedding convolutional neural network (DECNN). Specifically, we generate the feature maps from MR images, and then transform these feature maps forward through convolutional layers in the network. We can further compute a tentative CT synthesis from the midway of the flow of feature maps, and then embed this tentative CT synthesis back to the feature maps. This embedding operation results in better feature maps, which are further transformed forward in DECNN. After repeat-ing this embedding procedure for several times in the network, we can eventually synthesize a final CT image in the end of the DECNN. We have validated our proposed method on both brain and prostate datasets, by also compar-ing with the state-of-the-art methods. Experimental results suggest that our DECNN (with repeated embedding op-erations) demonstrates its superior performances, in terms of both the perceptive quality of the synthesized CT image and the run-time cost for synthesizing a CT image.
Tasks	Computed Tomography (CT), Image Generation
Published	2017-09-07
URL	http://arxiv.org/abs/1709.02073v2
PDF	http://arxiv.org/pdf/1709.02073v2.pdf
PWC	https://paperswithcode.com/paper/deep-embedding-convolutional-neural-network
Repo
Framework

A Lightweight Front-end Tool for Interactive Entity Population


Title	A Lightweight Front-end Tool for Interactive Entity Population
Authors	Hidekazu Oiwa, Yoshihiko Suhara, Jiyu Komiya, Andrei Lopatenko
Abstract	Entity population, a task of collecting entities that belong to a particular category, has attracted attention from vertical domains. There is still a high demand for creating entity dictionaries in vertical domains, which are not covered by existing knowledge bases. We develop a lightweight front-end tool for facilitating interactive entity population. We implement key components necessary for effective interactive entity population: 1) GUI-based dashboards to quickly modify an entity dictionary, and 2) entity highlighting on documents for quickly viewing the current progress. We aim to reduce user cost from beginning to end, including package installation and maintenance. The implementation enables users to use this tool on their web browsers without any additional packages — users can focus on their missions to create entity dictionaries. Moreover, an entity expansion module is implemented as external APIs. This design makes it easy to continuously improve interactive entity population pipelines. We are making our demo publicly available (http://bit.ly/luwak-demo).
Tasks
Published	2017-08-01
URL	http://arxiv.org/abs/1708.00481v1
PDF	http://arxiv.org/pdf/1708.00481v1.pdf
PWC	https://paperswithcode.com/paper/a-lightweight-front-end-tool-for-interactive
Repo
Framework