July 29, 2019

3587 words 17 mins read

Paper Group ANR 144

Efficient Algorithms for Non-convex Isotonic Regression through Submodular Optimization. 4D Multi-atlas Label Fusion using Longitudinal Images. Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability. Going Deeper with Semantics: Video Activity Interpretation using Semant …

Efficient Algorithms for Non-convex Isotonic Regression through Submodular Optimization


Title	Efficient Algorithms for Non-convex Isotonic Regression through Submodular Optimization
Authors	Francis Bach
Abstract	We consider the minimization of submodular functions subject to ordering constraints. We show that this optimization problem can be cast as a convex optimization problem on a space of uni-dimensional measures, with ordering constraints corresponding to first-order stochastic dominance. We propose new discretization schemes that lead to simple and efficient algorithms based on zero-th, first, or higher order oracles; these algorithms also lead to improvements without isotonic constraints. Finally, our experiments show that non-convex loss functions can be much more robust to outliers for isotonic regression, while still leading to an efficient optimization problem.
Tasks
Published	2017-07-28
URL	http://arxiv.org/abs/1707.09157v1
PDF	http://arxiv.org/pdf/1707.09157v1.pdf
PWC	https://paperswithcode.com/paper/efficient-algorithms-for-non-convex-isotonic
Repo
Framework

4D Multi-atlas Label Fusion using Longitudinal Images


Title	4D Multi-atlas Label Fusion using Longitudinal Images
Authors	Yuankai Huo, Susan M. Resnick, Bennett A. Landman
Abstract	Longitudinal reproducibility is an essential concern in automated medical image segmentation, yet has proven to be an elusive objective as manual brain structure tracings have shown more than 10% variability. To improve reproducibility, lon-gitudinal segmentation (4D) approaches have been investigated to reconcile tem-poral variations with traditional 3D approaches. In the past decade, multi-atlas la-bel fusion has become a state-of-the-art segmentation technique for 3D image and many efforts have been made to adapt it to a 4D longitudinal fashion. However, the previous methods were either limited by using application specified energy function (e.g., surface fusion and multi model fusion) or only considered tem-poral smoothness on two consecutive time points (t and t+1) under sparsity as-sumption. Therefore, a 4D multi-atlas label fusion theory for general label fusion purpose and simultaneously considering temporal consistency on all time points is appealing. Herein, we propose a novel longitudinal label fusion algorithm, called 4D joint label fusion (4DJLF), to incorporate the temporal consistency modeling via non-local patch-intensity covariance models. The advantages of 4DJLF include: (1) 4DJLF is under the general label fusion framework by simul-taneously incorporating the spatial and temporal covariance on all longitudinal time points. (2) The proposed algorithm is a longitudinal generalization of a lead-ing joint label fusion method (JLF) that has proven adaptable to a wide variety of applications. (3) The spatial temporal consistency of atlases is modeled in a prob-abilistic model inspired from both voting based and statistical fusion. The pro-posed approach improves the consistency of the longitudinal segmentation while retaining sensitivity compared with original JLF approach using the same set of atlases. The method is available online in open-source.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2017-08-29
URL	http://arxiv.org/abs/1708.08825v1
PDF	http://arxiv.org/pdf/1708.08825v1.pdf
PWC	https://paperswithcode.com/paper/4d-multi-atlas-label-fusion-using
Repo
Framework

Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability


Title	Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability
Authors	Alberto Delmas, Sayeh Sharify, Patrick Judd, Andreas Moshovos
Abstract	Tartan (TRT), a hardware accelerator for inference with Deep Neural Networks (DNNs), is presented and evaluated on Convolutional Neural Networks. TRT exploits the variable per layer precision requirements of DNNs to deliver execution time that is proportional to the precision p in bits used per layer for convolutional and fully-connected layers. Prior art has demonstrated an accelerator with the same execution performance only for convolutional layers. Experiments on image classification CNNs show that on average across all networks studied, TRT outperforms a state-of-the-art bit-parallel accelerator by 1:90x without any loss in accuracy while it is 1:17x more energy efficient. TRT requires no network retraining while it enables trading off accuracy for additional improvements in execution performance and energy efficiency. For example, if a 1% relative loss in accuracy is acceptable, TRT is on average 2:04x faster and 1:25x more energy efficient than a conventional bit-parallel accelerator. A Tartan configuration that processes 2-bits at time, requires less area than the 1-bit configuration, improves efficiency to 1:24x over the bit-parallel baseline while being 73% faster for convolutional layers and 60% faster for fully-connected layers is also presented.
Tasks	Image Classification
Published	2017-07-27
URL	http://arxiv.org/abs/1707.09068v1
PDF	http://arxiv.org/pdf/1707.09068v1.pdf
PWC	https://paperswithcode.com/paper/tartan-accelerating-fully-connected-and
Repo
Framework

Going Deeper with Semantics: Video Activity Interpretation using Semantic Contextualization


Title	Going Deeper with Semantics: Video Activity Interpretation using Semantic Contextualization
Authors	Sathyanarayanan N. Aakur, Fillipe DM de Souza, Sudeep Sarkar
Abstract	A deeper understanding of video activities extends beyond recognition of underlying concepts such as actions and objects: constructing deep semantic representations requires reasoning about the semantic relationships among these concepts, often beyond what is directly observed in the data. To this end, we propose an energy minimization framework that leverages large-scale commonsense knowledge bases, such as ConceptNet, to provide contextual cues to establish semantic relationships among entities directly hypothesized from video signal. We mathematically express this using the language of Grenander’s canonical pattern generator theory. We show that the use of prior encoded commonsense knowledge alleviate the need for large annotated training datasets and help tackle imbalance in training through prior knowledge. Using three different publicly available datasets - Charades, Microsoft Visual Description Corpus and Breakfast Actions datasets, we show that the proposed model can generate video interpretations whose quality is better than those reported by state-of-the-art approaches, which have substantial training needs. Through extensive experiments, we show that the use of commonsense knowledge from ConceptNet allows the proposed approach to handle various challenges such as training data imbalance, weak features, and complex semantic relationships and visual scenes.
Tasks
Published	2017-08-11
URL	http://arxiv.org/abs/1708.03725v3
PDF	http://arxiv.org/pdf/1708.03725v3.pdf
PWC	https://paperswithcode.com/paper/going-deeper-with-semantics-video-activity
Repo
Framework

Learning a Variational Network for Reconstruction of Accelerated MRI Data


Title	Learning a Variational Network for Reconstruction of Accelerated MRI Data
Authors	Kerstin Hammernik, Teresa Klatzer, Erich Kobler, Michael P Recht, Daniel K Sodickson, Thomas Pock, Florian Knoll
Abstract	Purpose: To allow fast and high-quality reconstruction of clinical accelerated multi-coil MR data by learning a variational network that combines the mathematical structure of variational models with deep learning. Theory and Methods: Generalized compressed sensing reconstruction formulated as a variational model is embedded in an unrolled gradient descent scheme. All parameters of this formulation, including the prior model defined by filter kernels and activation functions as well as the data term weights, are learned during an offline training procedure. The learned model can then be applied online to previously unseen data. Results: The variational network approach is evaluated on a clinical knee imaging protocol. The variational network reconstructions outperform standard reconstruction algorithms in terms of image quality and residual artifacts for all tested acceleration factors and sampling patterns. Conclusion: Variational network reconstructions preserve the natural appearance of MR images as well as pathologies that were not included in the training data set. Due to its high computational performance, i.e., reconstruction time of 193 ms on a single graphics card, and the omission of parameter tuning once the network is trained, this new approach to image reconstruction can easily be integrated into clinical workflow.
Tasks	Image Reconstruction
Published	2017-04-03
URL	http://arxiv.org/abs/1704.00447v1
PDF	http://arxiv.org/pdf/1704.00447v1.pdf
PWC	https://paperswithcode.com/paper/learning-a-variational-network-for
Repo
Framework

When Neurons Fail


Title	When Neurons Fail
Authors	El Mahdi El Mhamdi, Rachid Guerraoui
Abstract	We view a neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact that neural activation functions are Lipschitz-continuous. Our bound is on a quantity, we call the \textit{Forward Error Propagation}, capturing how much error is propagated by a neural network when a given number of components is failing, computing this quantity only requires looking at the topology of the network, while experimentally assessing the robustness of a network requires the costly experiment of looking at all the possible inputs and testing all the possible configurations of the network corresponding to different failure situations, facing a discouraging combinatorial explosion. We distinguish the case of neurons that can fail and stop their activity (crashed neurons) from the case of neurons that can fail by transmitting arbitrary values (Byzantine neurons). Interestingly, as we show in the paper, our bound can easily be extended to the case where synapses can fail. We show how our bound can be leveraged to quantify the effect of memory cost reduction on the accuracy of a neural network, to estimate the amount of information any neuron needs from its preceding layer, enabling thereby a boosting scheme that prevents neurons from waiting for unnecessary signals. We finally discuss the trade-off between neural networks robustness and learning cost.
Tasks
Published	2017-06-27
URL	http://arxiv.org/abs/1706.08884v1
PDF	http://arxiv.org/pdf/1706.08884v1.pdf
PWC	https://paperswithcode.com/paper/when-neurons-fail
Repo
Framework

Resolution and Relevance Trade-offs in Deep Learning


Title	Resolution and Relevance Trade-offs in Deep Learning
Authors	Juyong Song, Matteo Marsili, Junghyo Jo
Abstract	Deep learning has been successfully applied to various tasks, but its underlying mechanism remains unclear. Neural networks associate similar inputs in the visible layer to the same state of hidden variables in deep layers. The fraction of inputs that are associated to the same state is a natural measure of similarity and is simply related to the cost in bits required to represent these inputs. The degeneracy of states with the same information cost provides instead a natural measure of noise and is simply related the entropy of the frequency of states, that we call relevance. Representations with minimal noise, at a given level of similarity (resolution), are those that maximise the relevance. A signature of such efficient representations is that frequency distributions follow power laws. We show, in extensive numerical experiments, that deep neural networks extract a hierarchy of efficient representations from data, because they i) achieve low levels of noise (i.e. high relevance) and ii) exhibit power law distributions. We also find that the layer that is most efficient to reliably generate patterns of training data is the one for which relevance and resolution are traded at the same price, which implies that frequency distribution follows Zipf’s law.
Tasks
Published	2017-10-31
URL	http://arxiv.org/abs/1710.11324v2
PDF	http://arxiv.org/pdf/1710.11324v2.pdf
PWC	https://paperswithcode.com/paper/resolution-and-relevance-trade-offs-in-deep
Repo
Framework

An LSTM-Based Dynamic Customer Model for Fashion Recommendation


Title	An LSTM-Based Dynamic Customer Model for Fashion Recommendation
Authors	Sebastian Heinz, Christian Bracher, Roland Vollgraf
Abstract	Online fashion sales present a challenging use case for personalized recommendation: Stores offer a huge variety of items in multiple sizes. Small stocks, high return rates, seasonality, and changing trends cause continuous turnover of articles for sale on all time scales. Customers tend to shop rarely, but often buy multiple items at once. We report on backtest experiments with sales data of 100k frequent shoppers at Zalando, Europe’s leading online fashion platform. To model changing customer and store environments, our recommendation method employs a pair of neural networks: To overcome the cold start problem, a feedforward network generates article embeddings in “fashion space,” which serve as input to a recurrent neural network that predicts a style vector in this space for each client, based on their past purchase sequence. We compare our results with a static collaborative filtering approach, and a popularity ranking baseline.
Tasks
Published	2017-08-24
URL	http://arxiv.org/abs/1708.07347v1
PDF	http://arxiv.org/pdf/1708.07347v1.pdf
PWC	https://paperswithcode.com/paper/an-lstm-based-dynamic-customer-model-for
Repo
Framework

Discovering More Precise Process Models from Event Logs by Filtering Out Chaotic Activities


Title	Discovering More Precise Process Models from Event Logs by Filtering Out Chaotic Activities
Authors	Niek Tax, Natalia Sidorova, Wil M. P. van der Aalst
Abstract	Process Discovery is concerned with the automatic generation of a process model that describes a business process from execution data of that business process. Real life event logs can contain chaotic activities. These activities are independent of the state of the process and can, therefore, happen at rather arbitrary points in time. We show that the presence of such chaotic activities in an event log heavily impacts the quality of the process models that can be discovered with process discovery techniques. The current modus operandi for filtering activities from event logs is to simply filter out infrequent activities. We show that frequency-based filtering of activities does not solve the problems that are caused by chaotic activities. Moreover, we propose a novel technique to filter out chaotic activities from event logs. We evaluate this technique on a collection of seventeen real-life event logs that originate from both the business process management domain and the smart home environment domain. As demonstrated, the developed activity filtering methods enable the discovery of process models that are more behaviorally specific compared to process models that are discovered using standard frequency-based filtering.
Tasks
Published	2017-11-03
URL	http://arxiv.org/abs/1711.01287v1
PDF	http://arxiv.org/pdf/1711.01287v1.pdf
PWC	https://paperswithcode.com/paper/discovering-more-precise-process-models-from
Repo
Framework

Truth and Regret in Online Scheduling


Title	Truth and Regret in Online Scheduling
Authors	Shuchi Chawla, Nikhil Devanur, Janardhan Kulkarni, Rad Niazadeh
Abstract	We consider a scheduling problem where a cloud service provider has multiple units of a resource available over time. Selfish clients submit jobs, each with an arrival time, deadline, length, and value. The service provider’s goal is to implement a truthful online mechanism for scheduling jobs so as to maximize the social welfare of the schedule. Recent work shows that under a stochastic assumption on job arrivals, there is a single-parameter family of mechanisms that achieves near-optimal social welfare. We show that given any such family of near-optimal online mechanisms, there exists an online mechanism that in the worst case performs nearly as well as the best of the given mechanisms. Our mechanism is truthful whenever the mechanisms in the given family are truthful and prompt, and achieves optimal (within constant factors) regret. We model the problem of competing against a family of online scheduling mechanisms as one of learning from expert advice. A primary challenge is that any scheduling decisions we make affect not only the payoff at the current step, but also the resource availability and payoffs in future steps. Furthermore, switching from one algorithm (a.k.a. expert) to another in an online fashion is challenging both because it requires synchronization with the state of the latter algorithm as well as because it affects the incentive structure of the algorithms. We further show how to adapt our algorithm to a non-clairvoyant setting where job lengths are unknown until jobs are run to completion. Once again, in this setting, we obtain truthfulness along with asymptotically optimal regret (within poly-logarithmic factors).
Tasks
Published	2017-03-01
URL	http://arxiv.org/abs/1703.00484v1
PDF	http://arxiv.org/pdf/1703.00484v1.pdf
PWC	https://paperswithcode.com/paper/truth-and-regret-in-online-scheduling
Repo
Framework

A Grassmannian Approach to Zero-Shot Learning for Network Intrusion Detection


Title	A Grassmannian Approach to Zero-Shot Learning for Network Intrusion Detection
Authors	Jorge Rivero, Bernardete Ribeiro, Ning Chen, Fátima Silva Leite
Abstract	One of the main problems in Network Intrusion Detection comes from constant rise of new attacks, so that not enough labeled examples are available for the new classes of attacks. Traditional Machine Learning approaches hardly address such problem. This can be overcome with Zero-Shot Learning, a new approach in the field of Computer Vision, which can be described in two stages: the Attribute Learning and the Inference Stage. The goal of this paper is to propose a new Inference Stage algorithm for Network Intrusion Detection. In order to attain this objective, we firstly put forward an experimental setup for the evaluation of the Zero-Shot Learning in Network Intrusion Detection related tasks. Secondly, a decision tree based algorithm is applied to extract rules for generating the attributes in the AL stage. Finally, using a representation of a Zero-Shot Class as a point in the Grassmann manifold, an explicit formula for the shortest distance between points in that manifold can be used to compute the geodesic distance between the Zero-Shot Classes which represent the new attacks and the Known Classes corresponding to the attack categories. The experimental results in the datasets KDD Cup 99 and NSL-KDD show that our approach with Zero-Shot Learning successfully addresses the Network Intrusion Detection problem.
Tasks	Intrusion Detection, Network Intrusion Detection, Zero-Shot Learning
Published	2017-09-23
URL	http://arxiv.org/abs/1709.07984v1
PDF	http://arxiv.org/pdf/1709.07984v1.pdf
PWC	https://paperswithcode.com/paper/a-grassmannian-approach-to-zero-shot-learning
Repo
Framework

An Efficient Evolutionary Based Method For Image Segmentation


Title	An Efficient Evolutionary Based Method For Image Segmentation
Authors	Roohollah Aslanzadeh, Kazem Qazanfari, Mohammad Rahmati
Abstract	The goal of this paper is to present a new efficient image segmentation method based on evolutionary computation which is a model inspired from human behavior. Based on this model, a four layer process for image segmentation is proposed using the split/merge approach. In the first layer, an image is split into numerous regions using the watershed algorithm. In the second layer, a co-evolutionary process is applied to form centers of finals segments by merging similar primary regions. In the third layer, a meta-heuristic process uses two operators to connect the residual regions to their corresponding determined centers. In the final layer, an evolutionary algorithm is used to combine the resulted similar and neighbor regions. Different layers of the algorithm are totally independent, therefore for certain applications a specific layer can be changed without constraint of changing other layers. Some properties of this algorithm like the flexibility of its method, the ability to use different feature vectors for segmentation (grayscale, color, texture, etc), the ability to control uniformity and the number of final segments using free parameters and also maintaining small regions, makes it possible to apply the algorithm to different applications. Moreover, the independence of each region from other regions in the second layer, and the independence of centers in the third layer, makes parallel implementation possible. As a result the algorithm speed will increase. The presented algorithm was tested on a standard dataset (BSDS 300) of images, and the region boundaries were compared with different people segmentation contours. Results show the efficiency of the algorithm and its improvement to similar methods. As an instance, in 70% of tested images, results are better than ACT algorithm, besides in 100% of tested images, we had better results in comparison with VSP algorithm.
Tasks	Semantic Segmentation
Published	2017-09-13
URL	http://arxiv.org/abs/1709.04393v2
PDF	http://arxiv.org/pdf/1709.04393v2.pdf
PWC	https://paperswithcode.com/paper/an-efficient-evolutionary-based-method-for
Repo
Framework

Model Selection for Anomaly Detection


Title	Model Selection for Anomaly Detection
Authors	Evgeny Burnaev, Pavel Erofeev, Dmitry Smolyakov
Abstract	Anomaly detection based on one-class classification algorithms is broadly used in many applied domains like image processing (e.g. detection of whether a patient is “cancerous” or “healthy” from mammography image), network intrusion detection, etc. Performance of an anomaly detection algorithm crucially depends on a kernel, used to measure similarity in a feature space. The standard approaches (e.g. cross-validation) for kernel selection, used in two-class classification problems, can not be used directly due to the specific nature of a data (absence of a second, abnormal, class data). In this paper we generalize several kernel selection methods from binary-class case to the case of one-class classification and perform extensive comparison of these approaches using both synthetic and real-world data.
Tasks	Anomaly Detection, Intrusion Detection, Model Selection, Network Intrusion Detection
Published	2017-07-12
URL	http://arxiv.org/abs/1707.03909v1
PDF	http://arxiv.org/pdf/1707.03909v1.pdf
PWC	https://paperswithcode.com/paper/model-selection-for-anomaly-detection
Repo
Framework

Outline Colorization through Tandem Adversarial Networks


Title	Outline Colorization through Tandem Adversarial Networks
Authors	Kevin Frans
Abstract	When creating digital art, coloring and shading are often time consuming tasks that follow the same general patterns. A solution to automatically colorize raw line art would have many practical applications. We propose a setup utilizing two networks in tandem: a color prediction network based only on outlines, and a shading network conditioned on both outlines and a color scheme. We present processing methods to limit information passed in the color scheme, improving generalization. Finally, we demonstrate natural-looking results when colorizing outlines from scratch, as well as from a messy, user-defined color scheme.
Tasks	Colorization
Published	2017-04-28
URL	http://arxiv.org/abs/1704.08834v1
PDF	http://arxiv.org/pdf/1704.08834v1.pdf
PWC	https://paperswithcode.com/paper/outline-colorization-through-tandem
Repo
Framework

Stacked Denoising Autoencoders and Transfer Learning for Immunogold Particles Detection and Recognition


Title	Stacked Denoising Autoencoders and Transfer Learning for Immunogold Particles Detection and Recognition
Authors	Ricardo Gamelas Sousa, Jorge M. Santos, Luís M. Silva, Luís A. Alexandre, Tiago Esteves, Sara Rocha, Paulo Monjardino, Joaquim Marques de Sá, Francisco Figueiredo, Pedro Quelhas
Abstract	In this paper we present a system for the detection of immunogold particles and a Transfer Learning (TL) framework for the recognition of these immunogold particles. Immunogold particles are part of a high-magnification method for the selective localization of biological molecules at the subcellular level only visible through Electron Microscopy. The number of immunogold particles in the cell walls allows the assessment of the differences in their compositions providing a tool to analise the quality of different plants. For its quantization one requires a laborious manual labeling (or annotation) of images containing hundreds of particles. The system that is proposed in this paper can leverage significantly the burden of this manual task. For particle detection we use a LoG filter coupled with a SDA. In order to improve the recognition, we also study the applicability of TL settings for immunogold recognition. TL reuses the learning model of a source problem on other datasets (target problems) containing particles of different sizes. The proposed system was developed to solve a particular problem on maize cells, namely to determine the composition of cell wall ingrowths in endosperm transfer cells. This novel dataset as well as the code for reproducing our experiments is made publicly available. We determined that the LoG detector alone attained more than 84% of accuracy with the F-measure. Developing immunogold recognition with TL also provided superior performance when compared with the baseline models augmenting the accuracy rates by 10%.
Tasks	Denoising, Quantization, Transfer Learning
Published	2017-12-07
URL	http://arxiv.org/abs/1712.02824v1
PDF	http://arxiv.org/pdf/1712.02824v1.pdf
PWC	https://paperswithcode.com/paper/stacked-denoising-autoencoders-and-transfer
Repo
Framework