October 19, 2019

3237 words 16 mins read

Paper Group ANR 171

Stability of Scattering Decoder For Nonlinear Diffractive Imaging. Remote Detection of Idling Cars Using Infrared Imaging and Deep Networks. Modified Apriori Graph Algorithm for Frequent Pattern Mining. Learning Neuron Non-Linearities with Kernel-Based Deep Neural Networks. Combinatorial and Structural Results for gamma-Psi-dimensions. Deep Fluids: …

Stability of Scattering Decoder For Nonlinear Diffractive Imaging


Title	Stability of Scattering Decoder For Nonlinear Diffractive Imaging
Authors	Yu Sun, Ulugbek S. Kamilov
Abstract	The problem of image reconstruction under multiple light scattering is usually formulated as a regularized non-convex optimization. A deep learning architecture, Scattering Decoder (ScaDec), was recently proposed to solve this problem in a purely data-driven fashion. The proposed method was shown to substantially outperform optimization-based baselines and achieve state-of-the-art results. In this paper, we thoroughly test the robustness of ScaDec to different permittivity contrasts, number of transmissions, and input signal-to-noise ratios. The results on high-fidelity simulated datasets show that the performance of ScaDec is stable in different settings.
Tasks	Image Reconstruction
Published	2018-06-20
URL	http://arxiv.org/abs/1806.08015v4
PDF	http://arxiv.org/pdf/1806.08015v4.pdf
PWC	https://paperswithcode.com/paper/stability-of-scattering-decoder-for-nonlinear
Repo
Framework

Remote Detection of Idling Cars Using Infrared Imaging and Deep Networks


Title	Remote Detection of Idling Cars Using Infrared Imaging and Deep Networks
Authors	Muhammet Bastan, Kim-Hui Yap, Lap-Pui Chau
Abstract	Idling vehicles waste energy and pollute the environment through exhaust emission. In some countries, idling a vehicle for more than a predefined duration is prohibited and automatic idling vehicle detection is desirable for law enforcement. We propose the first automatic system to detect idling cars, using infrared (IR) imaging and deep networks. We rely on the differences in spatio-temporal heat signatures of idling and stopped cars and monitor the car temperature with a long-wavelength IR camera. We formulate the idling car detection problem as spatio-temporal event detection in IR image sequences and employ deep networks for spatio-temporal modeling. We collected the first IR image sequence dataset for idling car detection. First, we detect the cars in each IR image using a convolutional neural network, which is pre-trained on regular RGB images and fine-tuned on IR images for higher accuracy. Then, we track the detected cars over time to identify the cars that are parked. Finally, we use the 3D spatio-temporal IR image volume of each parked car as input to convolutional and recurrent networks to classify them as idling or not. We carried out an extensive empirical evaluation of temporal and spatio-temporal modeling approaches with various convolutional and recurrent architectures. We present promising experimental results on our IR image sequence dataset.
Tasks
Published	2018-04-28
URL	http://arxiv.org/abs/1804.10805v1
PDF	http://arxiv.org/pdf/1804.10805v1.pdf
PWC	https://paperswithcode.com/paper/remote-detection-of-idling-cars-using
Repo
Framework

Modified Apriori Graph Algorithm for Frequent Pattern Mining


Title	Modified Apriori Graph Algorithm for Frequent Pattern Mining
Authors	Pritish Yuvraj, Suneetha K. R
Abstract	Web Usage Mining is an application of Data Mining Techniques to discover interesting usage patterns from web data in order to understand and better serve the needs of web-based applications. The paper proposes an algorithm for finding these usage patterns using a modified version of Apriori Algorithm called Apriori-Graph. These rules will help service providers to predict, which web pages, the user is likely to visit next. This will optimize the website in terms of efficiency, bandwidth and will have positive economic benefits for them. The proposed Apriori Graph Algorithm O((V)(E)) works faster compared to the existing Apriori Algorithm and is well suitable for real-time application.
Tasks
Published	2018-04-27
URL	http://arxiv.org/abs/1804.10711v1
PDF	http://arxiv.org/pdf/1804.10711v1.pdf
PWC	https://paperswithcode.com/paper/modified-apriori-graph-algorithm-for-frequent
Repo
Framework

Learning Neuron Non-Linearities with Kernel-Based Deep Neural Networks


Title	Learning Neuron Non-Linearities with Kernel-Based Deep Neural Networks
Authors	Giuseppe Marra, Dario Zanca, Alessandro Betti, Marco Gori
Abstract	The effectiveness of deep neural architectures has been widely supported in terms of both experimental and foundational principles. There is also clear evidence that the activation function (e.g. the rectifier and the LSTM units) plays a crucial role in the complexity of learning. Based on this remark, this paper discusses an optimal selection of the neuron non-linearity in a functional framework that is inspired from classic regularization arguments. It is shown that the best activation function is represented by a kernel expansion in the training set, that can be effectively approximated over an opportune set of points modeling 1-D clusters. The idea can be naturally extended to recurrent networks, where the expressiveness of kernel-based activation functions turns out to be a crucial ingredient to capture long-term dependencies. We give experimental evidence of this property by a set of challenging experiments, where we compare the results with neural architectures based on state of the art LSTM cells.
Tasks
Published	2018-07-17
URL	http://arxiv.org/abs/1807.06302v2
PDF	http://arxiv.org/pdf/1807.06302v2.pdf
PWC	https://paperswithcode.com/paper/learning-neuron-non-linearities-with-kernel
Repo
Framework

Combinatorial and Structural Results for gamma-Psi-dimensions


Title	Combinatorial and Structural Results for gamma-Psi-dimensions
Authors	Yann Guermeur
Abstract	This article deals with the generalization performance of margin multi-category classifiers, when minimal learnability hypotheses are made. In that context, the derivation of a guaranteed risk is based on the handling of capacity measures belonging to three main families: Rademacher/Gaussian complexities, metric entropies and scale-sensitive combinatorial dimensions. The scale-sensitive combinatorial dimensions dedicated to the classifiers of this kind are the gamma-Psi-dimensions. We introduce the combinatorial and structural results needed to involve them in the derivation of upper bounds on the metric entropies and the Rademacher complexity. Their incidence on the guaranteed risks is characterized, which establishes that in the theoretical framework of interest, performing the transition from the multi-class case to the binary one with combinatorial dimensions is a promising alternative to proceeding with covering numbers.
Tasks
Published	2018-09-19
URL	https://arxiv.org/abs/1809.07310v2
PDF	https://arxiv.org/pdf/1809.07310v2.pdf
PWC	https://paperswithcode.com/paper/combinatorial-and-structural-results-for
Repo
Framework

Deep Fluids: A Generative Network for Parameterized Fluid Simulations


Title	Deep Fluids: A Generative Network for Parameterized Fluid Simulations
Authors	Byungsoo Kim, Vinicius C. Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, Barbara Solenthaler
Abstract	This paper presents a novel generative model to synthesize fluid simulations from a set of reduced parameters. A convolutional neural network is trained on a collection of discrete, parameterizable fluid simulation velocity fields. Due to the capability of deep learning architectures to learn representative features of the data, our generative model is able to accurately approximate the training data set, while providing plausible interpolated in-betweens. The proposed generative model is optimized for fluids by a novel loss function that guarantees divergence-free velocity fields at all times. In addition, we demonstrate that we can handle complex parameterizations in reduced spaces, and advance simulations in time by integrating in the latent space with a second network. Our method models a wide variety of fluid behaviors, thus enabling applications such as fast construction of simulations, interpolation of fluids with different parameters, time re-sampling, latent space simulations, and compression of fluid simulation data. Reconstructed velocity fields are generated up to 700x faster than re-simulating the data with the underlying CPU solver, while achieving compression rates of up to 1300x.
Tasks
Published	2018-06-06
URL	http://arxiv.org/abs/1806.02071v2
PDF	http://arxiv.org/pdf/1806.02071v2.pdf
PWC	https://paperswithcode.com/paper/deep-fluids-a-generative-network-for
Repo
Framework

Efficient Structured Surrogate Loss and Regularization in Structured Prediction


Title	Efficient Structured Surrogate Loss and Regularization in Structured Prediction
Authors	Heejin Choi
Abstract	In this dissertation, we focus on several important problems in structured prediction. In structured prediction, the label has a rich intrinsic substructure, and the loss varies with respect to the predicted label and the true label pair. Structured SVM is an extension of binary SVM to adapt to such structured tasks. In the first part of the dissertation, we study the surrogate losses and its efficient methods. To minimize the empirical risk, a surrogate loss which upper bounds the loss, is used as a proxy to minimize the actual loss. Since the objective function is written in terms of the surrogate loss, the choice of the surrogate loss is important, and the performance depends on it. Another issue regarding the surrogate loss is the efficiency of the argmax label inference for the surrogate loss. Efficient inference is necessary for the optimization since it is often the most time-consuming step. We present a new class of surrogate losses named bi-criteria surrogate loss, which is a generalization of the popular surrogate losses. We first investigate an efficient method for a slack rescaling formulation as a starting point utilizing decomposability of the model. Then, we extend the algorithm to the bi-criteria surrogate loss, which is very efficient and also shows performance improvements. In the second part of the dissertation, another important issue of regularization is studied. Specifically, we investigate a problem of regularization in hierarchical classification when a structural imbalance exists in the label structure. We present a method to normalize the structure, as well as a new norm, namely shared Frobenius norm. It is suitable for hierarchical classification that adapts to the data in addition to the label structure.
Tasks	Structured Prediction
Published	2018-09-14
URL	http://arxiv.org/abs/1809.05550v1
PDF	http://arxiv.org/pdf/1809.05550v1.pdf
PWC	https://paperswithcode.com/paper/efficient-structured-surrogate-loss-and
Repo
Framework

Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videos


Title	Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videos
Authors	Bo Xiong, Suyog Dutt Jain, Kristen Grauman
Abstract	We propose an end-to-end learning framework for segmenting generic objects in both images and videos. Given a novel image or video, our approach produces a pixel-level mask for all “object-like” regions—even for object categories never seen during training. We formulate the task as a structured prediction problem of assigning an object/background label to each pixel, implemented using a deep fully convolutional network. When applied to a video, our model further incorporates a motion stream, and the network learns to combine both appearance and motion and attempts to extract all prominent objects whether they are moving or not. Beyond the core model, a second contribution of our approach is how it leverages varying strengths of training annotations. Pixel-level annotations are quite difficult to obtain, yet crucial for training a deep network approach for segmentation. Thus we propose ways to exploit weakly labeled data for learning dense foreground segmentation. For images, we show the value in mixing object category examples with image-level labels together with relatively few images with boundary-level annotations. For video, we show how to bootstrap weakly annotated videos together with the network trained for image segmentation. Through experiments on multiple challenging image and video segmentation benchmarks, our method offers consistently strong results and improves the state-of-the-art for fully automatic segmentation of generic (unseen) objects. In addition, we demonstrate how our approach benefits image retrieval and image retargeting, both of which flourish when given our high-quality foreground maps. Code, models, and videos are at:http://vision.cs.utexas.edu/projects/pixelobjectness/
Tasks	Image Retrieval, Semantic Segmentation, Structured Prediction, Video Semantic Segmentation
Published	2018-08-11
URL	http://arxiv.org/abs/1808.04702v2
PDF	http://arxiv.org/pdf/1808.04702v2.pdf
PWC	https://paperswithcode.com/paper/pixel-objectness-learning-to-segment-generic
Repo
Framework

Helix: Accelerating Human-in-the-loop Machine Learning


Title	Helix: Accelerating Human-in-the-loop Machine Learning
Authors	Doris Xin, Litian Ma, Jialin Liu, Stephen Macke, Shuchen Song, Aditya Parameswaran
Abstract	Data application developers and data scientists spend an inordinate amount of time iterating on machine learning (ML) workflows – by modifying the data pre-processing, model training, and post-processing steps – via trial-and-error to achieve the desired model performance. Existing work on accelerating machine learning focuses on speeding up one-shot execution of workflows, failing to address the incremental and dynamic nature of typical ML development. We propose Helix, a declarative machine learning system that accelerates iterative development by optimizing workflow execution end-to-end and across iterations. Helix minimizes the runtime per iteration via program analysis and intelligent reuse of previous results, which are selectively materialized – trading off the cost of materialization for potential future benefits – to speed up future iterations. Additionally, Helix offers a graphical interface to visualize workflow DAGs and compare versions to facilitate iterative development. Through two ML applications, in classification and in structured prediction, attendees will experience the succinctness of Helix programming interface and the speed and ease of iterative development using Helix. In our evaluations, Helix achieved up to an order of magnitude reduction in cumulative run time compared to state-of-the-art machine learning tools.
Tasks	Structured Prediction
Published	2018-08-03
URL	http://arxiv.org/abs/1808.01095v1
PDF	http://arxiv.org/pdf/1808.01095v1.pdf
PWC	https://paperswithcode.com/paper/helix-accelerating-human-in-the-loop-machine
Repo
Framework

Reservoir Computing based Neural Image Filters


Title	Reservoir Computing based Neural Image Filters
Authors	Samiran Ganguly, Yunfei Gu, Yunkun Xie, Mircea R. Stan, Avik W. Ghosh, Nibir K. Dhar
Abstract	Clean images are an important requirement for machine vision systems to recognize visual features correctly. However, the environment, optics, electronics of the physical imaging systems can introduce extreme distortions and noise in the acquired images. In this work, we explore the use of reservoir computing, a dynamical neural network model inspired from biological systems, in creating dynamic image filtering systems that extracts signal from noise using inverse modeling. We discuss the possibility of implementing these networks in hardware close to the sensors.
Tasks
Published	2018-09-07
URL	http://arxiv.org/abs/1809.02651v1
PDF	http://arxiv.org/pdf/1809.02651v1.pdf
PWC	https://paperswithcode.com/paper/reservoir-computing-based-neural-image
Repo
Framework

Weight Initialization without Local Minima in Deep Nonlinear Neural Networks


Title	Weight Initialization without Local Minima in Deep Nonlinear Neural Networks
Authors	Tohru Nitta
Abstract	In this paper, we propose a new weight initialization method called even initialization for wide and deep nonlinear neural networks with the ReLU activation function. We prove that no poor local minimum exists in the initial loss landscape in the wide and deep nonlinear neural network initialized by the even initialization method that we propose. Specifically, in the initial loss landscape of such a wide and deep ReLU neural network model, the following four statements hold true: 1) the loss function is non-convex and non-concave; 2) every local minimum is a global minimum; 3) every critical point that is not a global minimum is a saddle point; and 4) bad saddle points exist. We also show that the weight values initialized by the even initialization method are contained in those initialized by both of the (often used) standard initialization and He initialization methods.
Tasks
Published	2018-06-13
URL	http://arxiv.org/abs/1806.04884v1
PDF	http://arxiv.org/pdf/1806.04884v1.pdf
PWC	https://paperswithcode.com/paper/weight-initialization-without-local-minima-in
Repo
Framework

Information Constraints on Auto-Encoding Variational Bayes


Title	Information Constraints on Auto-Encoding Variational Bayes
Authors	Romain Lopez, Jeffrey Regier, Michael I. Jordan, Nir Yosef
Abstract	Parameterizing the approximate posterior of a generative model with neural networks has become a common theme in recent machine learning research. While providing appealing flexibility, this approach makes it difficult to impose or assess structural constraints such as conditional independence. We propose a framework for learning representations that relies on Auto-Encoding Variational Bayes and whose search space is constrained via kernel-based measures of independence. In particular, our method employs the $d$-variable Hilbert-Schmidt Independence Criterion (dHSIC) to enforce independence between the latent representations and arbitrary nuisance factors. We show how to apply this method to a range of problems, including the problems of learning invariant representations and the learning of interpretable representations. We also present a full-fledged application to single-cell RNA sequencing (scRNA-seq). In this setting the biological signal is mixed in complex ways with sequencing errors and sampling effects. We show that our method out-performs the state-of-the-art in this domain.
Tasks
Published	2018-05-22
URL	http://arxiv.org/abs/1805.08672v4
PDF	http://arxiv.org/pdf/1805.08672v4.pdf
PWC	https://paperswithcode.com/paper/information-constraints-on-auto-encoding
Repo
Framework

Video Synthesis from a Single Image and Motion Stroke


Title	Video Synthesis from a Single Image and Motion Stroke
Authors	Qiyang Hu, Adrian Waelchli, Tiziano Portenier, Matthias Zwicker, Paolo Favaro
Abstract	In this paper, we propose a new method to automatically generate a video sequence from a single image and a user provided motion stroke. Generating a video sequence based on a single input image has many applications in visual content creation, but it is tedious and time-consuming to produce even for experienced artists. Automatic methods have been proposed to address this issue, but most existing video prediction approaches require multiple input frames. In addition, generated sequences have limited variety since the output is mostly determined by the input frames, without allowing the user to provide additional constraints on the result. In our technique, users can control the generated animation using a sketch stroke on a single input image. We train our system such that the trajectory of the animated object follows the stroke, which makes it both more flexible and more controllable. From a single image, users can generate a variety of video sequences corresponding to different sketch inputs. Our method is the first system that, given a single frame and a motion stroke, can generate animations by recurrently generating videos frame by frame. An important benefit of the recurrent nature of our architecture is that it facilitates the synthesis of an arbitrary number of generated frames. Our architecture uses an autoencoder and a generative adversarial network (GAN) to generate sharp texture images, and we use another GAN to guarantee that transitions between frames are realistic and smooth. We demonstrate the effectiveness of our approach on the MNIST, KTH, and Human 3.6M datasets.
Tasks	Video Prediction
Published	2018-12-05
URL	http://arxiv.org/abs/1812.01874v1
PDF	http://arxiv.org/pdf/1812.01874v1.pdf
PWC	https://paperswithcode.com/paper/video-synthesis-from-a-single-image-and
Repo
Framework

Hierarchical clustering that takes advantage of both density-peak and density-connectivity


Title	Hierarchical clustering that takes advantage of both density-peak and density-connectivity
Authors	Ye Zhu, Kai Ming Ting, Yuan Jin, Maia Angelova
Abstract	This paper focuses on density-based clustering, particularly the Density Peak (DP) algorithm and the one based on density-connectivity DBSCAN; and proposes a new method which takes advantage of the individual strengths of these two methods to yield a density-based hierarchical clustering algorithm. Our investigation begins with formally defining the types of clusters DP and DBSCAN are designed to detect; and then identifies the kinds of distributions that DP and DBSCAN individually fail to detect all clusters in a dataset. These identified weaknesses inspire us to formally define a new kind of clusters and propose a new method called DC-HDP to overcome these weaknesses to identify clusters with arbitrary shapes and varied densities. In addition, the new method produces a richer clustering result in terms of hierarchy or dendrogram for better cluster structures understanding. Our empirical evaluation results show that DC-HDP produces the best clustering results on 14 datasets in comparison with 7 state-of-the-art clustering algorithms.
Tasks
Published	2018-10-08
URL	http://arxiv.org/abs/1810.03393v1
PDF	http://arxiv.org/pdf/1810.03393v1.pdf
PWC	https://paperswithcode.com/paper/hierarchical-clustering-that-takes-advantage
Repo
Framework

MaskConnect: Connectivity Learning by Gradient Descent


Title	MaskConnect: Connectivity Learning by Gradient Descent
Authors	Karim Ahmed, Lorenzo Torresani
Abstract	Although deep networks have recently emerged as the model of choice for many computer vision problems, in order to yield good results they often require time-consuming architecture search. To combat the complexity of design choices, prior work has adopted the principle of modularized design which consists in defining the network in terms of a composition of topologically identical or similar building blocks (a.k.a. modules). This reduces architecture search to the problem of determining the number of modules to compose and how to connect such modules. Again, for reasons of design complexity and training cost, previous approaches have relied on simple rules of connectivity, e.g., connecting each module to only the immediately preceding module or perhaps to all of the previous ones. Such simple connectivity rules are unlikely to yield the optimal architecture for the given problem. In this work we remove these predefined choices and propose an algorithm to learn the connections between modules in the network. Instead of being chosen a priori by the human designer, the connectivity is learned simultaneously with the weights of the network by optimizing the loss function of the end task using a modified version of gradient descent. We demonstrate our connectivity learning method on the problem of multi-class image classification using two popular architectures: ResNet and ResNeXt. Experiments on four different datasets show that connectivity learning using our approach yields consistently higher accuracy compared to relying on traditional predefined rules of connectivity. Furthermore, in certain settings it leads to significant savings in number of parameters.
Tasks	Image Classification, Neural Architecture Search
Published	2018-07-28
URL	http://arxiv.org/abs/1807.11473v1
PDF	http://arxiv.org/pdf/1807.11473v1.pdf
PWC	https://paperswithcode.com/paper/maskconnect-connectivity-learning-by-gradient
Repo
Framework