Paper Group ANR 376
Structured Pruning for Efficient ConvNets via Incremental Regularization. Simple stopping criteria for information theoretic feature selection. Three-Dimensional Dose Prediction for Lung IMRT Patients with Deep Neural Networks: Robust Learning from Heterogeneous Beam Configurations. On the Convergence of A Class of Adam-Type Algorithms for Non-Conv …
Structured Pruning for Efficient ConvNets via Incremental Regularization
| Title | Structured Pruning for Efficient ConvNets via Incremental Regularization |
| Authors | Huan Wang, Qiming Zhang, Yuehai Wang, Haoji Hu |
| Abstract | Parameter pruning is a promising approach for CNN compression and acceleration by eliminating redundant model parameters with tolerable performance loss. Despite its effectiveness, existing regularization-based parameter pruning methods usually drive weights towards zero with large and constant regularization factors, which neglects the fact that the expressiveness of CNNs is fragile and needs a more gentle way of regularization for the networks to adapt during pruning. To solve this problem, we propose a new regularization-based pruning method (named IncReg) to incrementally assign different regularization factors to different weight groups based on their relative importance, whose effectiveness is proved on popular CNNs compared with state-of-the-art methods. |
| Tasks | |
| Published | 2018-11-20 |
| URL | http://arxiv.org/abs/1811.08390v2 |
| http://arxiv.org/pdf/1811.08390v2.pdf | |
| PWC | https://paperswithcode.com/paper/structured-pruning-for-efficient-convnets-via |
| Repo | |
| Framework | |
Simple stopping criteria for information theoretic feature selection
| Title | Simple stopping criteria for information theoretic feature selection |
| Authors | Shujian Yu, Jose C. Principe |
| Abstract | Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual information between the selected features and the class labels is maximized. Despite the simplicity of this objective, there still remain several open problems in optimization. These include, for example, the automatic determination of the optimal subset size (i.e., the number of features) or a stopping criterion if the greedy searching strategy is adopted. In this paper, we suggest two stopping criteria by just monitoring the conditional mutual information (CMI) among groups of variables. Using the recently developed multivariate matrix-based Renyi’s \alpha-entropy functional, which can be directly estimated from data samples, we showed that the CMI among groups of variables can be easily computed without any decomposition or approximation, hence making our criteria easy to implement and seamlessly integrated into any existing information theoretic feature selection methods with a greedy search strategy. |
| Tasks | Feature Selection |
| Published | 2018-11-29 |
| URL | http://arxiv.org/abs/1811.11971v2 |
| http://arxiv.org/pdf/1811.11971v2.pdf | |
| PWC | https://paperswithcode.com/paper/simple-stopping-criteria-for-information |
| Repo | |
| Framework | |
Three-Dimensional Dose Prediction for Lung IMRT Patients with Deep Neural Networks: Robust Learning from Heterogeneous Beam Configurations
| Title | Three-Dimensional Dose Prediction for Lung IMRT Patients with Deep Neural Networks: Robust Learning from Heterogeneous Beam Configurations |
| Authors | Ana M. Barragan-Montero, Dan Nguyen, Weiguo Lu, Mu-Han Lin, Xavier Geets, Edmond Sterpin, Steve Jiang |
| Abstract | The use of neural networks to directly predict three-dimensional dose distributions for automatic planning is becoming popular. However, the existing methods only use patient anatomy as input and assume consistent beam configuration for all patients in the training database. The purpose of this work is to develop a more general model that, in addition to patient anatomy, also considers variable beam configurations, to achieve a more comprehensive automatic planning with a potentially easier clinical implementation, without the need of training specific models for different beam settings. |
| Tasks | |
| Published | 2018-12-17 |
| URL | http://arxiv.org/abs/1812.06934v2 |
| http://arxiv.org/pdf/1812.06934v2.pdf | |
| PWC | https://paperswithcode.com/paper/three-dimensional-dose-prediction-for-lung |
| Repo | |
| Framework | |
On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization
| Title | On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization |
| Authors | Xiangyi Chen, Sijia Liu, Ruoyu Sun, Mingyi Hong |
| Abstract | This paper studies a class of adaptive gradient based momentum algorithms that update the search directions and learning rates simultaneously using past gradients. This class, which we refer to as the “Adam-type”, includes the popular algorithms such as the Adam, AMSGrad and AdaGrad. Despite their popularity in training deep neural networks, the convergence of these algorithms for solving nonconvex problems remains an open question. This paper provides a set of mild sufficient conditions that guarantee the convergence for the Adam-type methods. We prove that under our derived conditions, these methods can achieve the convergence rate of order $O(\log{T}/\sqrt{T})$ for nonconvex stochastic optimization. We show the conditions are essential in the sense that violating them may make the algorithm diverge. Moreover, we propose and analyze a class of (deterministic) incremental adaptive gradient algorithms, which has the same $O(\log{T}/\sqrt{T})$ convergence rate. Our study could also be extended to a broader class of adaptive gradient methods in machine learning and optimization. |
| Tasks | Stochastic Optimization |
| Published | 2018-08-08 |
| URL | http://arxiv.org/abs/1808.02941v2 |
| http://arxiv.org/pdf/1808.02941v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-convergence-of-a-class-of-adam-type |
| Repo | |
| Framework | |
Adaptive Policies for Perimeter Surveillance Problems
| Title | Adaptive Policies for Perimeter Surveillance Problems |
| Authors | James A. Grant, David S. Leslie, Kevin Glazebrook, Roberto Szechtman, Adam N. Letchford |
| Abstract | Maximising the detection of intrusions is a fundamental and often critical aim of perimeter surveillance. Commonly, this requires a decision-maker to optimally allocate multiple searchers to segments of the perimeter. We consider a scenario where the decision-maker may sequentially update the searchers’ allocation, learning from the observed data to improve decisions over time. In this work we propose a formal model and solution methods for this sequential perimeter surveillance problem. Our model is a combinatorial multi-armed bandit (CMAB) with Poisson rewards and a novel filtered feedback mechanism - arising from the failure to detect certain intrusions. Our solution method is an upper confidence bound approach and we derive upper and lower bounds on its expected performance. We prove that the gap between these bounds is of constant order, and demonstrate empirically that our approach is more reliable in simulated problems than competing algorithms. |
| Tasks | |
| Published | 2018-10-04 |
| URL | https://arxiv.org/abs/1810.02176v2 |
| https://arxiv.org/pdf/1810.02176v2.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-policies-for-perimeter-surveillance |
| Repo | |
| Framework | |
Robust Conditional Generative Adversarial Networks
| Title | Robust Conditional Generative Adversarial Networks |
| Authors | Grigorios G. Chrysos, Jean Kossaifi, Stefanos Zafeiriou |
| Abstract | Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision. The major focus so far has been on performance improvement, while there has been little effort in making cGAN more robust to noise. The regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGAN unreliable for real-world applications. In this work, we introduce a novel conditional GAN model, called RoCGAN, which leverages structure in the target space of the model to address the issue. Our model augments the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold even in the presence of intense noise. We prove that RoCGAN share similar theoretical properties as GAN and experimentally verify that our model outperforms existing state-of-the-art cGAN architectures by a large margin in a variety of domains including images from natural scenes and faces. |
| Tasks | Conditional Image Generation, Image Generation |
| Published | 2018-05-22 |
| URL | http://arxiv.org/abs/1805.08657v2 |
| http://arxiv.org/pdf/1805.08657v2.pdf | |
| PWC | https://paperswithcode.com/paper/robust-conditional-generative-adversarial |
| Repo | |
| Framework | |
Semantic Segmentation of Human Thigh Quadriceps Muscle in Magnetic Resonance Images
| Title | Semantic Segmentation of Human Thigh Quadriceps Muscle in Magnetic Resonance Images |
| Authors | Ezak Ahmad, Manu Goyal, Jamie S. McPhee, Hans Degens, Moi Hoon Yap |
| Abstract | This paper presents an end-to-end solution for MRI thigh quadriceps segmentation. This is the first attempt that deep learning methods are used for the MRI thigh segmentation task. We use the state-of-the-art Fully Convolutional Networks with transfer learning approach for the semantic segmentation of regions of interest in MRI thigh scans. To further improve the performance of the segmentation, we propose a post-processing technique using basic image processing methods. With our proposed method, we have established a new benchmark for MRI thigh quadriceps segmentation with mean Jaccard Similarity Index of 0.9502 and processing time of 0.117 second per image. |
| Tasks | Semantic Segmentation, Transfer Learning |
| Published | 2018-01-01 |
| URL | http://arxiv.org/abs/1801.00415v2 |
| http://arxiv.org/pdf/1801.00415v2.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-segmentation-of-human-thigh |
| Repo | |
| Framework | |
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization
| Title | Theoretical Analysis of Auto Rate-Tuning by Batch Normalization |
| Authors | Sanjeev Arora, Zhiyuan Li, Kaifeng Lyu |
| Abstract | Batch Normalization (BN) has become a cornerstone of deep learning across diverse architectures, appearing to help optimization as well as generalization. While the idea makes intuitive sense, theoretical analysis of its effectiveness has been lacking. Here theoretical support is provided for one of its conjectured properties, namely, the ability to allow gradient descent to succeed with less tuning of learning rates. It is shown that even if we fix the learning rate of scale-invariant parameters (e.g., weights of each layer with BN) to a constant (say, $0.3$), gradient descent still approaches a stationary point (i.e., a solution where gradient is zero) in the rate of $T^{-1/2}$ in $T$ iterations, asymptotically matching the best bound for gradient descent with well-tuned learning rates. A similar result with convergence rate $T^{-1/4}$ is also shown for stochastic gradient descent. |
| Tasks | |
| Published | 2018-12-10 |
| URL | http://arxiv.org/abs/1812.03981v1 |
| http://arxiv.org/pdf/1812.03981v1.pdf | |
| PWC | https://paperswithcode.com/paper/theoretical-analysis-of-auto-rate-tuning-by |
| Repo | |
| Framework | |
SimArch: A Multi-agent System For Human Path Simulation In Architecture Design
| Title | SimArch: A Multi-agent System For Human Path Simulation In Architecture Design |
| Authors | Yen-Chia Hsu |
| Abstract | Human moving path is an important feature in architecture design. By studying the path, architects know where to arrange the basic elements (e.g. structures, glasses, furniture, etc.) in the space. This paper presents SimArch, a multi-agent system for human moving path simulation. It involves a behavior model built by using a Markov Decision Process. The model simulates human mental states, target range detection, and collision prediction when agents are on the floor, in a particular small gallery, looking at an exhibit, or leaving the floor. It also models different kinds of human characteristics by assigning different transition probabilities. A modified weighted A* search algorithm quickly plans the sub-optimal path of the agents. In an experiment, SimArch takes a series of preprocessed floorplans as inputs, simulates the moving path, and outputs a density map for evaluation. The density map provides the prediction that how likely a person will occur in a location. A following discussion illustrates how architects can use the density map to improve their floorplan design. |
| Tasks | |
| Published | 2018-07-10 |
| URL | http://arxiv.org/abs/1807.03760v1 |
| http://arxiv.org/pdf/1807.03760v1.pdf | |
| PWC | https://paperswithcode.com/paper/simarch-a-multi-agent-system-for-human-path |
| Repo | |
| Framework | |
A Simple Theoretical Model of Importance for Summarization
| Title | A Simple Theoretical Model of Importance for Summarization |
| Authors | Maxime Peyrard |
| Abstract | Research on summarization has mainly been driven by empirical approaches, crafting systems to perform well on standard datasets with the notion of information Importance remaining latent. We argue that establishing theoretical models of Importance will advance our understanding of the task and help to further improve summarization systems. To this end, we propose simple but rigorous definitions of several concepts that were previously used only intuitively in summarization: Redundancy, Relevance, and Informativeness. Importance arises as a single quantity naturally unifying these concepts. Additionally, we provide intuitions to interpret the proposed quantities and experiments to demonstrate the potential of the framework to inform and guide subsequent works. |
| Tasks | |
| Published | 2018-01-26 |
| URL | https://arxiv.org/abs/1801.08991v2 |
| https://arxiv.org/pdf/1801.08991v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-formal-definition-of-importance-for |
| Repo | |
| Framework | |
Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
| Title | Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping |
| Authors | Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Kun Zhang, Dacheng Tao |
| Abstract | Unsupervised domain mapping aims to learn a function to translate domain X to Y by a function GXY in the absence of paired examples. Finding the optimal GXY without paired data is an ill-posed problem, so appropriate constraints are required to obtain reasonable solutions. One of the most prominent constraints is cycle consistency, which enforces the translated image by GXY to be translated back to the input image by an inverse mapping GYX. While cycle consistency requires the simultaneous training of GXY and GY X, recent studies have shown that one-sided domain mapping can be achieved by preserving pairwise distances between images. Although cycle consistency and distance preservation successfully constrain the solution space, they overlook the special properties that simple geometric transformations do not change the semantic structure of images. Based on this special property, we develop a geometry-consistent generative adversarial network (GcGAN), which enables one-sided unsupervised domain mapping. GcGAN takes the original image and its counterpart image transformed by a predefined geometric transformation as inputs and generates two images in the new domain coupled with the corresponding geometry-consistency constraint. The geometry-consistency constraint reduces the space of possible solutions while keep the correct solutions in the search space. Quantitative and qualitative comparisons with the baseline (GAN alone) and the state-of-the-art methods including CycleGAN and DistanceGAN demonstrate the effectiveness of our method. |
| Tasks | |
| Published | 2018-09-16 |
| URL | http://arxiv.org/abs/1809.05852v2 |
| http://arxiv.org/pdf/1809.05852v2.pdf | |
| PWC | https://paperswithcode.com/paper/geometry-consistent-generative-adversarial |
| Repo | |
| Framework | |
Distinguishing Between Roles of Football Players in Play-by-play Match Event Data
| Title | Distinguishing Between Roles of Football Players in Play-by-play Match Event Data |
| Authors | Bart Aalbers, Jan Van Haaren |
| Abstract | Over the last few decades, the player recruitment process in professional football has evolved into a multi-billion industry and has thus become of vital importance. To gain insights into the general level of their candidate reinforcements, many professional football clubs have access to extensive video footage and advanced statistics. However, the question whether a given player would fit the team’s playing style often still remains unanswered. In this paper, we aim to bridge that gap by proposing a set of 21 player roles and introducing a method for automatically identifying the most applicable roles for each player from play-by-play event data collected during matches. |
| Tasks | |
| Published | 2018-09-13 |
| URL | http://arxiv.org/abs/1809.05173v1 |
| http://arxiv.org/pdf/1809.05173v1.pdf | |
| PWC | https://paperswithcode.com/paper/distinguishing-between-roles-of-football |
| Repo | |
| Framework | |
Deep In-GPU Experience Replay
| Title | Deep In-GPU Experience Replay |
| Authors | Ben Parr |
| Abstract | Experience replay allows a reinforcement learning agent to train on samples from a large amount of the most recent experiences. A simple in-RAM experience replay stores these most recent experiences in a list in RAM, and then copies sampled batches to the GPU for training. I moved this list to the GPU, thus creating an in-GPU experience replay, and a training step that no longer has inputs copied from the CPU. I trained an agent to play Super Smash Bros. Melee, using internal game memory values as inputs and outputting controller button presses. A single state in Melee contains 27 floats, so the full experience replay fits on a single GPU. For a batch size of 128, the in-GPU experience replay trained twice as fast as the in-RAM experience replay. As far as I know, this is the first in-GPU implementation of experience replay. Finally, I note a few ideas for fitting the experience replay inside the GPU when the environment state requires more memory. |
| Tasks | |
| Published | 2018-01-09 |
| URL | http://arxiv.org/abs/1801.03138v1 |
| http://arxiv.org/pdf/1801.03138v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-in-gpu-experience-replay |
| Repo | |
| Framework | |
Learning to discover and localize visual objects with open vocabulary
| Title | Learning to discover and localize visual objects with open vocabulary |
| Authors | Keren Ye, Mingda Zhang, Wei Li, Danfeng Qin, Adriana Kovashka, Jesse Berent |
| Abstract | To alleviate the cost of obtaining accurate bounding boxes for training today’s state-of-the-art object detection models, recent weakly supervised detection work has proposed techniques to learn from image-level labels. However, requiring discrete image-level labels is both restrictive and suboptimal. Real-world “supervision” usually consists of more unstructured text, such as captions. In this work we learn association maps between images and captions. We then use a novel objectness criterion to rank the resulting candidate boxes, such that high-ranking boxes have strong gradients along all edges. Thus, we can detect objects beyond a fixed object category vocabulary, if those objects are frequent and distinctive enough. We show that our objectness criterion improves the proposed bounding boxes in relation to prior weakly supervised detection methods. Further, we show encouraging results on object detection from image-level captions only. |
| Tasks | Object Detection |
| Published | 2018-11-25 |
| URL | http://arxiv.org/abs/1811.10080v1 |
| http://arxiv.org/pdf/1811.10080v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-discover-and-localize-visual |
| Repo | |
| Framework | |
How good is my GAN?
| Title | How good is my GAN? |
| Authors | Konstantin Shmelkov, Cordelia Schmid, Karteek Alahari |
| Abstract | Generative adversarial networks (GANs) are one of the most popular methods for generating images today. While impressive results have been validated by visual inspection, a number of quantitative criteria have emerged only recently. We argue here that the existing ones are insufficient and need to be in adequation with the task at hand. In this paper we introduce two measures based on image classification—GAN-train and GAN-test, which approximate the recall (diversity) and precision (quality of the image) of GANs respectively. We evaluate a number of recent GAN approaches based on these two measures and demonstrate a clear difference in performance. Furthermore, we observe that the increasing difficulty of the dataset, from CIFAR10 over CIFAR100 to ImageNet, shows an inverse correlation with the quality of the GANs, as clearly evident from our measures. |
| Tasks | Image Classification |
| Published | 2018-07-25 |
| URL | http://arxiv.org/abs/1807.09499v1 |
| http://arxiv.org/pdf/1807.09499v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-good-is-my-gan |
| Repo | |
| Framework | |
