October 21, 2019

2920 words 14 mins read

Paper Group AWR 103

Efficient Neural Architecture Search via Parameter Sharing. Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model. NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks. nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation. Interactive Semantic Parsing for If-Then R …


Title	Efficient Neural Architecture Search via Parameter Sharing
Authors	Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean
Abstract	We propose Efficient Neural Architecture Search (ENAS), a fast and inexpensive approach for automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph. The controller is trained with policy gradient to select a subgraph that maximizes the expected reward on the validation set. Meanwhile the model corresponding to the selected subgraph is trained to minimize a canonical cross entropy loss. Thanks to parameter sharing between child models, ENAS is fast: it delivers strong empirical performances using much fewer GPU-hours than all existing automatic model design approaches, and notably, 1000x less expensive than standard Neural Architecture Search. On the Penn Treebank dataset, ENAS discovers a novel architecture that achieves a test perplexity of 55.8, establishing a new state-of-the-art among all methods without post-training processing. On the CIFAR-10 dataset, ENAS designs novel architectures that achieve a test error of 2.89%, which is on par with NASNet (Zoph et al., 2018), whose test error is 2.65%.
Tasks	Language Modelling, Neural Architecture Search
Published	2018-02-09
URL	http://arxiv.org/abs/1802.03268v2
PDF	http://arxiv.org/pdf/1802.03268v2.pdf
PWC	https://paperswithcode.com/paper/efficient-neural-architecture-search-via-1
Repo	https://github.com/ahundt/enas
Framework	tf

Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model


Title	Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model
Authors	Tian Han, Erik Nijkamp, Xiaolin Fang, Mitch Hill, Song-Chun Zhu, Ying Nian Wu
Abstract	This paper proposes the divergence triangle as a framework for joint training of generator model, energy-based model and inference model. The divergence triangle is a compact and symmetric (anti-symmetric) objective function that seamlessly integrates variational learning, adversarial learning, wake-sleep algorithm, and contrastive divergence in a unified probabilistic formulation. This unification makes the processes of sampling, inference, energy evaluation readily available without the need for costly Markov chain Monte Carlo methods. Our experiments demonstrate that the divergence triangle is capable of learning (1) an energy-based model with well-formed energy landscape, (2) direct sampling in the form of a generator network, and (3) feed-forward inference that faithfully reconstructs observed as well as synthesized data. The divergence triangle is a robust training method that can learn from incomplete data.
Tasks
Published	2018-12-28
URL	http://arxiv.org/abs/1812.10907v2
PDF	http://arxiv.org/pdf/1812.10907v2.pdf
PWC	https://paperswithcode.com/paper/divergence-triangle-for-joint-training-of
Repo	https://github.com/enijkamp/triangle
Framework	tf

NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks


Title	NeuroX: A Toolkit for Analyzing Individual Neurons in Neural Networks
Authors	Fahim Dalvi, Avery Nortonsmith, D. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, James Glass
Abstract	We present a toolkit to facilitate the interpretation and understanding of neural network models. The toolkit provides several methods to identify salient neurons with respect to the model itself or an external task. A user can visualize selected neurons, ablate them to measure their effect on the model accuracy, and manipulate them to control the behavior of the model at the test time. Such an analysis has a potential to serve as a springboard in various research directions, such as understanding the model, better architectural choices, model distillation and controlling data biases.
Tasks
Published	2018-12-21
URL	http://arxiv.org/abs/1812.09359v1
PDF	http://arxiv.org/pdf/1812.09359v1.pdf
PWC	https://paperswithcode.com/paper/neurox-a-toolkit-for-analyzing-individual
Repo	https://github.com/fdalvi/NeuroX
Framework	pytorch

nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation


Title	nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation
Authors	Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F. Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, Klaus H. Maier-Hein
Abstract	The U-Net was presented in 2015. With its straight-forward and successful architecture it quickly evolved to a commonly used benchmark in medical image segmentation. The adaptation of the U-Net to novel problems, however, comprises several degrees of freedom regarding the exact architecture, preprocessing, training and inference. These choices are not independent of each other and substantially impact the overall performance. The present paper introduces the nnU-Net (‘no-new-Net’), which refers to a robust and self-adapting framework on the basis of 2D and 3D vanilla U-Nets. We argue the strong case for taking away superfluous bells and whistles of many proposed network designs and instead focus on the remaining aspects that make out the performance and generalizability of a method. We evaluate the nnU-Net in the context of the Medical Segmentation Decathlon challenge, which measures segmentation performance in ten disciplines comprising distinct entities, image modalities, image geometries and dataset sizes, with no manual adjustments between datasets allowed. At the time of manuscript submission, nnU-Net achieves the highest mean dice scores across all classes and seven phase 1 tasks (except class 1 in BrainTumour) in the online leaderboard of the challenge.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2018-09-27
URL	http://arxiv.org/abs/1809.10486v1
PDF	http://arxiv.org/pdf/1809.10486v1.pdf
PWC	https://paperswithcode.com/paper/nnu-net-self-adapting-framework-for-u-net
Repo	https://github.com/hibetterheyj/Paper-Everyday
Framework	pytorch

Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning


Title	Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
Authors	Ziyu Yao, Xiujun Li, Jianfeng Gao, Brian Sadler, Huan Sun
Abstract	Given a text description, most existing semantic parsers synthesize a program in one shot. However, it is quite challenging to produce a correct program solely based on the description, which in reality is often ambiguous or incomplete. In this paper, we investigate interactive semantic parsing, where the agent can ask the user clarification questions to resolve ambiguities via a multi-turn dialogue, on an important type of programs called “If-Then recipes.” We develop a hierarchical reinforcement learning (HRL) based agent that significantly improves the parsing performance with minimal questions to the user. Results under both simulation and human evaluation show that our agent substantially outperforms non-interactive semantic parsers and rule-based agents.
Tasks	Hierarchical Reinforcement Learning, Semantic Parsing
Published	2018-08-21
URL	http://arxiv.org/abs/1808.06740v2
PDF	http://arxiv.org/pdf/1808.06740v2.pdf
PWC	https://paperswithcode.com/paper/interactive-semantic-parsing-for-if-then
Repo	https://github.com/LittleYUYU/Interactive-Semantic-Parsing
Framework	tf

Neural-Brane: Neural Bayesian Personalized Ranking for Attributed Network Embedding


Title	Neural-Brane: Neural Bayesian Personalized Ranking for Attributed Network Embedding
Authors	Vachik S. Dave, Baichuan Zhang, Pin-Yu Chen, Mohammad Al Hasan
Abstract	Network embedding methodologies, which learn a distributed vector representation for each vertex in a network, have attracted considerable interest in recent years. Existing works have demonstrated that vertex representation learned through an embedding method provides superior performance in many real-world applications, such as node classification, link prediction, and community detection. However, most of the existing methods for network embedding only utilize topological information of a vertex, ignoring a rich set of nodal attributes (such as, user profiles of an online social network, or textual contents of a citation network), which is abundant in all real-life networks. A joint network embedding that takes into account both attributional and relational information entails a complete network information and could further enrich the learned vector representations. In this work, we present Neural-Brane, a novel Neural Bayesian Personalized Ranking based Attributed Network Embedding. For a given network, Neural-Brane extracts latent feature representation of its vertices using a designed neural network model that unifies network topological information and nodal attributes; Besides, it utilizes Bayesian personalized ranking objective, which exploits the proximity ordering between a similar node-pair and a dissimilar node-pair. We evaluate the quality of vertex embedding produced by Neural-Brane by solving the node classification and clustering tasks on four real-world datasets. Experimental results demonstrate the superiority of our proposed method over the state-of-the-art existing methods.
Tasks	Community Detection, Link Prediction, Network Embedding, Node Classification
Published	2018-04-23
URL	http://arxiv.org/abs/1804.08774v2
PDF	http://arxiv.org/pdf/1804.08774v2.pdf
PWC	https://paperswithcode.com/paper/neural-brane-neural-bayesian-personalized
Repo	https://github.com/Vachik-Dave/Neural-Brane-Neural-Bayesian-Personalized-Ranking-for-Attributed-Network-Embedding
Framework	tf

Super-Resolution using Convolutional Neural Networks without Any Checkerboard Artifacts


Title	Super-Resolution using Convolutional Neural Networks without Any Checkerboard Artifacts
Authors	Yusuke Sugawara, Sayaka Shiota, Hitoshi Kiya
Abstract	It is well-known that a number of excellent super-resolution (SR) methods using convolutional neural networks (CNNs) generate checkerboard artifacts. A condition to avoid the checkerboard artifacts is proposed in this paper. So far, checkerboard artifacts have been mainly studied for linear multirate systems, but the condition to avoid checkerboard artifacts can not be applied to CNNs due to the non-linearity of CNNs. We extend the avoiding condition for CNNs, and apply the proposed structure to some typical SR methods to confirm the effectiveness of the new scheme. Experiment results demonstrate that the proposed structure can perfectly avoid to generate checkerboard artifacts under two loss conditions: mean square error and perceptual loss, while keeping excellent properties that the SR methods have.
Tasks	Super-Resolution
Published	2018-06-07
URL	http://arxiv.org/abs/1806.02658v1
PDF	http://arxiv.org/pdf/1806.02658v1.pdf
PWC	https://paperswithcode.com/paper/super-resolution-using-convolutional-neural
Repo	https://github.com/r06922019/butt_lion_paper_notes
Framework	pytorch

Early hospital mortality prediction using vital signals


Title	Early hospital mortality prediction using vital signals
Authors	Reza Sadeghi, Tanvi Banerjee, William Romine
Abstract	Early hospital mortality prediction is critical as intensivists strive to make efficient medical decisions about the severely ill patients staying in intensive care units. As a result, various methods have been developed to address this problem based on clinical records. However, some of the laboratory test results are time-consuming and need to be processed. In this paper, we propose a novel method to predict mortality using features extracted from the heart signals of patients within the first hour of ICU admission. In order to predict the risk, quantitative features have been computed based on the heart rate signals of ICU patients. Each signal is described in terms of 12 statistical and signal-based features. The extracted features are fed into eight classifiers: decision tree, linear discriminant, logistic regression, support vector machine (SVM), random forest, boosted trees, Gaussian SVM, and K-nearest neighborhood (K-NN). To derive insight into the performance of the proposed method, several experiments have been conducted using the well-known clinical dataset named Medical Information Mart for Intensive Care III (MIMIC-III). The experimental results demonstrate the capability of the proposed method in terms of precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC). The decision tree classifier satisfies both accuracy and interpretability better than the other classifiers, producing an F1-score and AUC equal to 0.91 and 0.93, respectively. It indicates that heart rate signals can be used for predicting mortality in patients in the ICU, achieving a comparable performance with existing predictions that rely on high dimensional features from clinical records which need to be processed and may contain missing information.
Tasks	Mortality Prediction
Published	2018-03-18
URL	http://arxiv.org/abs/1803.06589v2
PDF	http://arxiv.org/pdf/1803.06589v2.pdf
PWC	https://paperswithcode.com/paper/early-hospital-mortality-prediction-using
Repo	https://github.com/RezaSadeghiWSU/Early-Hospital-Mortality-Prediction-using-Vital-Signals
Framework	none

Heuristic Framework for Multi-Scale Testing of the Multi-Manifold Hypothesis


Title	Heuristic Framework for Multi-Scale Testing of the Multi-Manifold Hypothesis
Authors	F. Patricia Medina, Linda Ness, Melanie Weber, Karamatou Yacoubou Djima
Abstract	When analyzing empirical data, we often find that global linear models overestimate the number of parameters required. In such cases, we may ask whether the data lies on or near a manifold or a set of manifolds (a so-called multi-manifold) of lower dimension than the ambient space. This question can be phrased as a (multi-) manifold hypothesis. The identification of such intrinsic multiscale features is a cornerstone of data analysis and representation and has given rise to a large body of work on manifold learning. In this work, we review key results on multi-scale data analysis and intrinsic dimension followed by the introduction of a heuristic, multiscale framework for testing the multi-manifold hypothesis. Our method implements a hypothesis test on a set of spline-interpolated manifolds constructed from variance-based intrinsic dimensions. The workflow is suitable for empirical data analysis as we demonstrate on two use cases.
Tasks
Published	2018-07-01
URL	http://arxiv.org/abs/1807.00349v1
PDF	http://arxiv.org/pdf/1807.00349v1.pdf
PWC	https://paperswithcode.com/paper/heuristic-framework-for-multi-scale-testing
Repo	https://github.com/MelWe/mm-hypothesis
Framework	none

PhotoShape: Photorealistic Materials for Large-Scale Shape Collections


Title	PhotoShape: Photorealistic Materials for Large-Scale Shape Collections
Authors	Keunhong Park, Konstantinos Rematas, Ali Farhadi, Steven M. Seitz
Abstract	Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data – shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).
Tasks
Published	2018-09-26
URL	http://arxiv.org/abs/1809.09761v1
PDF	http://arxiv.org/pdf/1809.09761v1.pdf
PWC	https://paperswithcode.com/paper/photoshape-photorealistic-materials-for-large
Repo	https://github.com/keunhong/photoshape
Framework	pytorch

VnCoreNLP: A Vietnamese Natural Language Processing Toolkit


Title	VnCoreNLP: A Vietnamese Natural Language Processing Toolkit
Authors	Thanh Vu, Dat Quoc Nguyen, Dai Quoc Nguyen, Mark Dras, Mark Johnson
Abstract	We present an easy-to-use and fast toolkit, namely VnCoreNLP—a Java NLP annotation pipeline for Vietnamese. Our VnCoreNLP supports key natural language processing (NLP) tasks including word segmentation, part-of-speech (POS) tagging, named entity recognition (NER) and dependency parsing, and obtains state-of-the-art (SOTA) results for these tasks. We release VnCoreNLP to provide rich linguistic annotations to facilitate research work on Vietnamese NLP. Our VnCoreNLP is open-source and available at: https://github.com/vncorenlp/VnCoreNLP
Tasks	Dependency Parsing, Named Entity Recognition, Part-Of-Speech Tagging
Published	2018-01-04
URL	http://arxiv.org/abs/1801.01331v2
PDF	http://arxiv.org/pdf/1801.01331v2.pdf
PWC	https://paperswithcode.com/paper/vncorenlp-a-vietnamese-natural-language
Repo	https://github.com/vncorenlp/VnCoreNLP
Framework	none

Leveraging Unlabeled Data for Crowd Counting by Learning to Rank


Title	Leveraging Unlabeled Data for Crowd Counting by Learning to Rank
Authors	Xialei Liu, Joost van de Weijer, Andrew D. Bagdanov
Abstract	We propose a novel crowd counting approach that leverages abundantly available unlabeled crowd imagery in a learning-to-rank framework. To induce a ranking of cropped images , we use the observation that any sub-image of a crowded scene image is guaranteed to contain the same number or fewer persons than the super-image. This allows us to address the problem of limited size of existing datasets for crowd counting. We collect two crowd scene datasets from Google using keyword searches and query-by-example image retrieval, respectively. We demonstrate how to efficiently learn from these unlabeled datasets by incorporating learning-to-rank in a multi-task network which simultaneously ranks images and estimates crowd density maps. Experiments on two of the most challenging crowd counting datasets show that our approach obtains state-of-the-art results.
Tasks	Crowd Counting, Image Retrieval, Learning-To-Rank
Published	2018-03-08
URL	http://arxiv.org/abs/1803.03095v1
PDF	http://arxiv.org/pdf/1803.03095v1.pdf
PWC	https://paperswithcode.com/paper/leveraging-unlabeled-data-for-crowd-counting
Repo	https://github.com/xialeiliu/CrowdCountingCVPR18
Framework	none

Merge Non-Dominated Sorting Algorithm for Many-Objective Optimization


Title	Merge Non-Dominated Sorting Algorithm for Many-Objective Optimization
Authors	Javier Moreno, Daniel Rodriguez, Antonio Nebro, Jose A. Lozano
Abstract	Many Pareto-based multi-objective evolutionary algorithms require to rank the solutions of the population in each iteration according to the dominance principle, what can become a costly operation particularly in the case of dealing with many-objective optimization problems. In this paper, we present a new efficient algorithm for computing the non-dominated sorting procedure, called Merge Non-Dominated Sorting (MNDS), which has a best computational complexity of $\Theta(NlogN)$ and a worst computational complexity of $\Theta(MN^2)$. Our approach is based on the computation of the dominance set of each solution by taking advantage of the characteristics of the merge sort algorithm. We compare the MNDS against four well-known techniques that can be considered as the state-of-the-art. The results indicate that the MNDS algorithm outperforms the other techniques in terms of number of comparisons as well as the total running time.
Tasks
Published	2018-09-17
URL	http://arxiv.org/abs/1809.06106v1
PDF	http://arxiv.org/pdf/1809.06106v1.pdf
PWC	https://paperswithcode.com/paper/merge-non-dominated-sorting-algorithm-for
Repo	https://github.com/jMetal/jMetal
Framework	none

The MOEADr Package - A Component-Based Framework for Multiobjective Evolutionary Algorithms Based on Decomposition


Title	The MOEADr Package - A Component-Based Framework for Multiobjective Evolutionary Algorithms Based on Decomposition
Authors	Felipe Campelo, Lucas S. Batista, Claus Aranha
Abstract	Multiobjective Evolutionary Algorithms based on Decomposition (MOEA/D) represent a widely used class of population-based metaheuristics for the solution of multicriteria optimization problems. We introduce the MOEADr package, which offers many of these variants as instantiations of a component-oriented framework. This approach contributes for easier reproducibility of existing MOEA/D variants from the literature, as well as for faster development and testing of new composite algorithms. The package offers an standardized, modular implementation of MOEA/D based on this framework, which was designed aiming at providing researchers and practitioners with a standard way to discuss and express MOEA/D variants. In this paper we introduce the design principles behind the MOEADr package, as well as its current components. Three case studies are provided to illustrate the main aspects of the package.
Tasks
Published	2018-07-18
URL	http://arxiv.org/abs/1807.06731v1
PDF	http://arxiv.org/pdf/1807.06731v1.pdf
PWC	https://paperswithcode.com/paper/the-moeadr-package-a-component-based
Repo	https://github.com/yclavinas/MOEADr
Framework	none

Stacked Pooling: Improving Crowd Counting by Boosting Scale Invariance


Title	Stacked Pooling: Improving Crowd Counting by Boosting Scale Invariance
Authors	Siyu Huang, Xi Li, Zhi-Qi Cheng, Zhongfei Zhang, Alexander Hauptmann
Abstract	In this work, we explore the cross-scale similarity in crowd counting scenario, in which the regions of different scales often exhibit high visual similarity. This feature is universal both within an image and across different images, indicating the importance of scale invariance of a crowd counting model. Motivated by this, in this paper we propose simple but effective variants of pooling module, i.e., multi-kernel pooling and stacked pooling, to boost the scale invariance of convolutional neural networks (CNNs), benefiting much the crowd density estimation and counting. Specifically, the multi-kernel pooling comprises of pooling kernels with multiple receptive fields to capture the responses at multi-scale local ranges. The stacked pooling is an equivalent form of multi-kernel pooling, while, it reduces considerable computing cost. Our proposed pooling modules do not introduce extra parameters into model and can easily take place of the vanilla pooling layer in implementation. In empirical study on two benchmark crowd counting datasets, the stacked pooling beats the vanilla pooling layer in most cases.
Tasks	Crowd Counting, Density Estimation
Published	2018-08-22
URL	http://arxiv.org/abs/1808.07456v1
PDF	http://arxiv.org/pdf/1808.07456v1.pdf
PWC	https://paperswithcode.com/paper/stacked-pooling-improving-crowd-counting-by
Repo	https://github.com/siyuhuang/crowdcount-stackpool
Framework	pytorch