January 31, 2020

3276 words 16 mins read

Paper Group ANR 79

Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction. The Difficulty of Training Sparse Neural Networks. T-Net: Encoder-Decoder in Encoder-Decoder architecture for the main vessel segmentation in coronary angiography. Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Maki …

Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction


Title	Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction
Authors	Wenjia Bai, Chen Chen, Giacomo Tarroni, Jinming Duan, Florian Guitton, Steffen E. Petersen, Yike Guo, Paul M. Matthews, Daniel Rueckert
Abstract	In the recent years, convolutional neural networks have transformed the field of medical image analysis due to their capacity to learn discriminative image features for a variety of classification and regression tasks. However, successfully learning these features requires a large amount of manually annotated data, which is expensive to acquire and limited by the available resources of expert image analysts. Therefore, unsupervised, weakly-supervised and self-supervised feature learning techniques receive a lot of attention, which aim to utilise the vast amount of available data, while at the same time avoid or substantially reduce the effort of manual annotation. In this paper, we propose a novel way for training a cardiac MR image segmentation network, in which features are learnt in a self-supervised manner by predicting anatomical positions. The anatomical positions serve as a supervisory signal and do not require extra manual annotation. We demonstrate that this seemingly simple task provides a strong signal for feature learning and with self-supervised learning, we achieve a high segmentation accuracy that is better than or comparable to a U-net trained from scratch, especially at a small data setting. When only five annotated subjects are available, the proposed method improves the mean Dice metric from 0.811 to 0.852 for short-axis image segmentation, compared to the baseline U-net.
Tasks	Semantic Segmentation
Published	2019-07-05
URL	https://arxiv.org/abs/1907.02757v1
PDF	https://arxiv.org/pdf/1907.02757v1.pdf
PWC	https://paperswithcode.com/paper/self-supervised-learning-for-cardiac-mr-image
Repo
Framework

The Difficulty of Training Sparse Neural Networks


Title	The Difficulty of Training Sparse Neural Networks
Authors	Utku Evci, Fabian Pedregosa, Aidan Gomez, Erich Elsen
Abstract	We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime. Recent work of \citep{Gale2019, Liu2018} has shown that sparse ResNet-50 architectures trained on ImageNet-2012 dataset converge to solutions that are significantly worse than those found by pruning. We show that, despite the failure of optimizers, there is a linear path with a monotonically decreasing objective from the initialization to the “good” solution. Additionally, our attempts to find a decreasing objective path from “bad” solutions to the “good” ones in the sparse subspace fail. However, if we allow the path to traverse the dense subspace, then we consistently find a path between two solutions. These findings suggest traversing extra dimensions may be needed to escape stationary points found in the sparse subspace.
Tasks
Published	2019-06-25
URL	https://arxiv.org/abs/1906.10732v2
PDF	https://arxiv.org/pdf/1906.10732v2.pdf
PWC	https://paperswithcode.com/paper/the-difficulty-of-training-sparse-neural
Repo
Framework

T-Net: Encoder-Decoder in Encoder-Decoder architecture for the main vessel segmentation in coronary angiography


Title	T-Net: Encoder-Decoder in Encoder-Decoder architecture for the main vessel segmentation in coronary angiography
Authors	Tae Joon Jun, Jihoon Kweon, Young-Hak Kim, Daeyoung Kim
Abstract	In this paper, we proposed T-Net containing a small encoder-decoder inside the encoder-decoder structure (EDiED). T-Net overcomes the limitation that U-Net can only have a single set of the concatenate layer between encoder and decoder block. To be more precise, the U-Net symmetrically forms the concatenate layers, so the low-level feature of the encoder is connected to the latter part of the decoder, and the high-level feature is connected to the beginning of the decoder. T-Net arranges the pooling and up-sampling appropriately during the encoder process, and likewise during the decoding process so that feature-maps of various sizes are obtained in a single block. As a result, all features from the low-level to the high-level extracted from the encoder are delivered from the beginning of the decoder to predict a more accurate mask. We evaluated T-Net for the problem of segmenting three main vessels in coronary angiography images. The experiment consisted of a comparison of U-Net and T-Nets under the same conditions, and an optimized T-Net for the main vessel segmentation. As a result, T-Net recorded a Dice Similarity Coefficient score (DSC) of 0.815, 0.095 higher than that of U-Net, and the optimized T-Net recorded a DSC of 0.890 which was 0.170 higher than that of U-Net. In addition, we visualized the weight activation of the convolutional layer of T-Net and U-Net to show that T-Net actually predicts the mask from earlier decoders. Therefore, we expect that T-Net can be effectively applied to other similar medical image segmentation problems.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2019-05-10
URL	https://arxiv.org/abs/1905.04197v1
PDF	https://arxiv.org/pdf/1905.04197v1.pdf
PWC	https://paperswithcode.com/paper/t-net-encoder-decoder-in-encoder-decoder
Repo
Framework

Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems


Title	Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
Authors	Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, Joydeep Ghosh
Abstract	Machine learning based decision making systems are increasingly affecting humans. An individual can suffer an undesirable outcome under such decision making systems (e.g. denied credit) irrespective of whether the decision is fair or accurate. Individual recourse pertains to the problem of providing an actionable set of changes a person can undertake in order to improve their outcome. We propose a recourse algorithm that models the underlying data distribution or manifold. We then provide a mechanism to generate the smallest set of changes that will improve an individual’s outcome. This mechanism can be easily used to provide recourse for any differentiable machine learning based decision making system. Further, the resulting algorithm is shown to be applicable to both supervised classification and causal decision making systems. Our work attempts to fill gaps in existing fairness literature that have primarily focused on discovering and/or algorithmically enforcing fairness constraints on decision making systems. This work also provides an alternative approach to generating counterfactual explanations.
Tasks	Decision Making
Published	2019-07-22
URL	https://arxiv.org/abs/1907.09615v1
PDF	https://arxiv.org/pdf/1907.09615v1.pdf
PWC	https://paperswithcode.com/paper/towards-realistic-individual-recourse-and
Repo
Framework

Communications and Networking Technologies for Intelligent Drone Cruisers


Title	Communications and Networking Technologies for Intelligent Drone Cruisers
Authors	Li-Chun Wang, Chuan-Chi Lai, Hong-Han Shuai, Hsin-Piao Lin, Chi-Yu Li, Teng-Hu Cheng, Chiun-Hsun Chen
Abstract	Future mobile communication networks require an Aerial Base Station (ABS) with fast mobility and long-term hovering capabilities. At present, unmanned aerial vehicles (UAV) or drones do not have long flight times and are mainly used for monitoring, surveillance, and image post-processing. On the other hand, the traditional airship is too large and not easy to take off and land. Therefore, we propose to develop an “Artificial Intelligence (AI) Drone-Cruiser” base station that can help 5G mobile communication systems and beyond quickly recover the network after a disaster and handle the instant communications by the flash crowd. The drone-cruiser base station can overcome the communications problem for three types of flash crowds, such as in stadiums, parades, and large plaza so that an appropriate number of aerial base stations can be accurately deployed to meet large and dynamic traffic demands. Artificial intelligence can solve these problems by analyzing the collected data, and then adjust the system parameters in the framework of Self-Organizing Network (SON) to achieve the goals of self-configuration, self-optimization, and self-healing. With the help of AI technologies, 5G networks can become more intelligent. This paper aims to provide a new type of service, On-Demand Aerial Base Station as a Service. This work needs to overcome the following five technical challenges: innovative design of drone-cruisers for the long-time hovering, crowd estimation and prediction, rapid 3D wireless channel learning and modeling, 3D placement of aerial base stations and the integration of WiFi front-haul and millimeter wave/WiGig back-haul networks.
Tasks
Published	2019-09-25
URL	https://arxiv.org/abs/1910.05309v1
PDF	https://arxiv.org/pdf/1910.05309v1.pdf
PWC	https://paperswithcode.com/paper/communications-and-networking-technologies
Repo
Framework

Quadratic Decomposable Submodular Function Minimization: Theory and Practice


Title	Quadratic Decomposable Submodular Function Minimization: Theory and Practice
Authors	Pan Li, Niao He, Olgica Milenkovic
Abstract	We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization (QDSFM), which allows to model a number of learning tasks on graphs and hypergraphs. The problem exhibits close ties to decomposable submodular function minimization (DSFM), yet is much more challenging to solve. We approach the problem via a new dual strategy and formulate an objective that can be optimized through a number of double-loop algorithms. The outer-loop uses either random coordinate descent (RCD) or alternative projection (AP) methods, for both of which we prove linear convergence rates. The inner-loop computes projections onto cones generated by base polytopes of the submodular functions, via the modified min-norm-point or Frank-Wolfe algorithm. We also describe two new applications of QDSFM: hypergraph-adapted PageRank and semi-supervised learning. The proposed hypergraph-based PageRank algorithm can be used for local hypergraph partitioning, and comes with provable performance guarantees. For hypergraph-adapted semi-supervised learning, we provide numerical experiments demonstrating the efficiency of our QDSFM solvers and their significant improvements on prediction accuracy when compared to state-of-the-art methods.
Tasks	hypergraph partitioning
Published	2019-02-26
URL	http://arxiv.org/abs/1902.10132v1
PDF	http://arxiv.org/pdf/1902.10132v1.pdf
PWC	https://paperswithcode.com/paper/quadratic-decomposable-submodular-function-1
Repo
Framework

A Fast, Semi-Automatic Brain Structure Segmentation Algorithm for Magnetic Resonance Imaging


Title	A Fast, Semi-Automatic Brain Structure Segmentation Algorithm for Magnetic Resonance Imaging
Authors	Kevin Karsch, Qing He, Ye Duan
Abstract	Medical image segmentation has become an essential technique in clinical and research-oriented applications. Because manual segmentation methods are tedious, and fully automatic segmentation lacks the flexibility of human intervention or correction, semi-automatic methods have become the preferred type of medical image segmentation. We present a hybrid, semi-automatic segmentation method in 3D that integrates both region-based and boundary-based procedures. Our method differs from previous hybrid methods in that we perform region-based and boundary-based approaches separately, which allows for more efficient segmentation. A region-based technique is used to generate an initial seed contour that roughly represents the boundary of a target brain structure, alleviating the local minima problem in the subsequent model deformation phase. The contour is deformed under a unique force equation independent of image edges. Experiments on MRI data show that this method can achieve high accuracy and efficiency primarily due to the unique seed initialization technique.
Tasks	Medical Image Segmentation, Semantic Segmentation
Published	2019-04-21
URL	http://arxiv.org/abs/1904.09978v1
PDF	http://arxiv.org/pdf/1904.09978v1.pdf
PWC	https://paperswithcode.com/paper/a-fast-semi-automatic-brain-structure
Repo
Framework

Temporal Network Representation Learning


Title	Temporal Network Representation Learning
Authors	John Boaz Lee, Giang Nguyen, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, Sungchul Kim
Abstract	Networks evolve continuously over time with the addition, deletion, and changing of links and nodes. Such temporal networks (or edge streams) consist of a sequence of timestamped edges and are seemingly ubiquitous. Despite the importance of accurately modeling the temporal information, most embedding methods ignore it entirely or approximate the temporal network using a sequence of static snapshot graphs. In this work, we introduce the notion of \emph{temporal walks} for learning dynamic embeddings from temporal networks. Temporal walks capture the temporally valid interactions (\eg, flow of information, spread of disease) in the dynamic network in a lossless fashion. Based on the notion of temporal walks, we describe a general class of embeddings called continuous-time dynamic network embeddings (CTDNEs) that completely avoid the issues and problems that arise when approximating the temporal network as a sequence of static snapshot graphs. Unlike previous work, CTDNEs learn dynamic node embeddings directly from the temporal network at the finest temporal granularity and thus use only temporally valid information. As such CTDNEs naturally support online learning of the node embeddings in a streaming real-time fashion. The experiments demonstrate the effectiveness of this class of embedding methods for prediction in temporal networks.
Tasks	Representation Learning
Published	2019-04-12
URL	http://arxiv.org/abs/1904.06449v1
PDF	http://arxiv.org/pdf/1904.06449v1.pdf
PWC	https://paperswithcode.com/paper/temporal-network-representation-learning
Repo
Framework

GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images


Title	GeoGAN: A Conditional GAN with Reconstruction and Style Loss to Generate Standard Layer of Maps from Satellite Images
Authors	Swetava Ganguli, Pedro Garzon, Noa Glaser
Abstract	Automatically generating maps from satellite images is an important task. There is a body of literature which tries to address this challenge. We created a more expansive survey of the task by experimenting with different models and adding new loss functions to improve results. We created a database of pairs of satellite images and the corresponding map of the area. Our model translates the satellite image to the corresponding standard layer map image using three main model architectures: (i) a conditional Generative Adversarial Network (GAN) which compresses the images down to a learned embedding, (ii) a generator which is trained as a normalizing flow (RealNVP) model, and (iii) a conditional GAN where the generator translates via a series of convolutions to the standard layer of a map and the discriminator input is the concatenation of the real/generated map and the satellite image. Model (iii) was by far the most promising of three models. To improve the results we also added a reconstruction loss and style transfer loss in addition to the GAN losses. The third model architecture produced the best quality of sampled images. In contrast to the other generative model where evaluation of the model is a challenging problem. since we have access to the real map for a given satellite image, we are able to assign a quantitative metric to the quality of the generated images in addition to inspecting them visually. While we are continuing to work on increasing the accuracy of the model, one challenge has been the coarse resolution of the data which upper-bounds the quality of the results of our model. Nevertheless, as will be seen in the results, the generated map is more accurate in the features it produces since the generator architecture demands a pixel-wise image translation/pixel-wise coloring. A video presentation summarizing this paper is available at: https://youtu.be/Ur0flOX-Ji0
Tasks	Style Transfer
Published	2019-02-14
URL	http://arxiv.org/abs/1902.05611v2
PDF	http://arxiv.org/pdf/1902.05611v2.pdf
PWC	https://paperswithcode.com/paper/geogan-a-conditional-gan-with-reconstruction
Repo
Framework

A scalable saliency-based Feature selection method with instance level information


Title	A scalable saliency-based Feature selection method with instance level information
Authors	Brais Cancela, Verónica Bolón-Canedo, Amparo Alonso-Betanzos, João Gama
Abstract	Classic feature selection techniques remove those features that are either irrelevant or redundant, achieving a subset of relevant features that help to provide a better knowledge extraction. This allows the creation of compact models that are easier to interpret. Most of these techniques work over the whole dataset, but they are unable to provide the user with successful information when only instance information is needed. In short, given any example, classic feature selection algorithms do not give any information about which the most relevant information is, regarding this sample. This work aims to overcome this handicap by developing a novel feature selection method, called Saliency-based Feature Selection (SFS), based in deep-learning saliency techniques. Our experimental results will prove that this algorithm can be successfully used not only in Neural Networks, but also under any given architecture trained by using Gradient Descent techniques.
Tasks	Feature Selection
Published	2019-04-30
URL	http://arxiv.org/abs/1904.13127v1
PDF	http://arxiv.org/pdf/1904.13127v1.pdf
PWC	https://paperswithcode.com/paper/a-scalable-saliency-based-feature-selection
Repo
Framework

Low-Rank Principal Eigenmatrix Analysis


Title	Low-Rank Principal Eigenmatrix Analysis
Authors	Krishna Balasubramanian, Elynn Y. Chen, Jianqing Fan, Xiang Wu
Abstract	Sparse PCA is a widely used technique for high-dimensional data analysis. In this paper, we propose a new method called low-rank principal eigenmatrix analysis. Different from sparse PCA, the dominant eigenvectors are allowed to be dense but are assumed to have a low-rank structure when matricized appropriately. Such a structure arises naturally in several practical cases: Indeed the top eigenvector of a circulant matrix, when matricized appropriately is a rank-1 matrix. We propose a matricized rank-truncated power method that could be efficiently implemented and establish its computational and statistical properties. Extensive experiments on several synthetic data sets demonstrate the competitive empirical performance of our method.
Tasks
Published	2019-04-28
URL	http://arxiv.org/abs/1904.12369v1
PDF	http://arxiv.org/pdf/1904.12369v1.pdf
PWC	https://paperswithcode.com/paper/low-rank-principal-eigenmatrix-analysis
Repo
Framework

On the Value of Bandit Feedback for Offline Recommender System Evaluation


Title	On the Value of Bandit Feedback for Offline Recommender System Evaluation
Authors	Olivier Jeunen, David Rohde, Flavian Vasile
Abstract	In academic literature, recommender systems are often evaluated on the task of next-item prediction. The procedure aims to give an answer to the question: “Given the natural sequence of user-item interactions up to time t, can we predict which item the user will interact with at time t+1?". Evaluation results obtained through said methodology are then used as a proxy to predict which system will perform better in an online setting. The online setting, however, poses a subtly different question: “Given the natural sequence of user-item interactions up to time t, can we get the user to interact with a recommended item at time t+1?". From a causal perspective, the system performs an intervention, and we want to measure its effect. Next-item prediction is often used as a fall-back objective when information about interventions and their effects (shown recommendations and whether they received a click) is unavailable. When this type of data is available, however, it can provide great value for reliably estimating online recommender system performance. Through a series of simulated experiments with the RecoGym environment, we show where traditional offline evaluation schemes fall short. Additionally, we show how so-called bandit feedback can be exploited for effective offline evaluation that more accurately reflects online performance.
Tasks	Recommendation Systems
Published	2019-07-26
URL	https://arxiv.org/abs/1907.12384v1
PDF	https://arxiv.org/pdf/1907.12384v1.pdf
PWC	https://paperswithcode.com/paper/on-the-value-of-bandit-feedback-for-offline
Repo
Framework

Scaling Laws for the Principled Design, Initialization and Preconditioning of ReLU Networks


Title	Scaling Laws for the Principled Design, Initialization and Preconditioning of ReLU Networks
Authors	Aaron Defazio, Léon Bottou
Abstract	In this work, we describe a set of rules for the design and initialization of well-conditioned neural networks, guided by the goal of naturally balancing the diagonal blocks of the Hessian at the start of training. Our design principle balances multiple sensible measures of the conditioning of neural networks. We prove that for a ReLU-based deep multilayer perceptron, a simple initialization scheme using the geometric mean of the fan-in and fan-out satisfies our scaling rule. For more sophisticated architectures, we show how our scaling principle can be used to guide design choices to produce well-conditioned neural networks, reducing guess-work.
Tasks
Published	2019-06-10
URL	https://arxiv.org/abs/1906.04267v2
PDF	https://arxiv.org/pdf/1906.04267v2.pdf
PWC	https://paperswithcode.com/paper/scaling-laws-for-the-principled-design
Repo
Framework

The Functional Neural Process


Title	The Functional Neural Process
Authors	Christos Louizos, Xiahan Shi, Klamer Schutte, Max Welling
Abstract	We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs). FNPs model distributions over functions by learning a graph of dependencies on top of latent representations of the points in the given dataset. In doing so, they define a Bayesian model without explicitly positing a prior distribution over latent global parameters; they instead adopt priors over the relational structure of the given dataset, a task that is much simpler. We show how we can learn such models from data, demonstrate that they are scalable to large datasets through mini-batch optimization and describe how we can make predictions for new points via their posterior predictive distribution. We experimentally evaluate FNPs on the tasks of toy regression and image classification and show that, when compared to baselines that employ global latent parameters, they offer both competitive predictions as well as more robust uncertainty estimates.
Tasks	Image Classification
Published	2019-06-19
URL	https://arxiv.org/abs/1906.08324v2
PDF	https://arxiv.org/pdf/1906.08324v2.pdf
PWC	https://paperswithcode.com/paper/the-functional-neural-process
Repo
Framework

Riconoscimento ortografico per apostrofo ed espressioni polirematiche


Title	Riconoscimento ortografico per apostrofo ed espressioni polirematiche
Authors	Massimiliano Polito
Abstract	The work presents two algorithms of manipulation and comparison between strings whose purpose is the orthographic recognition of the apostrophe and of the compound expressions. The theory supporting general reasoning refers to the basic concept of EditDistance, the improvements that ensure the achievement of the objective are achieved with the aid of tools borrowed from the use of techniques for processing large amounts of data on distributed platforms.
Tasks
Published	2019-01-31
URL	http://arxiv.org/abs/1902.00555v1
PDF	http://arxiv.org/pdf/1902.00555v1.pdf
PWC	https://paperswithcode.com/paper/riconoscimento-ortografico-per-apostrofo-ed
Repo
Framework