January 27, 2020

3087 words 15 mins read

Paper Group ANR 1222

Reducing the Model Variance of a Rectal Cancer Segmentation Network. Learning to Speak and Act in a Fantasy Text Adventure Game. An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks. Algorithms for $\ell_p$-based semi-supervised learning on graphs. From Visual to Acoustic Question Answering. Metric-based R …

Reducing the Model Variance of a Rectal Cancer Segmentation Network


Title	Reducing the Model Variance of a Rectal Cancer Segmentation Network
Authors	Joohyung Lee, Ji Eun Oh, Min Ju Kim, Bo Yun Hur, Dae Kyung Sohn
Abstract	In preoperative imaging, the demarcation of rectal cancer with magnetic resonance images provides an important basis for cancer staging and treatment planning. Recently, deep learning has greatly improved the state-of-the-art method in automatic segmentation. However, limitations in data availability in the medical field can cause large variance and consequent overfitting to medical image segmentation networks. In this study, we propose methods to reduce the model variance of a rectal cancer segmentation network by adding a rectum segmentation task and performing data augmentation; the geometric correlation between the rectum and rectal cancer motivated the former approach. Moreover, we propose a method to perform a bias-variance analysis within an arbitrary region-of-interest (ROI) of a segmentation network, which we applied to assess the efficacy of our approaches in reducing model variance. As a result, adding a rectum segmentation task reduced the model variance of the rectal cancer segmentation network within tumor regions by a factor of 0.90; data augmentation further reduced the variance by a factor of 0.89. These approaches also reduced the training duration by a factor of 0.96 and a further factor of 0.78, respectively. Our approaches will improve the quality of rectal cancer staging by increasing the accuracy of its automatic demarcation and by providing rectum boundary information since rectal cancer staging requires the demarcation of both rectum and rectal cancer. Besides such clinical benefits, our method also enables segmentation networks to be assessed with bias-variance analysis within an arbitrary ROI, such as a cancerous region.
Tasks	Data Augmentation, Medical Image Segmentation, Multi-Task Learning, Semantic Segmentation
Published	2019-01-22
URL	https://arxiv.org/abs/1901.07213v5
PDF	https://arxiv.org/pdf/1901.07213v5.pdf
PWC	https://paperswithcode.com/paper/multi-task-learning-with-a-fully
Repo
Framework

Learning to Speak and Act in a Fantasy Text Adventure Game


Title	Learning to Speak and Act in a Fantasy Text Adventure Game
Authors	Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, Jason Weston
Abstract	We introduce a large scale crowdsourced text adventure game as a research platform for studying grounded dialogue. In it, agents can perceive, emote, and act whilst conducting dialogue with other agents. Models and humans can both act as characters within the game. We describe the results of training state-of-the-art generative and retrieval models in this setting. We show that in addition to using past dialogue, these models are able to effectively use the state of the underlying world to condition their predictions. In particular, we show that grounding on the details of the local environment, including location descriptions, and the objects (and their affordances) and characters (and their previous actions) present within it allows better predictions of agent behavior and dialogue. We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.
Tasks
Published	2019-03-07
URL	http://arxiv.org/abs/1903.03094v1
PDF	http://arxiv.org/pdf/1903.03094v1.pdf
PWC	https://paperswithcode.com/paper/learning-to-speak-and-act-in-a-fantasy-text
Repo
Framework

An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks


Title	An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks
Authors	S. Ivvan Valdez, Alfonso Rojas-Domínguez
Abstract	The performance of deep neural networks, such as Deep Belief Networks formed by Restricted Boltzmann Machines (RBMs), strongly depends on their training, which is the process of adjusting their parameters. This process can be posed as an optimization problem over n dimensions. However, typical networks contain tens of thousands of parameters, making this a High-Dimensional Problem (HDP). Although different optimization methods have been employed for this goal, the use of most of the Evolutionary Algorithms (EAs) becomes prohibitive due to their inability to deal with HDPs. For instance, the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) which is regarded as one of the most effective EAs, exhibits the enormous disadvantage of requiring $O(n^2)$ memory and operations, making it unpractical for problems with more than a few hundred variables. In this paper, we introduce a novel EA that requires $O(n)$ operations and memory, but delivers competitive solutions for the training stage of RBMs with over one million variables, when compared against CMA-ES and the Contrastive Divergence algorithm, which is the standard method for training RBMs.
Tasks
Published	2019-07-12
URL	https://arxiv.org/abs/1907.05951v1
PDF	https://arxiv.org/pdf/1907.05951v1.pdf
PWC	https://paperswithcode.com/paper/an-evolutionary-algorithm-of-linear
Repo
Framework

Algorithms for $\ell_p$-based semi-supervised learning on graphs


Title	Algorithms for $\ell_p$-based semi-supervised learning on graphs
Authors	Mauricio Flores Rios, Jeff Calder, Gilad Lerman
Abstract	We develop fast algorithms for solving the variational and game-theoretic $p$-Laplace equations on weighted graphs for $p>2$. The graph $p$-Laplacian for $p>2$ has been proposed recently as a replacement for the standard ($p=2$) graph Laplacian in semi-supervised learning problems with very few labels, where the minimizer of the graph Laplacian becomes degenerate. We present several efficient and scalable algorithms for both the variational and game-theoretic formulations, and present numerical results on synthetic data and on classification and regression problems that illustrate the effectiveness of the $p$-Laplacian for semi-supervised learning with few labels.
Tasks
Published	2019-01-15
URL	http://arxiv.org/abs/1901.05031v1
PDF	http://arxiv.org/pdf/1901.05031v1.pdf
PWC	https://paperswithcode.com/paper/algorithms-for-ell_p-based-semi-supervised
Repo
Framework

From Visual to Acoustic Question Answering


Title	From Visual to Acoustic Question Answering
Authors	Jerome Abdelnour, Giampiero Salvi, Jean Rouat
Abstract	We introduce the new task of Acoustic Question Answering (AQA) to promote research in acoustic reasoning. The AQA task consists of analyzing an acoustic scene composed by a combination of elementary sounds and answering questions that relate the position and properties of these sounds. The kind of relational questions asked, require that the models perform non-trivial reasoning in order to answer correctly. Although similar problems have been extensively studied in the domain of visual reasoning, we are not aware of any previous studies addressing the problem in the acoustic domain. We propose a method for generating the acoustic scenes from elementary sounds and a number of relevant questions for each scene using templates. We also present preliminary results obtained with two models (FiLM and MAC) that have been shown to work for visual reasoning.
Tasks	Acoustic Question Answering, Question Answering, Visual Reasoning
Published	2019-02-28
URL	http://arxiv.org/abs/1902.11280v1
PDF	http://arxiv.org/pdf/1902.11280v1.pdf
PWC	https://paperswithcode.com/paper/from-visual-to-acoustic-question-answering
Repo
Framework

Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks


Title	Metric-based Regularization and Temporal Ensemble for Multi-task Learning using Heterogeneous Unsupervised Tasks
Authors	Dae Ha Kim, Seung Hyun Lee, Byung Cheol Song
Abstract	One of the ways to improve the performance of a target task is to learn the transfer of abundant knowledge of a pre-trained network. However, learning of the pre-trained network requires high computation capability and large-scale labeled dataset. To mitigate the burden of large-scale labeling, learning in un/self-supervised manner can be a solution. In addition, using unsupervised multi-task learning, a generalized feature representation can be learned. However, unsupervised multi-task learning can be biased to a specific task. To overcome this problem, we propose the metric-based regularization term and temporal task ensemble (TTE) for multi-task learning. Since these two techniques prevent the entire network from learning in a state deviated to a specific task, it is possible to learn a generalized feature representation that appropriately reflects the characteristics of each task without biasing. Experimental results for three target tasks such as classification, object detection and embedding clustering prove that the TTE-based multi-task framework is more effective than the state-of-the-art (SOTA) method in improving the performance of a target task.
Tasks	Multi-Task Learning, Object Detection
Published	2019-08-29
URL	https://arxiv.org/abs/1908.11024v1
PDF	https://arxiv.org/pdf/1908.11024v1.pdf
PWC	https://paperswithcode.com/paper/metric-based-regularization-and-temporal
Repo
Framework

Application of Deep Learning Long Short-Term Memory in Energy Demand Forecasting


Title	Application of Deep Learning Long Short-Term Memory in Energy Demand Forecasting
Authors	Nameer Al Khafaf, Mahdi Jalili, Peter Sokolowski
Abstract	The smart metering infrastructure has changed how electricity is measured in both residential and industrial application. The large amount of data collected by smart meter per day provides a huge potential for analytics to support the operation of a smart grid, an example of which is energy demand forecasting. Short term energy forecasting can be used by utilities to assess if any forecasted peak energy demand would have an adverse effect on the power system transmission and distribution infrastructure. It can also help in load scheduling and demand side management. Many techniques have been proposed to forecast time series including Support Vector Machine, Artificial Neural Network and Deep Learning. In this work we use Long Short Term Memory architecture to forecast 3-day ahead energy demand across each month in the year. The results show that 3-day ahead demand can be accurately forecasted with a Mean Absolute Percentage Error of 3.15%. In addition to that, the paper proposes way to quantify the time as a feature to be used in the training phase which is shown to affect the network performance.
Tasks	Time Series
Published	2019-03-04
URL	http://arxiv.org/abs/1903.11941v1
PDF	http://arxiv.org/pdf/1903.11941v1.pdf
PWC	https://paperswithcode.com/paper/application-of-deep-learning-long-short-term
Repo
Framework

Deep learning in ultrasound imaging


Title	Deep learning in ultrasound imaging
Authors	Ruud JG van Sloun, Regev Cohen, Yonina C Eldar
Abstract	We consider deep learning strategies in ultrasound systems, from the front-end to advanced applications. Our goal is to provide the reader with a broad understanding of the possible impact of deep learning methodologies on many aspects of ultrasound imaging. In particular, we discuss methods that lie at the interface of signal acquisition and machine learning, exploiting both data structure (e.g. sparsity in some domain) and data dimensionality (big data) already at the raw radio-frequency channel stage. As some examples, we outline efficient and effective deep learning solutions for adaptive beamforming and adaptive spectral Doppler through artificial agents, learn compressive encodings for color Doppler, and provide a framework for structured signal recovery by learning fast approximations of iterative minimization problems, with applications to clutter suppression and super-resolution ultrasound. These emerging technologies may have considerable impact on ultrasound imaging, showing promise across key components in the receive processing chain.
Tasks	Super-Resolution
Published	2019-07-05
URL	https://arxiv.org/abs/1907.02994v2
PDF	https://arxiv.org/pdf/1907.02994v2.pdf
PWC	https://paperswithcode.com/paper/deep-learning-in-ultrasound-imaging
Repo
Framework

Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship


Title	Higher Criticism for Discriminating Word-Frequency Tables and Testing Authorship
Authors	Alon Kipnis
Abstract	We adapt the Higher Criticism (HC) goodness-of-fit test to detect changes between word frequency tables. We apply the test to authorship attribution, where the goal is to identify the author of a document using other documents whose authorship is known. The method is simple yet performs well without handcrafting and tuning. As an inherent side effect, the HC calculation identifies a subset of discriminating words. In practice, the identified words have low variance across documents belonging to a corpus of homogeneous authorship. We conclude that in testing a new document against the corpus of an author, HC is mostly affected by words characteristic of that author and is relatively unaffected by topic structure.
Tasks
Published	2019-10-30
URL	https://arxiv.org/abs/1911.01208v2
PDF	https://arxiv.org/pdf/1911.01208v2.pdf
PWC	https://paperswithcode.com/paper/higher-criticism-for-discriminating-word
Repo
Framework

Fast Neural Network Predictions from Constrained Aerodynamics Datasets


Title	Fast Neural Network Predictions from Constrained Aerodynamics Datasets
Authors	Cristina White, Daniela Ushizima, Charbel Farhat
Abstract	Incorporating computational fluid dynamics in the design process of jets, spacecraft, or gas turbine engines is often challenged by the required computational resources and simulation time, which depend on the chosen physics-based computational models and grid resolutions. An ongoing problem in the field is how to simulate these systems faster but with sufficient accuracy. While many approaches involve simplified models of the underlying physics, others are model-free and make predictions based only on existing simulation data. We present a novel model-free approach in which we reformulate the simulation problem to effectively increase the size of constrained pre-computed datasets and introduce a novel neural network architecture (called a cluster network) with an inductive bias well-suited to highly nonlinear computational fluid dynamics solutions. Compared to the state-of-the-art in model-based approximations, we show that our approach is nearly as accurate, an order of magnitude faster, and easier to apply. Furthermore, we show that our method outperforms other model-free approaches.
Tasks
Published	2019-01-26
URL	https://arxiv.org/abs/1902.00091v5
PDF	https://arxiv.org/pdf/1902.00091v5.pdf
PWC	https://paperswithcode.com/paper/neural-networks-predict-fluid-dynamics
Repo
Framework

OASIS: ILP-Guided Synthesis of Loop Invariants


Title	OASIS: ILP-Guided Synthesis of Loop Invariants
Authors	Sahil Bhatia, Saswat Padhi, Nagarajan Natarajan, Rahul Sharma, Prateek Jain
Abstract	Finding appropriate inductive loop invariants for a program is a key challenge in verifying its functional properties. Although the problem is undecidable in general, several heuristics have been proposed to handle practical programs that tend to have simple control-flow structures. However, these heuristics only work well when the space of invariants is small. On the other hand, machine-learned techniques that use continuous optimization have a high sample complexity, i.e., the number of invariant guesses and the associated counterexamples, since the invariant is required to exactly satisfy a specification. We propose a novel technique that is able to solve complex verification problems involving programs with larger number of variables and non-linear specifications. We formulate an invariant as a piecewise low-degree polynomial, and reduce the problem of synthesizing it to a set of integer linear programming (ILP) problems. This enables the use of state-of-the-art ILP techniques that combine enumerative search with continuous optimization; thus ensuring fast convergence for a large class of verification tasks while still ensuring low sample complexity. We instantiate our technique as the open-source oasis tool using an off-the-shelf ILP solver, and evaluate it on more than 300 benchmark tasks collected from the annual SyGuS competition and recent prior work. Our experiments show that oasis outperforms the state-of-the-art tools, including the winner of last year’s SyGuS competition, and is able to solve 9 challenging tasks that existing tools fail on.
Tasks
Published	2019-11-26
URL	https://arxiv.org/abs/1911.11728v1
PDF	https://arxiv.org/pdf/1911.11728v1.pdf
PWC	https://paperswithcode.com/paper/oasis-ilp-guided-synthesis-of-loop-invariants
Repo
Framework

Beamforming Learning for mmWave Communication: Theory and Experimental Validation


Title	Beamforming Learning for mmWave Communication: Theory and Experimental Validation
Authors	ohaned Chraiti, Dmitry Chizhik, Jinfeng Du, Reinaldo A. Valenzuela, Ali Ghrayeb, Chadi Assi
Abstract	To establish reliable and long-range millimeter-wave (mmWave) communication, beamforming is deemed to be a promising solution. Although beamforming can be done in the digital and analog domains, both approaches are hindered by several constraints when it comes to mmWave communications. For example, performing fully digital beamforming in mmWave systems involves using many radio frequency (RF) chains, which are expensive and consume high power. This necessitates finding more efficient ways for using fewer RF chains while taking advantage of the large antenna arrays. One way to overcome this challenge is to employ (partially or fully) analog beamforming through proper configuration of phase-shifters. Existing works on mmWave analog beam design either rely on the knowledge of the channel state information (CSI) per antenna within the array, require a large search time (e.g., exhaustive search) or do not guarantee a minimum beamforming gain (e.g., codebook based beamforming). In this paper, we propose a beam design technique that reduces the search time and does not require CSI while guaranteeing a minimum beamforming gain. The key idea derives from observations drawn from real-life measurements. It was observed that for a given propagation environment (e.g., coverage area of a mmWave BS) the azimuthal angles of dominant signals could be more probable from certain angles than others. Thus, pre-collected measurements could used to build a beamforming codebook that regroups the most probable beam designs. We invoke Bayesian learning for measurements clustering. We evaluate the efficacy of the proposed scheme in terms of building the codebook and assessing its performance through real-life measurements. We demonstrate that the training time required by the proposed scheme is only 5% of that of exhaustive search. This crucial gain is obtained while achieving a minimum targeted beamforming gain.
Tasks
Published	2019-12-28
URL	https://arxiv.org/abs/1912.12406v1
PDF	https://arxiv.org/pdf/1912.12406v1.pdf
PWC	https://paperswithcode.com/paper/beamforming-learning-for-mmwave-communication
Repo
Framework

Crack Detection Using Enhanced Hierarchical Convolutional Neural Networks


Title	Crack Detection Using Enhanced Hierarchical Convolutional Neural Networks
Authors	Qiuchen Zhu, Manh Duong Phung, Quang Ha
Abstract	Unmanned aerial vehicles (UAV) are expected to replace human in hazardous tasks of surface inspection due to their flexibility in operating space and capability of collecting high quality visual data. In this study, we propose enhanced hierarchical convolutional neural networks (HCNN) to detect cracks from image data collected by UAVs. Unlike traditional HCNN, here a set of branch networks is utilised to reduce the obscuration in the down-sampling process. Moreover, the feature preserving blocks combine the current and previous terms from the convolutional blocks to provide input to the loss functions. As a result, the weights of resized images can be reduced to minimise the information loss. Experiments on images of different crack datasets have been carried out to demonstrate the effectiveness of proposed HCNN.
Tasks
Published	2019-12-13
URL	https://arxiv.org/abs/1912.12139v1
PDF	https://arxiv.org/pdf/1912.12139v1.pdf
PWC	https://paperswithcode.com/paper/crack-detection-using-enhanced-hierarchical
Repo
Framework

What can the brain teach us about building artificial intelligence?


Title	What can the brain teach us about building artificial intelligence?
Authors	Dileep George
Abstract	This paper is the preprint of an invited commentary on Lake et al’s Behavioral and Brain Sciences article titled “Building machines that learn and think like people”. Lake et al’s paper offers a timely critique on the recent accomplishments in artificial intelligence from the vantage point of human intelligence, and provides insightful suggestions about research directions for building more human-like intelligence. Since we agree with most of the points raised in that paper, we will offer a few points that are complementary.
Tasks
Published	2019-09-04
URL	https://arxiv.org/abs/1909.01561v1
PDF	https://arxiv.org/pdf/1909.01561v1.pdf
PWC	https://paperswithcode.com/paper/what-can-the-brain-teach-us-about-building
Repo
Framework

Sideways Transliteration: How to Transliterate Multicultural Person Names?


Title	Sideways Transliteration: How to Transliterate Multicultural Person Names?
Authors	Raphael Cohen, Michael Elhadad
Abstract	In a global setting, texts contain transliterated names from many cultural origins. Correct transliteration depends not only on target and source languages but also, on the source language of the name. We introduce a novel methodology for transliteration of names originating in different languages using only monolingual resources. Our method is based on a step of noisy transliteration and then ranking of the results based on origin specific letter models. The transliteration table used for noisy generation is learned in an unsupervised manner for each possible origin language. We present a solution for gathering monolingual training data used by our method by mining of social media sites such as Facebook and Wikipedia. We present results in the context of transliterating from English to Hebrew and provide an online web service for transliteration from English to Hebrew
Tasks	Transliteration
Published	2019-11-27
URL	https://arxiv.org/abs/1911.12022v1
PDF	https://arxiv.org/pdf/1911.12022v1.pdf
PWC	https://paperswithcode.com/paper/sideways-transliteration-how-to-transliterate
Repo
Framework