Paper Group ANR 819
What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks. DSIC: Deep Stereo Image Compression. Human Perceptual Evaluations for Image Compression. Joint embedding of structure and features via graph convolutional networks. Multiphase flow prediction with deep neural networks. Combining Model and Parameter Unc …
What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks
| Title | What is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks |
| Authors | Shashi Narayan, Shay B. Cohen, Mirella Lapata |
| Abstract | We introduce ‘extreme summarization’, a new single-document summarization task which aims at creating a short, one-sentence news summary answering the question ``What is the article about?''. We argue that extreme summarization, by nature, is not amenable to extractive strategies and requires an abstractive modeling approach. In the hope of driving research on this task further: (a) we collect a real-world, large scale dataset by harvesting online articles from the British Broadcasting Corporation (BBC); and (b) propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. We demonstrate experimentally that this architecture captures long-range dependencies in a document and recognizes pertinent content, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans on the extreme summarization dataset. | |
| Tasks | Document Summarization |
| Published | 2019-07-19 |
| URL | https://arxiv.org/abs/1907.08722v1 |
| https://arxiv.org/pdf/1907.08722v1.pdf | |
| PWC | https://paperswithcode.com/paper/what-is-this-article-about-extreme |
| Repo | |
| Framework | |
DSIC: Deep Stereo Image Compression
| Title | DSIC: Deep Stereo Image Compression |
| Authors | Jerry Liu, Shenlong Wang, Raquel Urtasun |
| Abstract | In this paper we tackle the problem of stereo image compression, and leverage the fact that the two images have overlapping fields of view to further compress the representations. Our approach leverages state-of-the-art single-image compression autoencoders and enhances the compression with novel parametric skip functions to feed fully differentiable, disparity-warped features at all levels to the encoder/decoder of the second image. Moreover, we model the probabilistic dependence between the image codes using a conditional entropy model. Our experiments show an impressive 30 - 50% reduction in the second image bitrate at low bitrates compared to deep single-image compression, and a 10 - 20% reduction at higher bitrates. |
| Tasks | Image Compression |
| Published | 2019-08-09 |
| URL | https://arxiv.org/abs/1908.03631v1 |
| https://arxiv.org/pdf/1908.03631v1.pdf | |
| PWC | https://paperswithcode.com/paper/dsic-deep-stereo-image-compression |
| Repo | |
| Framework | |
Human Perceptual Evaluations for Image Compression
| Title | Human Perceptual Evaluations for Image Compression |
| Authors | Yash Patel, Srikar Appalaraju, R. Manmatha |
| Abstract | Recently, there has been much interest in deep learning techniques to do image compression and there have been claims that several of these produce better results than engineered compression schemes (such as JPEG, JPEG2000 or BPG). A standard way of comparing image compression schemes today is to use perceptual similarity metrics such as PSNR or MS-SSIM (multi-scale structural similarity). This has led to some deep learning techniques which directly optimize for MS-SSIM by choosing it as a loss function. While this leads to a higher MS-SSIM for such techniques, we demonstrate using user studies that the resulting improvement may be misleading. Deep learning techniques for image compression with a higher MS-SSIM may actually be perceptually worse than engineered compression schemes with a lower MS-SSIM. |
| Tasks | Image Compression |
| Published | 2019-08-09 |
| URL | https://arxiv.org/abs/1908.04187v1 |
| https://arxiv.org/pdf/1908.04187v1.pdf | |
| PWC | https://paperswithcode.com/paper/human-perceptual-evaluations-for-image |
| Repo | |
| Framework | |
Joint embedding of structure and features via graph convolutional networks
| Title | Joint embedding of structure and features via graph convolutional networks |
| Authors | Sébastien Lerique, Jacob Levy Abitbol, Márton Karsai |
| Abstract | The creation of social ties is largely determined by the entangled effects of people’s similarities in terms of individual characters and friends. However, feature and structural characters of people usually appear to be correlated, making it difficult to determine which has greater responsibility in the formation of the emergent network structure. We propose \emph{AN2VEC}, a node embedding method which ultimately aims at disentangling the information shared by the structure of a network and the features of its nodes. Building on the recent developments of Graph Convolutional Networks (GCN), we develop a multitask GCN Variational Autoencoder where different dimensions of the generated embeddings can be dedicated to encoding feature information, network structure, and shared feature-network information. We explore the interaction between these disentangled characters by comparing the embedding reconstruction performance to a baseline case where no shared information is extracted. We use synthetic datasets with different levels of interdependency between feature and network characters and show (i) that shallow embeddings relying on shared information perform better than the corresponding reference with unshared information, (ii) that this performance gap increases with the correlation between network and feature structure, and (iii) that our embedding is able to capture joint information of structure and features. Our method can be relevant for the analysis and prediction of any featured network structure ranging from online social systems to network medicine. |
| Tasks | |
| Published | 2019-05-21 |
| URL | https://arxiv.org/abs/1905.08636v4 |
| https://arxiv.org/pdf/1905.08636v4.pdf | |
| PWC | https://paperswithcode.com/paper/joint-embedding-of-structure-and-features-via |
| Repo | |
| Framework | |
Multiphase flow prediction with deep neural networks
| Title | Multiphase flow prediction with deep neural networks |
| Authors | Gege Wen, Meng Tang, Sally M. Benson |
| Abstract | This paper proposes a deep neural network approach for predicting multiphase flow in heterogeneous domains with high computational efficiency. The deep neural network model is able to handle permeability heterogeneity in high dimensional systems, and can learn the interplay of viscous, gravity, and capillary forces from small data sets. Using the example of carbon dioxide (CO2) storage, we demonstrate that the model can generate highly accurate predictions of a CO2 saturation distribution given a permeability field, injection duration, injection rate, and injection location. The trained neural network model has an excellent ability to interpolate and to a limited extent, the ability to extrapolate beyond the training data ranges. To improve the prediction accuracy when the neural network model needs to extrapolate, we propose a transfer learning (fine-tuning) procedure that can quickly teach the neural network model new information without going through massive data collection and retraining. Based on this trained neural network model, a web-based tool is provided that allows users to perform CO2-water multiphase flow calculations online. With the tools provided in this paper, the deep neural network approach can provide a computationally efficient substitute for repetitive forward multiphase flow simulations, which can be adopted to the context of history matching and uncertainty quantification. |
| Tasks | Transfer Learning |
| Published | 2019-10-21 |
| URL | https://arxiv.org/abs/1910.09657v1 |
| https://arxiv.org/pdf/1910.09657v1.pdf | |
| PWC | https://paperswithcode.com/paper/multiphase-flow-prediction-with-deep-neural |
| Repo | |
| Framework | |
Combining Model and Parameter Uncertainty in Bayesian Neural Networks
| Title | Combining Model and Parameter Uncertainty in Bayesian Neural Networks |
| Authors | Aliaksandr Hubin, Geir Storvik |
| Abstract | Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs. |
| Tasks | Bayesian Inference, Model Selection |
| Published | 2019-03-18 |
| URL | https://arxiv.org/abs/1903.07594v3 |
| https://arxiv.org/pdf/1903.07594v3.pdf | |
| PWC | https://paperswithcode.com/paper/combining-model-and-parameter-uncertainty-in |
| Repo | |
| Framework | |
Task-assisted Motion Planning in Partially Observable Domains
| Title | Task-assisted Motion Planning in Partially Observable Domains |
| Authors | Antony Thomas, Sunny Amatya, Fulvio Mastrogiovanni, Marco Baglietto |
| Abstract | We present an integrated Task-Motion Planning framework for robot navigation in belief space. Autonomous robots operating in real world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. To this end, we propose a framework for integrating belief space reasoning within a hybrid task planner. The expressive power of PDDL+ combined with heuristic-driven semantic attachments performs the propagated and posterior belief estimates while planning. The underlying methodology for the development of the combined hybrid planner is discussed, providing suggestions for improvements and future work. Furthermore we validate key aspects of our approach using a realistic scenario in simulation. |
| Tasks | Motion Planning, Robot Navigation |
| Published | 2019-08-27 |
| URL | https://arxiv.org/abs/1908.10227v1 |
| https://arxiv.org/pdf/1908.10227v1.pdf | |
| PWC | https://paperswithcode.com/paper/task-assisted-motion-planning-in-partially |
| Repo | |
| Framework | |
Transfer Learning for Sequences via Learning to Collocate
| Title | Transfer Learning for Sequences via Learning to Collocate |
| Authors | Wanyun Cui, Guangyu Zheng, Zhiqiang Shen, Sihang Jiang, Wei Wang |
| Abstract | Transfer learning aims to solve the data sparsity for a target domain by applying information of the source domain. Given a sequence (e.g. a natural language sentence), the transfer learning, usually enabled by recurrent neural network (RNN), represents the sequential information transfer. RNN uses a chain of repeating cells to model the sequence data. However, previous studies of neural network based transfer learning simply represents the whole sentence by a single vector, which is unfeasible for seq2seq and sequence labeling. Meanwhile, such layer-wise transfer learning mechanisms lose the fine-grained cell-level information from the source domain. In this paper, we proposed the aligned recurrent transfer, ART, to achieve cell-level information transfer. ART is under the pre-training framework. Each cell attentively accepts transferred information from a set of positions in the source domain. Therefore, ART learns the cross-domain word collocations in a more flexible way. We conducted extensive experiments on both sequence labeling tasks (POS tagging, NER) and sentence classification (sentiment analysis). ART outperforms the state-of-the-arts over all experiments. |
| Tasks | Sentence Classification, Sentiment Analysis, Transfer Learning |
| Published | 2019-02-25 |
| URL | http://arxiv.org/abs/1902.09092v1 |
| http://arxiv.org/pdf/1902.09092v1.pdf | |
| PWC | https://paperswithcode.com/paper/transfer-learning-for-sequences-via-learning |
| Repo | |
| Framework | |
Fully Convolutional Network for Removing DCT Artefacts From Images
| Title | Fully Convolutional Network for Removing DCT Artefacts From Images |
| Authors | Patryk Najgebauer, Rafal Scherer |
| Abstract | Deep learning methods achieve excellent results in image transformations as well as image noise reduction or super-resolution methods. Based on these solutions, we present a deep-learning method of block reconstruction of images compressed with the JPEG format. Images compressed with the discrete cosine transform (DCT) contain visible artefacts in the form of blocks, which in some cases spoil the aesthetics of the image mostly on the edges of the contrasting elements. This is unavoidable, and the discernibility of the block artefacts can be adjusted by the degree of image compression, which profoundly affects the output image size. We use a fully convolutional network which operates directly on 8x8-pixel blocks in the same way as the JPEG encoder. Thanks to that, we do not modify the input image; we only divide it into separately processed blocks. The purpose of our neural model is to modify the pixels in the blocks to reduce artefact visibility %against the background of the neighbouring image and to recreate the original pattern of the image distorted by the DCT transform. We trained our model on a dataset created from vector images transformed to the JPEG and PNG formats, as the input and output data, respectively. |
| Tasks | Image Compression, Super-Resolution |
| Published | 2019-07-08 |
| URL | https://arxiv.org/abs/1907.03798v1 |
| https://arxiv.org/pdf/1907.03798v1.pdf | |
| PWC | https://paperswithcode.com/paper/fully-convolutional-network-for-removing-dct |
| Repo | |
| Framework | |
Twins Recognition with Multi Biometric System: Handcrafted-Deep Learning Based Multi Algorithm with Voice-Ear Recognition Based Multi Modal
| Title | Twins Recognition with Multi Biometric System: Handcrafted-Deep Learning Based Multi Algorithm with Voice-Ear Recognition Based Multi Modal |
| Authors | Cihan Akın, Umit Kacar, Murvet Kirci |
| Abstract | With the development of technology, the usage areas and importance of biometric systems have started to increase. Since the characteristics of each person are different from each other, a single model biometric system can yield successful results. However, because the characteristics of twin people are very close to each other, multiple biometric systems including multiple characteristics of individuals will be more appropriate and will increase the recognition rate. In this study, a multiple biometric recognition system consisting of a combination of multiple algorithms and multiple models was developed to distinguish people from other people and their twins. Ear and voice biometric data were used for the multimodal model and 38 pair of twin ear images and sound recordings were used in the data set. Sound and ear recognition rates were obtained using classical (hand-crafted) and deep learning algorithms. The results obtained were combined with the score level fusion method to achieve a success rate of 94.74% in rank-1 and 100% in rank -2. |
| Tasks | |
| Published | 2019-03-15 |
| URL | http://arxiv.org/abs/1903.07981v1 |
| http://arxiv.org/pdf/1903.07981v1.pdf | |
| PWC | https://paperswithcode.com/paper/twins-recognition-with-multi-biometric-system |
| Repo | |
| Framework | |
Task-Guided Pair Embedding in Heterogeneous Network
| Title | Task-Guided Pair Embedding in Heterogeneous Network |
| Authors | Chanyoung Park, Donghyun Kim, Qi Zhu, Jiawei Han, Hwanjo Yu |
| Abstract | Many real-world tasks solved by heterogeneous network embedding methods can be cast as modeling the likelihood of pairwise relationship between two nodes. For example, the goal of author identification task is to model the likelihood of a paper being written by an author (paper-author pairwise relationship). Existing task-guided embedding methods are node-centric in that they simply measure the similarity between the node embeddings to compute the likelihood of a pairwise relationship between two nodes. However, we claim that for task-guided embeddings, it is crucial to focus on directly modeling the pairwise relationship. In this paper, we propose a novel task-guided pair embedding framework in heterogeneous network, called TaPEm, that directly models the relationship between a pair of nodes that are related to a specific task (e.g., paper-author relationship in author identification). To this end, we 1) propose to learn a pair embedding under the guidance of its associated context path, i.e., a sequence of nodes between the pair, and 2) devise the pair validity classifier to distinguish whether the pair is valid with respect to the specific task at hand. By introducing pair embeddings that capture the semantics behind the pairwise relationships, we are able to learn the fine-grained pairwise relationship between two nodes, which is paramount for task-guided embedding methods. Extensive experiments on author identification task demonstrate that TaPEm outperforms the state-of-the-art methods, especially for authors with few publication records. |
| Tasks | Network Embedding |
| Published | 2019-06-04 |
| URL | https://arxiv.org/abs/1906.01546v3 |
| https://arxiv.org/pdf/1906.01546v3.pdf | |
| PWC | https://paperswithcode.com/paper/task-guided-pair-embedding-in-heterogeneous |
| Repo | |
| Framework | |
Empirical Study of Diachronic Word Embeddings for Scarce Data
| Title | Empirical Study of Diachronic Word Embeddings for Scarce Data |
| Authors | Syrielle Montariol, Alexandre Allauzen |
| Abstract | Word meaning change can be inferred from drifts of time-varying word embeddings. However, temporal data may be too sparse to build robust word embeddings and to discriminate significant drifts from noise. In this paper, we compare three models to learn diachronic word embeddings on scarce data: incremental updating of a Skip-Gram from Kim et al. (2014), dynamic filtering from Bamler and Mandt (2017), and dynamic Bernoulli embeddings from Rudolph and Blei (2018). In particular, we study the performance of different initialisation schemes and emphasise what characteristics of each model are more suitable to data scarcity, relying on the distribution of detected drifts. Finally, we regularise the loss of these models to better adapt to scarce data. |
| Tasks | Word Embeddings |
| Published | 2019-09-04 |
| URL | https://arxiv.org/abs/1909.01863v1 |
| https://arxiv.org/pdf/1909.01863v1.pdf | |
| PWC | https://paperswithcode.com/paper/empirical-study-of-diachronic-word-embeddings |
| Repo | |
| Framework | |
Hydra: A method for strain-minimizing hyperbolic embedding of network- and distance-based data
| Title | Hydra: A method for strain-minimizing hyperbolic embedding of network- and distance-based data |
| Authors | Martin Keller-Ressel, Stephanie Nargang |
| Abstract | We introduce hydra (hyperbolic distance recovery and approximation), a new method for embedding network- or distance-based data into hyperbolic space. We show mathematically that hydra satisfies a certain optimality guarantee: It minimizes the `hyperbolic strain’ between original and embedded data points. Moreover, it recovers points exactly, when they are located on a hyperbolic submanifold of the feature space. Testing on real network data we show that the embedding quality of hydra is competitive with existing hyperbolic embedding methods, but achieved at substantially shorter computation time. An extended method, termed hydra+, outperforms existing methods in both computation time and embedding quality. | |
| Tasks | |
| Published | 2019-03-21 |
| URL | https://arxiv.org/abs/1903.08977v2 |
| https://arxiv.org/pdf/1903.08977v2.pdf | |
| PWC | https://paperswithcode.com/paper/hydra-a-method-for-strain-minimizing |
| Repo | |
| Framework | |
Long-distance Detection of Bioacoustic Events with Per-channel Energy Normalization
| Title | Long-distance Detection of Bioacoustic Events with Per-channel Energy Normalization |
| Authors | Vincent Lostanlen, Kaitlin Palmer, Elly Knight, Christopher Clark, Holger Klinck, Andrew Farnsworth, Tina Wong, Jason Cramer, Juan Pablo Bello |
| Abstract | This paper proposes to perform unsupervised detection of bioacoustic events by pooling the magnitudes of spectrogram frames after per-channel energy normalization (PCEN). Although PCEN was originally developed for speech recognition, it also has beneficial effects in enhancing animal vocalizations, despite the presence of atmospheric absorption and intermittent noise. We prove that PCEN generalizes logarithm-based spectral flux, yet with a tunable time scale for background noise estimation. In comparison with pointwise logarithm, PCEN reduces false alarm rate by 50x in the near field and 5x in the far field, both on avian and marine bioacoustic datasets. Such improvements come at moderate computational cost and require no human intervention, thus heralding a promising future for PCEN in bioacoustics. |
| Tasks | Speech Recognition |
| Published | 2019-11-01 |
| URL | https://arxiv.org/abs/1911.00417v1 |
| https://arxiv.org/pdf/1911.00417v1.pdf | |
| PWC | https://paperswithcode.com/paper/long-distance-detection-of-bioacoustic-events |
| Repo | |
| Framework | |
Evolving Self-supervised Neural Networks: Autonomous Intelligence from Evolved Self-teaching
| Title | Evolving Self-supervised Neural Networks: Autonomous Intelligence from Evolved Self-teaching |
| Authors | Nam Le |
| Abstract | This paper presents a technique called evolving self-supervised neural networks - neural networks that can teach themselves, intrinsically motivated, without external supervision or reward. The proposed method presents some sort-of paradigm shift, and differs greatly from both traditional gradient-based learning and evolutionary algorithms in that it combines the metaphor of evolution and learning, more specifically self-learning, together, rather than treating these phenomena alternatively. I simulate a multi-agent system in which neural networks are used to control autonomous foraging agents with little domain knowledge. Experimental results show that only evolved self-supervised agents can demonstrate some sort of intelligent behaviour, but not evolution or self-learning alone. Indications for future work on evolving intelligence are also presented. |
| Tasks | |
| Published | 2019-05-27 |
| URL | https://arxiv.org/abs/1906.08865v1 |
| https://arxiv.org/pdf/1906.08865v1.pdf | |
| PWC | https://paperswithcode.com/paper/evolving-self-supervised-neural-networks |
| Repo | |
| Framework | |
