Paper Group ANR 207
How far from automatically interpreting deep learning. Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms. A telescoping Bregmanian proximal gradient method without the global Lipschitz continuity assumption. The loss surface of deep linear networks viewed through the algebraic geometry lens. Gene …
How far from automatically interpreting deep learning
| Title | How far from automatically interpreting deep learning |
| Authors | Jinwei Zhao, Qizhou Wang, Yufei Wang, Xinhong Hei, Yu Liu |
| Abstract | In recent years, deep learning researchers have focused on how to find the interpretability behind deep learning models. However, today cognitive competence of human has not completely covered the deep learning model. In other words, there is a gap between the deep learning model and the cognitive mode. How to evaluate and shrink the cognitive gap is a very important issue. In this paper, the interpretability evaluation, the relationship between the generalization performance and the interpretability of the model and the method for improving the interpretability are concerned. A universal learning framework is put forward to solve the equilibrium problem between the two performances. The uniqueness of solution of the problem is proved and condition of unique solution is obtained. Probability upper bound of the sum of the two performances is analyzed. |
| Tasks | |
| Published | 2018-11-19 |
| URL | http://arxiv.org/abs/1811.07747v1 |
| http://arxiv.org/pdf/1811.07747v1.pdf | |
| PWC | https://paperswithcode.com/paper/how-far-from-automatically-interpreting-deep |
| Repo | |
| Framework | |
Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms
| Title | Exploring the Use of Attention within an Neural Machine Translation Decoder States to Translate Idioms |
| Authors | Giancarlo D. Salton, Robert J. Ross, John D. Kelleher |
| Abstract | Idioms pose problems to almost all Machine Translation systems. This type of language is very frequent in day-to-day language use and cannot be simply ignored. The recent interest in memory augmented models in the field of Language Modelling has aided the systems to achieve good results by bridging long-distance dependencies. In this paper we explore the use of such techniques into a Neural Machine Translation system to help in translation of idiomatic language. |
| Tasks | Language Modelling, Machine Translation |
| Published | 2018-10-10 |
| URL | http://arxiv.org/abs/1810.06695v1 |
| http://arxiv.org/pdf/1810.06695v1.pdf | |
| PWC | https://paperswithcode.com/paper/exploring-the-use-of-attention-within-an |
| Repo | |
| Framework | |
A telescoping Bregmanian proximal gradient method without the global Lipschitz continuity assumption
| Title | A telescoping Bregmanian proximal gradient method without the global Lipschitz continuity assumption |
| Authors | Daniel Reem, Simeon Reich, Alvaro De Pierro |
| Abstract | The problem of minimization of the sum of two convex functions has various theoretical and real-world applications. One of the popular methods for solving this problem is the proximal gradient method (proximal forward-backward algorithm). A very common assumption in the use of this method is that the gradient of the smooth term is globally Lipschitz continuous. However, this assumption is not always satisfied in practice, thus casting a limitation on the method. In this paper, we discuss, in a wide class of finite and infinite-dimensional spaces, a new variant of the proximal gradient method which does not impose the above-mentioned global Lipschitz continuity assumption. A key contribution of the method is the dependence of the iterative steps on a certain telescopic decomposition of the constraint set into subsets. Moreover, we use a Bregman divergence in the proximal forward-backward operation. Under certain practical conditions, a non-asymptotic rate of convergence (that is, in the function values) is established, as well as the weak convergence of the whole sequence to a minimizer. We also obtain a few auxiliary results of independent interest. |
| Tasks | |
| Published | 2018-04-19 |
| URL | http://arxiv.org/abs/1804.10273v4 |
| http://arxiv.org/pdf/1804.10273v4.pdf | |
| PWC | https://paperswithcode.com/paper/a-telescoping-bregmanian-proximal-gradient |
| Repo | |
| Framework | |
The loss surface of deep linear networks viewed through the algebraic geometry lens
| Title | The loss surface of deep linear networks viewed through the algebraic geometry lens |
| Authors | Dhagash Mehta, Tianran Chen, Tingting Tang, Jonathan D. Hauenstein |
| Abstract | By using the viewpoint of modern computational algebraic geometry, we explore properties of the optimization landscapes of the deep linear neural network models. After clarifying on the various definitions of “flat” minima, we show that the geometrically flat minima, which are merely artifacts of residual continuous symmetries of the deep linear networks, can be straightforwardly removed by a generalized $L_2$ regularization. Then, we establish upper bounds on the number of isolated stationary points of these networks with the help of algebraic geometry. Using these upper bounds and utilizing a numerical algebraic geometry method, we find all stationary points of modest depth and matrix size. We show that in the presence of the non-zero regularization, deep linear networks indeed possess local minima which are not the global minima. Our computational results clarify certain aspects of the loss surfaces of deep linear networks and provide novel insights. |
| Tasks | |
| Published | 2018-10-17 |
| URL | http://arxiv.org/abs/1810.07716v1 |
| http://arxiv.org/pdf/1810.07716v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-loss-surface-of-deep-linear-networks |
| Repo | |
| Framework | |
Generalized Latent Variable Recovery for Generative Adversarial Networks
| Title | Generalized Latent Variable Recovery for Generative Adversarial Networks |
| Authors | Nicholas Egan, Jeffrey Zhang, Kevin Shen |
| Abstract | The Generator of a Generative Adversarial Network (GAN) is trained to transform latent vectors drawn from a prior distribution into realistic looking photos. These latent vectors have been shown to encode information about the content of their corresponding images. Projecting input images onto the latent space of a GAN is non-trivial, but previous work has successfully performed this task for latent spaces with a uniform prior. We extend these techniques to latent spaces with a Gaussian prior, and demonstrate our technique’s effectiveness. |
| Tasks | |
| Published | 2018-10-09 |
| URL | http://arxiv.org/abs/1810.03764v1 |
| http://arxiv.org/pdf/1810.03764v1.pdf | |
| PWC | https://paperswithcode.com/paper/generalized-latent-variable-recovery-for |
| Repo | |
| Framework | |
A review of neuro-fuzzy systems based on intelligent control
| Title | A review of neuro-fuzzy systems based on intelligent control |
| Authors | Fatemeh Zahedi, Zahra Zahedi |
| Abstract | The system’s ability to adapt and self-organize are two key factors when it comes to how well the system can survive the changes to the environment and the plant they work within. Intelligent control improves these two factors in controllers. Considering the increasing complexity of dynamic systems along with their need for feedback controls, using more complicated controls has become necessary and intelligent control can be a suitable response to this necessity. This paper briefly describes the structure of intelligent control and provides a review on fuzzy logic and neural networks which are some of the base methods for intelligent control. The different aspects of these two methods are then compared together and an example of a combined method is presented. |
| Tasks | |
| Published | 2018-05-06 |
| URL | http://arxiv.org/abs/1805.03138v1 |
| http://arxiv.org/pdf/1805.03138v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-review-of-neuro-fuzzy-systems-based-on |
| Repo | |
| Framework | |
Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning
| Title | Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning |
| Authors | Benjamin Kellenberger, Diego Marcos, Devis Tuia |
| Abstract | Knowledge over the number of animals in large wildlife reserves is a vital necessity for park rangers in their efforts to protect endangered species. Manual animal censuses are dangerous and expensive, hence Unmanned Aerial Vehicles (UAVs) with consumer level digital cameras are becoming a popular alternative tool to estimate livestock. Several works have been proposed that semi-automatically process UAV images to detect animals, of which some employ Convolutional Neural Networks (CNNs), a recent family of deep learning algorithms that proved very effective in object detection in large datasets from computer vision. However, the majority of works related to wildlife focuses only on small datasets (typically subsets of UAV campaigns), which might be detrimental when presented with the sheer scale of real study areas for large mammal census. Methods may yield thousands of false alarms in such cases. In this paper, we study how to scale CNNs to large wildlife census tasks and present a number of recommendations to train a CNN on a large UAV dataset. We further introduce novel evaluation protocols that are tailored to censuses and model suitability for subsequent human verification of detections. Using our recommendations, we are able to train a CNN reducing the number of false positives by an order of magnitude compared to previous state-of-the-art. Setting the requirements at 90% recall, our CNN allows to reduce the amount of data required for manual verification by three times, thus making it possible for rangers to screen all the data acquired efficiently and to detect almost all animals in the reserve automatically. |
| Tasks | Object Detection |
| Published | 2018-06-29 |
| URL | http://arxiv.org/abs/1806.11368v1 |
| http://arxiv.org/pdf/1806.11368v1.pdf | |
| PWC | https://paperswithcode.com/paper/detecting-mammals-in-uav-images-best |
| Repo | |
| Framework | |
Techniques for Interpretable Machine Learning
| Title | Techniques for Interpretable Machine Learning |
| Authors | Mengnan Du, Ninghao Liu, Xia Hu |
| Abstract | Interpretable machine learning tackles the important problem that humans cannot understand the behaviors of complex machine learning models and how these models arrive at a particular decision. Although many approaches have been proposed, a comprehensive understanding of the achievements and challenges is still lacking. We provide a survey covering existing techniques to increase the interpretability of machine learning models. We also discuss crucial issues that the community should consider in future work such as designing user-friendly explanations and developing comprehensive evaluation metrics to further push forward the area of interpretable machine learning. |
| Tasks | Interpretable Machine Learning |
| Published | 2018-07-31 |
| URL | https://arxiv.org/abs/1808.00033v3 |
| https://arxiv.org/pdf/1808.00033v3.pdf | |
| PWC | https://paperswithcode.com/paper/techniques-for-interpretable-machine-learning |
| Repo | |
| Framework | |
Towards Abstraction in ASP with an Application on Reasoning about Agent Policies
| Title | Towards Abstraction in ASP with an Application on Reasoning about Agent Policies |
| Authors | Zeynep G. Saribatur, Thomas Eiter |
| Abstract | ASP programs are a convenient tool for problem solving, whereas with large problem instances the size of the state space can be prohibitive. We consider abstraction as a means of over-approximation and introduce a method to automatically abstract (possibly non-ground) ASP programs that preserves their structure, while reducing the size of the problem. One particular application case is the problem of defining declarative policies for reactive agents and reasoning about them, which we illustrate on examples. |
| Tasks | |
| Published | 2018-09-18 |
| URL | http://arxiv.org/abs/1809.06638v1 |
| http://arxiv.org/pdf/1809.06638v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-abstraction-in-asp-with-an |
| Repo | |
| Framework | |
On the Complexity of Detecting Convexity over a Box
| Title | On the Complexity of Detecting Convexity over a Box |
| Authors | Amir Ali Ahmadi, Georgina Hall |
| Abstract | It has recently been shown that the problem of testing global convexity of polynomials of degree four is {strongly} NP-hard, answering an open question of N.Z. Shor. This result is minimal in the degree of the polynomial when global convexity is of concern. In a number of applications however, one is interested in testing convexity only over a compact region, most commonly a box (i.e., hyper-rectangle). In this paper, we show that this problem is also strongly NP-hard, in fact for polynomials of degree as low as three. This result is minimal in the degree of the polynomial and in some sense justifies why convexity detection in nonlinear optimization solvers is limited to quadratic functions or functions with special structure. As a byproduct, our proof shows that the problem of testing whether all matrices in an interval family are positive semidefinite is strongly NP-hard. This problem, which was previously shown to be (weakly) NP-hard by Nemirovski, is of independent interest in the theory of robust control. |
| Tasks | |
| Published | 2018-06-16 |
| URL | http://arxiv.org/abs/1806.06173v2 |
| http://arxiv.org/pdf/1806.06173v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-complexity-of-detecting-convexity-over |
| Repo | |
| Framework | |
Biometric Recognition System (Algorithm)
| Title | Biometric Recognition System (Algorithm) |
| Authors | Rahul Kumar Jaiswal, Gaurav Saxena |
| Abstract | Fingerprints are the most widely deployed form of biometric identification. No two individuals share the same fingerprint because they have unique biometric identifiers. This paper presents an efficient fingerprint verification algorithm which improves matching accuracy. Fingerprint images get degraded and corrupted due to variations in skin and impression conditions. Thus, image enhancement techniques are employed prior to singular point detection and minutiae extraction. Singular point is the point of maximum curvature. It is determined by the normal of each fingerprint ridge, and then following them inward towards the centre. The local ridge features known as minutiae is extracted using cross-number method to find ridge endings and ridge bifurcations. The proposed algorithm chooses a radius and draws a circle with core point as centre, making fingerprint images rotationally invariant and uniform. The radius can be varied according to the accuracy depending on the particular application. Morphological techniques such as clean, spur and H-break is employed to remove noise, followed by removing spurious minutiae. Templates are created based on feature vector extraction and databases are made for verification and identification for the fingerprint images taken from Fingerprint Verification Competition (FVC2002). Minimum Euclidean distance is calculated between saved template and the test fingerprint image template and compared with the set threshold for matching decision. For the performance evaluation of the proposed algorithm various measures, equal error rate (EER), Dmin at EER, accuracy and threshold are evaluated and plotted. The measures demonstrate that the proposed algorithm is more effective and robust. |
| Tasks | Image Enhancement |
| Published | 2018-12-08 |
| URL | http://arxiv.org/abs/1812.03385v1 |
| http://arxiv.org/pdf/1812.03385v1.pdf | |
| PWC | https://paperswithcode.com/paper/biometric-recognition-system-algorithm |
| Repo | |
| Framework | |
Provably Accelerated Randomized Gossip Algorithms
| Title | Provably Accelerated Randomized Gossip Algorithms |
| Authors | Nicolas Loizou, Michael Rabbat, Peter Richtárik |
| Abstract | In this work we present novel provably accelerated gossip algorithms for solving the average consensus problem. The proposed protocols are inspired from the recently developed accelerated variants of the randomized Kaczmarz method - a popular method for solving linear systems. In each gossip iteration all nodes of the network update their values but only a pair of them exchange their private information. Numerical experiments on popular wireless sensor networks showing the benefits of our protocols are also presented. |
| Tasks | |
| Published | 2018-10-31 |
| URL | http://arxiv.org/abs/1810.13084v1 |
| http://arxiv.org/pdf/1810.13084v1.pdf | |
| PWC | https://paperswithcode.com/paper/provably-accelerated-randomized-gossip |
| Repo | |
| Framework | |
Learning to Map Context-Dependent Sentences to Executable Formal Queries
| Title | Learning to Map Context-Dependent Sentences to Executable Formal Queries |
| Authors | Alane Suhr, Srinivasan Iyer, Yoav Artzi |
| Abstract | We propose a context-dependent model to map utterances within an interaction to executable formal queries. To incorporate interaction history, the model maintains an interaction-level encoder that updates after each turn, and can copy sub-sequences of previously predicted queries during generation. Our approach combines implicit and explicit modeling of references between utterances. We evaluate our model on the ATIS flight planning interactions, and demonstrate the benefits of modeling context and explicit references. |
| Tasks | |
| Published | 2018-04-18 |
| URL | http://arxiv.org/abs/1804.06868v2 |
| http://arxiv.org/pdf/1804.06868v2.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-map-context-dependent-sentences |
| Repo | |
| Framework | |
Rapid Bayesian optimisation for synthesis of short polymer fiber materials
| Title | Rapid Bayesian optimisation for synthesis of short polymer fiber materials |
| Authors | Cheng Li, David Rubin de Celis Leal, Santu Rana, Sunil Gupta, Alessandra Sutti, Stewart Greenhill, Teo Slezak, Murray Height, Svetha Venkatesh |
| Abstract | The discovery of processes for the synthesis of new materials involves many decisions about process design, operation, and material properties. Experimentation is crucial but as complexity increases, exploration of variables can become impractical using traditional combinatorial approaches. We describe an iterative method which uses machine learning to optimise process development, incorporating multiple qualitative and quantitative objectives. We demonstrate the method with a novel fluid processing platform for synthesis of short polymer fibers, and show how the synthesis process can be efficiently directed to achieve material and process objectives. |
| Tasks | Bayesian Optimisation |
| Published | 2018-02-16 |
| URL | http://arxiv.org/abs/1802.05841v1 |
| http://arxiv.org/pdf/1802.05841v1.pdf | |
| PWC | https://paperswithcode.com/paper/rapid-bayesian-optimisation-for-synthesis-of |
| Repo | |
| Framework | |
IEGAN: Multi-purpose Perceptual Quality Image Enhancement Using Generative Adversarial Network
| Title | IEGAN: Multi-purpose Perceptual Quality Image Enhancement Using Generative Adversarial Network |
| Authors | Soumya Shubhra Ghosh, Yang Hua, Sankha Subhra Mukherjee, Neil Robertson |
| Abstract | Despite the breakthroughs in quality of image enhancement, an end-to-end solution for simultaneous recovery of the finer texture details and sharpness for degraded images with low resolution is still unsolved. Some existing approaches focus on minimizing the pixel-wise reconstruction error which results in a high peak signal-to-noise ratio. The enhanced images fail to provide high-frequency details and are perceptually unsatisfying, i.e., they fail to match the quality expected in a photo-realistic image. In this paper, we present Image Enhancement Generative Adversarial Network (IEGAN), a versatile framework capable of inferring photo-realistic natural images for both artifact removal and super-resolution simultaneously. Moreover, we propose a new loss function consisting of a combination of reconstruction loss, feature loss and an edge loss counterpart. The feature loss helps to push the output image to the natural image manifold and the edge loss preserves the sharpness of the output image. The reconstruction loss provides low-level semantic information to the generator regarding the quality of the generated images compared to the original. Our approach has been experimentally proven to recover photo-realistic textures from heavily compressed low-resolution images on public benchmarks and our proposed high-resolution World100 dataset. |
| Tasks | Image Enhancement, Super-Resolution |
| Published | 2018-11-22 |
| URL | http://arxiv.org/abs/1811.09134v1 |
| http://arxiv.org/pdf/1811.09134v1.pdf | |
| PWC | https://paperswithcode.com/paper/iegan-multi-purpose-perceptual-quality-image |
| Repo | |
| Framework | |
