Paper Group ANR 1145
Neural Network Memorization Dissection. Neural Re-Simulation for Generating Bounces in Single Images. Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images. A priori generalization error for two-layer ReLU neural network through minimum norm solution. Significance of parallel computing on the performance of Digital Image Correlat …
Neural Network Memorization Dissection
| Title | Neural Network Memorization Dissection |
| Authors | Jindong Gu, Volker Tresp |
| Abstract | Deep neural networks (DNNs) can easily fit a random labeling of the training data with zero training error. What is the difference between DNNs trained with random labels and the ones trained with true labels? Our paper answers this question with two contributions. First, we study the memorization properties of DNNs. Our empirical experiments shed light on how DNNs prioritize the learning of simple input patterns. In the second part, we propose to measure the similarity between what different DNNs have learned and memorized. With the proposed approach, we analyze and compare DNNs trained on data with true labels and random labels. The analysis shows that DNNs have \textit{One way to Learn} and \textit{N ways to Memorize}. We also use gradient information to gain an understanding of the analysis results. |
| Tasks | |
| Published | 2019-11-21 |
| URL | https://arxiv.org/abs/1911.09537v1 |
| https://arxiv.org/pdf/1911.09537v1.pdf | |
| PWC | https://paperswithcode.com/paper/neural-network-memorization-dissection |
| Repo | |
| Framework | |
Neural Re-Simulation for Generating Bounces in Single Images
| Title | Neural Re-Simulation for Generating Bounces in Single Images |
| Authors | Carlo Innamorati, Bryan Russell, Danny M. Kaufman, and Niloy J. Mitra |
| Abstract | We introduce a method to generate videos of dynamic virtual objects plausibly interacting via collisions with a still image’s environment. Given a starting trajectory, physically simulated with the estimated geometry of a single, static input image, we learn to ‘correct’ this trajectory to a visually plausible one via a neural network. The neural network can then be seen as learning to ‘correct’ traditional simulation output, generated with incomplete and imprecise world information, to obtain context-specific, visually plausible re-simulated output, a process we call neural re-simulation. We train our system on a set of 50k synthetic scenes where a virtual moving object (ball) has been physically simulated. We demonstrate our approach on both our synthetic dataset and a collection of real-life images depicting everyday scenes, obtaining consistent improvement over baseline alternatives throughout. |
| Tasks | |
| Published | 2019-08-17 |
| URL | https://arxiv.org/abs/1908.06217v3 |
| https://arxiv.org/pdf/1908.06217v3.pdf | |
| PWC | https://paperswithcode.com/paper/neural-re-simulation-for-generating-bounces |
| Repo | |
| Framework | |
Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images
| Title | Uncertainty-Guided Domain Alignment for Layer Segmentation in OCT Images |
| Authors | Jiexiang Wang, Cheng Bian, Meng Li, Xin Yang, Kai Ma, Wenao Ma, Jin Yuan, Xinghao Ding, Yefeng Zheng |
| Abstract | Automatic and accurate segmentation for retinal and choroidal layers of Optical Coherence Tomography (OCT) is crucial for detection of various ocular diseases. However, because of the variations in different equipments, OCT data obtained from different manufacturers might encounter appearance discrepancy, which could lead to performance fluctuation to a deep neural network. In this paper, we propose an uncertainty-guided domain alignment method to aim at alleviating this problem to transfer discriminative knowledge across distinct domains. We disign a novel uncertainty-guided cross-entropy loss for boosting the performance over areas with high uncertainty. An uncertainty-guided curriculum transfer strategy is developed for the self-training (ST), which regards uncertainty as efficient and effective guidance to optimize the learning process in target domain. Adversarial learning with feature recalibration module (FRM) is applied to transfer informative knowledge from the domain feature spaces adaptively. The experiments on two OCT datasets show that the proposed methods can obtain significant segmentation improvements compared with the baseline models. |
| Tasks | |
| Published | 2019-08-22 |
| URL | https://arxiv.org/abs/1908.08242v2 |
| https://arxiv.org/pdf/1908.08242v2.pdf | |
| PWC | https://paperswithcode.com/paper/uncertainty-guided-domain-alignment-for-layer |
| Repo | |
| Framework | |
A priori generalization error for two-layer ReLU neural network through minimum norm solution
| Title | A priori generalization error for two-layer ReLU neural network through minimum norm solution |
| Authors | Zhi-Qin John Xu, Jiwei Zhang, Yaoyu Zhang, Chengchao Zhao |
| Abstract | We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the target function, through the following research line. We first estimate \emph{a priori} generalization error of finite-width two-layer ReLU NN with constraint of minimal norm solution, which is proved by \cite{zhang2019type} to be an equivalent solution of a linearized (w.r.t. parameter) finite-width two-layer NN. As the width goes to infinity, the linearized NN converges to the NN in Neural Tangent Kernel (NTK) regime \citep{jacot2018neural}. Thus, we can derive the \emph{a priori} generalization error of two-layer ReLU NN in NTK regime. The distance between NN in a NTK regime and a finite-width NN with gradient training is estimated by \cite{arora2019exact}. Based on the results in \cite{arora2019exact}, our work proves an \emph{a priori} generalization error bound of two-layer ReLU NNs. This estimate uses the intrinsic implicit bias of the minimum norm solution without requiring extra regularity in the loss function. This \emph{a priori} estimate also implies that NN does not suffer from curse of dimensionality, and a small generalization error can be achieved without requiring exponentially large number of neurons. In addition the research line proposed in this paper can also be used to study other properties of the finite-width network, such as the posterior generalization error. |
| Tasks | |
| Published | 2019-12-06 |
| URL | https://arxiv.org/abs/1912.03011v2 |
| https://arxiv.org/pdf/1912.03011v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-priori-generalization-error-for-two-layer |
| Repo | |
| Framework | |
Significance of parallel computing on the performance of Digital Image Correlation algorithms in MATLAB
| Title | Significance of parallel computing on the performance of Digital Image Correlation algorithms in MATLAB |
| Authors | Andreas Thoma, Sridhar Ravi |
| Abstract | Digital Image Correlation (DIC) is a powerful tool used to evaluate displacements and deformations in a non-intrusive manner. By comparing two images, one of the undeformed reference state of a specimen and another of the deformed target state, the relative displacement between those two states is determined. DIC is well known and often used for post-processing analysis of in-plane displacements and deformation of specimen. Increasing the analysis speed to enable real-time DIC analysis will be beneficial and extend the field of use of this technique. Here we tested several combinations of the most common DIC methods in combination with different parallelization approaches in MATLAB and evaluated their performance to determine whether real-time analysis is possible with these methods. To reflect improvements in computing technology different hardware settings were also analysed. We found that implementation problems can reduce the efficiency of a theoretically superior algorithm such that it becomes practically slower than a sub-optimal algorithm. The Newton-Raphson algorithm in combination with a modified Particle Swarm algorithm in parallel image computation was found to be most effective. This is contrary to theory, suggesting that the inverse-compositional Gauss-Newton algorithm is superior. As expected, the Brute Force Search algorithm is the least effective method. We also found that the correct choice of parallelization tasks is crucial to achieve improvements in computing speed. A poorly chosen parallelisation approach with high parallel overhead leads to inferior performance. Finally, irrespective of the computing mode the correct choice of combinations of integer-pixel and sub-pixel search algorithms is decisive for an efficient analysis. Using currently available hardware real-time analysis at high framerates remains an aspiration. |
| Tasks | |
| Published | 2019-05-15 |
| URL | https://arxiv.org/abs/1905.06228v1 |
| https://arxiv.org/pdf/1905.06228v1.pdf | |
| PWC | https://paperswithcode.com/paper/significance-of-parallel-computing-on-the |
| Repo | |
| Framework | |
Semantic Web for Machine Translation: Challenges and Directions
| Title | Semantic Web for Machine Translation: Challenges and Directions |
| Authors | Diego Moussallem, Matthias Wauer, Axel-Cyrille Ngonga Ngomo |
| Abstract | A large number of machine translation approaches have recently been developed to facilitate the fluid migration of content across languages. However, the literature suggests that many obstacles must still be dealt with to achieve better automatic translations. One of these obstacles is lexical and syntactic ambiguity. A promising way of overcoming this problem is using Semantic Web technologies. This article is an extended abstract of our systematic review on machine translation approaches that rely on Semantic Web technologies for improving the translation of texts. Overall, we present the challenges and opportunities in the use of Semantic Web technologies in Machine Translation. Moreover, our research suggests that while Semantic Web technologies can enhance the quality of machine translation outputs for various problems, the combination of both is still in its infancy. |
| Tasks | Machine Translation |
| Published | 2019-07-23 |
| URL | https://arxiv.org/abs/1907.10676v1 |
| https://arxiv.org/pdf/1907.10676v1.pdf | |
| PWC | https://paperswithcode.com/paper/semantic-web-for-machine-translation |
| Repo | |
| Framework | |
Feedback Control for Online Training of Neural Networks
| Title | Feedback Control for Online Training of Neural Networks |
| Authors | Zilong Zhao, Sophie Cerf, Bogdan Robu, Nicolas Marchand |
| Abstract | Convolutional neural networks (CNNs) are commonly used for image classification tasks, raising the challenge of their application on data flows. During their training, adaptation is often performed by tuning the learning rate. Usual learning rate strategies are time-based i.e. monotonously decreasing. In this paper, we advocate switching to a performance-based adaptation, in order to improve the learning efficiency. We present E (Exponential)/PD (Proportional Derivative)-Control, a conditional learning rate strategy that combines a feedback PD controller based on the CNN loss function, with an exponential control signal to smartly boost the learning and adapt the PD parameters. Stability proof is provided as well as an experimental evaluation using two state of the art image datasets (CIFAR-10 and Fashion-MNIST). Results show better performances than the related works (faster network accuracy growth reaching higher levels) and robustness of the E/PD-Control regarding its parametrization. |
| Tasks | Image Classification |
| Published | 2019-11-18 |
| URL | https://arxiv.org/abs/1911.07710v1 |
| https://arxiv.org/pdf/1911.07710v1.pdf | |
| PWC | https://paperswithcode.com/paper/feedback-control-for-online-training-of |
| Repo | |
| Framework | |
HateMonitors: Language Agnostic Abuse Detection in Social Media
| Title | HateMonitors: Language Agnostic Abuse Detection in Social Media |
| Authors | Punyajoy Saha, Binny Mathew, Pawan Goyal, Animesh Mukherjee |
| Abstract | Reducing hateful and offensive content in online social media pose a dual problem for the moderators. On the one hand, rigid censorship on social media cannot be imposed. On the other, the free flow of such content cannot be allowed. Hence, we require efficient abusive language detection system to detect such harmful content in social media. In this paper, we present our machine learning model, HateMonitor, developed for Hate Speech and Offensive Content Identification in Indo-European Languages (HASOC), a shared task at FIRE 2019. We have used a Gradient Boosting model, along with BERT and LASER embeddings, to make the system language agnostic. Our model came at First position for the German sub-task A. We have also made our model public at https://github.com/punyajoy/HateMonitors-HASOC . |
| Tasks | Abuse Detection |
| Published | 2019-09-27 |
| URL | https://arxiv.org/abs/1909.12642v1 |
| https://arxiv.org/pdf/1909.12642v1.pdf | |
| PWC | https://paperswithcode.com/paper/hatemonitors-language-agnostic-abuse |
| Repo | |
| Framework | |
Count, Crop and Recognise: Fine-Grained Recognition in the Wild
| Title | Count, Crop and Recognise: Fine-Grained Recognition in the Wild |
| Authors | Max Bain, Arsha Nagrani, Daniel Schofield, Andrew Zisserman |
| Abstract | The goal of this paper is to label all the animal individuals present in every frame of a video. Unlike previous methods that have principally concentrated on labelling face tracks, we aim to label individuals even when their faces are not visible. We make the following contributions: (i) we introduce a ‘Count, Crop and Recognise’ (CCR) multistage recognition process for frame level labelling. The Count and Recognise stages involve specialised CNNs for the task, and we show that this simple staging gives a substantial boost in performance; (ii) we compare the recall using frame based labelling to both face and body track based labelling, and demonstrate the advantage of frame based with CCR for the specified goal; (iii) we introduce a new dataset for chimpanzee recognition in the wild; and (iv) we apply a high-granularity visualisation technique to further understand the learned CNN features for the recognition of chimpanzee individuals. |
| Tasks | |
| Published | 2019-09-19 |
| URL | https://arxiv.org/abs/1909.08950v2 |
| https://arxiv.org/pdf/1909.08950v2.pdf | |
| PWC | https://paperswithcode.com/paper/count-crop-and-recognise-fine-grained |
| Repo | |
| Framework | |
Mean estimation and regression under heavy-tailed distributions–a survey
| Title | Mean estimation and regression under heavy-tailed distributions–a survey |
| Authors | Gabor Lugosi, Shahar Mendelson |
| Abstract | We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni’s estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems–in particular, regression function estimation–in the presence of possibly heavy-tailed data. |
| Tasks | |
| Published | 2019-06-10 |
| URL | https://arxiv.org/abs/1906.04280v1 |
| https://arxiv.org/pdf/1906.04280v1.pdf | |
| PWC | https://paperswithcode.com/paper/mean-estimation-and-regression-under-heavy |
| Repo | |
| Framework | |
Optimization problems with low SWaP tactical Computing
| Title | Optimization problems with low SWaP tactical Computing |
| Authors | Mee Seong Im, Venkat R. Dasari, Lubjana Beshaj, Dale Shires |
| Abstract | In a resource-constrained, contested environment, computing resources need to be aware of possible size, weight, and power (SWaP) restrictions. SWaP-aware computational efficiency depends upon optimization of computational resources and intelligent time versus efficiency tradeoffs in decision making. In this paper we address the complexity of various optimization strategies related to low SWaP computing. Due to these restrictions, only a small subset of less complicated and fast computable algorithms can be used for tactical, adaptive computing. |
| Tasks | Decision Making |
| Published | 2019-02-13 |
| URL | http://arxiv.org/abs/1902.05070v1 |
| http://arxiv.org/pdf/1902.05070v1.pdf | |
| PWC | https://paperswithcode.com/paper/optimization-problems-with-low-swap-tactical |
| Repo | |
| Framework | |
Knowledge-Enhanced Attentive Learning for Answer Selection in Community Question Answering Systems
| Title | Knowledge-Enhanced Attentive Learning for Answer Selection in Community Question Answering Systems |
| Authors | Fengshi Jing, Qingpeng Zhang |
| Abstract | In the community question answering (CQA) system, the answer selection task aims to identify the best answer for a specific question, and thus is playing a key role in enhancing the service quality through recommending appropriate answers for new questions. Recent advances in CQA answer selection focus on enhancing the performance by incorporating the community information, particularly the expertise (previous answers) and authority (position in the social network) of an answerer. However, existing approaches for incorporating such information are limited in (a) only considering either the expertise or the authority, but not both; (b) ignoring the domain knowledge to differentiate topics of previous answers; and (c) simply using the authority information to adjust the similarity score, instead of fully utilizing it in the process of measuring the similarity between segments of the question and the answer. We propose the Knowledge-enhanced Attentive Answer Selection (KAAS) model, which enhances the performance through (a) considering both the expertise and the authority of the answerer; (b) utilizing the human-labeled tags, the taxonomy of the tags, and the votes as the domain knowledge to infer the expertise of the answer; (c) using matrix decomposition of the social network (formed by following-relationship) to infer the authority of the answerer and incorporating such information in the process of evaluating the similarity between segments. Besides, for vertical community, we incorporate an external knowledge graph to capture more professional information for vertical CQA systems. Then we adopt the attention mechanism to integrate the analysis of the text of questions and answers and the aforementioned community information. Experiments with both vertical and general CQA sites demonstrate the superior performance of the proposed KAAS model. |
| Tasks | Answer Selection, Community Question Answering, Question Answering |
| Published | 2019-12-17 |
| URL | https://arxiv.org/abs/1912.07915v1 |
| https://arxiv.org/pdf/1912.07915v1.pdf | |
| PWC | https://paperswithcode.com/paper/knowledge-enhanced-attentive-learning-for |
| Repo | |
| Framework | |
Deep Learning for Whole Slide Image Analysis: An Overview
| Title | Deep Learning for Whole Slide Image Analysis: An Overview |
| Authors | Neofytos Dimitriou, Ognjen Arandjelović, Peter D Caie |
| Abstract | The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artefacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies. |
| Tasks | |
| Published | 2019-10-18 |
| URL | https://arxiv.org/abs/1910.11097v1 |
| https://arxiv.org/pdf/1910.11097v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-for-whole-slide-image-analysis |
| Repo | |
| Framework | |
Tackling Online Abuse: A Survey of Automated Abuse Detection Methods
| Title | Tackling Online Abuse: A Survey of Automated Abuse Detection Methods |
| Authors | Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova |
| Abstract | Abuse on the Internet represents an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse on online platforms. The psychological effects of such abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abuse detection in the field of natural language processing (NLP). In this paper, we present a comprehensive survey of the methods that have been proposed to date, thus providing a platform for further development of this area. We describe the existing datasets and review the computational approaches to abuse detection, analyzing their strengths and limitations. We discuss the main trends that emerge, highlight the challenges that remain, outline possible solutions, and propose guidelines for ethics and explainability |
| Tasks | Abuse Detection |
| Published | 2019-08-13 |
| URL | https://arxiv.org/abs/1908.06024v1 |
| https://arxiv.org/pdf/1908.06024v1.pdf | |
| PWC | https://paperswithcode.com/paper/tackling-online-abuse-a-survey-of-automated |
| Repo | |
| Framework | |
Quantifying Perceptual Distortion of Adversarial Examples
| Title | Quantifying Perceptual Distortion of Adversarial Examples |
| Authors | Matt Jordan, Naren Manoj, Surbhi Goel, Alexandros G. Dimakis |
| Abstract | Recent work has shown that additive threat models, which only permit the addition of bounded noise to the pixels of an image, are insufficient for fully capturing the space of imperceivable adversarial examples. For example, small rotations and spatial transformations can fool classifiers, remain imperceivable to humans, but have large additive distance from the original images. In this work, we leverage quantitative perceptual metrics like LPIPS and SSIM to define a novel threat model for adversarial attacks. To demonstrate the value of quantifying the perceptual distortion of adversarial examples, we present and employ a unifying framework fusing different attack styles. We first prove that our framework results in images that are unattainable by attack styles in isolation. We then perform adversarial training using attacks generated by our framework to demonstrate that networks are only robust to classes of adversarial perturbations they have been trained against, and combination attacks are stronger than any of their individual components. Finally, we experimentally demonstrate that our combined attacks retain the same perceptual distortion but induce far higher misclassification rates when compared against individual attacks. |
| Tasks | |
| Published | 2019-02-21 |
| URL | http://arxiv.org/abs/1902.08265v1 |
| http://arxiv.org/pdf/1902.08265v1.pdf | |
| PWC | https://paperswithcode.com/paper/quantifying-perceptual-distortion-of |
| Repo | |
| Framework | |
