Paper Group NAWR 24
ChangeNet: A Deep Learning Architecture for Visual Change Detection. Team yeon-zi at SemEval-2019 Task 4: Hyperpartisan News Detection by De-noising Weakly-labeled Data. Training Image Estimators without Image Ground Truth. Neural Machine Translation with Soft Prototype. Reflection Separation using a Pair of Unpolarized and Polarized Images. Memory …
ChangeNet: A Deep Learning Architecture for Visual Change Detection
| Title | ChangeNet: A Deep Learning Architecture for Visual Change Detection |
| Authors | Ashley Varghese, Jayavardhana Gubbi, and Akshaya Ramaswamy Balamuralidhar P Embedding Systems and Robotics, TCS Research and Innovation, Bengaluru, India |
| Abstract | The increasing urban population in cities necessitates the need for the development of smart cities that can offer better services to its citizens. Drone technology plays a crucial role in the smart city environment and is already involved in a number of functions in smart cities such as traffic control and construction monitoring. A major challenge in fast growing cities is the encroachment of public spaces. A robotic solution using visual change detection can be used for such purposes. For the detection of encroachment, a drone can monitor outdoor urban areas over a period of time to infer the visual changes. Visual change detection is a higher level inference task that aims at accurately identifying variations between a reference image (historical) and a new test image depicting the current scenario. In case of images, the challenges are complex considering the variations caused by environmental conditions that are actually unchanged events. Human mind interprets the change by comparing the current status with historical data at intelligence level rather than using only visual information. In this paper, we present a deep architecture called ChangeNet for detecting changes between pairs of images and express the same semantically (label the change). A parallel deep convolutional neural network (CNN) architecture for localizing and identifying the changes between image pair has been proposed in this paper. The architecture is evaluated with VL-CMU-CD street view change detection, TSUNAMI and Google Street View (GSV) datasets that resemble drone captured images. The performance of the model for different lighting and seasonal conditions are experimented quantitatively and qualitatively. The result shows that ChangeNet outperforms the state of the art by achieving 98.3% pixel accuracy, 77.35% object based Intersection over Union (IoU) and 88.9% area under Receiver Operating Characteristics (RoC) curve. |
| Tasks | |
| Published | 2019-01-29 |
| URL | http://openaccess.thecvf.com/content_eccv_2018_workshops/w7/html/Varghese_ChangeNet_A_Deep_Learning_Architecture_for_Visual_Change_Detection_ECCVW_2018_paper.html |
| http://openaccess.thecvf.com/content_ECCVW_2018/papers/11130/Varghese_ChangeNet_A_Deep_Learning_Architecture_for_Visual_Change_Detection_ECCVW_2018_paper.pdf | |
| PWC | https://paperswithcode.com/paper/changenet-a-deep-learning-architecture-for |
| Repo | https://github.com/leonardoaraujosantos/ChangeNet |
| Framework | pytorch |
Team yeon-zi at SemEval-2019 Task 4: Hyperpartisan News Detection by De-noising Weakly-labeled Data
| Title | Team yeon-zi at SemEval-2019 Task 4: Hyperpartisan News Detection by De-noising Weakly-labeled Data |
| Authors | Nayeon Lee, Zihan Liu, Pascale Fung |
| Abstract | This paper describes our system that has been submitted to SemEval-2019 Task 4: Hyperpartisan News Detection. We focus on removing the noise inherent in the hyperpartisanship dataset from both data-level and model-level by leveraging semi-supervised pseudo-labels and the state-of-the-art BERT model. Our model achieves 75.8{%} accuracy in the final by-article dataset without ensemble learning. |
| Tasks | |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/S19-2184/ |
| https://www.aclweb.org/anthology/S19-2184 | |
| PWC | https://paperswithcode.com/paper/team-yeon-zi-at-semeval-2019-task-4 |
| Repo | https://github.com/zliucr/hyperpartisan-news-detection |
| Framework | pytorch |
Training Image Estimators without Image Ground Truth
| Title | Training Image Estimators without Image Ground Truth |
| Authors | Zhihao Xia, Ayan Chakrabarti |
| Abstract | Deep neural networks have been very successful in compressive-sensing and image restoration applications, as a means to estimate images from partial, blurry, or otherwise degraded measurements. These networks are trained on a large number of corresponding pairs of measurements and ground-truth images, and thus implicitly learn to exploit domain-specific image statistics. But unlike measurement data, it is often expensive or impractical to collect a large training set of ground-truth images in many application settings. In this paper, we introduce an unsupervised framework for training image estimation networks, from a training set that contains only measurements—with two varied measurements per image—but no ground-truth for the full images desired as output. We demonstrate that our framework can be applied for both regular and blind image estimation tasks, where in the latter case parameters of the measurement model (e.g., the blur kernel) are unknown: during inference, and potentially, also during training. We evaluate our framework for training networks for compressive-sensing and blind deconvolution, considering both non-blind and blind training for the latter. Our framework yields models that are nearly as accurate as those from fully supervised training, despite not having access to any ground-truth images. |
| Tasks | Compressive Sensing, Image Restoration |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8514-training-image-estimators-without-image-ground-truth |
| http://papers.nips.cc/paper/8514-training-image-estimators-without-image-ground-truth.pdf | |
| PWC | https://paperswithcode.com/paper/training-image-estimators-without-image-1 |
| Repo | https://github.com/likesum/unsupimg |
| Framework | tf |
Neural Machine Translation with Soft Prototype
| Title | Neural Machine Translation with Soft Prototype |
| Authors | Yiren Wang, Yingce Xia, Fei Tian, Fei Gao, Tao Qin, Cheng Xiang Zhai, Tie-Yan Liu |
| Abstract | Neural machine translation models usually use the encoder-decoder framework and generate translation from left to right (or right to left) without fully utilizing the target-side global information. A few recent approaches seek to exploit the global information through two-pass decoding, yet have limitations in translation quality and model efficiency. In this work, we propose a new framework that introduces a soft prototype into the encoder-decoder architecture, which allows the decoder to have indirect access to both past and future information, such that each target word can be generated based on the better global understanding. We further provide an efficient and effective method to generate the prototype. Empirical studies on various neural machine translation tasks show that our approach brings significant improvement in generation quality over the baseline model, with little extra cost in storage and inference time, demonstrating the effectiveness of our proposed framework. Specially, we achieve state-of-the-art results on WMT2014, 2015 and 2017 English to German translation. |
| Tasks | Machine Translation |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8861-neural-machine-translation-with-soft-prototype |
| http://papers.nips.cc/paper/8861-neural-machine-translation-with-soft-prototype.pdf | |
| PWC | https://paperswithcode.com/paper/neural-machine-translation-with-soft |
| Repo | https://github.com/ywang07/nmt_soft_prototype |
| Framework | pytorch |
Reflection Separation using a Pair of Unpolarized and Polarized Images
| Title | Reflection Separation using a Pair of Unpolarized and Polarized Images |
| Authors | Youwei Lyu, Zhaopeng Cui, Si Li, Marc Pollefeys, Boxin Shi |
| Abstract | When we take photos through glass windows or doors, the transmitted background scene is often blended with undesirable reflection. Separating two layers apart to enhance the image quality is of vital importance for both human and machine perception. In this paper, we propose to exploit physical constraints from a pair of unpolarized and polarized images to separate reflection and transmission layers. Due to the simplified capturing setup, the system becomes more underdetermined compared with existing polarization based solutions that take three or more images as input. We propose to solve semireflector orientation estimation first to make the physical image formation well-posed and then learn to reliably separate two layers using a refinement network with gradient loss. Quantitative and qualitative experimental results show our approach performs favorably over existing polarization and single image based solutions. |
| Tasks | |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9598-reflection-separation-using-a-pair-of-unpolarized-and-polarized-images |
| http://papers.nips.cc/paper/9598-reflection-separation-using-a-pair-of-unpolarized-and-polarized-images.pdf | |
| PWC | https://paperswithcode.com/paper/reflection-separation-using-a-pair-of |
| Repo | https://github.com/YouweiLyu/reflection_separation_with_un-polarized_images |
| Framework | pytorch |
Memory Efficient Adaptive Optimization
| Title | Memory Efficient Adaptive Optimization |
| Authors | Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer |
| Abstract | Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter, thus introducing significant memory overheads that restrict the size of the model being used as well as the number of examples in a mini-batch. We describe an effective and flexible adaptive optimization method with greatly reduced memory overhead. Our method retains the benefits of per-parameter adaptivity while allowing significantly larger models and batch sizes. We give convergence guarantees for our method, and demonstrate its effectiveness in training very large translation and language models with up to 2-fold speedups compared to the state-of-the-art. |
| Tasks | Language Modelling, Machine Translation |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9168-memory-efficient-adaptive-optimization |
| http://papers.nips.cc/paper/9168-memory-efficient-adaptive-optimization.pdf | |
| PWC | https://paperswithcode.com/paper/memory-efficient-adaptive-optimization |
| Repo | https://github.com/google-research/google-research |
| Framework | tf |
Spectral Modification of Graphs for Improved Spectral Clustering
| Title | Spectral Modification of Graphs for Improved Spectral Clustering |
| Authors | Ioannis Koutis, Huong Le |
| Abstract | Spectral clustering algorithms provide approximate solutions to hard optimization problems that formulate graph partitioning in terms of the graph conductance. It is well understood that the quality of these approximate solutions is negatively affected by a possibly significant gap between the conductance and the second eigenvalue of the graph. In this paper we show that for \textbf{any} graph $G$, there exists a `spectral maximizer’ graph $H$ which is cut-similar to $G$, but has eigenvalues that are near the theoretical limit implied by the cut structure of $G$. Applying then spectral clustering on $H$ has the potential to produce improved cuts that also exist in $G$ due to the cut similarity. This leads to the second contribution of this work: we describe a practical spectral modification algorithm that raises the eigenvalues of the input graph, while preserving its cuts. Combined with spectral clustering on the modified graph, this yields demonstrably improved cuts. | |
| Tasks | graph partitioning |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8732-spectral-modification-of-graphs-for-improved-spectral-clustering |
| http://papers.nips.cc/paper/8732-spectral-modification-of-graphs-for-improved-spectral-clustering.pdf | |
| PWC | https://paperswithcode.com/paper/spectral-modification-of-graphs-for-improved |
| Repo | https://github.com/ikoutis/spectral-modification |
| Framework | none |
Complex Question Decomposition for Semantic Parsing
| Title | Complex Question Decomposition for Semantic Parsing |
| Authors | Haoyu Zhang, Jingjing Cai, Jianjun Xu, Ji Wang |
| Abstract | In this work, we focus on complex question semantic parsing and propose a novel Hierarchical Semantic Parsing (HSP) method, which utilizes the decompositionality of complex questions for semantic parsing. Our model is designed within a three-stage parsing architecture based on the idea of decomposition-integration. In the first stage, we propose a question decomposer which decomposes a complex question into a sequence of sub-questions. In the second stage, we design an information extractor to derive the type and predicate information of these questions. In the last stage, we integrate the generated information from previous stages and generate a logical form for the complex question. We conduct experiments on COMPLEXWEBQUESTIONS which is a large scale complex question semantic parsing dataset, results show that our model achieves significant improvement compared to state-of-the-art methods. |
| Tasks | Semantic Parsing |
| Published | 2019-07-01 |
| URL | https://www.aclweb.org/anthology/P19-1440/ |
| https://www.aclweb.org/anthology/P19-1440 | |
| PWC | https://paperswithcode.com/paper/complex-question-decomposition-for-semantic |
| Repo | https://github.com/cairohy/hsp |
| Framework | tf |
Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation
| Title | Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation |
| Authors | Zeping Yu, Jianxun Lian, Ahmad Mahmoody, Gongshen Liu, Xing Xie |
| Abstract | User modeling is an essential task for online rec- ommender systems. In the past few decades, col- laborative filtering (CF) techniques have been well studied to model users’ long term preferences. Recently, recurrent neural networks (RNN) have shown a great advantage in modeling users’ short term preference. A natural way to improve the rec- ommender is to combine both long-term and short- term modeling. Previous approaches neglect the importance of dynamically integrating these two user modeling paradigms. Moreover, users’ be- haviors are much more complex than sentences in language modeling or images in visual computing, thus the classical structures of RNN such as Long Short-Term Memory (LSTM) need to be upgraded for better user modeling. In this paper, we im- prove the traditional RNN structure by proposing a time-aware controller and a content-aware con- troller, so that contextual information can be well considered to control the state transition. We fur- ther propose an attention-based framework to com- bine users’ long-term and short-term preferences, thus users’ representation can be generated adap- tively according to the specific context. We con- duct extensive experiments on both public and in- dustrial datasets. The results demonstrate that our proposed method outperforms several state-of-art methods consistently. |
| Tasks | Language Modelling |
| Published | 2019-01-01 |
| URL | https://www.ijcai.org/proceedings/2019/0585.pdf |
| https://www.ijcai.org/proceedings/2019/0585.pdf | |
| PWC | https://paperswithcode.com/paper/adaptive-user-modeling-with-long-and-short |
| Repo | https://github.com/zepingyu0512/sli_rec |
| Framework | tf |
Attentive State-Space Modeling of Disease Progression
| Title | Attentive State-Space Modeling of Disease Progression |
| Authors | Ahmed M. Alaa, Mihaela Van Der Schaar |
| Abstract | Models of disease progression are instrumental for predicting patient outcomes and understanding disease dynamics. Existing models provide the patient with pragmatic (supervised) predictions of risk, but do not provide the clinician with intelligible (unsupervised) representations of disease pathophysiology. In this paper, we develop the attentive state-space model, a deep probabilistic model that learns accurate and interpretable structured representations for disease trajectories. Unlike Markovian state-space models, in which the dynamics are memoryless, our model uses an attention mechanism to create “memoryful” dynamics, whereby attention weights determine the dependence of future disease states on past medical history. To learn the model parameters from medical records, we develop an infer ence algorithm that simultaneously learns a compiled inference network and the model parameters, leveraging the attentive state-space representation to construct a “Rao-Blackwellized” variational approximation of the posterior state distribution. Experiments on data from the UK Cystic Fibrosis registry show that our model demonstrates superior predictive accuracy and provides insights into the progression of chronic disease. |
| Tasks | Predicting Patient Outcomes |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/9311-attentive-state-space-modeling-of-disease-progression |
| http://papers.nips.cc/paper/9311-attentive-state-space-modeling-of-disease-progression.pdf | |
| PWC | https://paperswithcode.com/paper/attentive-state-space-modeling-of-disease |
| Repo | https://github.com/ahmedmalaa/attentive-state-space-models |
| Framework | none |
Modeling the Long Term Future in Model-Based Reinforcement Learning
| Title | Modeling the Long Term Future in Model-Based Reinforcement Learning |
| Authors | Nan Rosemary Ke, Amanpreet Singh, Ahmed Touati, Anirudh Goyal, Yoshua Bengio, Devi Parikh, Dhruv Batra |
| Abstract | In model-based reinforcement learning, the agent interleaves between model learning and planning. These two components are inextricably intertwined. If the model is not able to provide sensible long-term prediction, the executed planer would exploit model flaws, which can yield catastrophic failures. This paper focuses on building a model that reasons about the long-term future and demonstrates how to use this for efficient planning and exploration. To this end, we build a latent-variable autoregressive model by leveraging recent ideas in variational inference. We argue that forcing latent variables to carry future information through an auxiliary task substantially improves long-term predictions. Moreover, by planning in the latent space, the planner’s solution is ensured to be within regions where the model is valid. An exploration strategy can be devised by searching for unlikely trajectories under the model. Our methods achieves higher reward faster compared to baselines on a variety of tasks and environments in both the imitation learning and model-based reinforcement learning settings. |
| Tasks | Imitation Learning |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=SkgQBn0cF7 |
| https://openreview.net/pdf?id=SkgQBn0cF7 | |
| PWC | https://paperswithcode.com/paper/modeling-the-long-term-future-in-model-based |
| Repo | https://github.com/maximecb/gym-minigrid |
| Framework | pytorch |
Sliced Wasserstein Auto-Encoders
| Title | Sliced Wasserstein Auto-Encoders |
| Authors | Soheil Kolouri, Phillip E. Pope, Charles E. Martin, Gustavo K. Rohde |
| Abstract | In this paper we use the geometric properties of the optimal transport (OT) problem and the Wasserstein distances to define a prior distribution for the latent space of an auto-encoder. We introduce Sliced-Wasserstein Auto-Encoders (SWAE), that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or having a likelihood function specified. In short, we regularize the auto-encoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a samplable prior distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Auto-Encoders (WAE) and Variational Auto-Encoders (VAE), while benefiting from an embarrassingly simple implementation. We provide extensive error analysis for our algorithm, and show its merits on three benchmark datasets. |
| Tasks | |
| Published | 2019-05-01 |
| URL | https://openreview.net/forum?id=H1xaJn05FQ |
| https://openreview.net/pdf?id=H1xaJn05FQ | |
| PWC | https://paperswithcode.com/paper/sliced-wasserstein-auto-encoders |
| Repo | https://github.com/AntixK/PyTorch-VAE |
| Framework | pytorch |
Deliberative Explanations: visualizing network insecurities
| Title | Deliberative Explanations: visualizing network insecurities |
| Authors | Pei Wang, Nuno Nvasconcelos |
| Abstract | A new approach to explainable AI, denoted {\it deliberative explanations,/} is proposed. Deliberative explanations are a visualization technique that aims to go beyond the simple visualization of the image regions (or, more generally, input variables) responsible for a network prediction. Instead, they aim to expose the deliberations carried by the network to arrive at that prediction, by uncovering the insecurities of the network about the latter. The explanation consists of a list of insecurities, each composed of 1) an image region (more generally, a set of input variables), and 2) an ambiguity formed by the pair of classes responsible for the network uncertainty about the region. Since insecurity detection requires quantifying the difficulty of network predictions, deliberative explanations combine ideas from the literatures on visual explanations and assessment of classification difficulty. More specifically, the proposed implementation combines attributions with respect to both class predictions and a difficulty score. An evaluation protocol that leverages object recognition (CUB200) and scene classification (ADE20K) datasets that combine part and attribute annotations is also introduced to evaluate the accuracy of deliberative explanations. Finally, an experimental evaluation shows that the most accurate explanations are achieved by combining non self-referential difficulty scores and second-order attributions. The resulting insecurities are shown to correlate with regions of attributes that are shared by different classes. Since these regions are also ambiguous for humans, deliberative explanations are intuitive, suggesting that the deliberative process of modern networks correlates with human reasoning. |
| Tasks | Object Recognition, Scene Classification |
| Published | 2019-12-01 |
| URL | http://papers.nips.cc/paper/8418-deliberative-explanations-visualizing-network-insecurities |
| http://papers.nips.cc/paper/8418-deliberative-explanations-visualizing-network-insecurities.pdf | |
| PWC | https://paperswithcode.com/paper/deliberative-explanations-visualizing-network |
| Repo | https://github.com/peiwang062/Deliberative-explanation |
| Framework | pytorch |
Language-Agnostic Visual-Semantic Embeddings
| Title | Language-Agnostic Visual-Semantic Embeddings |
| Authors | Jonatas Wehrmann, Douglas M. Souza, Mauricio A. Lopes, Rodrigo C. Barros |
| Abstract | This paper proposes a framework for training language-invariant cross-modal retrieval models. We also introduce a novel character-based word-embedding approach, allowing the model to project similar words across languages into the same word-embedding space. In addition, by performing cross-modal retrieval at the character level, the storage requirements for a text encoder decrease substantially, allowing for lighter and more scalable retrieval architectures. The proposed language-invariant textual encoder based on characters is virtually unaffected in terms of storage requirements when novel languages are added to the system. Our contributions include new methods for building character-level-based word-embeddings, an improved loss function, and a novel cross-language alignment module that not only makes the architecture language-invariant, but also presents better predictive performance. We show that our models outperform the current state-of-the-art in both single and multi-language scenarios. This work can be seen as the basis of a new path on retrieval research, now allowing for the effective use of captions in multiple-language scenarios. Code is available at https://github.com/jwehrmann/lavse. |
| Tasks | Cross-Modal Retrieval, Word Embeddings |
| Published | 2019-10-01 |
| URL | http://openaccess.thecvf.com/content_ICCV_2019/html/Wehrmann_Language-Agnostic_Visual-Semantic_Embeddings_ICCV_2019_paper.html |
| http://openaccess.thecvf.com/content_ICCV_2019/papers/Wehrmann_Language-Agnostic_Visual-Semantic_Embeddings_ICCV_2019_paper.pdf | |
| PWC | https://paperswithcode.com/paper/language-agnostic-visual-semantic-embeddings |
| Repo | https://github.com/jwehrmann/lavse |
| Framework | pytorch |
Target-oriented Opinion Words Extraction with Target-fused Neural Sequence Labeling
| Title | Target-oriented Opinion Words Extraction with Target-fused Neural Sequence Labeling |
| Authors | Zhifang Fan, Zhen Wu, Xin-Yu Dai, Shujian Huang, Jiajun Chen |
| Abstract | Opinion target extraction and opinion words extraction are two fundamental subtasks in Aspect Based Sentiment Analysis (ABSA). Recently, many methods have made progress on these two tasks. However, few works aim at extracting opinion targets and opinion words as pairs. In this paper, we propose a novel sequence labeling subtask for ABSA named TOWE (Target-oriented Opinion Words Extraction), which aims at extracting the corresponding opinion words for a given opinion target. A target-fused sequence labeling neural network model is designed to perform this task. The opinion target information is well encoded into context by an Inward-Outward LSTM. Then left and right contexts of the opinion target and the global context are combined to find the corresponding opinion words. We build four datasets for TOWE based on several popular ABSA benchmarks from laptop and restaurant reviews. The experimental results show that our proposed model outperforms the other compared methods significantly. We believe that our work may not only be helpful for downstream sentiment analysis task, but can also be used for pair-wise opinion summarization. |
| Tasks | Aspect-Based Sentiment Analysis, Sentiment Analysis |
| Published | 2019-06-01 |
| URL | https://www.aclweb.org/anthology/N19-1259/ |
| https://www.aclweb.org/anthology/N19-1259 | |
| PWC | https://paperswithcode.com/paper/target-oriented-opinion-words-extraction-with |
| Repo | https://github.com/NJUNLP/TOWE |
| Framework | pytorch |
