Paper Group ANR 1555
No Adjective Ordering Mystery, and No Raven Paradox, Just an Ontological Mishap. Can SGD Learn Recurrent Neural Networks with Provable Generalization?. Finding new routes for integrating Multi-Agent Systems using Apache Camel. Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language. Cloze-driven Pretraining …
No Adjective Ordering Mystery, and No Raven Paradox, Just an Ontological Mishap
| Title | No Adjective Ordering Mystery, and No Raven Paradox, Just an Ontological Mishap |
| Authors | Walid S. Saba |
| Abstract | In the concluding remarks of Ontological Promiscuity Hobbs (1985) made what we believe to be a very insightful observation: given that semantics is an attempt at specifying the relation between language and the world, if “one can assume a theory of the world that is isomorphic to the way we talk about it … then semantics becomes nearly trivial”. But how exactly can we rectify our logical formalisms so that semantics, an endeavor that has occupied the most penetrating minds for over two centuries, can become (nearly) trivial, and what exactly does it mean to assume a theory of the world in our semantics? In this paper we hope to provide answers for both questions. First, we believe that a commonsense theory of the world can (and should) be embedded in our semantic formalisms resulting in a logical semantics grounded in commonsense metaphysics. Moreover, we believe the first step to accomplishing this vision is rectifying what we think was a crucial oversight in logical semantics, namely the failure to distinguish between two fundamentally different types of concepts: (i) ontological concepts, that correspond to what Cocchiarella (2001) calls first-intension concepts and are types in a strongly-typed ontology; and (ii) logical concepts (or second intension concepts), that are predicates corresponding to properties of (and relations between) objects of various ontological types1. In such a framework, which we will refer to henceforth by ontologik, it will be shown how type unification and other type operations can be used to account for the `missing text phenomenon’ (MTP) (see Saba, 2019a) that is at the heart of most challenges in the semantics of natural language, by uncovering the significant amount of missing text that is never explicitly stated in everyday discourse, but is often implicitly assumed as shared background knowledge. | |
| Tasks | |
| Published | 2019-04-14 |
| URL | http://arxiv.org/abs/1904.06779v1 |
| http://arxiv.org/pdf/1904.06779v1.pdf | |
| PWC | https://paperswithcode.com/paper/no-adjective-ordering-mystery-and-no-raven |
| Repo | |
| Framework | |
Can SGD Learn Recurrent Neural Networks with Provable Generalization?
| Title | Can SGD Learn Recurrent Neural Networks with Provable Generalization? |
| Authors | Zeyuan Allen-Zhu, Yuanzhi Li |
| Abstract | Recurrent Neural Networks (RNNs) are among the most popular models in sequential data analysis. Yet, in the foundational PAC learning language, what concept class can it learn? Moreover, how can the same recurrent unit simultaneously learn functions from different input tokens to different output tokens, without affecting each other? Existing generalization bounds for RNN scale exponentially with the input length, significantly limiting their practical implications. In this paper, we show using the vanilla stochastic gradient descent (SGD), RNN can actually learn some notable concept class efficiently, meaning that both time and sample complexity scale polynomially in the input length (or almost polynomially, depending on the concept). This concept class at least includes functions where each output token is generated from inputs of earlier tokens using a smooth two-layer neural network. |
| Tasks | |
| Published | 2019-02-04 |
| URL | https://arxiv.org/abs/1902.01028v2 |
| https://arxiv.org/pdf/1902.01028v2.pdf | |
| PWC | https://paperswithcode.com/paper/can-sgd-learn-recurrent-neural-networks-with |
| Repo | |
| Framework | |
Finding new routes for integrating Multi-Agent Systems using Apache Camel
| Title | Finding new routes for integrating Multi-Agent Systems using Apache Camel |
| Authors | Cleber Jorge Amaral, Sérgio Pereira Bernardes, Mateus Conceição, Jomi Fred Hübner, Luis Pedro Arenhart Lampert, Otávio Arruda Matoso, Maicon Rafael Zatelli |
| Abstract | In Multi-Agent Systems (MAS) there are two main models of interaction: among agents, and between agents and the environment. Although there are studies considering these models, there is no practical tool to afford the interaction with external entities with both models. This paper presents a proposal for such a tool based on the Apache Camel framework by designing two new components, namely camel-jason and camel-artifact. By means of these components, an external entity is modelled according to its nature, i.e., whether it is autonomous or non-autonomous, interacting with the MAS respectively as an agent or an artifact. It models coherently external entities whereas Camel provides interoperability with several communication protocols. |
| Tasks | |
| Published | 2019-05-25 |
| URL | https://arxiv.org/abs/1905.10490v1 |
| https://arxiv.org/pdf/1905.10490v1.pdf | |
| PWC | https://paperswithcode.com/paper/finding-new-routes-for-integrating-multi |
| Repo | |
| Framework | |
Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language
| Title | Predicting Behavior in Cancer-Afflicted Patient and Spouse Interactions using Speech and Language |
| Authors | Sandeep Nallan Chakravarthula, Haoqi Li, Shao-Yen Tseng, Maija Reblin, Panayiotis Georgiou |
| Abstract | Cancer impacts the quality of life of those diagnosed as well as their spouse caregivers, in addition to potentially influencing their day-to-day behaviors. There is evidence that effective communication between spouses can improve well-being related to cancer but it is difficult to efficiently evaluate the quality of daily life interactions using manual annotation frameworks. Automated recognition of behaviors based on the interaction cues of speakers can help analyze interactions in such couples and identify behaviors which are beneficial for effective communication. In this paper, we present and detail a dataset of dyadic interactions in 85 real-life cancer-afflicted couples and a set of observational behavior codes pertaining to interpersonal communication attributes. We describe and employ neural network-based systems for classifying these behaviors based on turn-level acoustic and lexical speech patterns. Furthermore, we investigate the effect of controlling for factors such as gender, patient/caregiver role and conversation content on behavior classification. Analysis of our preliminary results indicates the challenges in this task due to the nature of the targeted behaviors and suggests that techniques incorporating contextual processing might be better suited to tackle this problem. |
| Tasks | |
| Published | 2019-08-02 |
| URL | https://arxiv.org/abs/1908.00908v1 |
| https://arxiv.org/pdf/1908.00908v1.pdf | |
| PWC | https://paperswithcode.com/paper/predicting-behavior-in-cancer-afflicted |
| Repo | |
| Framework | |
Cloze-driven Pretraining of Self-attention Networks
| Title | Cloze-driven Pretraining of Self-attention Networks |
| Authors | Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, Michael Auli |
| Abstract | We present a new approach for pretraining a bi-directional transformer model that provides significant performance gains across a variety of language understanding problems. Our model solves a cloze-style word reconstruction task, where each word is ablated and must be predicted given the rest of the text. Experiments demonstrate large performance gains on GLUE and new state of the art results on NER as well as constituency parsing benchmarks, consistent with the concurrently introduced BERT model. We also present a detailed analysis of a number of factors that contribute to effective pretraining, including data domain and size, model capacity, and variations on the cloze objective. |
| Tasks | Constituency Parsing, Named Entity Recognition, Sentiment Analysis, Text Classification |
| Published | 2019-03-19 |
| URL | http://arxiv.org/abs/1903.07785v1 |
| http://arxiv.org/pdf/1903.07785v1.pdf | |
| PWC | https://paperswithcode.com/paper/cloze-driven-pretraining-of-self-attention |
| Repo | |
| Framework | |
Unveiling phase transitions with machine learning
| Title | Unveiling phase transitions with machine learning |
| Authors | Askery Canabarro, Felipe Fernandes Fanchini, André Luiz Malvezzi, Rodrigo Pereira, Rafael Chaves |
| Abstract | The classification of phase transitions is a central and challenging task in condensed matter physics. Typically, it relies on the identification of order parameters and the analysis of singularities in the free energy and its derivatives. Here, we propose an alternative framework to identify quantum phase transitions, employing both unsupervised and supervised machine learning techniques. Using the axial next-nearest neighbor Ising (ANNNI) model as a benchmark, we show how unsupervised learning can detect three phases (ferromagnetic, paramagnetic, and a cluster of the antiphase with the floating phase) as well as two distinct regions within the paramagnetic phase. Employing supervised learning we show that transfer learning becomes possible: a machine trained only with nearest-neighbour interactions can learn to identify a new type of phase occurring when next-nearest-neighbour interactions are introduced. All our results rely on few and low dimensional input data (up to twelve lattice sites), thus providing a computational friendly and general framework for the study of phase transitions in many-body systems. |
| Tasks | Transfer Learning |
| Published | 2019-04-02 |
| URL | http://arxiv.org/abs/1904.01486v1 |
| http://arxiv.org/pdf/1904.01486v1.pdf | |
| PWC | https://paperswithcode.com/paper/unveiling-phase-transitions-with-machine |
| Repo | |
| Framework | |
Identifying and Reducing Gender Bias in Word-Level Language Models
| Title | Identifying and Reducing Gender Bias in Word-Level Language Models |
| Authors | Shikha Bordia, Samuel R. Bowman |
| Abstract | Many text corpora exhibit socially problematic biases, which can be propagated or amplified in the models trained on such data. For example, doctor cooccurs more frequently with male pronouns than female pronouns. In this study we (i) propose a metric to measure gender bias; (ii) measure bias in a text corpus and the text generated from a recurrent neural network language model trained on the text corpus; (iii) propose a regularization loss term for the language model that minimizes the projection of encoder-trained embeddings onto an embedding subspace that encodes gender; (iv) finally, evaluate efficacy of our proposed method on reducing gender bias. We find this regularization method to be effective in reducing gender bias up to an optimal weight assigned to the loss term, beyond which the model becomes unstable as the perplexity increases. We replicate this study on three training corpora—Penn Treebank, WikiText-2, and CNN/Daily Mail—resulting in similar conclusions. |
| Tasks | Language Modelling |
| Published | 2019-04-05 |
| URL | http://arxiv.org/abs/1904.03035v1 |
| http://arxiv.org/pdf/1904.03035v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-and-reducing-gender-bias-in-word |
| Repo | |
| Framework | |
Efficient order picking methods in robotic mobile fulfillment systems
| Title | Efficient order picking methods in robotic mobile fulfillment systems |
| Authors | Lin Xie, Nils Thieme, Ruslan Krenzler, Hanyi Li |
| Abstract | Robotic mobile fulfillment systems (RMFSs) are a new type of warehousing system, which has received more attention recently, due to increasing growth in the e-commerce sector. Instead of sending pickers to the inventory area to search for and pick the ordered items, robots carry shelves (called “pods”) including ordered items from the inventory area to picking stations. In the picking stations, human pickers put ordered items into totes; then these items are transported by a conveyor to the packing stations. This type of warehousing system relieves the human pickers and improves the picking process. In this paper, we concentrate on decisions about the assignment of pods to stations and orders to stations to fulfill picking for each incoming customer’s order. In previous research for an RMFS with multiple picking stations, these decisions are made sequentially. Instead, we present a new integrated model. To improve the system performance even more, we extend our model by splitting orders. This means parts of an order are allowed to be picked at different stations. To the best of the authors’ knowledge, this is the first publication on split orders in an RMFS. We analyze different performance metrics, such as pile-on, pod-station visits, robot moving distance and order turn-over time. We compare the results of our models in different instances with the sequential method in our open-source simulation framework RAWSim-O. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1902.03092v1 |
| http://arxiv.org/pdf/1902.03092v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-order-picking-methods-in-robotic |
| Repo | |
| Framework | |
Land Use Classification Using Multi-neighborhood LBPs
| Title | Land Use Classification Using Multi-neighborhood LBPs |
| Authors | Harjot Singh Parmar |
| Abstract | In this paper we propose the use of multiple local binary patterns(LBPs) to effectively classify land use images. We use the UC Merced 21 class land use image dataset. Task is challenging for classification as the dataset contains intra class variability and inter class similarities. Our proposed method of using multi-neighborhood LBPs combined with nearest neighbor classifier is able to achieve an accuracy of 77.76%. Further class wise analysis is conducted and suitable suggestion are made for further improvements to classification accuracy. |
| Tasks | |
| Published | 2019-02-07 |
| URL | http://arxiv.org/abs/1902.03240v1 |
| http://arxiv.org/pdf/1902.03240v1.pdf | |
| PWC | https://paperswithcode.com/paper/land-use-classification-using-multi |
| Repo | |
| Framework | |
OVSNet : Towards One-Pass Real-Time Video Object Segmentation
| Title | OVSNet : Towards One-Pass Real-Time Video Object Segmentation |
| Authors | Peng Sun, Peiwen Lin, Guangliang Cheng, Jianping Shi, Jiawan Zhang, Xi Li |
| Abstract | Video object segmentation aims at accurately segmenting the target object regions across consecutive frames. It is technically challenging for coping with complicated factors (e.g., shape deformations, occlusion and out of the lens). Recent approaches have largely solved them by using backforth re-identification and bi-directional mask propagation. However, their methods are extremely slow and only support offline inference, which in principle cannot be applied in real time. Motivated by this observation, we propose a efficient detection-based paradigm for video object segmentation. We propose an unified One-Pass Video Segmentation framework (OVS-Net) for modeling spatial-temporal representation in a unified pipeline, which seamlessly integrates object detection, object segmentation, and object re-identification. The proposed framework lends itself to one-pass inference that effectively and efficiently performs video object segmentation. Moreover, we propose a maskguided attention module for modeling the multi-scale object boundary and multi-level feature fusion. Experiments on the challenging DAVIS 2017 demonstrate the effectiveness of the proposed framework with comparable performance to the state-of-the-art, and the great efficiency about 11.5 FPS towards pioneering real-time work to our knowledge, more than 5 times faster than other state-of-the-art methods. |
| Tasks | Object Detection, Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation |
| Published | 2019-05-24 |
| URL | https://arxiv.org/abs/1905.10064v2 |
| https://arxiv.org/pdf/1905.10064v2.pdf | |
| PWC | https://paperswithcode.com/paper/ovsnet-towards-one-pass-real-time-video |
| Repo | |
| Framework | |
Active Exploration in Markov Decision Processes
| Title | Active Exploration in Markov Decision Processes |
| Authors | Jean Tarbouriech, Alessandro Lazaric |
| Abstract | We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as possible. Similarly to active exploration in multi-armed bandit (MAB), states may have different levels of noise, so that the higher the noise, the more samples are needed. As the noise level is initially unknown, we need to trade off the exploration of the environment to estimate the noise and the exploitation of these estimates to compute a policy maximizing the accuracy of the mean predictions. We introduce a novel learning algorithm to solve this problem showing that active exploration in MDPs may be significantly more difficult than in MAB. We also derive a heuristic procedure to mitigate the negative effect of slowly mixing policies. Finally, we validate our findings on simple numerical simulations. |
| Tasks | |
| Published | 2019-02-28 |
| URL | http://arxiv.org/abs/1902.11199v1 |
| http://arxiv.org/pdf/1902.11199v1.pdf | |
| PWC | https://paperswithcode.com/paper/active-exploration-in-markov-decision |
| Repo | |
| Framework | |
Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference
| Title | Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference |
| Authors | Yue Wang, Jianghao Shen, Ting-Kuei Hu, Pengfei Xu, Tan Nguyen, Richard Baraniuk, Zhangyang Wang, Yingyan Lin |
| Abstract | State-of-the-art convolutional neural networks (CNNs) yield record-breaking predictive performance, yet at the cost of high-energy-consumption inference, that prohibits their widely deployments in resource-constrained Internet of Things (IoT) applications. We propose a dual dynamic inference (DDI) framework that highlights the following aspects: 1) we integrate both input-dependent and resource-dependent dynamic inference mechanisms under a unified framework in order to fit the varying IoT resource requirements in practice. DDI is able to both constantly suppress unnecessary costs for easy samples, and to halt inference for all samples to meet hard resource constraints enforced; 2) we propose a flexible multi-grained learning to skip (MGL2S) approach for input-dependent inference which allows simultaneous layer-wise and channel-wise skipping; 3) we extend DDI to complex CNN backbones such as DenseNet and show that DDI can be applied towards optimizing any specific resource goals including inference latency or energy cost. Extensive experiments demonstrate the superior inference accuracy-resource trade-off achieved by DDI, as well as the flexibility to control such trade-offs compared to existing peer methods. Specifically, DDI can achieve up to 4 times computational savings with the same or even higher accuracy as compared to existing competitive baselines. |
| Tasks | |
| Published | 2019-07-10 |
| URL | https://arxiv.org/abs/1907.04523v3 |
| https://arxiv.org/pdf/1907.04523v3.pdf | |
| PWC | https://paperswithcode.com/paper/dual-dynamic-inference-enabling-more |
| Repo | |
| Framework | |
Rate-Memory Trade-off for Multi-access Coded Caching with Uncoded Placement
| Title | Rate-Memory Trade-off for Multi-access Coded Caching with Uncoded Placement |
| Authors | Kota Srinivas Reddy, Nikhil Karamchandani |
| Abstract | We study a multi-access variant of the popular coded caching framework, which consists of a central server with a catalog of $N$ files, $K$ caches with limited memory $M$, and $K$ users such that each user has access to $L$ consecutive caches with a cyclic wrap-around and requests one file from the central server’s catalog. The server assists in file delivery by transmitting a message of size $R$ over a shared error-free link and the goal is to characterize the optimal rate-memory trade-off. This setup was studied previously by Hachem et al., where an achievable rate and an information-theoretic lower bound were derived. However, the multiplicative gap between them was shown to scale linearly with the access degree $L$ and thus order-optimality could not be established. A series of recent works have used a natural mapping of the coded caching problem to the well-known index coding problem to derive tighter characterizations of the optimal rate-memory trade-off under the additional assumption that the caches store uncoded content. We follow a similar strategy for the multi-access framework and provide new bounds for the optimal rate-memory trade-off $R^*(M)$ over all uncoded placement policies. In particular, we derive a new achievable rate for any $L \ge 1$ and a new lower bound, which works for any uncoded placement policy and $L \ge K/2$. We then establish that the (multiplicative) gap between the new achievable rate and the lower bound is at most $2$ independent of all parameters, thus establishing an order-optimal characterization of $R^*(M)$ for any $L\ge K/2$. This is a significant improvement over the previously known gap result, albeit under the restriction of uncoded placement policies. Finally, we also characterize $R^*(M)$ exactly for a few special cases. |
| Tasks | |
| Published | 2019-09-04 |
| URL | https://arxiv.org/abs/1909.01756v2 |
| https://arxiv.org/pdf/1909.01756v2.pdf | |
| PWC | https://paperswithcode.com/paper/rate-memory-trade-off-for-multi-access-coded |
| Repo | |
| Framework | |
A Fourier Perspective on Model Robustness in Computer Vision
| Title | A Fourier Perspective on Model Robustness in Computer Vision |
| Authors | Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, Justin Gilmer |
| Abstract | Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision. Data augmentation is a commonly used approach for improving robustness, however robustness gains are typically not uniform across corruption types. Indeed increasing performance in the presence of random noise is often met with reduced performance on other corruptions such as contrast change. Understanding when and why these sorts of trade-offs occur is a crucial step towards mitigating them. Towards this end, we investigate recently observed trade-offs caused by Gaussian data augmentation and adversarial training. We find that both methods improve robustness to corruptions that are concentrated in the high frequency domain while reducing robustness to corruptions that are concentrated in the low frequency domain. This suggests that one way to mitigate these trade-offs via data augmentation is to use a more diverse set of augmentations. Towards this end we observe that AutoAugment, a recently proposed data augmentation policy optimized for clean accuracy, achieves state-of-the-art robustness on the CIFAR-10-C benchmark. |
| Tasks | Data Augmentation |
| Published | 2019-06-21 |
| URL | https://arxiv.org/abs/1906.08988v2 |
| https://arxiv.org/pdf/1906.08988v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-fourier-perspective-on-model-robustness-in |
| Repo | |
| Framework | |
A Kalman filtering induced heuristic optimization based partitional data clustering
| Title | A Kalman filtering induced heuristic optimization based partitional data clustering |
| Authors | Arjun Pakrashi, Bidyut B. Chaudhuri |
| Abstract | Clustering algorithms have regained momentum with recent popularity of data mining and knowledge discovery approaches. To obtain good clustering in reasonable amount of time, various meta-heuristic approaches and their hybridization, sometimes with K-Means technique, have been employed. A Kalman Filtering based heuristic approach called Heuristic Kalman Algorithm (HKA) has been proposed a few years ago, which may be used for optimizing an objective function in data/feature space. In this paper at first HKA is employed in partitional data clustering. Then an improved approach named HKA-K is proposed, which combines the benefits of global exploration of HKA and the fast convergence of K-Means method. Implemented and tested on several datasets from UCI machine learning repository, the results obtained by HKA-K were compared with other hybrid meta-heuristic clustering approaches. It is shown that HKA-K is atleast as good as and often better than the other compared algorithms. |
| Tasks | |
| Published | 2019-01-25 |
| URL | http://arxiv.org/abs/1901.09082v1 |
| http://arxiv.org/pdf/1901.09082v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-kalman-filtering-induced-heuristic |
| Repo | |
| Framework | |
