Paper Group ANR 1081
A Power Efficient Artificial Neuron Using Superconducting Nanowires. A joint text mining-rank size investigation of the rhetoric structures of the US Presidents’ speeches. NeuralHydrology – Interpreting LSTMs in Hydrology. Bayesian Optimization Approach for Analog Circuit Synthesis Using Neural Network. Annotation and Classification of Sentence-le …
A Power Efficient Artificial Neuron Using Superconducting Nanowires
| Title | A Power Efficient Artificial Neuron Using Superconducting Nanowires |
| Authors | Emily Toomey, Ken Segall, Karl K. Berggren |
| Abstract | With the rising societal demand for more information-processing capacity with lower power consumption, alternative architectures inspired by the parallelism and robustness of the human brain have recently emerged as possible solutions. In particular, spiking neural networks (SNNs) offer a bio-realistic approach, relying on pulses analogous to action potentials as units of information. While software encoded networks provide flexibility and precision, they are often computationally expensive. As a result, hardware SNNs based on the spiking dynamics of a device or circuit represent an increasingly appealing direction. Here, we propose to use superconducting nanowires as a platform for the development of an artificial neuron. Building on an architecture first proposed for Josephson junctions, we rely on the intrinsic nonlinearity of two coupled nanowires to generate spiking behavior, and use electrothermal circuit simulations to demonstrate that the nanowire neuron reproduces multiple characteristics of biological neurons. Furthermore, by harnessing the nonlinearity of the superconducting nanowire’s inductance, we develop a design for a variable inductive synapse capable of both excitatory and inhibitory control. We demonstrate that this synapse design supports direct fanout, a feature that has been difficult to achieve in other superconducting architectures, and that the nanowire neuron’s nominal energy performance is competitive with that of current technologies. |
| Tasks | |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.00263v1 |
| https://arxiv.org/pdf/1907.00263v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-power-efficient-artificial-neuron-using |
| Repo | |
| Framework | |
A joint text mining-rank size investigation of the rhetoric structures of the US Presidents’ speeches
| Title | A joint text mining-rank size investigation of the rhetoric structures of the US Presidents’ speeches |
| Authors | Valerio Ficcadenti, Roy Cerqueti, Marcel Ausloos |
| Abstract | This work presents a text mining context and its use for a deep analysis of the messages delivered by the politicians. Specifically, we deal with an expert systems-based exploration of the rhetoric dynamics of a large collection of US Presidents’ speeches, ranging from Washington to Trump. In particular, speeches are viewed as complex expert systems whose structures can be effectively analyzed through rank-size laws. The methodological contribution of the paper is twofold. First, we develop a text mining-based procedure for the construction of the dataset by using a web scraping routine on the Miller Center website – the repository collecting the speeches. Second, we explore the implicit structure of the discourse data by implementing a rank-size procedure over the individual speeches, being the words of each speech ranked in terms of their frequencies. The scientific significance of the proposed combination of text-mining and rank-size approaches can be found in its flexibility and generality, which let it be reproducible to a wide set of expert systems and text mining contexts. The usefulness of the proposed method and the speech subsequent analysis is demonstrated by the findings themselves. Indeed, in terms of impact, it is worth noting that interesting conclusions of social, political and linguistic nature on how 45 United States Presidents, from April 30, 1789 till February 28, 2017 delivered political messages can be carried out. Indeed, the proposed analysis shows some remarkable regularities, not only inside a given speech, but also among different speeches. Moreover, under a purely methodological perspective, the presented contribution suggests possible ways of generating a linguistic decision-making algorithm. |
| Tasks | Decision Making |
| Published | 2019-05-09 |
| URL | https://arxiv.org/abs/1905.04705v1 |
| https://arxiv.org/pdf/1905.04705v1.pdf | |
| PWC | https://paperswithcode.com/paper/190504705 |
| Repo | |
| Framework | |
NeuralHydrology – Interpreting LSTMs in Hydrology
| Title | NeuralHydrology – Interpreting LSTMs in Hydrology |
| Authors | Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer |
| Abstract | Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in the field of hydrology, in which the river discharge has to be predicted from meteorological observations. LSTMs are particularly well-suited for this problem since memory cells can represent dynamic reservoirs and storages, which are essential components in state-space modelling approaches of the hydrological system. On basis of two different catchments, one with snow influence and one without, we demonstrate how the trained model can be analyzed and interpreted. In the process, we show that the network internally learns to represent patterns that are consistent with our qualitative understanding of the hydrological system. |
| Tasks | |
| Published | 2019-03-19 |
| URL | https://arxiv.org/abs/1903.07903v2 |
| https://arxiv.org/pdf/1903.07903v2.pdf | |
| PWC | https://paperswithcode.com/paper/neuralhydrology-interpreting-lstms-in |
| Repo | |
| Framework | |
Bayesian Optimization Approach for Analog Circuit Synthesis Using Neural Network
| Title | Bayesian Optimization Approach for Analog Circuit Synthesis Using Neural Network |
| Authors | Shuhan Zhang, Wenlong Lyu, Fan Yang, Changhao Yan, Dian Zhou, Xuan Zeng |
| Abstract | Bayesian optimization with Gaussian process as surrogate model has been successfully applied to analog circuit synthesis. In the traditional Gaussian process regression model, the kernel functions are defined explicitly. The computational complexity of training is O(N 3 ), and the computation complexity of prediction is O(N 2 ), where N is the number of training data. Gaussian process model can also be derived from a weight space view, where the original data are mapped to feature space, and the kernel function is defined as the inner product of nonlinear features. In this paper, we propose a Bayesian optimization approach for analog circuit synthesis using neural network. We use deep neural network to extract good feature representations, and then define Gaussian process using the extracted features. Model averaging method is applied to improve the quality of uncertainty prediction. Compared to Gaussian process model with explicitly defined kernel functions, the neural-network-based Gaussian process model can automatically learn a kernel function from data, which makes it possible to provide more accurate predictions and thus accelerate the follow-up optimization procedure. Also, the neural-network-based model has O(N) training time and constant prediction time. The efficiency of the proposed method has been verified by two real-world analog circuits. |
| Tasks | |
| Published | 2019-12-01 |
| URL | https://arxiv.org/abs/1912.00402v1 |
| https://arxiv.org/pdf/1912.00402v1.pdf | |
| PWC | https://paperswithcode.com/paper/bayesian-optimization-approach-for-analog |
| Repo | |
| Framework | |
Annotation and Classification of Sentence-level Revision Improvement
| Title | Annotation and Classification of Sentence-level Revision Improvement |
| Authors | Tazin Afrin, Diane Litman |
| Abstract | Studies of writing revisions rarely focus on revision quality. To address this issue, we introduce a corpus of between-draft revisions of student argumentative essays, annotated as to whether each revision improves essay quality. We demonstrate a potential usage of our annotations by developing a machine learning model to predict revision improvement. With the goal of expanding training data, we also extract revisions from a dataset edited by expert proofreaders. Our results indicate that blending expert and non-expert revisions increases model performance, with expert data particularly important for predicting low-quality revisions. |
| Tasks | |
| Published | 2019-09-03 |
| URL | https://arxiv.org/abs/1909.05309v1 |
| https://arxiv.org/pdf/1909.05309v1.pdf | |
| PWC | https://paperswithcode.com/paper/annotation-and-classification-of-sentence-1 |
| Repo | |
| Framework | |
Capacity allocation analysis of neural networks: A tool for principled architecture design
| Title | Capacity allocation analysis of neural networks: A tool for principled architecture design |
| Authors | Jonathan Donier |
| Abstract | Designing neural network architectures is a task that lies somewhere between science and art. For a given task, some architectures are eventually preferred over others, based on a mix of intuition, experience, experimentation and luck. For many tasks, the final word is attributed to the loss function, while for some others a further perceptual evaluation is necessary to assess and compare performance across models. In this paper, we introduce the concept of capacity allocation analysis, with the aim of shedding some light on what network architectures focus their modelling capacity on, when used on a given task. We focus more particularly on spatial capacity allocation, which analyzes a posteriori the effective number of parameters that a given model has allocated for modelling dependencies on a given point or region in the input space, in linear settings. We use this framework to perform a quantitative comparison between some classical architectures on various synthetic tasks. Finally, we consider how capacity allocation might translate in non-linear settings. |
| Tasks | |
| Published | 2019-02-12 |
| URL | http://arxiv.org/abs/1902.04485v1 |
| http://arxiv.org/pdf/1902.04485v1.pdf | |
| PWC | https://paperswithcode.com/paper/capacity-allocation-analysis-of-neural |
| Repo | |
| Framework | |
AI for Social Impact: Learning and Planning in the Data-to-Deployment Pipeline
| Title | AI for Social Impact: Learning and Planning in the Data-to-Deployment Pipeline |
| Authors | Andrew Perrault, Fei Fang, Arunesh Sinha, Milind Tambe |
| Abstract | With the maturing of AI and multiagent systems research, we have a tremendous opportunity to direct these advances towards addressing complex societal problems. In pursuit of this goal of AI for Social Impact, we as AI researchers must go beyond improvements in computational methodology; it is important to step out in the field to demonstrate social impact. To this end, we focus on the problems of public safety and security, wildlife conservation, and public health in low-resource communities, and present research advances in multiagent systems to address one key cross-cutting challenge: how to effectively deploy our limited intervention resources in these problem domains. We present case studies from our deployments around the world as well as lessons learned that we hope are of use to researchers who are interested in AI for Social Impact. In pushing this research agenda, we believe AI can indeed play an important role in fighting social injustice and improving society. |
| Tasks | |
| Published | 2019-12-16 |
| URL | https://arxiv.org/abs/2001.00088v1 |
| https://arxiv.org/pdf/2001.00088v1.pdf | |
| PWC | https://paperswithcode.com/paper/ai-for-social-impact-learning-and-planning-in |
| Repo | |
| Framework | |
Natural Language Generation Using Reinforcement Learning with External Rewards
| Title | Natural Language Generation Using Reinforcement Learning with External Rewards |
| Authors | Vidhushini Srinivasan, Sashank Santhanam, Samira Shaikh |
| Abstract | We propose an approach towards natural language generation using a bidirectional encoder-decoder which incorporates external rewards through reinforcement learning (RL). We use attention mechanism and maximum mutual information as an initial objective function using RL. Using a two-part training scheme, we train an external reward analyzer to predict the external rewards and then use the predicted rewards to maximize the expected rewards (both internal and external). We evaluate the system on two standard dialogue corpora - Cornell Movie Dialog Corpus and Yelp Restaurant Review Corpus. We report standard evaluation metrics including BLEU, ROUGE-L, and perplexity as well as human evaluation to validate our approach. |
| Tasks | Text Generation |
| Published | 2019-11-26 |
| URL | https://arxiv.org/abs/1911.11404v1 |
| https://arxiv.org/pdf/1911.11404v1.pdf | |
| PWC | https://paperswithcode.com/paper/natural-language-generation-using |
| Repo | |
| Framework | |
Context-Aware Local Differential Privacy
| Title | Context-Aware Local Differential Privacy |
| Authors | Jayadev Acharya, Keith Bonawitz, Peter Kairouz, Daniel Ramage, Ziteng Sun |
| Abstract | Local differential privacy (LDP) is a strong notion of privacy for individual users that often comes at the expense of a significant drop in utility. The classical definition of LDP assumes that all elements in the data domain are equally sensitive. However, in many applications, some symbols are more sensitive than others. This work proposes a context-aware framework of local differential privacy that allows a privacy designer to incorporate the application’s context into the privacy definition. For binary data domains, we provide a universally optimal privatization scheme and highlight its connections to Warner’s randomized response (RR) and Mangat’s improved response. Motivated by geolocation and web search applications, for $k$-ary data domains, we consider two special cases of context-aware LDP: block-structured LDP and high-low LDP. We study discrete distribution estimation and provide communication-efficient, sample-optimal schemes and information-theoretic lower bounds for both models. We show that using contextual information can require fewer samples than classical LDP to achieve the same accuracy. |
| Tasks | |
| Published | 2019-10-31 |
| URL | https://arxiv.org/abs/1911.00038v1 |
| https://arxiv.org/pdf/1911.00038v1.pdf | |
| PWC | https://paperswithcode.com/paper/context-aware-local-differential-privacy |
| Repo | |
| Framework | |
Registration of retinal images from Public Health by minimising an error between vessels using an affine model with radial distortions
| Title | Registration of retinal images from Public Health by minimising an error between vessels using an affine model with radial distortions |
| Authors | Guillaume Noyel, R Thomas, S Iles, G Bhakta, A Crowder, D. Owens, P. Boyle |
| Abstract | In order to estimate a registration model of eye fundus images made of an affinity and two radial distortions, we introduce an estimation criterion based on an error between the vessels. In [1], we estimated this model by minimising the error between characteristics points. In this paper, the detected vessels are selected using the circle and ellipse equations of the overlap area boundaries deduced from our model. Our method successfully registers 96 % of the 271 pairs in a Public Health dataset acquired mostly with different cameras. This is better than our previous method [1] and better than three other state-of-the-art methods. On a publicly available dataset, ours still better register the images than the reference method. |
| Tasks | |
| Published | 2019-04-17 |
| URL | http://arxiv.org/abs/1904.12733v1 |
| http://arxiv.org/pdf/1904.12733v1.pdf | |
| PWC | https://paperswithcode.com/paper/190412733 |
| Repo | |
| Framework | |
Symbolic regression by uniform random global search
| Title | Symbolic regression by uniform random global search |
| Authors | Sohrab Towfighi |
| Abstract | Symbolic regression (SR) is a data analysis problem where we search for the mathematical expression that best fits a numerical dataset. It is a global optimization problem. The most popular approach to SR is by genetic programming (SRGP). It is a common paradigm to compare an algorithm’s performance to that of random search, but the data comparing SRGP to random search is lacking. We describe a novel algorithm for SR, namely SR by uniform random global search (SRURGS), also known as pure random search. We conduct experiments comparing SRURGS with SRGP using 100 randomly generated equations. Our results suggest that a SRGP is faster than SRURGS in producing equations with good R^2 for simple problems. However, our experiments suggest that SRURGS is more robust than SRGP, able to produce good output in more challenging problems. As SRURGS is arguably the simplest global search algorithm, we believe it should serve as a control algorithm against which other symbolic regression algorithms are compared. SRURGS has only one tuning parameter, and is conceptually very simple, making it a useful tool in solving SR problems. The method produces random equations, which is useful for the generation of symbolic regression benchmark problems. We have released well documented and open-source python code, currently under formal peer-review, so that interested researchers can deploy the tool in practice. |
| Tasks | |
| Published | 2019-06-18 |
| URL | https://arxiv.org/abs/1906.07848v4 |
| https://arxiv.org/pdf/1906.07848v4.pdf | |
| PWC | https://paperswithcode.com/paper/a-general-methodology-to-assess-symbolic |
| Repo | |
| Framework | |
Controlled CNN-based Sequence Labeling for Aspect Extraction
| Title | Controlled CNN-based Sequence Labeling for Aspect Extraction |
| Authors | Lei Shu, Hu Xu, Bing Liu |
| Abstract | One key task of fine-grained sentiment analysis on reviews is to extract aspects or features that users have expressed opinions on. This paper focuses on supervised aspect extraction using a modified CNN called controlled CNN (Ctrl). The modified CNN has two types of control modules. Through asynchronous parameter updating, it prevents over-fitting and boosts CNN’s performance significantly. This model achieves state-of-the-art results on standard aspect extraction datasets. To the best of our knowledge, this is the first paper to apply control modules to aspect extraction. |
| Tasks | Aspect Extraction, Sentiment Analysis |
| Published | 2019-05-15 |
| URL | https://arxiv.org/abs/1905.06407v2 |
| https://arxiv.org/pdf/1905.06407v2.pdf | |
| PWC | https://paperswithcode.com/paper/controlled-cnn-based-sequence-labeling-for |
| Repo | |
| Framework | |
Combining Physically-Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn from Mismatch?
| Title | Combining Physically-Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn from Mismatch? |
| Authors | Alexander Y. Sun, Bridget R. Scanlon, Zizhan Zhang, David Walling, Soumendra N. Bhanja, Abhijit Mukherjee, Zhi Zhong |
| Abstract | Global hydrological and land surface models are increasingly used for tracking terrestrial total water storage (TWS) dynamics, but the utility of existing models is hampered by conceptual and/or data uncertainties related to various underrepresented and unrepresented processes, such as groundwater storage. The gravity recovery and climate experiment (GRACE) satellite mission provided a valuable independent data source for tracking TWS at regional and continental scales. Strong interests exist in fusing GRACE data into global hydrological models to improve their predictive performance. Here we develop and apply deep convolutional neural network (CNN) models to learn the spatiotemporal patterns of mismatch between TWS anomalies (TWSA) derived from GRACE and those simulated by NOAH, a widely used land surface model. Once trained, our CNN models can be used to correct the NOAH simulated TWSA without requiring GRACE data, potentially filling the data gap between GRACE and its follow-on mission, GRACE-FO. Our methodology is demonstrated over India, which has experienced significant groundwater depletion in recent decades that is nevertheless not being captured by the NOAH model. Results show that the CNN models significantly improve the match with GRACE TWSA, achieving a country-average correlation coefficient of 0.94 and Nash-Sutcliff efficient of 0.87, or 14% and 52% improvement respectively over the original NOAH TWSA. At the local scale, the learned mismatch pattern correlates well with the observed in situ groundwater storage anomaly data for most parts of India, suggesting that deep learning models effectively compensate for the missing groundwater component in NOAH for this study region. |
| Tasks | |
| Published | 2019-01-31 |
| URL | http://arxiv.org/abs/1902.01933v1 |
| http://arxiv.org/pdf/1902.01933v1.pdf | |
| PWC | https://paperswithcode.com/paper/combining-physically-based-modeling-and-deep |
| Repo | |
| Framework | |
Understanding Human Judgments of Causality
| Title | Understanding Human Judgments of Causality |
| Authors | Masahiro Kazama, Yoshihiko Suhara, Andrey Bogomolov, Alex `Sandy’ Pentland | |
| Abstract | Discriminating between causality and correlation is a major problem in machine learning, and theoretical tools for determining causality are still being developed. However, people commonly make causality judgments and are often correct, even in unfamiliar domains. What are humans doing to make these judgments? This paper examines differences in human experts’ and non-experts’ ability to attribute causality by comparing their performances to those of machine-learning algorithms. We collected human judgments by using Amazon Mechanical Turk (MTurk) and then divided the human subjects into two groups: experts and non-experts. We also prepared expert and non-expert machine algorithms based on different training of convolutional neural network (CNN) models. The results showed that human experts’ judgments were similar to those made by an “expert” CNN model trained on a large number of examples from the target domain. The human non-experts’ judgments resembled the prediction outputs of the CNN model that was trained on only the small number of examples used during the MTurk instruction. We also analyzed the differences between the expert and non-expert machine algorithms based on their neural representations to evaluate the performances, providing insight into the human experts’ and non-experts’ cognitive abilities. |
| Tasks | |
| Published | 2019-12-19 |
| URL | https://arxiv.org/abs/1912.08998v1 |
| https://arxiv.org/pdf/1912.08998v1.pdf | |
| PWC | https://paperswithcode.com/paper/understanding-human-judgments-of-causality |
| Repo | |
| Framework | |
Misleading Failures of Partial-input Baselines
| Title | Misleading Failures of Partial-input Baselines |
| Authors | Shi Feng, Eric Wallace, Jordan Boyd-Graber |
| Abstract | Recent work establishes dataset difficulty and removes annotation artifacts via partial-input baselines (e.g., hypothesis-only models for SNLI or question-only models for VQA). When a partial-input baseline gets high accuracy, a dataset is cheatable. However, the converse is not necessarily true: the failure of a partial-input baseline does not mean a dataset is free of artifacts. To illustrate this, we first design artificial datasets which contain trivial patterns in the full input that are undetectable by any partial-input model. Next, we identify such artifacts in the SNLI dataset - a hypothesis-only model augmented with trivial patterns in the premise can solve 15% of the examples that are previously considered “hard”. Our work provides a caveat for the use of partial-input baselines for dataset verification and creation. |
| Tasks | Natural Language Inference, Visual Question Answering |
| Published | 2019-05-14 |
| URL | https://arxiv.org/abs/1905.05778v3 |
| https://arxiv.org/pdf/1905.05778v3.pdf | |
| PWC | https://paperswithcode.com/paper/misleading-failures-of-partial-input |
| Repo | |
| Framework | |
