Paper Group ANR 572
Mining Procedures from Technical Support Documents. Multi-label Multi-task Deep Learning for Behavioral Coding. Comparative study of Discrete Wavelet Transforms and Wavelet Tensor Train decomposition to feature extraction of FTIR data of medicinal plants. Light Gated Recurrent Units for Speech Recognition. On The Differential Privacy of Thompson Sa …
Mining Procedures from Technical Support Documents
| Title | Mining Procedures from Technical Support Documents |
| Authors | Abhirut Gupta, Abhay Khosla, Gautam Singh, Gargi Dasgupta |
| Abstract | Guided troubleshooting is an inherent task in the domain of technical support services. When a customer experiences an issue with the functioning of a technical service or a product, an expert user helps guide the customer through a set of steps comprising a troubleshooting procedure. The objective is to identify the source of the problem through a set of diagnostic steps and observations, and arrive at a resolution. Procedures containing these set of diagnostic steps and observations in response to different problems are common artifacts in the body of technical support documentation. The ability to use machine learning and linguistics to understand and leverage these procedures for applications like intelligent chatbots or robotic process automation, is crucial. Existing research on question answering or intelligent chatbots does not look within procedures or deep-understand them. In this paper, we outline a system for mining procedures from technical support documents. We create models for solving important subproblems like extraction of procedures, identifying decision points within procedures, identifying blocks of instructions corresponding to these decision points and mapping instructions within a decision block. We also release a dataset containing our manual annotations on publicly available support documents, to promote further research on the problem. |
| Tasks | Question Answering |
| Published | 2018-05-24 |
| URL | http://arxiv.org/abs/1805.09780v1 |
| http://arxiv.org/pdf/1805.09780v1.pdf | |
| PWC | https://paperswithcode.com/paper/mining-procedures-from-technical-support |
| Repo | |
| Framework | |
Multi-label Multi-task Deep Learning for Behavioral Coding
| Title | Multi-label Multi-task Deep Learning for Behavioral Coding |
| Authors | James Gibson, David C. Atkins, Torrey Creed, Zac Imel, Panayiotis Georgiou, Shrikanth Narayanan |
| Abstract | We propose a methodology for estimating human behaviors in psychotherapy sessions using mutli-label and multi-task learning paradigms. We discuss the problem of behavioral coding in which data of human interactions is the annotated with labels to describe relevant human behaviors of interest. We describe two related, yet distinct, corpora consisting of therapist client interactions in psychotherapy sessions. We experimentally compare the proposed learning approaches for estimating behaviors of interest in these datasets. Specifically, we compare single and multiple label learning approaches, single and multiple task learning approaches, and evaluate the performance of these approaches when incorporating turn context. We demonstrate the prediction performance gains which can be achieved by using the proposed paradigms and discuss the insights these models provide into these complex interactions. |
| Tasks | Multi-Task Learning |
| Published | 2018-10-29 |
| URL | http://arxiv.org/abs/1810.12349v2 |
| http://arxiv.org/pdf/1810.12349v2.pdf | |
| PWC | https://paperswithcode.com/paper/multi-label-multi-task-deep-learning-for |
| Repo | |
| Framework | |
Comparative study of Discrete Wavelet Transforms and Wavelet Tensor Train decomposition to feature extraction of FTIR data of medicinal plants
| Title | Comparative study of Discrete Wavelet Transforms and Wavelet Tensor Train decomposition to feature extraction of FTIR data of medicinal plants |
| Authors | Pavel Kharyuk, Dmitry Nazarenko, Ivan Oseledets |
| Abstract | Fourier-transform infra-red (FTIR) spectra of samples from 7 plant species were used to explore the influence of preprocessing and feature extraction on efficiency of machine learning algorithms. Wavelet Tensor Train (WTT) and Discrete Wavelet Transforms (DWT) were compared as feature extraction techniques for FTIR data of medicinal plants. Various combinations of signal processing steps showed different behavior when applied to classification and clustering tasks. Best results for WTT and DWT found through grid search were similar, significantly improving quality of clustering as well as classification accuracy for tuned logistic regression in comparison to original spectra. Unlike DWT, WTT has only one parameter to be tuned (rank), making it a more versatile and easier to use as a data processing tool in various signal processing applications. |
| Tasks | |
| Published | 2018-07-18 |
| URL | http://arxiv.org/abs/1807.07099v1 |
| http://arxiv.org/pdf/1807.07099v1.pdf | |
| PWC | https://paperswithcode.com/paper/comparative-study-of-discrete-wavelet |
| Repo | |
| Framework | |
Light Gated Recurrent Units for Speech Recognition
| Title | Light Gated Recurrent Units for Speech Recognition |
| Authors | Mirco Ravanelli, Philemon Brakel, Maurizio Omologo, Yoshua Bengio |
| Abstract | A field that has directly benefited from the recent advances in deep learning is Automatic Speech Recognition (ASR). Despite the great achievements of the past decades, however, a natural and robust human-machine speech interaction still appears to be out of reach, especially in challenging environments characterized by significant noise and reverberation. To improve robustness, modern speech recognizers often employ acoustic models based on Recurrent Neural Networks (RNNs), that are naturally able to exploit large time contexts and long-term speech modulations. It is thus of great interest to continue the study of proper techniques for improving the effectiveness of RNNs in processing speech signals. In this paper, we revise one of the most popular RNN models, namely Gated Recurrent Units (GRUs), and propose a simplified architecture that turned out to be very effective for ASR. The contribution of this work is two-fold: First, we analyze the role played by the reset gate, showing that a significant redundancy with the update gate occurs. As a result, we propose to remove the former from the GRU design, leading to a more efficient and compact single-gate model. Second, we propose to replace hyperbolic tangent with ReLU activations. This variation couples well with batch normalization and could help the model learn long-term dependencies without numerical issues. Results show that the proposed architecture, called Light GRU (Li-GRU), not only reduces the per-epoch training time by more than 30% over a standard GRU, but also consistently improves the recognition accuracy across different tasks, input features, noisy conditions, as well as across different ASR paradigms, ranging from standard DNN-HMM speech recognizers to end-to-end CTC models. |
| Tasks | Speech Recognition |
| Published | 2018-03-26 |
| URL | http://arxiv.org/abs/1803.10225v1 |
| http://arxiv.org/pdf/1803.10225v1.pdf | |
| PWC | https://paperswithcode.com/paper/light-gated-recurrent-units-for-speech |
| Repo | |
| Framework | |
On The Differential Privacy of Thompson Sampling With Gaussian Prior
| Title | On The Differential Privacy of Thompson Sampling With Gaussian Prior |
| Authors | Aristide C. Y. Tossou, Christos Dimitrakakis |
| Abstract | We show that Thompson Sampling with Gaussian Prior as detailed by Algorithm 2 in (Agrawal & Goyal, 2013) is already differentially private. Theorem 1 show that it enjoys a very competitive privacy loss of only $\mathcal{O}(\ln^2 T)$ after T rounds. Finally, Theorem 2 show that one can control the privacy loss to any desirable $\epsilon$ level by appropriately increasing the variance of the samples from the Gaussian posterior. And this increases the regret only by a term of $\mathcal{O}(\frac{\ln^2 T}{\epsilon})$. This compares favorably to the previous result for Thompson Sampling in the literature ((Mishra & Thakurta, 2015)) which adds a term of $\mathcal{O}(\frac{K \ln^3 T}{\epsilon^2})$ to the regret in order to achieve the same privacy level. Furthermore, our result use the basic Thompson Sampling with few modifications whereas the result of (Mishra & Thakurta, 2015) required sophisticated constructions. |
| Tasks | |
| Published | 2018-06-24 |
| URL | http://arxiv.org/abs/1806.09192v1 |
| http://arxiv.org/pdf/1806.09192v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-differential-privacy-of-thompson |
| Repo | |
| Framework | |
A Meta-Learning Approach for Custom Model Training
| Title | A Meta-Learning Approach for Custom Model Training |
| Authors | Amir Erfan Eshratifar, Mohammad Saeed Abrishami, David Eigen, Massoud Pedram |
| Abstract | Transfer-learning and meta-learning are two effective methods to apply knowledge learned from large data sources to new tasks. In few-class, few-shot target task settings (i.e. when there are only a few classes and training examples available in the target task), meta-learning approaches that optimize for future task learning have outperformed the typical transfer approach of initializing model weights from a pre-trained starting point. But as we experimentally show, meta-learning algorithms that work well in the few-class setting do not generalize well in many-shot and many-class cases. In this paper, we propose a joint training approach that combines both transfer-learning and meta-learning. Benefiting from the advantages of each, our method obtains improved generalization performance on unseen target tasks in both few- and many-class and few- and many-shot scenarios. |
| Tasks | Meta-Learning, Transfer Learning |
| Published | 2018-09-21 |
| URL | http://arxiv.org/abs/1809.08346v2 |
| http://arxiv.org/pdf/1809.08346v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-meta-learning-approach-for-custom-model |
| Repo | |
| Framework | |
Optimal approximation of continuous functions by very deep ReLU networks
| Title | Optimal approximation of continuous functions by very deep ReLU networks |
| Authors | Dmitry Yarotsky |
| Abstract | We consider approximations of general continuous functions on finite-dimensional cubes by general deep ReLU neural networks and study the approximation rates with respect to the modulus of continuity of the function and the total number of weights $W$ in the network. We establish the complete phase diagram of feasible approximation rates and show that it includes two distinct phases. One phase corresponds to slower approximations that can be achieved with constant-depth networks and continuous weight assignments. The other phase provides faster approximations at the cost of depths necessarily growing as a power law $L\sim W^{\alpha}, 0<\alpha\le 1,$ and with necessarily discontinuous weight assignments. In particular, we prove that constant-width fully-connected networks of depth $L\sim W$ provide the fastest possible approximation rate $\f-\widetilde f_\infty = O(\omega_f(O(W^{-2/\nu})))$ that cannot be achieved with less deep networks. |
| Tasks | |
| Published | 2018-02-10 |
| URL | http://arxiv.org/abs/1802.03620v2 |
| http://arxiv.org/pdf/1802.03620v2.pdf | |
| PWC | https://paperswithcode.com/paper/optimal-approximation-of-continuous-functions |
| Repo | |
| Framework | |
Rate-Adaptive Neural Networks for Spatial Multiplexers
| Title | Rate-Adaptive Neural Networks for Spatial Multiplexers |
| Authors | Suhas Lohit, Rajhans Singh, Kuldeep Kulkarni, Pavan Turaga |
| Abstract | In resource-constrained environments, one can employ spatial multiplexing cameras to acquire a small number of measurements of a scene, and perform effective reconstruction or high-level inference using purely data-driven neural networks. However, once trained, the measurement matrix and the network are valid only for a single measurement rate (MR) chosen at training time. To overcome this drawback, we answer the following question: How can we jointly design the measurement operator and the reconstruction/inference network so that the system can operate over a \textit{range} of MRs? To this end, we present a novel training algorithm, for learning \textbf{\textit{rate-adaptive}} networks. Using standard datasets, we demonstrate that, when tested over a range of MRs, a rate-adaptive network can provide high quality reconstruction over a the entire range, resulting in up to about 15 dB improvement over previous methods, where the network is valid for only one MR. We demonstrate the effectiveness of our approach for sample-efficient object tracking where video frames are acquired at dynamically varying MRs. We also extend this algorithm to learn the measurement operator in conjunction with image recognition networks. Experiments on MNIST and CIFAR-10 confirm the applicability of our algorithm to different tasks. |
| Tasks | Object Tracking |
| Published | 2018-09-08 |
| URL | http://arxiv.org/abs/1809.02850v1 |
| http://arxiv.org/pdf/1809.02850v1.pdf | |
| PWC | https://paperswithcode.com/paper/rate-adaptive-neural-networks-for-spatial |
| Repo | |
| Framework | |
Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells
| Title | Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells |
| Authors | Kelvin Loh, Pejman Shoeibi Omrani, Ruud van der Linden |
| Abstract | The prediction of the gas production from mature gas wells, due to their complex end-of-life behavior, is challenging and crucial for operational decision making. In this paper, we apply a modified deep LSTM model for prediction of the gas flow rates in mature gas wells, including the uncertainties in input parameters. Additionally, due to changes in the system in time and in order to increase the accuracy and robustness of the prediction, the Ensemble Kalman Filter (EnKF) is used to update the flow rate predictions based on new observations. The developed approach was tested on the data from two mature gas production wells in which their production is highly dynamic and suffering from salt deposition. The results show that the flow predictions using the EnKF updated model leads to better Jeffreys’ J-divergences than the predictions without the EnKF model updating scheme. |
| Tasks | Decision Making |
| Published | 2018-02-14 |
| URL | http://arxiv.org/abs/1802.05141v2 |
| http://arxiv.org/pdf/1802.05141v2.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-and-data-assimilation-for-real |
| Repo | |
| Framework | |
Text-mining and ontologies: new approaches to knowledge discovery of microbial diversity
| Title | Text-mining and ontologies: new approaches to knowledge discovery of microbial diversity |
| Authors | Claire Nédellec, Robert Bossy, Estelle Chaix, Louise Deléger |
| Abstract | Microbiology research has access to a very large amount of public information on the habitats of microorganisms. Many areas of microbiology research uses this information, primarily in biodiversity studies. However the habitat information is expressed in unstructured natural language form, which hinders its exploitation at large-scale. It is very common for similar habitats to be described by different terms, which makes them hard to compare automatically, e.g. intestine and gut. The use of a common reference to standardize these habitat descriptions as claimed by (Ivana et al., 2010) is a necessity. We propose the ontology called OntoBiotope that we have been developing since 2010. The OntoBiotope ontology is in a formal machine-readable representation that enables indexing of information as well as conceptualization and reasoning. |
| Tasks | |
| Published | 2018-05-10 |
| URL | http://arxiv.org/abs/1805.04107v2 |
| http://arxiv.org/pdf/1805.04107v2.pdf | |
| PWC | https://paperswithcode.com/paper/text-mining-and-ontologies-new-approaches-to |
| Repo | |
| Framework | |
Machine learning-based colon deformation estimation method for colonoscope tracking
| Title | Machine learning-based colon deformation estimation method for colonoscope tracking |
| Authors | Masahiro Oda, Takayuki Kitasaka, Kazuhiro Furukawa, Ryoji Miyahara, Yoshiki Hirooka, Hidemi Goto, Nassir Navab, Kensaku Mori |
| Abstract | This paper presents a colon deformation estimation method, which can be used to estimate colon deformations during colonoscope insertions. Colonoscope tracking or navigation system that navigates a physician to polyp positions during a colonoscope insertion is required to reduce complications such as colon perforation. A previous colonoscope tracking method obtains a colonoscope position in the colon by registering a colonoscope shape and a colon shape. The colonoscope shape is obtained using an electromagnetic sensor, and the colon shape is obtained from a CT volume. However, large tracking errors were observed due to colon deformations occurred during colonoscope insertions. Such deformations make the registration difficult. Because the colon deformation is caused by a colonoscope, there is a strong relationship between the colon deformation and the colonoscope shape. An estimation method of colon deformations occur during colonoscope insertions is necessary to reduce tracking errors. We propose a colon deformation estimation method. This method is used to estimate a deformed colon shape from a colonoscope shape. We use the regression forests algorithm to estimate a deformed colon shape. The regression forests algorithm is trained using pairs of colon and colonoscope shapes, which contains deformations occur during colonoscope insertions. As a preliminary study, we utilized the method to estimate deformations of a colon phantom. In our experiments, the proposed method correctly estimated deformed colon phantom shapes. |
| Tasks | |
| Published | 2018-06-08 |
| URL | http://arxiv.org/abs/1806.03014v1 |
| http://arxiv.org/pdf/1806.03014v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-learning-based-colon-deformation |
| Repo | |
| Framework | |
Feedback alignment in deep convolutional networks
| Title | Feedback alignment in deep convolutional networks |
| Authors | Theodore H. Moskovitz, Ashok Litwin-Kumar, L. F. Abbott |
| Abstract | Ongoing studies have identified similarities between neural representations in biological networks and in deep artificial neural networks. This has led to renewed interest in developing analogies between the backpropagation learning algorithm used to train artificial networks and the synaptic plasticity rules operative in the brain. These efforts are challenged by biologically implausible features of backpropagation, one of which is a reliance on symmetric forward and backward synaptic weights. A number of methods have been proposed that do not rely on weight symmetry but, thus far, these have failed to scale to deep convolutional networks and complex data. We identify principal obstacles to the scalability of such algorithms and introduce several techniques to mitigate them. We demonstrate that a modification of the feedback alignment method that enforces a weaker form of weight symmetry, one that requires agreement of weight sign but not magnitude, can achieve performance competitive with backpropagation. Our results complement those of Bartunov et al. (2018) and Xiao et al. (2018b) and suggest that mechanisms that promote alignment of feedforward and feedback weights are critical for learning in deep networks. |
| Tasks | |
| Published | 2018-12-12 |
| URL | https://arxiv.org/abs/1812.06488v2 |
| https://arxiv.org/pdf/1812.06488v2.pdf | |
| PWC | https://paperswithcode.com/paper/feedback-alignment-in-deep-convolutional |
| Repo | |
| Framework | |
Provably convergent acceleration in factored gradient descent with applications in matrix sensing
| Title | Provably convergent acceleration in factored gradient descent with applications in matrix sensing |
| Authors | Tayo Ajayi, David Mildebrath, Anastasios Kyrillidis, Shashanka Ubaru, Georgios Kollias, Kristofer Bouchard |
| Abstract | We present theoretical results on the convergence of \emph{non-convex} accelerated gradient descent in matrix factorization models with $\ell_2$-norm loss. The purpose of this work is to study the effects of acceleration in non-convex settings, where provable convergence with acceleration should not be considered a \emph{de facto} property. The technique is applied to matrix sensing problems, for the estimation of a rank $r$ optimal solution $X^\star \in \mathbb{R}^{n \times n}$. Our contributions can be summarized as follows. $i)$ We show that acceleration in factored gradient descent converges at a linear rate; this fact is novel for non-convex matrix factorization settings, under common assumptions. $ii)$ Our proof technique requires the acceleration parameter to be carefully selected, based on the properties of the problem, such as the condition number of $X^\star$ and the condition number of objective function. $iii)$ Currently, our proof leads to the same dependence on the condition number(s) in the contraction parameter, similar to recent results on non-accelerated algorithms. $iv)$ Acceleration is observed in practice, both in synthetic examples and in two real applications: neuronal multi-unit activities recovery from single electrode recordings, and quantum state tomography on quantum computing simulators. |
| Tasks | Quantum State Tomography |
| Published | 2018-06-01 |
| URL | https://arxiv.org/abs/1806.00534v5 |
| https://arxiv.org/pdf/1806.00534v5.pdf | |
| PWC | https://paperswithcode.com/paper/run-procrustes-run-on-the-convergence-of |
| Repo | |
| Framework | |
Feature Fusion Effects of Tensor Product Representation on (De)Compositional Network for Caption Generation for Images
| Title | Feature Fusion Effects of Tensor Product Representation on (De)Compositional Network for Caption Generation for Images |
| Authors | Chiranjib Sur |
| Abstract | Progress in image captioning is gradually getting complex as researchers try to generalized the model and define the representation between visual features and natural language processing. This work tried to define such kind of relationship in the form of representation called Tensor Product Representation (TPR) which generalized the scheme of language modeling and structuring the linguistic attributes (related to grammar and parts of speech of language) which will provide a much better structure and grammatically correct sentence. TPR enables better and unique representation and structuring of the feature space and will enable better sentence composition from these representations. A large part of the different ways of defining and improving these TPR are discussed and their performance with respect to the traditional procedures and feature representations are evaluated for image captioning application. The new models achieved considerable improvement than the corresponding previous architectures. |
| Tasks | Image Captioning, Language Modelling |
| Published | 2018-12-17 |
| URL | http://arxiv.org/abs/1812.06624v1 |
| http://arxiv.org/pdf/1812.06624v1.pdf | |
| PWC | https://paperswithcode.com/paper/feature-fusion-effects-of-tensor-product |
| Repo | |
| Framework | |
A rule-based system proposal to aid in the evaluation and decision-making in external beam radiation treatment planning
| Title | A rule-based system proposal to aid in the evaluation and decision-making in external beam radiation treatment planning |
| Authors | R. C. Fernandes, T. M. Machado, H. J. Onisto, A. D. Muñoz, R. O. Silva, L. R. Domingues, G. C. Fonseca, J. E. Bertuzzo, M. T. Pereira, B. Biazotto, E. T. Costa |
| Abstract | As part of a plan launched by the Ministry of Health of Brazil to increase the availability of linear accelerators for radiotherapy treatment for the whole country, for which Varian Medical Systems company has won the bidding, a technical cooperation agreement was signed inviting Brazilian Scientific and Technological Institutions to participate in a technology transfer program. As a result, jointly, the Eldorado Research Institute and the Center for Biomedical Engineering of the University of Campinas presents in this work, the concepts behind of a proposed rule engine to aid in the evaluation and decision-making in radiotherapy treatment planning. Normally, the determination of the radiation dose for a given patient is a complex and intensive procedure, which requires a lot of domain knowledge and subjective experience from the oncologists’ team. In order to help them in this complex task, and additionally, provide an auxiliary tool for less experienced oncologists, it is presented a project conception of a software system that will make use of a hybrid data-oriented approach. The proposed rule engine will apply both inference mechanism and expression evaluation to verify and accredit the quality of an external beam radiation treatment plan by considering, at first, the 3D-conformal radiotherapy (3DCRT) technique. |
| Tasks | Decision Making |
| Published | 2018-11-29 |
| URL | http://arxiv.org/abs/1811.12454v1 |
| http://arxiv.org/pdf/1811.12454v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-rule-based-system-proposal-to-aid-in-the |
| Repo | |
| Framework | |
