Paper Group ANR 24
Unsupervised Extraction of Market Moving Events with Neural Attention. Compatible Learning for Deep Photonic Neural Network. Towards Probability-based Safety Verification of Systems with Components from Machine Learning. Eigenresiduals for improved Parametric Speech Synthesis. Deformation Flow Based Two-Stream Network for Lip Reading. On the comput …
Unsupervised Extraction of Market Moving Events with Neural Attention
| Title | Unsupervised Extraction of Market Moving Events with Neural Attention |
| Authors | Luciano Del Corro, Johannes Hoffart |
| Abstract | We present a method to identify relevant events associated with stock price movements without manually labeled data. We train an attention-based neural network, which given a set of news headlines for a given time frame, predicts the price movement of a given stock index (i.e., DOWN, STAY, UP). An attention layer acts as an input selector; it computes a normalized weight for each headline embedding. The weighted average of the embeddings is used to predict the price movement. We present an analysis to understand if, after the network has been trained, the attention layer is capable of generating a global ranking of news events through its unnormalized weights. The ranking should be able to rank relevant financial events higher. In this initial study we use news categories as a proxy for relevance: news belonging to more relevant categories should be ranked higher. Our experiments on four indices suggest that there is an indication that the weights indeed skew the global set of events towards those categories that are more relevant to explain the price change; this effect reflects the performance of the network on stock prediction. |
| Tasks | Stock Prediction |
| Published | 2020-01-26 |
| URL | https://arxiv.org/abs/2001.09466v1 |
| https://arxiv.org/pdf/2001.09466v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-extraction-of-market-moving |
| Repo | |
| Framework | |
Compatible Learning for Deep Photonic Neural Network
| Title | Compatible Learning for Deep Photonic Neural Network |
| Authors | Yong-Liang Xiao, Rongguang Liang, Jianxin Zhong, Xianyu Su, Zhisheng You |
| Abstract | Realization of deep learning with coherent optical field has attracted remarkably attentions presently, which benefits on the fact that optical matrix manipulation can be executed at speed of light with inherent parallel computation as well as low latency. Photonic neural network has a significant potential for prediction-oriented tasks. Yet, real-value Backpropagation behaves somewhat intractably for coherent photonic intelligent training. We develop a compatible learning protocol in complex space, of which nonlinear activation could be selected efficiently depending on the unveiled compatible condition. Compatibility indicates that matrix representation in complex space covers its real counterpart, which could enable a single channel mingled training in real and complex space as a unified model. The phase logical XOR gate with Mach-Zehnder interferometers and diffractive neural network with optical modulation mechanism, implementing intelligent weight learned from compatible learning, are presented to prove the availability. Compatible learning opens an envisaged window for deep photonic neural network. |
| Tasks | |
| Published | 2020-03-14 |
| URL | https://arxiv.org/abs/2003.08360v1 |
| https://arxiv.org/pdf/2003.08360v1.pdf | |
| PWC | https://paperswithcode.com/paper/compatible-learning-for-deep-photonic-neural |
| Repo | |
| Framework | |
Towards Probability-based Safety Verification of Systems with Components from Machine Learning
| Title | Towards Probability-based Safety Verification of Systems with Components from Machine Learning |
| Authors | Hermann Kaindl, Stefan Kramer |
| Abstract | Machine learning (ML) has recently created many new success stories. Hence, there is a strong motivation to use ML technology in software-intensive systems, including safety-critical systems. This raises the issue of safety verification of ML-based systems, which is currently thought to be infeasible or, at least, very hard. We think that it requires taking into account specific properties of ML technology such as: (i) Most ML approaches are inductive, which is both their power and their source of failure. (ii) Neural networks (NN) resulting from deep learning are at the current state of the art not transparent. Consequently, there will always be errors remaining and, at least for deep NNs (DNNs), verification of their internal structure is extremely hard. However, also traditional safety engineering cannot provide full guarantees that no harm will ever occur. That is why probabilities are used, e.g., for specifying a Risk or a Tolerable Hazard Rate (THR). Recent theoretical work has extended the scope of formal verification to probabilistic model-checking, but this requires behavioral models. Hence, we propose verification based on probabilities of errors both estimated for by controlled experiments and output by the inductively learned classifier itself. Generalization error bounds may propagate to the probabilities of a hazard, which must not exceed a THR. As a result, the quantitatively determined bound on the probability of a classification error of an ML component in a safety-critical system contributes in a well-defined way to the latter’s overall safety verification. |
| Tasks | |
| Published | 2020-03-02 |
| URL | https://arxiv.org/abs/2003.01155v1 |
| https://arxiv.org/pdf/2003.01155v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-probability-based-safety-verification |
| Repo | |
| Framework | |
Eigenresiduals for improved Parametric Speech Synthesis
| Title | Eigenresiduals for improved Parametric Speech Synthesis |
| Authors | Thomas Drugman, Geoffrey Wilfart, Thierry Dutoit |
| Abstract | Statistical parametric speech synthesizers have recently shown their ability to produce natural-sounding and flexible voices. Unfortunately the delivered quality suffers from a typical buzziness due to the fact that speech is vocoded. This paper proposes a new excitation model in order to reduce this undesirable effect. This model is based on the decomposition of pitch-synchronous residual frames on an orthonormal basis obtained by Principal Component Analysis. This basis contains a limited number of eigenresiduals and is computed on a relatively small speech database. A stream of PCA-based coefficients is added to our HMM-based synthesizer and allows to generate the voiced excitation during the synthesis. An improvement compared to the traditional excitation is reported while the synthesis engine footprint remains under about 1Mb. |
| Tasks | Speech Synthesis |
| Published | 2020-01-02 |
| URL | https://arxiv.org/abs/2001.00581v1 |
| https://arxiv.org/pdf/2001.00581v1.pdf | |
| PWC | https://paperswithcode.com/paper/eigenresiduals-for-improved-parametric-speech |
| Repo | |
| Framework | |
Deformation Flow Based Two-Stream Network for Lip Reading
| Title | Deformation Flow Based Two-Stream Network for Lip Reading |
| Authors | Jingyun Xiao, Shuang Yang, Yuanhang Zhang, Shiguang Shan, Xilin Chen |
| Abstract | Lip reading is the task of recognizing the speech content by analyzing movements in the lip region when people are speaking. Observing on the continuity in adjacent frames in the speaking process, and the consistency of the motion patterns among different speakers when they pronounce the same phoneme, we model the lip movements in the speaking process as a sequence of apparent deformations in the lip region. Specifically, we introduce a Deformation Flow Network (DFN) to learn the deformation flow between adjacent frames, which directly captures the motion information within the lip region. The learned deformation flow is then combined with the original grayscale frames with a two-stream network to perform lip reading. Different from previous two-stream networks, we make the two streams learn from each other in the learning process by introducing a bidirectional knowledge distillation loss to train the two branches jointly. Owing to the complementary cues provided by different branches, the two-stream network shows a substantial improvement over using either single branch. A thorough experimental evaluation on two large-scale lip reading benchmarks is presented with detailed analysis. The results accord with our motivation, and show that our method achieves state-of-the-art or comparable performance on these two challenging datasets. |
| Tasks | |
| Published | 2020-03-12 |
| URL | https://arxiv.org/abs/2003.05709v2 |
| https://arxiv.org/pdf/2003.05709v2.pdf | |
| PWC | https://paperswithcode.com/paper/deformation-flow-based-two-stream-network-for |
| Repo | |
| Framework | |
On the computational power and complexity of Spiking Neural Networks
| Title | On the computational power and complexity of Spiking Neural Networks |
| Authors | Johan Kwisthout, Nils Donselaar |
| Abstract | The last decade has seen the rise of neuromorphic architectures based on artificial spiking neural networks, such as the SpiNNaker, TrueNorth, and Loihi systems. The massive parallelism and co-locating of computation and memory in these architectures potentially allows for an energy usage that is orders of magnitude lower compared to traditional Von Neumann architectures. However, to date a comparison with more traditional computational architectures (particularly with respect to energy usage) is hampered by the lack of a formal machine model and a computational complexity theory for neuromorphic computation. In this paper we take the first steps towards such a theory. We introduce spiking neural networks as a machine model where—in contrast to the familiar Turing machine—information and the manipulation thereof are co-located in the machine. We introduce canonical problems, define hierarchies of complexity classes and provide some first completeness results. |
| Tasks | |
| Published | 2020-01-23 |
| URL | https://arxiv.org/abs/2001.08439v1 |
| https://arxiv.org/pdf/2001.08439v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-computational-power-and-complexity-of |
| Repo | |
| Framework | |
Graph Convolutional Gaussian Processes For Link Prediction
| Title | Graph Convolutional Gaussian Processes For Link Prediction |
| Authors | Felix L. Opolka, Pietro Liò |
| Abstract | Link prediction aims to reveal missing edges in a graph. We address this task with a Gaussian process that is transformed using simplified graph convolutions to better leverage the inductive bias of the domain. To scale the Gaussian process model to large graphs, we introduce a variational inducing point method that places pseudo inputs on a graph-structured domain. We evaluate our model on eight large graphs with up to thousands of nodes and report consistent improvements over existing Gaussian process models as well as competitive performance when compared to state-of-the-art graph neural network approaches. |
| Tasks | Gaussian Processes, Link Prediction |
| Published | 2020-02-11 |
| URL | https://arxiv.org/abs/2002.04337v1 |
| https://arxiv.org/pdf/2002.04337v1.pdf | |
| PWC | https://paperswithcode.com/paper/graph-convolutional-gaussian-processes-for |
| Repo | |
| Framework | |
Do optimization methods in deep learning applications matter?
| Title | Do optimization methods in deep learning applications matter? |
| Authors | Buse Melis Ozyildirim, Mariam Kiran |
| Abstract | With advances in deep learning, exponential data growth and increasing model complexity, developing efficient optimization methods are attracting much research attention. Several implementations favor the use of Conjugate Gradient (CG) and Stochastic Gradient Descent (SGD) as being practical and elegant solutions to achieve quick convergence, however, these optimization processes also present many limitations in learning across deep learning applications. Recent research is exploring higher-order optimization functions as better approaches, but these present very complex computational challenges for practical use. Comparing first and higher-order optimization functions, in this paper, our experiments reveal that Levemberg-Marquardt (LM) significantly supersedes optimal convergence but suffers from very large processing time increasing the training complexity of both, classification and reinforcement learning problems. Our experiments compare off-the-shelf optimization functions(CG, SGD, LM and L-BFGS) in standard CIFAR, MNIST, CartPole and FlappyBird experiments.The paper presents arguments on which optimization functions to use and further, which functions would benefit from parallelization efforts to improve pretraining time and learning rate convergence. |
| Tasks | |
| Published | 2020-02-28 |
| URL | https://arxiv.org/abs/2002.12642v1 |
| https://arxiv.org/pdf/2002.12642v1.pdf | |
| PWC | https://paperswithcode.com/paper/do-optimization-methods-in-deep-learning |
| Repo | |
| Framework | |
Meta-learning framework with applications to zero-shot time-series forecasting
| Title | Meta-learning framework with applications to zero-shot time-series forecasting |
| Authors | Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, Yoshua Bengio |
| Abstract | Can meta-learning discover generic ways of processing time-series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets? This work provides positive evidence to demonstrate this using a broad meta-learning framework which we show subsumes many existing meta-learning algorithms as specific cases. We further identify via theoretical analysis the meta-learning adaptation mechanisms within N-BEATS, a recent neural TS forecasting model. Our meta-learning theory predicts that N-BEATS iteratively generates a subset of its task-specific parameters based on a given TS input, thus gradually expanding the expressive power of the architecture on-the-fly. Our empirical results emphasize the importance of meta-learning for successful zero-shot forecasting to new sources of TS, supporting the claim that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, resulting in performance that is at least as good as that of state-of-practice univariate forecasting models. |
| Tasks | Meta-Learning, Time Series, Time Series Forecasting |
| Published | 2020-02-07 |
| URL | https://arxiv.org/abs/2002.02887v1 |
| https://arxiv.org/pdf/2002.02887v1.pdf | |
| PWC | https://paperswithcode.com/paper/meta-learning-framework-with-applications-to |
| Repo | |
| Framework | |
EOLO: Embedded Object Segmentation only Look Once
| Title | EOLO: Embedded Object Segmentation only Look Once |
| Authors | Longfei Zeng, Mohammed Sabah |
| Abstract | In this paper, we introduce an anchor-free and single-shot instance segmentation method, which is conceptually simple with 3 independent branches, fully convolutional and can be used by easily embedding it into mobile and embedded devices. Our method, refer as EOLO, reformulates the instance segmentation problem as predicting semantic segmentation and distinguishing overlapping objects problem, through instance center classification and 4D distance regression on each pixel. Moreover, we propose one effective loss function to deal with sampling a high-quality center of gravity examples and optimization for 4D distance regression, which can significantly improve the mAP performance. Without any bells and whistles, EOLO achieves 27.7$%$ in mask mAP under IoU50 and reaches 30 FPS on 1080Ti GPU, with a single-model and single-scale training/testing on the challenging COCO2017 dataset. For the first time, we show the different comprehension of instance segmentation in recent methods, in terms of both up-bottom, down-up, and direct-predict paradigms. Then we illustrate our model and present related experiments and results. We hope that the proposed EOLO framework can serve as a fundamental baseline for a single-shot instance segmentation task in Real-time Industrial Scenarios. |
| Tasks | Instance Segmentation, Semantic Segmentation |
| Published | 2020-03-31 |
| URL | https://arxiv.org/abs/2004.00123v1 |
| https://arxiv.org/pdf/2004.00123v1.pdf | |
| PWC | https://paperswithcode.com/paper/eolo-embedded-object-segmentation-only-look |
| Repo | |
| Framework | |
HistomicsML2.0: Fast interactive machine learning for whole slide imaging data
| Title | HistomicsML2.0: Fast interactive machine learning for whole slide imaging data |
| Authors | Sanghoon Lee, Mohamed Amgad, Deepak R. Chittajallu, Matt McCormick, Brian P Pollack, Habiba Elfandy, Hagar Hussein, David A Gutman, Lee AD Cooper |
| Abstract | Extracting quantitative phenotypic information from whole-slide images presents significant challenges for investigators who are not experienced in developing image analysis algorithms. We present new software that enables rapid learn-by-example training of machine learning classifiers for detection of histologic patterns in whole-slide imaging datasets. HistomicsML2.0 uses convolutional networks to be readily adaptable to a variety of applications, provides a web-based user interface, and is available as a software container to simplify deployment. |
| Tasks | |
| Published | 2020-01-30 |
| URL | https://arxiv.org/abs/2001.11547v1 |
| https://arxiv.org/pdf/2001.11547v1.pdf | |
| PWC | https://paperswithcode.com/paper/histomicsml20-fast-interactive-machine |
| Repo | |
| Framework | |
Scalable learning for bridging the species gap in image-based plant phenotyping
| Title | Scalable learning for bridging the species gap in image-based plant phenotyping |
| Authors | Daniel Ward, Peyman Moghadam |
| Abstract | The traditional paradigm of applying deep learning – collect, annotate and train on data – is not applicable to image-based plant phenotyping as almost 400,000 different plant species exists. Data costs include growing physical samples, imaging and labelling them. Model performance is impacted by the species gap between the domain of each plant species, it is not generalisable and may not transfer to unseen plant species. In this paper, we investigate the use of synthetic data for leaf instance segmentation. We study multiple synthetic data training regimes using Mask-RCNN when few or no annotated real data is available. We also present UPGen: a Universal Plant Generator for bridging the species gap. UPGen leverages domain randomisation to produce widely distributed data samples and models stochastic biological variation. Our methods outperform standard practices, such as transfer learning from publicly available plant data, by 26.6% and 51.46% on two unseen plant species respectively. We benchmark UPGen by competing in the CVPPP Leaf Segmentation Challenge and set a new state-of-the-art, a mean of 88% across A1-4 test datasets. This study is applicable to use of synthetic data for automating the measurement of phenotypic traits. Our synthetic dataset and pretrained model are available at https://danielcward.github.io/UPGen/. |
| Tasks | Instance Segmentation, Semantic Segmentation, Transfer Learning |
| Published | 2020-03-24 |
| URL | https://arxiv.org/abs/2003.10757v1 |
| https://arxiv.org/pdf/2003.10757v1.pdf | |
| PWC | https://paperswithcode.com/paper/scalable-learning-for-bridging-the-species |
| Repo | |
| Framework | |
Deep Learning for Musculoskeletal Image Analysis
| Title | Deep Learning for Musculoskeletal Image Analysis |
| Authors | Ismail Irmakci, Syed Muhammad Anwar, Drew A. Torigian, Ulas Bagci |
| Abstract | The diagnosis, prognosis, and treatment of patients with musculoskeletal (MSK) disorders require radiology imaging (using computed tomography, magnetic resonance imaging(MRI), and ultrasound) and their precise analysis by expert radiologists. Radiology scans can also help assessment of metabolic health, aging, and diabetes. This study presents how machinelearning, specifically deep learning methods, can be used for rapidand accurate image analysis of MRI scans, an unmet clinicalneed in MSK radiology. As a challenging example, we focus on automatic analysis of knee images from MRI scans and study machine learning classification of various abnormalities including meniscus and anterior cruciate ligament tears. Using widely used convolutional neural network (CNN) based architectures, we comparatively evaluated the knee abnormality classification performances of different neural network architectures under limited imaging data regime and compared single and multi-view imaging when classifying the abnormalities. Promising results indicated the potential use of multi-view deep learning based classification of MSK abnormalities in routine clinical assessment. |
| Tasks | |
| Published | 2020-03-01 |
| URL | https://arxiv.org/abs/2003.00541v1 |
| https://arxiv.org/pdf/2003.00541v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-for-musculoskeletal-image |
| Repo | |
| Framework | |
Region of Interest Identification for Brain Tumors in Magnetic Resonance Images
| Title | Region of Interest Identification for Brain Tumors in Magnetic Resonance Images |
| Authors | Fateme Mostafaie, Reihaneh Teimouri, Zahra Nabizadeh, Nader Karimi, Shadrokh Samavi |
| Abstract | Glioma is a common type of brain tumor, and accurate detection of it plays a vital role in the diagnosis and treatment process. Despite advances in medical image analyzing, accurate tumor segmentation in brain magnetic resonance (MR) images remains a challenge due to variations in tumor texture, position, and shape. In this paper, we propose a fast, automated method, with light computational complexity, to find the smallest bounding box around the tumor region. This region-of-interest can be used as a preprocessing step in training networks for subregion tumor segmentation. By adopting the outputs of this algorithm, redundant information is removed; hence the network can focus on learning notable features related to subregions’ classes. The proposed method has six main stages, in which the brain segmentation is the most vital step. Expectation-maximization (EM) and K-means algorithms are used for brain segmentation. The proposed method is evaluated on the BraTS 2015 dataset, and the average gained DICE score is 0.73, which is an acceptable result for this application. |
| Tasks | Brain Segmentation |
| Published | 2020-02-26 |
| URL | https://arxiv.org/abs/2002.11509v1 |
| https://arxiv.org/pdf/2002.11509v1.pdf | |
| PWC | https://paperswithcode.com/paper/region-of-interest-identification-for-brain |
| Repo | |
| Framework | |
Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles
| Title | Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles |
| Authors | Dylan J. Foster, Alexander Rakhlin |
| Abstract | A fundamental challenge in contextual bandits is to develop flexible, general-purpose algorithms with computational requirements no worse than classical supervised learning tasks such as classification and regression. Algorithms based on regression have shown promising empirical success, but theoretical guarantees have remained elusive except in special cases. We provide the first universal and optimal reduction from contextual bandits to online regression. We show how to transform any oracle for online regression with a given value function class into an algorithm for contextual bandits with the induced policy class, with no overhead in runtime or memory requirements. We characterize the minimax rates for contextual bandits with general, potentially nonparametric function classes, and show that our algorithm is minimax optimal whenever the oracle obtains the optimal rate for regression. Compared to previous results, our algorithm requires no distributional assumptions beyond realizability, and works even when contexts are chosen adversarially. |
| Tasks | Multi-Armed Bandits |
| Published | 2020-02-12 |
| URL | https://arxiv.org/abs/2002.04926v1 |
| https://arxiv.org/pdf/2002.04926v1.pdf | |
| PWC | https://paperswithcode.com/paper/beyond-ucb-optimal-and-efficient-contextual |
| Repo | |
| Framework | |
