Paper Group ANR 1252
Speeding up Deep Learning with Transient Servers. ragamAI: A Network Based Recommender System to Arrange a Indian Classical Music Concert. A Scalable Handwritten Text Recognition System. On the Hardness of Robust Classification. Evaluating Generative Models Using Divergence Frontiers. SMP Challenge: An Overview of Social Media Prediction Challenge …
Speeding up Deep Learning with Transient Servers
| Title | Speeding up Deep Learning with Transient Servers |
| Authors | Shijian Li, Robert J. Walls, Lijie Xu, Tian Guo |
| Abstract | Distributed training frameworks, like TensorFlow, have been proposed as a means to reduce the training time of deep learning models by using a cluster of GPU servers. While such speedups are often desirable—e.g., for rapidly evaluating new model designs—they often come with significantly higher monetary costs due to sublinear scalability. In this paper, we investigate the feasibility of using training clusters composed of cheaper transient GPU servers to get the benefits of distributed training without the high costs. We conduct the first large-scale empirical analysis, launching more than a thousand GPU servers of various capacities, aimed at understanding the characteristics of transient GPU servers and their impact on distributed training performance. Our study demonstrates the potential of transient servers with a speedup of 7.7X with more than 62.9% monetary savings for some cluster configurations. We also identify a number of important challenges and opportunities for redesigning distributed training frameworks to be transient-aware. For example, the dynamic cost and availability characteristics of transient servers suggest the need for frameworks to dynamically change cluster configurations to best take advantage of current conditions. |
| Tasks | |
| Published | 2019-02-28 |
| URL | https://arxiv.org/abs/1903.00045v2 |
| https://arxiv.org/pdf/1903.00045v2.pdf | |
| PWC | https://paperswithcode.com/paper/speeding-up-deep-learning-with-transient |
| Repo | |
| Framework | |
ragamAI: A Network Based Recommender System to Arrange a Indian Classical Music Concert
| Title | ragamAI: A Network Based Recommender System to Arrange a Indian Classical Music Concert |
| Authors | Arunkumar Bagavathi, Siddharth Krishnan, Sanjay Subrahmanyan, S. L. Narasimhan |
| Abstract | South Indian classical music (Carnatic music) is best consumed through live concerts. A carnatic recital requires meticulous planning accounting for several parameters like the performers’ repertoire, composition variety, musical versatility, thematic structure, the recital’s arrangement, etc. to ensure that the audience have a comprehensive listening experience. In this work, we present ragamAI a novel machine learning framework that utilizes the tonic nuances and musical structures in the carnatic music to generate a concert recital that melodically captures the entire range in an octave. Utilizing the underlying idea of playlist and session-based recommender models, the proposed model studies the mathematical structure present in past concerts and recommends relevant items for the playlist/concert. ragamAI ensembles recommendations given by multiple models to learn user idea and past preference of sequences in concerts to extract recommendations. Our experiments on a vast collection of concert show that our model performs 25%-50% better than baseline models. ragamAI’s applications are two-fold. 1) it will assist musicians to customize their performance with the necessary variety required to sustain the interest of the audience for the entirety of the concert 2) it will generate carefully curated lists of south Indian classical music so that the listener can discover the wide range of melody that the musical system can offer. |
| Tasks | Recommendation Systems |
| Published | 2019-12-08 |
| URL | https://arxiv.org/abs/1912.03769v1 |
| https://arxiv.org/pdf/1912.03769v1.pdf | |
| PWC | https://paperswithcode.com/paper/ragamai-a-network-based-recommender-system-to |
| Repo | |
| Framework | |
A Scalable Handwritten Text Recognition System
| Title | A Scalable Handwritten Text Recognition System |
| Authors | R. Reeve Ingle, Yasuhisa Fujii, Thomas Deselaers, Jonathan Baccash, Ashok C. Popat |
| Abstract | Many studies on (Offline) Handwritten Text Recognition (HTR) systems have focused on building state-of-the-art models for line recognition on small corpora. However, adding HTR capability to a large scale multilingual OCR system poses new challenges. This paper addresses three problems in building such systems: data, efficiency, and integration. Firstly, one of the biggest challenges is obtaining sufficient amounts of high quality training data. We address the problem by using online handwriting data collected for a large scale production online handwriting recognition system. We describe our image data generation pipeline and study how online data can be used to build HTR models. We show that the data improve the models significantly under the condition where only a small number of real images is available, which is usually the case for HTR models. It enables us to support a new script at substantially lower cost. Secondly, we propose a line recognition model based on neural networks without recurrent connections. The model achieves a comparable accuracy with LSTM-based models while allowing for better parallelism in training and inference. Finally, we present a simple way to integrate HTR models into an OCR system. These constitute a solution to bring HTR capability into a large scale OCR system. |
| Tasks | Optical Character Recognition |
| Published | 2019-04-19 |
| URL | https://arxiv.org/abs/1904.09150v2 |
| https://arxiv.org/pdf/1904.09150v2.pdf | |
| PWC | https://paperswithcode.com/paper/a-scalable-handwritten-text-recognition |
| Repo | |
| Framework | |
On the Hardness of Robust Classification
| Title | On the Hardness of Robust Classification |
| Authors | Pascale Gourdeau, Varun Kanade, Marta Kwiatkowska, James Worrell |
| Abstract | It is becoming increasingly important to understand the vulnerability of machine learning models to adversarial attacks. In this paper we study the feasibility of robust learning from the perspective of computational learning theory, considering both sample and computational complexity. In particular, our definition of robust learnability requires polynomial sample complexity. We start with two negative results. We show that no non-trivial concept class can be robustly learned in the distribution-free setting against an adversary who can perturb just a single input bit. We show moreover that the class of monotone conjunctions cannot be robustly learned under the uniform distribution against an adversary who can perturb $\omega(\log n)$ input bits. However if the adversary is restricted to perturbing $O(\log n)$ bits, then the class of monotone conjunctions can be robustly learned with respect to a general class of distributions (that includes the uniform distribution). Finally, we provide a simple proof of the computational hardness of robust learning on the boolean hypercube. Unlike previous results of this nature, our result does not rely on another computational model (e.g. the statistical query model) nor on any hardness assumption other than the existence of a hard learning problem in the PAC framework. |
| Tasks | |
| Published | 2019-09-12 |
| URL | https://arxiv.org/abs/1909.05822v1 |
| https://arxiv.org/pdf/1909.05822v1.pdf | |
| PWC | https://paperswithcode.com/paper/on-the-hardness-of-robust-classification |
| Repo | |
| Framework | |
Evaluating Generative Models Using Divergence Frontiers
| Title | Evaluating Generative Models Using Divergence Frontiers |
| Authors | Josip Djolonga, Mario Lucic, Marco Cuturi, Olivier Bachem, Olivier Bousquet, Sylvain Gelly |
| Abstract | Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a much slower pace. Recent developments have investigated metrics that quantify which parts of the true distribution are modeled well, and, on the contrary, what the model fails to capture, akin to precision and recall in information retrieval. In this paper, we present a general evaluation framework for generative models that measures the trade-off between precision and recall using R'enyi divergences. Our framework provides a novel perspective on existing techniques and extends them to more general domains. As a key advantage, it allows for efficient algorithms that are directly applicable to continuous distributions directly without discretization. We further showcase the proposed techniques on a set of image synthesis models. |
| Tasks | Image Generation, Information Retrieval |
| Published | 2019-05-26 |
| URL | https://arxiv.org/abs/1905.10768v1 |
| https://arxiv.org/pdf/1905.10768v1.pdf | |
| PWC | https://paperswithcode.com/paper/evaluating-generative-models-using-divergence |
| Repo | |
| Framework | |
SMP Challenge: An Overview of Social Media Prediction Challenge 2019
| Title | SMP Challenge: An Overview of Social Media Prediction Challenge 2019 |
| Authors | Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo |
| Abstract | “SMP Challenge” aims to discover novel prediction tasks for numerous data on social multimedia and seek excellent research teams. Making predictions via social multimedia data (e.g. photos, videos or news) is not only helps us to make better strategic decisions for the future, but also explores advanced predictive learning and analytic methods on various problems and scenarios, such as multimedia recommendation, advertising system, fashion analysis etc. In the SMP Challenge at ACM Multimedia 2019, we introduce a novel prediction task Temporal Popularity Prediction, which focuses on predicting future interaction or attractiveness (in terms of clicks, views or likes etc.) of new online posts in social media feeds before uploading. We also collected and released a large-scale SMPD benchmark with over 480K posts from 69K users. In this paper, we define the challenge problem, give an overview of the dataset, present statistics of rich information for data and annotation and design the accuracy and correlation evaluation metrics for temporal popularity prediction to the challenge. |
| Tasks | |
| Published | 2019-10-04 |
| URL | https://arxiv.org/abs/1910.01795v2 |
| https://arxiv.org/pdf/1910.01795v2.pdf | |
| PWC | https://paperswithcode.com/paper/smp-challenge-an-overview-of-social-media |
| Repo | |
| Framework | |
The Weight Function in the Subtree Kernel is Decisive
| Title | The Weight Function in the Subtree Kernel is Decisive |
| Authors | Romain Azaïs, Florian Ingels |
| Abstract | Tree data are ubiquitous because they model a large variety of situations, e.g., the architecture of plants, the secondary structure of RNA, or the hierarchy of XML files. Nevertheless, the analysis of these non-Euclidean data is difficult per se. In this paper, we focus on the subtree kernel that is a convolution kernel for tree data introduced by Vishwanathan and Smola in the early 2000’s. More precisely, we investigate the influence of the weight function from a theoretical perspective and in real data applications. We establish on a 2-classes stochastic model that the performance of the subtree kernel is improved when the weight of leaves vanishes, which motivates the definition of a new weight function, learned from the data and not fixed by the user as usually done. To this end, we define a unified framework for computing the subtree kernel from ordered or unordered trees, that is particularly suitable for tuning parameters. We show through eight real data classification problems the great efficiency of our approach, in particular for small datasets, which also states the high importance of the weight function. Finally, a visualization tool of the significant features is derived. |
| Tasks | |
| Published | 2019-04-10 |
| URL | https://arxiv.org/abs/1904.05421v3 |
| https://arxiv.org/pdf/1904.05421v3.pdf | |
| PWC | https://paperswithcode.com/paper/the-weight-function-in-the-subtree-kernel-is |
| Repo | |
| Framework | |
Text Classification Components for Detecting Descriptions and Names of CAD models
| Title | Text Classification Components for Detecting Descriptions and Names of CAD models |
| Authors | Thomas Köllmer, Jens Hasselbach, Patrick Aichroth |
| Abstract | We apply text analysis approaches for a specialized search engine for 3D CAD models and associated products. The main goals are to distinguish between actual product descriptions and other text on a website, as well as to decide whether a given text is or contains a product name. For this we use paragraph vectors for text classification, a character-level long short-term memory network (LSTM) for a single word classification and an LSTM tagger based on word embeddings for detecting product names within sentences. Despite the need to collect bigger datasets in our specific problem domain, the first results are promising and partially fit for production use. |
| Tasks | Text Classification, Word Embeddings |
| Published | 2019-04-04 |
| URL | http://arxiv.org/abs/1904.12587v1 |
| http://arxiv.org/pdf/1904.12587v1.pdf | |
| PWC | https://paperswithcode.com/paper/190412587 |
| Repo | |
| Framework | |
Computational design of organic solar cell active layer through genetic algorithm
| Title | Computational design of organic solar cell active layer through genetic algorithm |
| Authors | Caine Ardayfio |
| Abstract | The active layer microstructure of organic solar cells is critical to efficiency. By studying the photovoltaic properties of organic solar cell’s microstructure, it is possible to increase the efficiency of the solar cell. A graph-based microstructure model was employed to approximate the efficiency, measured as short circuit current, of a solar cell given a microstructure. Through probabilistic graph-based optimization, a class of microstructures were found with an efficiency surpassing that of more conventional morphologies. These optimized solar cells surpass the efficiency of more conventional photovoltaic devices as they better facilitate charge transport, generation, and dissociation. A device was designed with a 40.29% increase in short circuit current from the solar cells with the currently believed optimal morphology. The designed morphologies feature two dendritic clusters of the donor material poly(3-hexylthiophene-2,5-diyl) (P3HT) and the acceptor material phenyl-C61-Butyric-Acid-Methyl Ester (PCBM). The designed microstructure’s increase in performance contrasts with more conventional structures featuring interdigitated or bilayer strands of P3HT and PCBM. The change of microstructure morphology through graph-based evolution obtains an organic solar cell with an efficiency significantly greater than conventional organic solar cells, proves the validity of graph-based microstructure models for simulation in materials science, and advances the vision of an inexpensive, efficient form of renewable energy. |
| Tasks | |
| Published | 2019-10-28 |
| URL | https://arxiv.org/abs/1910.12401v1 |
| https://arxiv.org/pdf/1910.12401v1.pdf | |
| PWC | https://paperswithcode.com/paper/computational-design-of-organic-solar-cell |
| Repo | |
| Framework | |
New insights on Multi-Solution Distribution of the P3P Problem
| Title | New insights on Multi-Solution Distribution of the P3P Problem |
| Authors | Bo Wang, Hao Hu, Caixia Zhang |
| Abstract | Traditionally, the P3P problem is solved by firstly transforming its 3 quadratic equations into a quartic one, then by locating the roots of the resulting quartic equation and verifying whether a root does really correspond to a true solution of the P3P problem itself. However, a root of the quartic equation does not always correspond to a solution of the P3P problem. In this work, we show that when the optical center is outside of all the 6 toroids defined by the control point triangle, each positive root of the Grunert’s quartic equation must correspond to a true solution of the P3P problem, and the corresponding P3P problem cannot have a unique solution, it must have either 2 positive solutions or 4 positive solutions. In addition, we show that when the optical center passes through any one of the 3 toroids among these 6 toroids ( except possibly for two concentric circles) , the number of the solutions of the corresponding P3P problem always changes by 1, either increased by 1 or decreased by 1.Furthermore we show that such changed solutions always locate in a small neighborhood of control points, hence the 3 toroids are critical surfaces of the P3P problem and the 3 control points are 3 singular points of solutions. A notable example is that when the optical center passes through the outer surface of the union of the 6 toroids from the outside to inside, the number of the solutions must always decrease by 1. Our results are the first to give an explicit and geometrically intuitive relationship between the P3P solutions and the roots of its quartic equation. It could act as some theoretical guidance for P3P practitioners to properly arrange their control points to avoid undesirable solutions. |
| Tasks | |
| Published | 2019-01-30 |
| URL | http://arxiv.org/abs/1901.11464v1 |
| http://arxiv.org/pdf/1901.11464v1.pdf | |
| PWC | https://paperswithcode.com/paper/new-insights-on-multi-solution-distribution |
| Repo | |
| Framework | |
3D Multi-Robot Patrolling with a Two-Level Coordination Strategy
| Title | 3D Multi-Robot Patrolling with a Two-Level Coordination Strategy |
| Authors | Luigi Freda, Mario Gianni, Fiora Pirri, Abel Gawel, Renaud Dube, Roland Siegwart, Cesar Cadena |
| Abstract | Teams of UGVs patrolling harsh and complex 3D environments can experience interference and spatial conflicts with one another. Neglecting the occurrence of these events crucially hinders both soundness and reliability of a patrolling process. This work presents a distributed multi-robot patrolling technique, which uses a two-level coordination strategy to minimize and explicitly manage the occurrence of conflicts and interference. The first level guides the agents to single out exclusive target nodes on a topological map. This target selection relies on a shared idleness representation and a coordination mechanism preventing topological conflicts. The second level hosts coordination strategies based on a metric representation of space and is supported by a 3D SLAM system. Here, each robot path planner negotiates spatial conflicts by applying a multi-robot traversability function. Continuous interactions between these two levels ensure coordination and conflicts resolution. Both simulations and real-world experiments are presented to validate the performances of the proposed patrolling strategy in 3D environments. Results show this is a promising solution for managing spatial conflicts and preventing deadlocks. |
| Tasks | |
| Published | 2019-06-23 |
| URL | https://arxiv.org/abs/1906.09591v1 |
| https://arxiv.org/pdf/1906.09591v1.pdf | |
| PWC | https://paperswithcode.com/paper/3d-multi-robot-patrolling-with-a-two-level |
| Repo | |
| Framework | |
Does Interpretability of Neural Networks Imply Adversarial Robustness?
| Title | Does Interpretability of Neural Networks Imply Adversarial Robustness? |
| Authors | Adam Noack, Isaac Ahern, Dejing Dou, Boyang Li |
| Abstract | The success of deep neural networks is clouded by two issues that largely remain open to this day: the abundance of adversarial attacks that fool neural networks with small perturbations and the lack of interpretation for the predictions they make. Empirical evidence in the literature as well as theoretical analysis on simple models suggest these two seemingly disparate issues may actually be connected, as robust models tend to be more interpretable than non-robust models. In this paper, we provide evidence for the claim that this relationship is bidirectional. Viz., models that are forced to have interpretable gradients are more robust to adversarial examples than models trained in a standard manner. With further analysis and experiments, we identify two factors behind this phenomenon, namely the suppression of the gradient and the selective use of features guided by high-quality interpretations, which explain model behaviors under various regularization and target interpretation settings. |
| Tasks | |
| Published | 2019-12-07 |
| URL | https://arxiv.org/abs/1912.03430v1 |
| https://arxiv.org/pdf/1912.03430v1.pdf | |
| PWC | https://paperswithcode.com/paper/does-interpretability-of-neural-networks |
| Repo | |
| Framework | |
Alterations in Structural Correlation Networks with Prior Concussion in Collision-Sport Athletes
| Title | Alterations in Structural Correlation Networks with Prior Concussion in Collision-Sport Athletes |
| Authors | Muhammad Usman Sadiq, Diana Svaldi, Trey Shenk, Evan Breedlove, Victoria Poole, Greg Tamer, Kausar Abbas, Thomas Talavage |
| Abstract | Several studies have used structural correlation networks, derived from anatomical covariance of brain regions, to analyze neurologic changes associated with multiple sclerosis, schizophrenia and breast cancer [1][2]. Graph-theoretical analyses of human brain structural networks have consistently shown the characteristic of small-worldness that reflects a network with both high segregation and high integration. A large neuroimaging literature on football players, with and without history of concussion, has shown both functional and anatomical changes. Here we use graph-based topological properties of anatomical correlation networks to study the effect of prior concussion in collision-sport athletes. 40 high school collision-sport athletes (23 male football, 17 female soccer; CSA) without self-reported history of concussion (HOC-), 18 athletes (13 male football, 5 female soccer) with self-reported history of concussion (HOC+) and 24 healthy controls (19 male, 5 female; CN) participated in imaging sessions before the beginning of a competition season. The extracted residual volumes for each group were used for building the correlation networks and their small-worldness, , is calculated. The small-worldness of CSA without prior history of concussion, , is significantly greater than that of controls, . CSA with prior history have significantly higher (vs. 95% confidence interval) small-worldness compared to HOC+, over a range of network densities. The longer path lengths in HOC+ group could indicate disrupted neuronal integration relative to healthy controls. |
| Tasks | |
| Published | 2019-04-18 |
| URL | http://arxiv.org/abs/1904.10924v1 |
| http://arxiv.org/pdf/1904.10924v1.pdf | |
| PWC | https://paperswithcode.com/paper/190410924 |
| Repo | |
| Framework | |
Interpreting Age Effects of Human Fetal Brain from Spontaneous fMRI using Deep 3D Convolutional Neural Networks
| Title | Interpreting Age Effects of Human Fetal Brain from Spontaneous fMRI using Deep 3D Convolutional Neural Networks |
| Authors | Xiangrui Li, Jasmine Hect, Moriah Thomason, Dongxiao Zhu |
| Abstract | Understanding human fetal neurodevelopment is of great clinical importance as abnormal development is linked to adverse neuropsychiatric outcomes after birth. Recent advances in functional Magnetic Resonance Imaging (fMRI) have provided new insight into development of the human brain before birth, but these studies have predominately focused on brain functional connectivity (i.e. Fisher z-score), which requires manual processing steps for feature extraction from fMRI images. Deep learning approaches (i.e., Convolutional Neural Networks) have achieved remarkable success on learning directly from image data, yet have not been applied on fetal fMRI for understanding fetal neurodevelopment. Here, we bridge this gap by applying a novel application of deep 3D CNN to fetal blood oxygen-level dependence (BOLD) resting-state fMRI data. Specifically, we test a supervised CNN framework as a data-driven approach to isolate variation in fMRI signals that relate to younger v.s. older fetal age groups. Based on the learned CNN, we further perform sensitivity analysis to identify brain regions in which changes in BOLD signal are strongly associated with fetal brain age. The findings demonstrate that deep CNNs are a promising approach for identifying spontaneous functional patterns in fetal brain activity that discriminate age groups. Further, we discovered that regions that most strongly differentiate groups are largely bilateral, share similar distribution in older and younger age groups, and are areas of heightened metabolic activity in early human development. |
| Tasks | |
| Published | 2019-06-09 |
| URL | https://arxiv.org/abs/1906.03691v1 |
| https://arxiv.org/pdf/1906.03691v1.pdf | |
| PWC | https://paperswithcode.com/paper/interpreting-age-effects-of-human-fetal-brain |
| Repo | |
| Framework | |
Adversarially robust transfer learning
| Title | Adversarially robust transfer learning |
| Authors | Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, Tom Goldstein |
| Abstract | Transfer learning, in which a network is trained on one task and re-purposed on another, is often used to produce neural network classifiers when data is scarce or full-scale training is too costly. When the goal is to produce a model that is not only accurate but also adversarially robust, data scarcity and computational limitations become even more cumbersome. We consider robust transfer learning, in which we transfer not only performance but also robustness from a source model to a target domain. We start by observing that robust networks contain robust feature extractors. By training classifiers on top of these feature extractors, we produce new models that inherit the robustness of their parent networks. We then consider the case of fine tuning a network by re-training end-to-end in the target domain. When using lifelong learning strategies, this process preserves the robustness of the source network while achieving high accuracy. By using such strategies, it is possible to produce accurate and robust models with little data, and without the cost of adversarial training. Additionally, we can improve the generalization of adversarially trained models, while maintaining their robustness. |
| Tasks | Transfer Learning |
| Published | 2019-05-20 |
| URL | https://arxiv.org/abs/1905.08232v2 |
| https://arxiv.org/pdf/1905.08232v2.pdf | |
| PWC | https://paperswithcode.com/paper/adversarially-robust-transfer-learning |
| Repo | |
| Framework | |
