Paper Group ANR 1150
Activation Functions: Comparison of trends in Practice and Research for Deep Learning. See far with TPNET: a Tile Processor and a CNN Symbiosis. Machine Translation between Vietnamese and English: an Empirical Study. Estimating Achilles tendon healing progress with convolutional neural networks. On Looking for Local Expansion Invariants in Argument …
Activation Functions: Comparison of trends in Practice and Research for Deep Learning
| Title | Activation Functions: Comparison of trends in Practice and Research for Deep Learning |
| Authors | Chigozie Nwankpa, Winifred Ijomah, Anthony Gachagan, Stephen Marshall |
| Abstract | Deep neural networks have been successfully used in diverse emerging domains to solve real world complex problems with may more deep learning(DL) architectures, being developed to date. To achieve these state-of-the-art performances, the DL architectures use activation functions (AFs), to perform diverse computations between the hidden layers and the output layers of any given DL architecture. This paper presents a survey on the existing AFs used in deep learning applications and highlights the recent trends in the use of the activation functions for deep learning applications. The novelty of this paper is that it compiles majority of the AFs used in DL and outlines the current trends in the applications and usage of these functions in practical deep learning deployments against the state-of-the-art research results. This compilation will aid in making effective decisions in the choice of the most suitable and appropriate activation function for any given application, ready for deployment. This paper is timely because most research papers on AF highlights similar works and results while this paper will be the first, to compile the trends in AF applications in practice against the research results from literature, found in deep learning research to date. |
| Tasks | |
| Published | 2018-11-08 |
| URL | http://arxiv.org/abs/1811.03378v1 |
| http://arxiv.org/pdf/1811.03378v1.pdf | |
| PWC | https://paperswithcode.com/paper/activation-functions-comparison-of-trends-in |
| Repo | |
| Framework | |
See far with TPNET: a Tile Processor and a CNN Symbiosis
| Title | See far with TPNET: a Tile Processor and a CNN Symbiosis |
| Authors | Andrey Filippov, Oleg Dzhimiev |
| Abstract | Throughout the evolution of the neural networks more specialized cells were added to the set of basic building blocks. These cells aim to improve training convergence, increase the overall performance, and reduce the number of required labels, all while preserving the expressive power of the universal network. Inspired by the partitioning of the human visual perception system between the eyes and the cerebral cortex, we present TPNET, which offloads universal and application-specific CNN from the bulk processing of the high resolution pixel data and performs the translation-variant image correction while delegating all non-linear decision making to the network. In this work, we explore application of TPNET to 3D perception with a narrow-baseline (0.0001-0.0025) quad stereo camera and prove that a trained network provides a disparity prediction from the 2D phase correlation output by the Tile Processor (TP) that is twice as accurate as the prediction from a carefully hand-crafted algorithm. The TP in turn reduces the dimensions of the input features of the network and provides instrument-invariant and translation-invariant data, making real-time high resolution stereo 3D perception feasible and easing the requirement to have a complete end-to-end network. |
| Tasks | Decision Making |
| Published | 2018-11-20 |
| URL | http://arxiv.org/abs/1811.08032v1 |
| http://arxiv.org/pdf/1811.08032v1.pdf | |
| PWC | https://paperswithcode.com/paper/see-far-with-tpnet-a-tile-processor-and-a-cnn |
| Repo | |
| Framework | |
Machine Translation between Vietnamese and English: an Empirical Study
| Title | Machine Translation between Vietnamese and English: an Empirical Study |
| Authors | Hong-Hai Phan-Vu, Viet-Trung Tran, Van-Nam Nguyen, Hoang-Vu Dang, Phan-Thuan Do |
| Abstract | Machine translation is shifting to an end-to-end approach based on deep neural networks. The state of the art achieves impressive results for popular language pairs such as English - French or English - Chinese. However for English - Vietnamese the shortage of parallel corpora and expensive hyper-parameter search present practical challenges to neural-based approaches. This paper highlights our efforts on improving English-Vietnamese translations in two directions: (1) Building the largest open Vietnamese - English corpus to date, and (2) Extensive experiments with the latest neural models to achieve the highest BLEU scores. Our experiments provide practical examples of effectively employing different neural machine translation models with low-resource language pairs. |
| Tasks | Machine Translation |
| Published | 2018-10-30 |
| URL | http://arxiv.org/abs/1810.12557v1 |
| http://arxiv.org/pdf/1810.12557v1.pdf | |
| PWC | https://paperswithcode.com/paper/machine-translation-between-vietnamese-and |
| Repo | |
| Framework | |
Estimating Achilles tendon healing progress with convolutional neural networks
| Title | Estimating Achilles tendon healing progress with convolutional neural networks |
| Authors | Norbert Kapinski, Jakub Zielinski, Bartosz A. Borucki, Tomasz Trzcinski, Beata Ciszkowska-Lyson, Krzysztof S. Nowinski |
| Abstract | Quantitative assessment of a treatment progress in the Achilles tendon healing process - one of the most common musculoskeletal disorder in modern medical practice - is typically a long and complex process: multiple MRI protocols need to be acquired and analysed by radiology experts. In this paper, we propose to significantly reduce the complexity of this assessment using a novel method based on a pre-trained convolutional neural network. We first train our neural network on over 500,000 2D axial cross-sections from over 3000 3D MRI studies to classify MRI images as belonging to a healthy or injured class, depending on the patient’s condition. We then take the outputs of modified pre-trained network and apply linear regression on the PCA-reduced space of the features to assess treatment progress. Our method allows to reduce up to 5-fold the amount of data needed to be registered during the MRI scan without any information loss. Furthermore, we are able to predict the healing process phase with equal accuracy to human experts in 3 out of 6 main criteria. Finally, contrary to the current approaches to regeneration assessment that rely on radiologist subjective opinion, our method allows to objectively compare different treatments methods which can lead to improved diagnostics and patient’s recovery. |
| Tasks | |
| Published | 2018-06-13 |
| URL | http://arxiv.org/abs/1806.05091v2 |
| http://arxiv.org/pdf/1806.05091v2.pdf | |
| PWC | https://paperswithcode.com/paper/estimating-achilles-tendon-healing-progress |
| Repo | |
| Framework | |
On Looking for Local Expansion Invariants in Argumentation Semantics: a Preliminary Report
| Title | On Looking for Local Expansion Invariants in Argumentation Semantics: a Preliminary Report |
| Authors | Stefano Bistarelli, Francesco Santini, Carlo Taticchi |
| Abstract | We study invariant local expansion operators for conflict-free and admissible sets in Abstract Argumentation Frameworks (AFs). Such operators are directly applied on AFs, and are invariant with respect to a chosen “semantics” (that is w.r.t. each of the conflict free/admissible set of arguments). Accordingly, we derive a definition of robustness for AFs in terms of the number of times such operators can be applied without producing any change in the chosen semantics. |
| Tasks | Abstract Argumentation |
| Published | 2018-02-22 |
| URL | http://arxiv.org/abs/1802.08328v2 |
| http://arxiv.org/pdf/1802.08328v2.pdf | |
| PWC | https://paperswithcode.com/paper/on-looking-for-local-expansion-invariants-in |
| Repo | |
| Framework | |
Image Pre-processing Using OpenCV Library on MORPH-II Face Database
| Title | Image Pre-processing Using OpenCV Library on MORPH-II Face Database |
| Authors | Benjamin Yip, Rachel Towner, Troy Kling, Cuixian Chen, Yishi Wang |
| Abstract | This paper outlines the steps taken toward pre-processing the 55,134 images of the MORPH-II non-commercial dataset. Following the introduction, section two begins with an overview of each step in the pre-processing pipeline. Section three expands upon each stage of the process and includes details on all calculations made, by providing the OpenCV functionality paired with each step. The last portion of this paper discusses the potential improvements to this pre-processing pipeline that became apparent in retrospect. |
| Tasks | |
| Published | 2018-11-16 |
| URL | http://arxiv.org/abs/1811.06934v1 |
| http://arxiv.org/pdf/1811.06934v1.pdf | |
| PWC | https://paperswithcode.com/paper/image-pre-processing-using-opencv-library-on |
| Repo | |
| Framework | |
Towards an Unequivocal Representation of Actions
| Title | Towards an Unequivocal Representation of Actions |
| Authors | Michael Wray, Davide Moltisanti, Dima Damen |
| Abstract | This work introduces verb-only representations for actions and interactions; the problem of describing similar motions (e.g. ‘open door’, ‘open cupboard’), and distinguish differing ones (e.g. ‘open door’ vs ‘open bottle’) using verb-only labels. Current approaches for action recognition neglect legitimate semantic ambiguities and class overlaps between verbs (Fig. 1), relying on the objects to disambiguate interactions. We deviate from single-verb labels and introduce a mapping between observations and multiple verb labels - in order to create an Unequivocal Representation of Actions. The new representation benefits from increased vocabulary and a soft assignment to an enriched space of verb labels. We learn these representations as multi-output regression, using a two-stream fusion CNN. The proposed approach outperforms conventional single-verb labels (also known as majority voting) on three egocentric datasets for both recognition and retrieval. |
| Tasks | Temporal Action Localization |
| Published | 2018-05-10 |
| URL | http://arxiv.org/abs/1805.04026v1 |
| http://arxiv.org/pdf/1805.04026v1.pdf | |
| PWC | https://paperswithcode.com/paper/towards-an-unequivocal-representation-of |
| Repo | |
| Framework | |
A probabilistic constrained clustering for transfer learning and image category discovery
| Title | A probabilistic constrained clustering for transfer learning and image category discovery |
| Authors | Yen-Chang Hsu, Zhaoyang Lv, Joel Schlosser, Phillip Odom, Zsolt Kira |
| Abstract | Neural network-based clustering has recently gained popularity, and in particular a constrained clustering formulation has been proposed to perform transfer learning and image category discovery using deep learning. The core idea is to formulate a clustering objective with pairwise constraints that can be used to train a deep clustering network; therefore the cluster assignments and their underlying feature representations are jointly optimized end-to-end. In this work, we provide a novel clustering formulation to address scalability issues of previous work in terms of optimizing deeper networks and larger amounts of categories. The proposed objective directly minimizes the negative log-likelihood of cluster assignment with respect to the pairwise constraints, has no hyper-parameters, and demonstrates improved scalability and performance on both supervised learning and unsupervised transfer learning. |
| Tasks | Transfer Learning |
| Published | 2018-06-28 |
| URL | http://arxiv.org/abs/1806.11078v1 |
| http://arxiv.org/pdf/1806.11078v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-probabilistic-constrained-clustering-for |
| Repo | |
| Framework | |
An Integrated Transfer Learning and Multitask Learning Approach for Pharmacokinetic Parameter Prediction
| Title | An Integrated Transfer Learning and Multitask Learning Approach for Pharmacokinetic Parameter Prediction |
| Authors | Zhuyifan Ye, Yilong Yang, Xiaoshan Li, Dongsheng Cao, Defang Ouyang |
| Abstract | Background: Pharmacokinetic evaluation is one of the key processes in drug discovery and development. However, current absorption, distribution, metabolism, excretion prediction models still have limited accuracy. Aim: This study aims to construct an integrated transfer learning and multitask learning approach for developing quantitative structure-activity relationship models to predict four human pharmacokinetic parameters. Methods: A pharmacokinetic dataset included 1104 U.S. FDA approved small molecule drugs. The dataset included four human pharmacokinetic parameter subsets (oral bioavailability, plasma protein binding rate, apparent volume of distribution at steady-state and elimination half-life). The pre-trained model was trained on over 30 million bioactivity data. An integrated transfer learning and multitask learning approach was established to enhance the model generalization. Results: The pharmacokinetic dataset was split into three parts (60:20:20) for training, validation and test by the improved Maximum Dissimilarity algorithm with the representative initial set selection algorithm and the weighted distance function. The multitask learning techniques enhanced the model predictive ability. The integrated transfer learning and multitask learning model demonstrated the best accuracies, because deep neural networks have the general feature extraction ability, transfer learning and multitask learning improved the model generalization. Conclusions: The integrated transfer learning and multitask learning approach with the improved dataset splitting algorithm was firstly introduced to predict the pharmacokinetic parameters. This method can be further employed in drug discovery and development. |
| Tasks | Drug Discovery, Transfer Learning |
| Published | 2018-12-21 |
| URL | http://arxiv.org/abs/1812.09073v1 |
| http://arxiv.org/pdf/1812.09073v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-integrated-transfer-learning-and-multitask |
| Repo | |
| Framework | |
Road Segmentation Using CNN with GRU
| Title | Road Segmentation Using CNN with GRU |
| Authors | Yecheng Lyu, Xinming Huang |
| Abstract | This paper presents an accurate and fast algorithm for road segmentation using convolutional neural network (CNN) and gated recurrent units (GRU). For autonomous vehicles, road segmentation is a fundamental task that can provide the drivable area for path planning. The existing deep neural network based segmentation algorithms usually take a very deep encoder-decoder structure to fuse pixels, which requires heavy computations, large memory and long processing time. Hereby, a CNN-GRU network model is proposed and trained to perform road segmentation using data captured by the front camera of a vehicle. GRU network obtains a long spatial sequence with lower computational complexity, comparing to traditional encoder-decoder architecture. The proposed road detector is evaluated on the KITTI road benchmark and achieves high accuracy for road segmentation at real-time processing speed. |
| Tasks | Autonomous Vehicles |
| Published | 2018-04-14 |
| URL | http://arxiv.org/abs/1804.05164v1 |
| http://arxiv.org/pdf/1804.05164v1.pdf | |
| PWC | https://paperswithcode.com/paper/road-segmentation-using-cnn-with-gru |
| Repo | |
| Framework | |
Evaluating Actuators in a Purely Information-Theory Based Reward Model
| Title | Evaluating Actuators in a Purely Information-Theory Based Reward Model |
| Authors | Wojciech Skaba |
| Abstract | AGINAO builds its cognitive engine by applying self-programming techniques to create a hierarchy of interconnected codelets - the tiny pieces of code executed on a virtual machine. These basic processing units are evaluated for their applicability and fitness with a notion of reward calculated from self-information gain of binary partitioning of the codelet’s input state-space. This approach, however, is useless for the evaluation of actuators. Instead, a model is proposed in which actuators are evaluated by measuring the impact that an activation of an effector, and consequently the feedback from the robot sensors, has on average reward received by the processing units. |
| Tasks | |
| Published | 2018-04-10 |
| URL | http://arxiv.org/abs/1804.03439v1 |
| http://arxiv.org/pdf/1804.03439v1.pdf | |
| PWC | https://paperswithcode.com/paper/evaluating-actuators-in-a-purely-information |
| Repo | |
| Framework | |
LSTM-Based ECG Classification for Continuous Monitoring on Personal Wearable Devices
| Title | LSTM-Based ECG Classification for Continuous Monitoring on Personal Wearable Devices |
| Authors | Saeed Saadatnejad, Mohammadhosein Oveisi, Matin Hashemi |
| Abstract | Objective: A novel ECG classification algorithm is proposed for continuous cardiac monitoring on wearable devices with limited processing capacity. Methods: The proposed solution employs a novel architecture consisting of wavelet transform and multiple LSTM recurrent neural networks. Results: Experimental evaluations show superior ECG classification performance compared to previous works. Measurements on different hardware platforms show the proposed algorithm meets timing requirements for continuous and real-time execution on wearable devices. Conclusion: In contrast to many compute-intensive deep-learning based approaches, the proposed algorithm is lightweight, and therefore, brings continuous monitoring with accurate LSTM-based ECG classification to wearable devices. Significance: The proposed algorithm is both accurate and lightweight. The source code is available online [1]. |
| Tasks | ECG Classification |
| Published | 2018-12-12 |
| URL | https://arxiv.org/abs/1812.04818v3 |
| https://arxiv.org/pdf/1812.04818v3.pdf | |
| PWC | https://paperswithcode.com/paper/lstm-based-ecg-classification-for-continuous |
| Repo | |
| Framework | |
Fairness Under Composition
| Title | Fairness Under Composition |
| Authors | Cynthia Dwork, Christina Ilvento |
| Abstract | Algorithmic fairness, and in particular the fairness of scoring and classification algorithms, has become a topic of increasing social concern and has recently witnessed an explosion of research in theoretical computer science, machine learning, statistics, the social sciences, and law. Much of the literature considers the case of a single classifier (or scoring function) used once, in isolation. In this work, we initiate the study of the fairness properties of systems composed of algorithms that are fair in isolation; that is, we study fairness under composition. We identify pitfalls of naive composition and give general constructions for fair composition, demonstrating both that classifiers that are fair in isolation do not necessarily compose into fair systems and also that seemingly unfair components may be carefully combined to construct fair systems. We focus primarily on the individual fairness setting proposed in [Dwork, Hardt, Pitassi, Reingold, Zemel, 2011], but also extend our results to a large class of group fairness definitions popular in the recent literature, exhibiting several cases in which group fairness definitions give misleading signals under composition. |
| Tasks | |
| Published | 2018-06-15 |
| URL | http://arxiv.org/abs/1806.06122v2 |
| http://arxiv.org/pdf/1806.06122v2.pdf | |
| PWC | https://paperswithcode.com/paper/fairness-under-composition |
| Repo | |
| Framework | |
Modeling Customer Engagement from Partial Observations
| Title | Modeling Customer Engagement from Partial Observations |
| Authors | Jelena Stojanovic, Djordje Gligorijevic, Zoran Obradovic |
| Abstract | It is of high interest for a company to identify customers expected to bring the largest profit in the upcoming period. Knowing as much as possible about each customer is crucial for such predictions. However, their demographic data, preferences, and other information that might be useful for building loyalty programs is often missing. Additionally, modeling relations among different customers as a network can be beneficial for predictions at an individual level, as similar customers tend to have similar purchasing patterns. We address this problem by proposing a robust framework for structured regression on deficient data in evolving networks with a supervised representation learning based on neural features embedding. The new method is compared to several unstructured and structured alternatives for predicting customer behavior (e.g. purchasing frequency and customer ticket) on user networks generated from customer databases of two companies from different industries. The obtained results show $4%$ to $130%$ improvement in accuracy over alternatives when all customer information is known. Additionally, the robustness of our method is demonstrated when up to $80%$ of demographic information was missing where it was up to several folds more accurate as compared to alternatives that are either ignoring cases with missing values or learn their feature representation in an unsupervised manner. |
| Tasks | Representation Learning |
| Published | 2018-03-28 |
| URL | http://arxiv.org/abs/1803.10799v1 |
| http://arxiv.org/pdf/1803.10799v1.pdf | |
| PWC | https://paperswithcode.com/paper/modeling-customer-engagement-from-partial |
| Repo | |
| Framework | |
Francy - An Interactive Discrete Mathematics Framework for GAP
| Title | Francy - An Interactive Discrete Mathematics Framework for GAP |
| Authors | Manuel Machado Martins, Markus Pfeiffer |
| Abstract | Data visualization and interaction with large data sets is known to be essential and critical in many businesses today, and the same applies to research and teaching, in this case, when exploring large and complex mathematical objects. GAP is a computer algebra system for computational discrete algebra with an emphasis on computational group theory. The existing XGAP package for GAP works exclusively on the X Window System. It lacks abstraction between its mathematical and graphical cores, making it difficult to extend, maintain, or port. In this paper, we present Francy, a graphical semantics package for GAP. Francy is responsible for creating a representational structure that can be rendered using many GUI frameworks independent from any particular programming language or operating system. Building on this, we use state of the art web technologies that take advantage of an improved REPL environment, which is currently under development for GAP. The integration of this project with Jupyter provides a rich graphical environment full of features enhancing the usability and accessibility of GAP. |
| Tasks | |
| Published | 2018-06-19 |
| URL | http://arxiv.org/abs/1806.08648v1 |
| http://arxiv.org/pdf/1806.08648v1.pdf | |
| PWC | https://paperswithcode.com/paper/francy-an-interactive-discrete-mathematics |
| Repo | |
| Framework | |
