Paper Group ANR 1218
Minimizing a Sum of Clipped Convex Functions. Cognitive Functions of the Brain: Perception, Attention and Memory. An Embedding Framework for Consistent Polyhedral Surrogates. An Effective Upperbound on Treewidth Using Partial Fill-in of Separators. KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks. Metamorphic Testing of a Deep …
Minimizing a Sum of Clipped Convex Functions
| Title | Minimizing a Sum of Clipped Convex Functions |
| Authors | Shane Barratt, Guillermo Angeris, Stephen Boyd |
| Abstract | We consider the problem of minimizing a sum of clipped convex functions; applications include clipped empirical risk minimization and clipped control. While the problem of minimizing the sum of clipped convex functions is NP-hard, we present some heuristics for approximately solving instances of these problems. These heuristics can be used to find good, if not global, solutions and appear to work well in practice. We also describe an alternative formulation, based on the perspective transformation, which makes the problem amenable to mixed-integer convex programming and yields computationally tractable lower bounds. We illustrate one of our heuristic methods by applying it to various examples and use the perspective transformation to certify that the solutions are relatively close to the global optimum. This paper is accompanied by an open-source implementation. |
| Tasks | |
| Published | 2019-10-27 |
| URL | https://arxiv.org/abs/1910.12342v2 |
| https://arxiv.org/pdf/1910.12342v2.pdf | |
| PWC | https://paperswithcode.com/paper/minimizing-a-sum-of-clipped-convex-functions |
| Repo | |
| Framework | |
Cognitive Functions of the Brain: Perception, Attention and Memory
| Title | Cognitive Functions of the Brain: Perception, Attention and Memory |
| Authors | Jiawei Zhang |
| Abstract | This is a follow-up tutorial article of [17] and [16], in this paper, we will introduce several important cognitive functions of the brain. Brain cognitive functions are the mental processes that allow us to receive, select, store, transform, develop, and recover information that we’ve received from external stimuli. This process allows us to understand and to relate to the world more effectively. Cognitive functions are brain-based skills we need to carry out any task from the simplest to the most complex. They are related with the mechanisms of how we learn, remember, problem-solve, and pay attention, etc. To be more specific, in this paper, we will talk about the perception, attention and memory functions of the human brain. Several other brain cognitive functions, e.g., arousal, decision making, natural language, motor coordination, planning, problem solving and thinking, will be added to this paper in the later versions, respectively. Many of the materials used in this paper are from wikipedia and several other neuroscience introductory articles, which will be properly cited in this paper. This is the last of the three tutorial articles about the brain. The readers are suggested to read this paper after the previous two tutorial articles on brain structure and functions [17] as well as the brain basic neural units [16]. |
| Tasks | Decision Making |
| Published | 2019-05-30 |
| URL | https://arxiv.org/abs/1907.02863v1 |
| https://arxiv.org/pdf/1907.02863v1.pdf | |
| PWC | https://paperswithcode.com/paper/cognitive-functions-of-the-brain-perception |
| Repo | |
| Framework | |
An Embedding Framework for Consistent Polyhedral Surrogates
| Title | An Embedding Framework for Consistent Polyhedral Surrogates |
| Authors | Jessie Finocchiaro, Rafael Frongillo, Bo Waggoner |
| Abstract | We formalize and study the natural approach of designing convex surrogate loss functions via embeddings for problems such as classification or ranking. In this approach, one embeds each of the finitely many predictions (e.g. classes) as a point in R^d, assigns the original loss values to these points, and convexifies the loss in between to obtain a surrogate. We prove that this approach is equivalent, in a strong sense, to working with polyhedral (piecewise linear convex) losses. Moreover, given any polyhedral loss $L$, we give a construction of a link function through which $L$ is a consistent surrogate for the loss it embeds. We go on to illustrate the power of this embedding framework with succinct proofs of consistency or inconsistency of various polyhedral surrogates in the literature. |
| Tasks | |
| Published | 2019-07-17 |
| URL | https://arxiv.org/abs/1907.07330v1 |
| https://arxiv.org/pdf/1907.07330v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-embedding-framework-for-consistent |
| Repo | |
| Framework | |
An Effective Upperbound on Treewidth Using Partial Fill-in of Separators
| Title | An Effective Upperbound on Treewidth Using Partial Fill-in of Separators |
| Authors | Boi Faltings, Martin Charles Golumbic |
| Abstract | Partitioning a graph using graph separators, and particularly clique separators, are well-known techniques to decompose a graph into smaller units which can be treated independently. It was previously known that the treewidth was bounded above by the sum of the size of the separator plus the treewidth of disjoint components, and this was obtained by the heuristic of filling in all edges of the separator making it into a clique. In this paper, we present a new, tighter upper bound on the treewidth of a graph obtained by only partially filling in the edges of a separator. In particular, the method completes just those pairs of separator vertices that are adjacent to a common component, and indicates a more effective heuristic than filling in the entire separator. We discuss the relevance of this result for combinatorial algorithms and give an example of how the tighter bound can be exploited in the domain of constraint satisfaction problems. |
| Tasks | |
| Published | 2019-09-06 |
| URL | https://arxiv.org/abs/1909.02789v1 |
| https://arxiv.org/pdf/1909.02789v1.pdf | |
| PWC | https://paperswithcode.com/paper/an-effective-upperbound-on-treewidth-using |
| Repo | |
| Framework | |
KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks
| Title | KNEEL: Knee Anatomical Landmark Localization Using Hourglass Networks |
| Authors | Aleksei Tiulpin, Iaroslav Melekhov, Simo Saarakkala |
| Abstract | This paper addresses the challenge of localization of anatomical landmarks in knee X-ray images at different stages of osteoarthritis (OA). Landmark localization can be viewed as regression problem, where the landmark position is directly predicted by using the region of interest or even full-size images leading to large memory footprint, especially in case of high resolution medical images. In this work, we propose an efficient deep neural networks framework with an hourglass architecture utilizing a soft-argmax layer to directly predict normalized coordinates of the landmark points. We provide an extensive evaluation of different regularization techniques and various loss functions to understand their influence on the localization performance. Furthermore, we introduce the concept of transfer learning from low-budget annotations, and experimentally demonstrate that such approach is improving the accuracy of landmark localization. Compared to the prior methods, we validate our model on two datasets that are independent from the train data and assess the performance of the method for different stages of OA severity. The proposed approach demonstrates better generalization performance compared to the current state-of-the-art. |
| Tasks | Transfer Learning |
| Published | 2019-07-29 |
| URL | https://arxiv.org/abs/1907.12237v2 |
| https://arxiv.org/pdf/1907.12237v2.pdf | |
| PWC | https://paperswithcode.com/paper/kneel-knee-anatomical-landmark-localization |
| Repo | |
| Framework | |
Metamorphic Testing of a Deep Learning based Forecaster
| Title | Metamorphic Testing of a Deep Learning based Forecaster |
| Authors | Anurag Dwarakanath, Manish Ahuja, Sanjay Podder, Silja Vinu, Arijit Naskar, Koushik MV |
| Abstract | In this paper, we present the Metamorphic Testing of an in-use deep learning based forecasting application. The application looks at the past data of system characteristics (e.g. `memory allocation’) to predict outages in the future. We focus on two statistical / machine learning based components - a) detection of co-relation between system characteristics and b) estimating the future value of a system characteristic using an LSTM (a deep learning architecture). In total, 19 Metamorphic Relations have been developed and we provide proofs & algorithms where applicable. We evaluated our method through two settings. In the first, we executed the relations on the actual application and uncovered 8 issues not known before. Second, we generated hypothetical bugs, through Mutation Testing, on a reference implementation of the LSTM based forecaster and found that 65.9% of the bugs were caught through the relations. | |
| Tasks | |
| Published | 2019-07-13 |
| URL | https://arxiv.org/abs/1907.06632v1 |
| https://arxiv.org/pdf/1907.06632v1.pdf | |
| PWC | https://paperswithcode.com/paper/metamorphic-testing-of-a-deep-learning-based |
| Repo | |
| Framework | |
Egocentric Visitors Localization in Cultural Sites
| Title | Egocentric Visitors Localization in Cultural Sites |
| Authors | Francesco Ragusa, Antonino Furnari, Sebastiano Battiato, Giovanni Signorello, Giovanni Maria Farinella |
| Abstract | We consider the problem of localizing visitors in a cultural site from egocentric (first person) images. Localization information can be useful both to assist the user during his visit (e.g., by suggesting where to go and what to see next) and to provide behavioral information to the manager of the cultural site (e.g., how much time has been spent by visitors at a given location? What has been liked most?). To tackle the problem, we collected a large dataset of egocentric videos using two cameras: a head-mounted HoloLens device and a chest-mounted GoPro. Each frame has been labeled according to the location of the visitor and to what he was looking at. The dataset is freely available in order to encourage research in this domain. The dataset is complemented with baseline experiments performed considering a state-of-the-art method for location-based temporal segmentation of egocentric videos. Experiments show that compelling results can be achieved to extract useful information for both the visitor and the site-manager. |
| Tasks | |
| Published | 2019-04-10 |
| URL | http://arxiv.org/abs/1904.05264v1 |
| http://arxiv.org/pdf/1904.05264v1.pdf | |
| PWC | https://paperswithcode.com/paper/egocentric-visitors-localization-in-cultural |
| Repo | |
| Framework | |
Diversifying Database Activity Monitoring with Bandits
| Title | Diversifying Database Activity Monitoring with Bandits |
| Authors | Hagit Grushka-Cohen, Ofer Biller, Oded Sofer, Lior Rokach, Bracha Shapira |
| Abstract | Database activity monitoring (DAM) systems are commonly used by organizations to protect the organizational data, knowledge and intellectual properties. In order to protect organizations database DAM systems have two main roles, monitoring (documenting activity) and alerting to anomalous activity. Due to high-velocity streams and operating costs, such systems are restricted to examining only a sample of the activity. Current solutions use policies, manually crafted by experts, to decide which transactions to monitor and log. This limits the diversity of the data collected. Bandit algorithms, which use reward functions as the basis for optimization while adding diversity to the recommended set, have gained increased attention in recommendation systems for improving diversity. In this work, we redefine the data sampling problem as a special case of the multi-armed bandit (MAB) problem and present a novel algorithm, which combines expert knowledge with random exploration. We analyze the effect of diversity on coverage and downstream event detection tasks using a simulated dataset. In doing so, we find that adding diversity to the sampling using the bandit-based approach works well for this task and maximizing population coverage without decreasing the quality in terms of issuing alerts about events. |
| Tasks | Recommendation Systems |
| Published | 2019-10-23 |
| URL | https://arxiv.org/abs/1910.10777v1 |
| https://arxiv.org/pdf/1910.10777v1.pdf | |
| PWC | https://paperswithcode.com/paper/diversifying-database-activity-monitoring |
| Repo | |
| Framework | |
Distributed Bandit Learning: Near-Optimal Regret with Efficient Communication
| Title | Distributed Bandit Learning: Near-Optimal Regret with Efficient Communication |
| Authors | Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, Liwei Wang |
| Abstract | We study the problem of regret minimization for distributed bandits learning, in which $M$ agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the total amount of transmitted data. For distributed multi-armed bandits, we propose a protocol with near-optimal regret and only $O(M\log(MK))$ communication cost, where $K$ is the number of arms. The communication cost is independent of the time horizon $T$, has only logarithmic dependence on the number of arms, and matches the lower bound except for a logarithmic factor. For distributed $d$-dimensional linear bandits, we propose a protocol that achieves near-optimal regret and has communication cost of order $\tilde{O}(Md)$, which has only logarithmic dependence on $T$. |
| Tasks | Multi-Armed Bandits |
| Published | 2019-04-12 |
| URL | https://arxiv.org/abs/1904.06309v2 |
| https://arxiv.org/pdf/1904.06309v2.pdf | |
| PWC | https://paperswithcode.com/paper/distributed-bandit-learning-how-much |
| Repo | |
| Framework | |
Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time
| Title | Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time |
| Authors | Jeongho Kim, Insoon Yang |
| Abstract | In this paper, we introduce Hamilton-Jacobi-Bellman (HJB) equations for Q-functions in continuous time optimal control problems with Lipschitz continuous controls. The standard Q-function used in reinforcement learning is shown to be the unique viscosity solution of the HJB equation. A necessary and sufficient condition for optimality is provided using the viscosity solution framework. By using the HJB equation, we develop a Q-learning method for continuous-time dynamical systems. A DQN-like algorithm is also proposed for high-dimensional state and control spaces. The performance of the proposed Q-learning algorithm is demonstrated using 1-, 10- and 20-dimensional dynamical systems. |
| Tasks | Q-Learning |
| Published | 2019-12-23 |
| URL | https://arxiv.org/abs/1912.10697v1 |
| https://arxiv.org/pdf/1912.10697v1.pdf | |
| PWC | https://paperswithcode.com/paper/hamilton-jacobi-bellman-equations-for-q |
| Repo | |
| Framework | |
A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications
| Title | A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications |
| Authors | Leonardo Enzo Brito da Silva, Islam Elnabarawy, Donald C. Wunsch II |
| Abstract | This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers. |
| Tasks | |
| Published | 2019-05-04 |
| URL | https://arxiv.org/abs/1905.11437v1 |
| https://arxiv.org/pdf/1905.11437v1.pdf | |
| PWC | https://paperswithcode.com/paper/190511437 |
| Repo | |
| Framework | |
The CUED’s Grammatical Error Correction Systems for BEA-2019
| Title | The CUED’s Grammatical Error Correction Systems for BEA-2019 |
| Authors | Felix Stahlberg, Bill Byrne |
| Abstract | We describe two entries from the Cambridge University Engineering Department to the BEA 2019 Shared Task on grammatical error correction. Our submission to the low-resource track is based on prior work on using finite state transducers together with strong neural language models. Our system for the restricted track is a purely neural system consisting of neural language models and neural machine translation models trained with back-translation and a combination of checkpoint averaging and fine-tuning – without the help of any additional tools like spell checkers. The latter system has been used inside a separate system combination entry in cooperation with the Cambridge University Computer Lab. |
| Tasks | Grammatical Error Correction, Machine Translation |
| Published | 2019-06-29 |
| URL | https://arxiv.org/abs/1907.00168v1 |
| https://arxiv.org/pdf/1907.00168v1.pdf | |
| PWC | https://paperswithcode.com/paper/the-cueds-grammatical-error-correction |
| Repo | |
| Framework | |
Identifying Cancer Patients at Risk for Heart Failure Using Machine Learning Methods
| Title | Identifying Cancer Patients at Risk for Heart Failure Using Machine Learning Methods |
| Authors | Xi Yang, Yan Gong, Nida Waheed, Keith March, Jiang Bian, William R. Hogan, Yonghui Wu |
| Abstract | Cardiotoxicity related to cancer therapies has become a serious issue, diminishing cancer treatment outcomes and quality of life. Early detection of cancer patients at risk for cardiotoxicity before cardiotoxic treatments and providing preventive measures are potential solutions to improve cancer patients’s quality of life. This study focuses on predicting the development of heart failure in cancer patients after cancer diagnoses using historical electronic health record (EHR) data. We examined four machine learning algorithms using 143,199 cancer patients from the University of Florida Health (UF Health) Integrated Data Repository (IDR). We identified a total number of 1,958 qualified cases and matched them to 15,488 controls by gender, age, race, and major cancer type. Two feature encoding strategies were compared to encode variables as machine learning features. The gradient boosting (GB) based model achieved the best AUC score of 0.9077 (with a sensitivity of 0.8520 and a specificity of 0.8138), outperforming other machine learning methods. We also looked into the subgroup of cancer patients with exposure to chemotherapy drugs and observed a lower specificity score (0.7089). The experimental results show that machine learning methods are able to capture clinical factors that are known to be associated with heart failure and that it is feasible to use machine learning methods to identify cancer patients at risk for cancer therapy-related heart failure. |
| Tasks | |
| Published | 2019-10-01 |
| URL | https://arxiv.org/abs/1910.00582v1 |
| https://arxiv.org/pdf/1910.00582v1.pdf | |
| PWC | https://paperswithcode.com/paper/identifying-cancer-patients-at-risk-for-heart |
| Repo | |
| Framework | |
Zeroth-order Stochastic Compositional Algorithms for Risk-Aware Learning
| Title | Zeroth-order Stochastic Compositional Algorithms for Risk-Aware Learning |
| Authors | Dionysios S. Kalogerias, Warren B. Powell |
| Abstract | We present Free-MESSAGEp, the first zeroth-order algorithm for convex mean-semideviation-based risk-aware learning, which is also the first three-level zeroth-order compositional stochastic optimization algorithm, whatsoever. Using a non-trivial extension of Nesterov’s classical results on Gaussian smoothing, we develop the Free-MESSAGEp algorithm from first principles, and show that it essentially solves a smoothed surrogate to the original problem, the former being a uniform approximation of the latter, in a useful, convenient sense. We then present a complete analysis of the Free-MESSAGEp algorithm, which establishes convergence in a user-tunable neighborhood of the optimal solutions of the original problem, as well as explicit convergence rates for both convex and strongly convex costs. Orderwise, and for fixed problem parameters, our results demonstrate no sacrifice in convergence speed compared to existing first-order methods, while striking a certain balance among the condition of the problem, its dimensionality, as well as the accuracy of the obtained results, naturally extending previous results in zeroth-order risk-neutral learning. |
| Tasks | Stochastic Optimization |
| Published | 2019-12-19 |
| URL | https://arxiv.org/abs/1912.09484v1 |
| https://arxiv.org/pdf/1912.09484v1.pdf | |
| PWC | https://paperswithcode.com/paper/zeroth-order-stochastic-compositional |
| Repo | |
| Framework | |
Temporal Network Embedding with Micro- and Macro-dynamics
| Title | Temporal Network Embedding with Micro- and Macro-dynamics |
| Authors | Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S. Yu, Yanfang Ye |
| Abstract | Network embedding aims to embed nodes into a low-dimensional space, while capturing the network structures and properties. Although quite a few promising network embedding methods have been proposed, most of them focus on static networks. In fact, temporal networks, which usually evolve over time in terms of microscopic and macroscopic dynamics, are ubiquitous. The micro-dynamics describe the formation process of network structures in a detailed manner, while the macro-dynamics refer to the evolution pattern of the network scale. Both micro- and macro-dynamics are the key factors to network evolution; however, how to elegantly capture both of them for temporal network embedding, especially macro-dynamics, has not yet been well studied. In this paper, we propose a novel temporal network embedding method with micro- and macro-dynamics, named $\rm{M^2DNE}$. Specifically, for micro-dynamics, we regard the establishments of edges as the occurrences of chronological events and propose a temporal attention point process to capture the formation process of network structures in a fine-grained manner. For macro-dynamics, we define a general dynamics equation parameterized with network embeddings to capture the inherent evolution pattern and impose constraints in a higher structural level on network embeddings. Mutual evolutions of micro- and macro-dynamics in a temporal network alternately affect the process of learning node embeddings. Extensive experiments on three real-world temporal networks demonstrate that $\rm{M^2DNE}$ significantly outperforms the state-of-the-arts not only in traditional tasks, e.g., network reconstruction, but also in temporal tendency-related tasks, e.g., scale prediction. |
| Tasks | Network Embedding |
| Published | 2019-09-10 |
| URL | https://arxiv.org/abs/1909.04246v1 |
| https://arxiv.org/pdf/1909.04246v1.pdf | |
| PWC | https://paperswithcode.com/paper/temporal-network-embedding-with-micro-and |
| Repo | |
| Framework | |
