Paper Group ANR 225
A Deep, Forgetful Novelty-Seeking Movie Recommender Model. Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare. A Convergence Proof of Projected Fast Iterative Soft-thresholding Algorithm for Parallel Magnetic Resonance Imaging. Multiple-Kernel Dictionary Learning for Reconstruction and Clustering of Unseen Multivariate Time-series …
A Deep, Forgetful Novelty-Seeking Movie Recommender Model
| Title | A Deep, Forgetful Novelty-Seeking Movie Recommender Model |
| Authors | Ruomu Zou |
| Abstract | As more and more people shift their movie watching online, competition between movie viewing websites are getting more and more intense. Therefore, it has become incredibly important to accurately predict a given user’s watching list to maximize the chances of keeping the user on the platform. Recent studies have suggested that the novelty-seeking propensity of users can impact their viewing behavior. In this paper, we aim to accurately model and describe this novelty-seeking trait across many users and timestamps driven by data, taking into consideration user forgetfulness. Compared to previous studies, we propose a more robust measure for novelty. Our model, termed Deep Forgetful Novelty-Seeking Model (DFNSM), leverages demographic information about users, genre information about movies, and novelty-seeking traits to predict the most likely next actions of a user. To evaluate the performance of our model, we conducted extensive experiments on a large movie rating dataset. The results reveal that DFNSM is very effective for movie recommendation. |
| Tasks | |
| Published | 2019-09-02 |
| URL | https://arxiv.org/abs/1909.01811v1 |
| https://arxiv.org/pdf/1909.01811v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-forgetful-novelty-seeking-movie |
| Repo | |
| Framework | |
Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare
| Title | Fairness With Minimal Harm: A Pareto-Optimal Approach For Healthcare |
| Authors | Natalia Martinez, Martin Bertran, Guillermo Sapiro |
| Abstract | Common fairness definitions in machine learning focus on balancing notions of disparity and utility. In this work, we study fairness in the context of risk disparity among sub-populations. We are interested in learning models that minimize performance discrepancies across sensitive groups without causing unnecessary harm. This is relevant to high-stakes domains such as healthcare, where non-maleficence is a core principle. We formalize this objective using Pareto frontiers, and provide analysis, based on recent works in fairness, to exemplify scenarios were perfect fairness might not be feasible without doing unnecessary harm. We present a methodology for training neural networks that achieve our goal by dynamically re-balancing subgroups risks. We argue that even in domains where fairness at cost is required, finding a non-unnecessary-harm fairness model is the optimal initial step. We demonstrate this methodology on real case-studies of predicting ICU patient mortality, and classifying skin lesions from dermatoscopic images. |
| Tasks | |
| Published | 2019-11-16 |
| URL | https://arxiv.org/abs/1911.06935v1 |
| https://arxiv.org/pdf/1911.06935v1.pdf | |
| PWC | https://paperswithcode.com/paper/fairness-with-minimal-harm-a-pareto-optimal |
| Repo | |
| Framework | |
A Convergence Proof of Projected Fast Iterative Soft-thresholding Algorithm for Parallel Magnetic Resonance Imaging
| Title | A Convergence Proof of Projected Fast Iterative Soft-thresholding Algorithm for Parallel Magnetic Resonance Imaging |
| Authors | Xinlin Zhang, Hengfa Lu, Di Guo, Lijun Bao, Feng Huang, Xiaobo Qu |
| Abstract | The boom of non-uniform sampling and compressed sensing techniques dramatically alleviates the prolonged data acquisition problem of magnetic resonance imaging. Sparse reconstruction, thanks to its fast computation and promising performance, has attracted researchers to put numerous efforts on it and has been adopted in commercial scanners. Algorithms for solving the sparse reconstruction models play an essential role in sparse reconstruction. Being a simple and efficient algorithm for sparse reconstruction, pFISTA has been successfully extended to parallel imaging, however, its convergence criterion is still an open question, confusing users on the setting of the parameter which assures the convergence of the algorithm. In this work, we prove the convergence of the parallel imaging version pFISTA. Specifically, the convergences of two well-known parallel imaging reconstruction models, SENSE and SPIRiT, solved by pFISTA are proved. Experiments on brain images demonstrate the validity of the convergence criterion. The convergence criterion proofed in this work can help users quickly obtain the satisfy parameter that admits faithful results and fast convergence speeds. |
| Tasks | |
| Published | 2019-09-17 |
| URL | https://arxiv.org/abs/1909.07600v1 |
| https://arxiv.org/pdf/1909.07600v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-convergence-proof-of-projected-fast |
| Repo | |
| Framework | |
Multiple-Kernel Dictionary Learning for Reconstruction and Clustering of Unseen Multivariate Time-series
| Title | Multiple-Kernel Dictionary Learning for Reconstruction and Clustering of Unseen Multivariate Time-series |
| Authors | Babak Hosseini, Barbara Hammer |
| Abstract | There exist many approaches for description and recognition of unseen classes in datasets. Nevertheless, it becomes a challenging problem when we deal with multivariate time-series (MTS) (e.g., motion data), where we cannot apply the vectorial algorithms directly to the inputs. In this work, we propose a novel multiple-kernel dictionary learning (MKD) which learns semantic attributes based on specific combinations of MTS dimensions in the feature space. Hence, MKD can fully/partially reconstructs the unseen classes based on the training data (seen classes). Furthermore, we obtain sparse encodings for unseen classes based on the learned MKD attributes, and upon which we propose a simple but effective incremental clustering algorithm to categorize the unseen MTS classes in an unsupervised way. According to the empirical evaluation of our MKD framework on real benchmarks, it provides an interpretable reconstruction of unseen MTS data as well as a high performance regarding their online clustering. |
| Tasks | Dictionary Learning, Time Series |
| Published | 2019-03-05 |
| URL | http://arxiv.org/abs/1903.01867v3 |
| http://arxiv.org/pdf/1903.01867v3.pdf | |
| PWC | https://paperswithcode.com/paper/multiple-kernel-dictionary-learning-for |
| Repo | |
| Framework | |
Sequential training algorithm for neural networks
| Title | Sequential training algorithm for neural networks |
| Authors | Jongrae Kim |
| Abstract | A sequential training method for large-scale feedforward neural networks is presented. Each layer of the neural network is decoupled and trained separately. After the training is completed for each layer, they are combined together. The performance of the network would be sub-optimal compared to the full network training if the optimal solution would be achieved. However, achieving the optimal solution for the full network would be infeasible or require long computing time. The proposed sequential approach reduces the required computer resources significantly and would have better convergences as a single layer is optimised for each optimisation step. The required modifications of existing algorithms to implement the sequential training are minimal. The performance is verified by a simple example. |
| Tasks | |
| Published | 2019-05-17 |
| URL | https://arxiv.org/abs/1905.07490v1 |
| https://arxiv.org/pdf/1905.07490v1.pdf | |
| PWC | https://paperswithcode.com/paper/sequential-training-algorithm-for-neural |
| Repo | |
| Framework | |
Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures
| Title | Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures |
| Authors | Yashar Kiarashinejad, Sajjad Abdollahramezani, Ali Adibi |
| Abstract | In this paper, we demonstrate a computationally efficient new approach based on deep learning (DL) techniques for analysis, design, and optimization of electromagnetic (EM) nanostructures. We use the strong correlation among features of a generic EM problem to considerably reduce the dimensionality of the problem and thus, the computational complexity, without imposing considerable errors. By employing the dimensionality reduction concept using the more recently demonstrated autoencoder technique, we redefine the conventional many-to-one design problem in EM nanostructures into a one-to-one problem plus a much simpler many-to-one problem, which can be simply solved using an analytic formulation. This approach reduces the computational complexity in solving both the forward problem (i.e., analysis) and the inverse problem (i.e., design) by orders of magnitude compared to conventional approaches. In addition, it provides analytic formulations that, despite their complexity, can be used to obtain intuitive understanding of the physics and dynamics of EM wave interaction with nanostructures with minimal computation requirements. As a proof-of-concept, we applied such an efficacious method to design a new class of on-demand reconfigurable optical metasurfaces based on phase-change materials (PCM). We envision that the integration of such a DL-based technique with full-wave commercial software packages offers a powerful toolkit to facilitate the analysis, design, and optimization of the EM nanostructures as well as explaining, understanding, and predicting the observed responses in such structures. |
| Tasks | Dimensionality Reduction |
| Published | 2019-02-11 |
| URL | http://arxiv.org/abs/1902.03865v3 |
| http://arxiv.org/pdf/1902.03865v3.pdf | |
| PWC | https://paperswithcode.com/paper/deep-learning-approach-based-on |
| Repo | |
| Framework | |
Self-Imitation Learning of Locomotion Movements through Termination Curriculum
| Title | Self-Imitation Learning of Locomotion Movements through Termination Curriculum |
| Authors | Amin Babadi, Kourosh Naderi, Perttu Hämäläinen |
| Abstract | Animation and machine learning research have shown great advancements in the past decade, leading to robust and powerful methods for learning complex physically-based animations. However, learning can take hours or days, especially if no reference movement data is available. In this paper, we propose and evaluate a novel combination of techniques for accelerating the learning of stable locomotion movements through self-imitation learning of synthetic animations. First, we produce synthetic and cyclic reference movement using a recent online tree search approach that can discover stable walking gaits in a few minutes. This allows us to use reinforcement learning with Reference State Initialization (RSI) to find a neural network controller for imitating the synthesized reference motion. We further accelerate the learning using a novel curriculum learning approach called Termination Curriculum (TC), that adapts the episode termination threshold over time. The combination of the RSI and TC ensures that simulation budget is not wasted in regions of the state space not visited by the final policy. As a result, our agents can learn locomotion skills in just a few hours on a modest 4-core computer. We demonstrate this by producing locomotion movements for a variety of characters. |
| Tasks | Imitation Learning |
| Published | 2019-07-27 |
| URL | https://arxiv.org/abs/1907.11842v2 |
| https://arxiv.org/pdf/1907.11842v2.pdf | |
| PWC | https://paperswithcode.com/paper/self-imitation-learning-of-locomotion |
| Repo | |
| Framework | |
A Geometry-Sensitive Approach for Photographic Style Classification
| Title | A Geometry-Sensitive Approach for Photographic Style Classification |
| Authors | Koustav Ghosal, Mukta Prasad, Aljosa Smolic |
| Abstract | Photographs are characterized by different compositional attributes like the Rule of Thirds, depth of field, vanishing-lines etc. The presence or absence of one or more of these attributes contributes to the overall artistic value of an image. In this work, we analyze the ability of deep learning based methods to learn such photographic style attributes. We observe that although a standard CNN learns the texture and appearance based features reasonably well, its understanding of global and geometric features is limited by two factors. First, the data-augmentation strategies (cropping, warping, etc.) distort the composition of a photograph and affect the performance. Secondly, the CNN features, in principle, are translation-invariant and appearance-dependent. But some geometric properties important for aesthetics, e.g. the Rule of Thirds (RoT), are position-dependent and appearance-invariant. Therefore, we propose a novel input representation which is geometry-sensitive, position-cognizant and appearance-invariant. We further introduce a two-column CNN architecture that performs better than the state-of-the-art (SoA) in photographic style classification. From our results, we observe that the proposed network learns both the geometric and appearance-based attributes better than the SoA. |
| Tasks | Data Augmentation |
| Published | 2019-09-03 |
| URL | https://arxiv.org/abs/1909.01040v1 |
| https://arxiv.org/pdf/1909.01040v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-geometry-sensitive-approach-for |
| Repo | |
| Framework | |
NOODL: Provable Online Dictionary Learning and Sparse Coding
| Title | NOODL: Provable Online Dictionary Learning and Sparse Coding |
| Authors | Sirisha Rambhatla, Xingguo Li, Jarvis Haupt |
| Abstract | We consider the dictionary learning problem, where the aim is to model the given data as a linear combination of a few columns of a matrix known as a dictionary, where the sparse weights forming the linear combination are known as coefficients. Since the dictionary and coefficients, parameterizing the linear model are unknown, the corresponding optimization is inherently non-convex. This was a major challenge until recently, when provable algorithms for dictionary learning were proposed. Yet, these provide guarantees only on the recovery of the dictionary, without explicit recovery guarantees on the coefficients. Moreover, any estimation error in the dictionary adversely impacts the ability to successfully localize and estimate the coefficients. This potentially limits the utility of existing provable dictionary learning methods in applications where coefficient recovery is of interest. To this end, we develop NOODL: a simple Neurally plausible alternating Optimization-based Online Dictionary Learning algorithm, which recovers both the dictionary and coefficients exactly at a geometric rate, when initialized appropriately. Our algorithm, NOODL, is also scalable and amenable for large scale distributed implementations in neural architectures, by which we mean that it only involves simple linear and non-linear operations. Finally, we corroborate these theoretical results via experimental evaluation of the proposed algorithm with the current state-of-the-art techniques. Keywords: dictionary learning, provable dictionary learning, online dictionary learning, non-convex, sparse coding, support recovery, iterative hard thresholding, matrix factorization, neural architectures, neural networks, noodl, sparse representations, sparse signal processing. |
| Tasks | Dictionary Learning |
| Published | 2019-02-28 |
| URL | https://arxiv.org/abs/1902.11261v5 |
| https://arxiv.org/pdf/1902.11261v5.pdf | |
| PWC | https://paperswithcode.com/paper/noodl-provable-online-dictionary-learning-and |
| Repo | |
| Framework | |
Benchmarking Model-Based Reinforcement Learning
| Title | Benchmarking Model-Based Reinforcement Learning |
| Authors | Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba |
| Abstract | Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL. However, research in model-based RL has not been very standardized. It is fairly common for authors to experiment with self-designed environments, and there are several separate lines of research, which are sometimes closed-sourced or not reproducible. Accordingly, it is an open question how these various existing MBRL algorithms perform relative to each other. To facilitate research in MBRL, in this paper we gather a wide collection of MBRL algorithms and propose over 18 benchmarking environments specially designed for MBRL. We benchmark these algorithms with unified problem settings, including noisy environments. Beyond cataloguing performance, we explore and unify the underlying algorithmic differences across MBRL algorithms. We characterize three key research challenges for future MBRL research: the dynamics bottleneck, the planning horizon dilemma, and the early-termination dilemma. Finally, to maximally facilitate future research on MBRL, we open-source our benchmark in http://www.cs.toronto.edu/~tingwuwang/mbrl.html. |
| Tasks | |
| Published | 2019-07-03 |
| URL | https://arxiv.org/abs/1907.02057v1 |
| https://arxiv.org/pdf/1907.02057v1.pdf | |
| PWC | https://paperswithcode.com/paper/benchmarking-model-based-reinforcement |
| Repo | |
| Framework | |
Characterizing the Shape of Activation Space in Deep Neural Networks
| Title | Characterizing the Shape of Activation Space in Deep Neural Networks |
| Authors | Thomas Gebhart, Paul Schrater, Alan Hylton |
| Abstract | The representations learned by deep neural networks are difficult to interpret in part due to their large parameter space and the complexities introduced by their multi-layer structure. We introduce a method for computing persistent homology over the graphical activation structure of neural networks, which provides access to the task-relevant substructures activated throughout the network for a given input. This topological perspective provides unique insights into the distributed representations encoded by neural networks in terms of the shape of their activation structures. We demonstrate the value of this approach by showing an alternative explanation for the existence of adversarial examples. By studying the topology of network activations across multiple architectures and datasets, we find that adversarial perturbations do not add activations that target the semantic structure of the adversarial class as previously hypothesized. Rather, adversarial examples are explainable as alterations to the dominant activation structures induced by the original image, suggesting the class representations learned by deep networks are problematically sparse on the input space. |
| Tasks | |
| Published | 2019-01-28 |
| URL | https://arxiv.org/abs/1901.09496v2 |
| https://arxiv.org/pdf/1901.09496v2.pdf | |
| PWC | https://paperswithcode.com/paper/adversarial-examples-target-topological-holes |
| Repo | |
| Framework | |
Learning Goal-Oriented Visual Dialog Agents: Imitating and Surpassing Analytic Experts
| Title | Learning Goal-Oriented Visual Dialog Agents: Imitating and Surpassing Analytic Experts |
| Authors | Yen-Wei Chang, Wen-Hsiao Peng |
| Abstract | This paper tackles the problem of learning a questioner in the goal-oriented visual dialog task. Several previous works adopt model-free reinforcement learning. Most pretrain the model from a finite set of human-generated data. We argue that using limited demonstrations to kick-start the questioner is insufficient due to the large policy search space. Inspired by a recently proposed information theoretic approach, we develop two analytic experts to serve as a source of high-quality demonstrations for imitation learning. We then take advantage of reinforcement learning to refine the model towards the goal-oriented objective. Experimental results on the GuessWhat?! dataset show that our method has the combined merits of imitation and reinforcement learning, achieving the state-of-the-art performance. |
| Tasks | Imitation Learning, Visual Dialog |
| Published | 2019-07-24 |
| URL | https://arxiv.org/abs/1907.10500v1 |
| https://arxiv.org/pdf/1907.10500v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-goal-oriented-visual-dialog-agents |
| Repo | |
| Framework | |
Bias and variance reduction and denoising for CTF Estimation
| Title | Bias and variance reduction and denoising for CTF Estimation |
| Authors | Ayelet Heimowitz, Joakim andén, Amit Singer |
| Abstract | When using an electron microscope for imaging of particles embedded in vitreous ice, the objective lens will inevitably corrupt the projection images. This corruption manifests as a band-pass filter on the micrograph. In addition, it causes the phase of several frequency bands to be flipped and distorts frequency bands. As a precursor to compensating for this distortion, the corrupting point spread function, which is termed the contrast transfer function (CTF) in reciprocal space, must be estimated. In this paper, we will present a novel method for CTF estimation. Our method is based on the multi-taper method for power spectral density estimation, which aims to reduce the bias and variance of the estimator. Furthermore, we use known properties of the CTF and of the background of the power spectrum to increase the accuracy of our estimation. We will show that the resulting estimates capture the zero-crossings of the CTF in the low-mid frequency range. |
| Tasks | Denoising, Density Estimation |
| Published | 2019-08-09 |
| URL | https://arxiv.org/abs/1908.03454v3 |
| https://arxiv.org/pdf/1908.03454v3.pdf | |
| PWC | https://paperswithcode.com/paper/bias-and-variance-reduction-and-denoising-for |
| Repo | |
| Framework | |
Dynamic Scheduling of MPI-based Distributed Deep Learning Training Jobs
| Title | Dynamic Scheduling of MPI-based Distributed Deep Learning Training Jobs |
| Authors | Tim Capes, Vishal Raheja, Mete Kemertas, Iqbal Mohomed |
| Abstract | There is a general trend towards solving problems suited to deep learning with more complex deep learning architectures trained on larger training sets. This requires longer compute times and greater data parallelization or model parallelization. Both data and model parallelism have been historically faster in parameter server architectures, but data parallelism is starting to be faster in ring architectures due to algorithmic improvements. In this paper, we analyze the math behind ring architectures and make an informed adaptation of dynamic scheduling to ring architectures. To do so, we formulate a non-convex, non-linear, NP-hard integer programming problem and a new efficient doubling heuristic for its solution. We build upon Horovod: an open source ring architecture framework over TensorFlow. We show that Horovod jobs have a low cost to stop and restart and that stopping and restarting ring architecture jobs leads to faster completion times. These two facts make dynamic scheduling of ring architecture jobs feasible. Lastly, we simulate a scheduler using these runs and show a more than halving of average job time on some workload patterns. |
| Tasks | |
| Published | 2019-08-21 |
| URL | https://arxiv.org/abs/1908.08082v1 |
| https://arxiv.org/pdf/1908.08082v1.pdf | |
| PWC | https://paperswithcode.com/paper/dynamic-scheduling-of-mpi-based-distributed |
| Repo | |
| Framework | |
Mutual Context Network for Jointly Estimating Egocentric Gaze and Actions
| Title | Mutual Context Network for Jointly Estimating Egocentric Gaze and Actions |
| Authors | Yifei Huang, Zhenqiang Li, Minjie Cai, Yoichi Sato |
| Abstract | In this work, we address two coupled tasks of gaze prediction and action recognition in egocentric videos by exploring their mutual context. Our assumption is that in the procedure of performing a manipulation task, what a person is doing determines where the person is looking at, and the gaze point reveals gaze and non-gaze regions which contain important and complementary information about the undergoing action. We propose a novel mutual context network (MCN) that jointly learns action-dependent gaze prediction and gaze-guided action recognition in an end-to-end manner. Experiments on public egocentric video datasets demonstrate that our MCN achieves state-of-the-art performance of both gaze prediction and action recognition. |
| Tasks | Gaze Prediction, Temporal Action Localization |
| Published | 2019-01-07 |
| URL | http://arxiv.org/abs/1901.01874v3 |
| http://arxiv.org/pdf/1901.01874v3.pdf | |
| PWC | https://paperswithcode.com/paper/mutual-context-network-for-jointly-estimating |
| Repo | |
| Framework | |
