Paper Group ANR 74
Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning. Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks. HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing. Using Automated Theorem Provers for Mistake Diagnosis in the Didact …
Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning
| Title | Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learning |
| Authors | Alberto Maria Metelli, Flavio Mazzolini, Lorenzo Bisi, Luca Sabbioni, Marcello Restelli |
| Abstract | The choice of the control frequency of a system has a relevant impact on the ability of reinforcement learning algorithms to learn a highly performing policy. In this paper, we introduce the notion of action persistence that consists in the repetition of an action for a fixed number of decision steps, having the effect of modifying the control frequency. We start analyzing how action persistence affects the performance of the optimal policy, and then we present a novel algorithm, Persistent Fitted Q-Iteration (PFQI), that extends FQI, with the goal of learning the optimal value function at a given persistence. After having provided a theoretical study of PFQI and a heuristic approach to identify the optimal persistence, we present an experimental campaign on benchmark domains to show the advantages of action persistence and proving the effectiveness of our persistence selection method. |
| Tasks | |
| Published | 2020-02-17 |
| URL | https://arxiv.org/abs/2002.06836v1 |
| https://arxiv.org/pdf/2002.06836v1.pdf | |
| PWC | https://paperswithcode.com/paper/control-frequency-adaptation-via-action |
| Repo | |
| Framework | |
Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks
| Title | Deep Adaptive Semantic Logic (DASL): Compiling Declarative Knowledge into Deep Neural Networks |
| Authors | Karan Sikka, Andrew Silberfarb, John Byrnes, Indranil Sur, Ed Chow, Ajay Divakaran, Richard Rohwer |
| Abstract | We introduce Deep Adaptive Semantic Logic (DASL), a novel framework for automating the generation of deep neural networks that incorporates user-provided formal knowledge to improve learning from data. We provide formal semantics that demonstrate that our knowledge representation captures all of first order logic and that finite sampling from infinite domains converges to correct truth values. DASL’s representation improves on prior neural-symbolic work by avoiding vanishing gradients, allowing deeper logical structure, and enabling richer interactions between the knowledge and learning components. We illustrate DASL through a toy problem in which we add structure to an image classification problem and demonstrate that knowledge of that structure reduces data requirements by a factor of $1000$. We then evaluate DASL on a visual relationship detection task and demonstrate that the addition of commonsense knowledge improves performance by $10.7%$ in a data scarce setting. |
| Tasks | Image Classification |
| Published | 2020-03-16 |
| URL | https://arxiv.org/abs/2003.07344v1 |
| https://arxiv.org/pdf/2003.07344v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-adaptive-semantic-logic-dasl-compiling |
| Repo | |
| Framework | |
HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing
| Title | HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in Mobile-Edge-Cloud Computing |
| Authors | Deyin Liu, Xu Chen, Zhi Zhou, Qing Ling |
| Abstract | Nowadays, deep neural networks (DNNs) are the core enablers for many emerging edge AI applications. Conventional approaches to training DNNs are generally implemented at central servers or cloud centers for centralized learning, which is typically time-consuming and resource-demanding due to the transmission of a large amount of data samples from the device to the remote cloud. To overcome these disadvantages, we consider accelerating the learning process of DNNs on the Mobile-Edge-Cloud Computing (MECC) paradigm. In this paper, we propose HierTrain, a hierarchical edge AI learning framework, which efficiently deploys the DNN training task over the hierarchical MECC architecture. We develop a novel \textit{hybrid parallelism} method, which is the key to HierTrain, to adaptively assign the DNN model layers and the data samples across the three levels of edge device, edge server and cloud center. We then formulate the problem of scheduling the DNN training tasks at both layer-granularity and sample-granularity. Solving this optimization problem enables us to achieve the minimum training time. We further implement a hardware prototype consisting of an edge device, an edge server and a cloud server, and conduct extensive experiments on it. Experimental results demonstrate that HierTrain can achieve up to 6.9x speedup compared to the cloud-based hierarchical training approach. |
| Tasks | |
| Published | 2020-03-22 |
| URL | https://arxiv.org/abs/2003.09876v1 |
| https://arxiv.org/pdf/2003.09876v1.pdf | |
| PWC | https://paperswithcode.com/paper/hiertrain-fast-hierarchical-edge-ai-learning |
| Repo | |
| Framework | |
Using Automated Theorem Provers for Mistake Diagnosis in the Didactics of Mathematics
| Title | Using Automated Theorem Provers for Mistake Diagnosis in the Didactics of Mathematics |
| Authors | Merlin Carl |
| Abstract | The Diproche system, an automated proof checker for natural language proofs specifically adapted to the context of exercises for beginner’s students similar to the Naproche system by Koepke, Schr"oder, Cramer and others, uses a modification of an automated theorem prover which uses common formal fallacies intead of sound deduction rules for mistake diagnosis. We briefly describe the concept of such an `Anti-ATP’ and explain the basic techniques used in its implementation. | |
| Tasks | |
| Published | 2020-02-12 |
| URL | https://arxiv.org/abs/2002.05083v1 |
| https://arxiv.org/pdf/2002.05083v1.pdf | |
| PWC | https://paperswithcode.com/paper/using-automated-theorem-provers-for-mistake |
| Repo | |
| Framework | |
Depressed individuals express more distorted thinking on social media
| Title | Depressed individuals express more distorted thinking on social media |
| Authors | Krishna C. Bathina, Marijn ten Thij, Lorenzo Lorenzo-Luaces, Lauren A. Rutter, Johan Bollen |
| Abstract | Depression is a leading cause of disability worldwide, but is often under-diagnosed and under-treated. One of the tenets of cognitive-behavioral therapy (CBT) is that individuals who are depressed exhibit distorted modes of thinking, so-called cognitive distortions, which can negatively affect their emotions and motivation. Here, we show that individuals with a self-reported diagnosis of depression on social media express higher levels of distorted thinking than a random sample. Some types of distorted thinking were found to be more than twice as prevalent in our depressed cohort, in particular Personalizing and Emotional Reasoning. This effect is specific to the distorted content of the expression and can not be explained by the presence of specific topics, sentiment, or first-person pronouns. Our results point towards the detection, and possibly mitigation, of patterns of online language that are generally deemed depressogenic. They may also provide insight into recent observations that social media usage can have a negative impact on mental health. |
| Tasks | |
| Published | 2020-02-07 |
| URL | https://arxiv.org/abs/2002.02800v1 |
| https://arxiv.org/pdf/2002.02800v1.pdf | |
| PWC | https://paperswithcode.com/paper/depressed-individuals-express-more-distorted |
| Repo | |
| Framework | |
High Accuracy Face Geometry Capture using a Smartphone Video
| Title | High Accuracy Face Geometry Capture using a Smartphone Video |
| Authors | Shubham Agrawal, Anuj Pahuja, Simon Lucey |
| Abstract | What’s the most accurate 3D model of your face you can obtain while sitting at your desk? We attempt to answer this question in our work. High fidelity face reconstructions have so far been limited to either studio settings or through expensive 3D scanners. On the other hand, unconstrained reconstruction methods are typically limited by low-capacity models. Our method reconstructs accurate face geometry of a subject using a video shot from a smartphone in an unconstrained environment. Our approach takes advantage of recent advances in visual SLAM, keypoint detection, and object detection to improve accuracy and robustness. By not being constrained to a model subspace, our reconstructed meshes capture important details while being robust to noise and being topologically consistent. Our evaluations show that our method outperforms current single and multi-view baselines by a significant margin, both in terms of geometric accuracy and in capturing person-specific details important for making realistic looking models. |
| Tasks | Keypoint Detection, Object Detection |
| Published | 2020-03-19 |
| URL | https://arxiv.org/abs/2003.08583v1 |
| https://arxiv.org/pdf/2003.08583v1.pdf | |
| PWC | https://paperswithcode.com/paper/high-accuracy-face-geometry-capture-using-a |
| Repo | |
| Framework | |
Learning to Simulate Complex Physics with Graph Networks
| Title | Learning to Simulate Complex Physics with Graph Networks |
| Authors | Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, Peter W. Battaglia |
| Abstract | Here we present a general framework for learning simulation, and provide a single model implementation that yields state-of-the-art performance across a variety of challenging physical domains, involving fluids, rigid solids, and deformable materials interacting with one another. Our framework—which we term “Graph Network-based Simulators” (GNS)—represents the state of a physical system with particles, expressed as nodes in a graph, and computes dynamics via learned message-passing. Our results show that our model can generalize from single-timestep predictions with thousands of particles during training, to different initial conditions, thousands of timesteps, and at least an order of magnitude more particles at test time. Our model was robust to hyperparameter choices across various evaluation metrics: the main determinants of long-term performance were the number of message-passing steps, and mitigating the accumulation of error by corrupting the training data with noise. Our GNS framework is the most accurate general-purpose learned physics simulator to date, and holds promise for solving a wide range of complex forward and inverse problems. |
| Tasks | |
| Published | 2020-02-21 |
| URL | https://arxiv.org/abs/2002.09405v1 |
| https://arxiv.org/pdf/2002.09405v1.pdf | |
| PWC | https://paperswithcode.com/paper/learning-to-simulate-complex-physics-with |
| Repo | |
| Framework | |
BABO: Background Activation Black-Out for Efficient Object Detection
| Title | BABO: Background Activation Black-Out for Efficient Object Detection |
| Authors | Byungseok Roh, Han-Cheol Cho, Myung-Ho Ju, Soon Hyung Pyo |
| Abstract | Recent advances in deep learning have enabled complex real-world use cases comprised of multiple vision tasks and detection tasks are being shifted to the edge side as a pre-processing step of the entire workload. Since running a deep model on resource-constraint devices is challenging, techniques for efficient inference methods are demanded. In this paper, we present an objectness-aware object detection method to reduce computational cost by sparsifying activation values on background regions where target objects don’t exist. Sparsified activation can be exploited to increase inference speed by software or hardware accelerated sparse convolution techniques. To accomplish this goal, we incorporate a light-weight objectness mask generation (OMG) network in front of an object detection (OD) network so that it can zero out unnecessary background areas of an input image before being fed into the OD network. In experiments, by switching background activation values to zero, the average number of zero values increases further from 36% to 68% on MobileNetV2-SSDLite even with ReLU activation while maintaining accuracy on MS-COCO. This result indicates that the total MAC including both OMG and OD networks can be reduced to 62% of the original OD model when only non-zero multiply-accumulate operations are considered. Moreover, we show a similar tendency in heavy networks (VGG and RetinaNet) and an additional dataset (PASCAL VOC). |
| Tasks | Object Detection |
| Published | 2020-02-05 |
| URL | https://arxiv.org/abs/2002.01609v2 |
| https://arxiv.org/pdf/2002.01609v2.pdf | |
| PWC | https://paperswithcode.com/paper/accelerating-object-detection-by-erasing |
| Repo | |
| Framework | |
Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering
| Title | Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering |
| Authors | Hui Tang, Ke Chen, Kui Jia |
| Abstract | Unsupervised domain adaptation (UDA) is to make predictions for unlabeled data on a target domain, given labeled data on a source domain whose distribution shifts from the target one. Mainstream UDA methods learn aligned features between the two domains, such that a classifier trained on the source features can be readily applied to the target ones. However, such a transferring strategy has a potential risk of damaging the intrinsic discrimination of target data. To alleviate this risk, we are motivated by the assumption of structural domain similarity, and propose to directly uncover the intrinsic target discrimination via discriminative clustering of target data. We constrain the clustering solutions using structural source regularization that hinges on our assumed structural domain similarity. Technically, we use a flexible framework of deep network based discriminative clustering that minimizes the KL divergence between predictive label distribution of the network and an introduced auxiliary one; replacing the auxiliary distribution with that formed by ground-truth labels of source data implements the structural source regularization via a simple strategy of joint network training. We term our proposed method as Structurally Regularized Deep Clustering (SRDC), where we also enhance target discrimination with clustering of intermediate network features, and enhance structural regularization with soft selection of less divergent source examples. Careful ablation studies show the efficacy of our proposed SRDC. Notably, with no explicit domain alignment, SRDC outperforms all existing methods on three UDA benchmarks. |
| Tasks | Domain Adaptation, Unsupervised Domain Adaptation |
| Published | 2020-03-19 |
| URL | https://arxiv.org/abs/2003.08607v1 |
| https://arxiv.org/pdf/2003.08607v1.pdf | |
| PWC | https://paperswithcode.com/paper/unsupervised-domain-adaptation-via-4 |
| Repo | |
| Framework | |
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches
| Title | Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches |
| Authors | Shuai Yang, Zhangyang Wang, Jiaying Liu, Zongming Guo |
| Abstract | Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Since sketches are difficult to collect, previous methods mainly use edge maps instead of sketches to train models (referred to as edge-based models). However, sketches display great structural discrepancy with edge maps, thus failing edge-based models. Moreover, sketches often demonstrate huge variety among different users, demanding even higher generalizability and robustness for the editing model to work. In this paper, we propose Deep Plastic Surgery, a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs. We present a sketch refinement strategy, as inspired by the coarse-to-fine drawing process of the artists, which we show can help our model well adapt to casual and varied sketches without the need for real sketch training data. Our model further provides a refinement level control parameter that enables users to flexibly define how “reliable” the input sketch should be considered for the final output, balancing between sketch faithfulness and output verisimilitude (as the two goals might contradict if the input sketch is drawn poorly). To achieve the multi-level refinement, we introduce a style-based module for level conditioning, which allows adaptive feature representations for different levels in a singe network. Extensive experimental results demonstrate the superiority of our approach in improving the visual quality and user controllablity of image editing over the state-of-the-art methods. |
| Tasks | |
| Published | 2020-01-09 |
| URL | https://arxiv.org/abs/2001.02890v1 |
| https://arxiv.org/pdf/2001.02890v1.pdf | |
| PWC | https://paperswithcode.com/paper/deep-plastic-surgery-robust-and-controllable |
| Repo | |
| Framework | |
A Deep Conditioning Treatment of Neural Networks
| Title | A Deep Conditioning Treatment of Neural Networks |
| Authors | Naman Agarwal, Pranjal Awasthi, Satyen Kale |
| Abstract | We study the role of depth in training randomly initialized overparameterized neural networks. We give the first general result showing that depth improves trainability of neural networks by improving the {\em conditioning} of certain kernel matrices of the input data. This result holds for arbitrary non-linear activation functions, and we provide a characterization of the improvement in conditioning as a function of the degree of non-linearity and the depth of the network. We provide versions of the result that hold for training just the top layer of the neural network, as well as for training all layers, via the neural tangent kernel. As applications of these general results, we provide a generalization of the results of Das et al. (2019) showing that learnability of deep random neural networks with arbitrary non-linear activations (under mild assumptions) degrades exponentially with depth. Additionally, we show how benign overfitting can occur in deep neural networks via the results of Bartlett et al. (2019b). |
| Tasks | |
| Published | 2020-02-04 |
| URL | https://arxiv.org/abs/2002.01523v1 |
| https://arxiv.org/pdf/2002.01523v1.pdf | |
| PWC | https://paperswithcode.com/paper/a-deep-conditioning-treatment-of-neural |
| Repo | |
| Framework | |
Efficient Nonnegative Tensor Factorization via Saturating Coordinate Descent
| Title | Efficient Nonnegative Tensor Factorization via Saturating Coordinate Descent |
| Authors | Thirunavukarasu Balasubramaniam, Richi Nayak, Chau Yuen |
| Abstract | With the advancements in computing technology and web-based applications, data is increasingly generated in multi-dimensional form. This data is usually sparse due to the presence of a large number of users and fewer user interactions. To deal with this, the Nonnegative Tensor Factorization (NTF) based methods have been widely used. However existing factorization algorithms are not suitable to process in all three conditions of size, density, and rank of the tensor. Consequently, their applicability becomes limited. In this paper, we propose a novel fast and efficient NTF algorithm using the element selection approach. We calculate the element importance using Lipschitz continuity and propose a saturation point based element selection method that chooses a set of elements column-wise for updating to solve the optimization problem. Empirical analysis reveals that the proposed algorithm is scalable in terms of tensor size, density, and rank in comparison to the relevant state-of-the-art algorithms. |
| Tasks | |
| Published | 2020-03-07 |
| URL | https://arxiv.org/abs/2003.03572v1 |
| https://arxiv.org/pdf/2003.03572v1.pdf | |
| PWC | https://paperswithcode.com/paper/efficient-nonnegative-tensor-factorization |
| Repo | |
| Framework | |
Malicious Experts versus the multiplicative weights algorithm in online prediction
| Title | Malicious Experts versus the multiplicative weights algorithm in online prediction |
| Authors | Erhan Bayraktar, H. Vincent Poor, Xin Zhang |
| Abstract | We consider a prediction problem with two experts and a forecaster. We assume that one of the experts is honest and makes correct prediction with probability $\mu$ at each round. The other one is malicious, who knows true outcomes at each round and makes predictions in order to maximize the loss of the forecaster. Assuming the forecaster adopts the classical multiplicative weights algorithm, we find upper and lower bounds for the value function of the malicious expert. Our results imply that the multiplicative weights algorithm cannot resist the corruption of malicious experts. We also show that an adaptive multiplicative weights algorithm is asymptotically optimal for the forecaster, and hence more resistant to the corruption of malicious experts. |
| Tasks | |
| Published | 2020-03-18 |
| URL | https://arxiv.org/abs/2003.08457v1 |
| https://arxiv.org/pdf/2003.08457v1.pdf | |
| PWC | https://paperswithcode.com/paper/malicious-experts-versus-the-multiplicative |
| Repo | |
| Framework | |
Poisson Kernel Avoiding Self-Smoothing in Graph Convolutional Networks
| Title | Poisson Kernel Avoiding Self-Smoothing in Graph Convolutional Networks |
| Authors | Ziqing Yang, Shoudong Han, Jun Zhao |
| Abstract | Graph convolutional network (GCN) is now an effective tool to deal with non-Euclidean data, such as social networks in social behavior analysis, molecular structure analysis in the field of chemistry, and skeleton-based action recognition. Graph convolutional kernel is one of the most significant factors in GCN to extract nodes’ feature, and some improvements of it have reached promising performance theoretically and experimentally. However, there is limited research about how exactly different data types and graph structures influence the performance of these kernels. Most existing methods used an adaptive convolutional kernel to deal with a given graph structure, which still not reveals the internal reasons. In this paper, we started from theoretical analysis of the spectral graph and studied the properties of existing graph convolutional kernels. While taking some designed datasets with specific parameters into consideration, we revealed the self-smoothing phenomenon of convolutional kernels. After that, we proposed the Poisson kernel that can avoid self-smoothing without training any adaptive kernel. Experimental results demonstrate that our Poisson kernel not only works well on the benchmark dataset where state-of-the-art methods work fine, but also is evidently superior to them in synthetic datasets. |
| Tasks | Skeleton Based Action Recognition |
| Published | 2020-02-07 |
| URL | https://arxiv.org/abs/2002.02589v2 |
| https://arxiv.org/pdf/2002.02589v2.pdf | |
| PWC | https://paperswithcode.com/paper/poisson-kernel-avoiding-self-smoothing-in |
| Repo | |
| Framework | |
Joint COCO and Mapillary Workshop at ICCV 2019 Keypoint Detection Challenge Track Technical Report: Distribution-Aware Coordinate Representation for Human Pose Estimation
| Title | Joint COCO and Mapillary Workshop at ICCV 2019 Keypoint Detection Challenge Track Technical Report: Distribution-Aware Coordinate Representation for Human Pose Estimation |
| Authors | Hanbin Dai, Liangbo Zhou, Feng Zhang, Zhengyu Zhang, Hong Hu, Xiatian Zhu, Mao Ye |
| Abstract | In this paper, we focus on the coordinate representation in human pose estimation. While being the standard choice, heatmap based representation has not been systematically investigated. We found that the process of coordinate decoding (i.e. transforming the predicted heatmaps to the coordinates) is surprisingly significant for human pose estimation performance, which nevertheless was not recognised before. In light of the discovered importance, we further probe the design limitations of the standard coordinate decoding method and propose a principled distribution-aware decoding method. Meanwhile, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating accurate heatmap distributions for unbiased model training. Taking them together, we formulate a novel Distribution-Aware coordinate Representation for Keypoint (DARK) method. Serving as a model-agnostic plug-in, DARK significantly improves the performance of a variety of state-of-the-art human pose estimation models. Extensive experiments show that DARK yields the best results on COCO keypoint detection challenge, validating the usefulness and effectiveness of our novel coordinate representation idea. The project page containing more details is at https://ilovepose.github.io/coco |
| Tasks | Keypoint Detection, Pose Estimation |
| Published | 2020-03-13 |
| URL | https://arxiv.org/abs/2003.07232v1 |
| https://arxiv.org/pdf/2003.07232v1.pdf | |
| PWC | https://paperswithcode.com/paper/joint-coco-and-mapillary-workshop-at-iccv |
| Repo | |
| Framework | |
