Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning
notes of Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning
.
Problem Statement
(big background)Model compression is an essential technique for depolying model on power and memory-constrained resources. (problem statement)Existing method often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs
.
Main Idea
In this paper, author propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy.
- graph neural network to identify
DNN topologies
- RL to optimize compression policy
Main Procedure:
- hierarchical graph representation
- multi-stage graph embedding
- learning-based pooling
- RL-based policy(use m-GNN)
Pros and Cons
Pros:
- hierarchical graph representation keep the DNN topologies information
- multi-staged graph embedding keep the graph structure as much as possible
- use RL approximate the best pruning policy
Cons:
- ?(to be done)
All articles in this blog are licensed under CC BY-NC-SA 4.0 unless stating additionally.