notes of Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning.

Problem Statement

(big background)Model compression is an essential technique for depolying model on power and memory-constrained resources. (problem statement)Existing method often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs.

Main Idea

In this paper, author propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy.

  • graph neural network to identify DNN topologies
  • RL to optimize compression policy

overview

Main Procedure:

  • hierarchical graph representation
  • multi-stage graph embedding
  • learning-based pooling
  • RL-based policy(use m-GNN)

Pros and Cons

Pros:

  • hierarchical graph representation keep the DNN topologies information
  • multi-staged graph embedding keep the graph structure as much as possible
  • use RL approximate the best pruning policy

Cons:

  • ?(to be done)