APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design

This repository contains the implementation of APSQ, as presented in our DAC 2025 paper: "APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design".

Introduction

DNN accelerators have made significant progress through model compression and specialized dataflow techniques. However, the frequent access of high-precision partial sums (PSUMs) often leads to excessive memory demands, especially in architectures utilizing input/weight stationary dataflows. Traditional compression strategies have typically overlooked PSUM quantization, which may account for 69% of power consumption.

APSQ introduces a novel Additive Partial Sum Quantization method that seamlessly integrates PSUM accumulation into the quantization framework. We propose a grouping strategy that combines APSQ with PSUM quantization, enhanced by a reconfigurable architecture. Our method achieves nearly lossless performance on NLP and CV tasks (BERT, Segformer, EfficientViT) while compressing PSUMs to INT8, leading to a notable reduction in energy costs by 28–87%. Extended experiments on LLaMA2-7B demonstrate the potential of APSQ for large language models.

Setup

1. Requirements

• Python 3.8+
• PyTorch 1.13.0
• Transformers 4.30.2

Install dependencies via:

pip install -r requirements.txt

2. Data Preparation

Please download the necessary datasets:

• GLUE dataset

For data augmentation on GLUE, please refer to the instructions in TinyBERT.

3. Pretrained Models

Download the pretrained models from the Hugging Face model zoo and place them in the models/ directory.

Dataset	Model Link
MNLI	bert-base-uncased-MNLI
QQP	bert-base-uncased-QQP
QNLI	bert-base-uncased-QNLI
SST-2	bert-base-uncased-SST-2
CoLA	bert-base-uncased-CoLA
STS-B	bert-base-uncased-STS-B
MRPC	bert-base-uncased-MRPC
RTE	bert-base-uncased-RTE

Usage

To reproduce the results, you can use the provided shell scripts in the scripts/ directory.

Configuration

You can configure the quantization and tiling parameters directly in the script (e.g., scripts/run_glue.sh).

Key parameters include:

• gs: Group size for the grouping strategy (corresponding to the group size in the paper).
• Tt, Tn, Tm: Tiling parameters for the hardware dataflow simulation.

Running the Code

To run the GLUE benchmark:

bash scripts/run_glue.sh

Ensure you have set the TASK_NAME and other paths correctly within the script before running.

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{APSQ2025,
  author = {Tan, Yonghao and Dong, Pingcheng and Wu, Yongkun and Liu, Yu and Liu, Xuejiao and Luo, Peng and Liu, Shih-Yang and Huang, Xijie and Zhang, Dong and Liang, Luhong and Cheng, Kwang-Ting},
  title = {APSQ: Additive Partial Sum Quantization with Algorithm-Hardware Co-Design},
  booktitle = {Proceedings of the 62nd Annual ACM/IEEE Design Automation Conference (DAC)},
  year = {2025}
}

Acknowledgement

This repository is built upon the BiT codebase. We thank the authors for open-sourcing their work.

License

APSQ is CC-BY-NC 4.0 licensed.