ViT-FPGA-TPU
所属分类：内容生成
开发工具：SystemVerilog
文件大小：0KB
下载次数：0
上传日期：2023-06-08 18:40:01
上 传 者：
sh-1993
说明： 基于FPGA的视觉transformer加速器（哈佛CS205），
(FPGA based Vision Transformer accelerator (Harvard CS205),)
文件列表:
code/ (0, 2023-12-11)
code/.vscode/ (0, 2023-12-11)
code/.vscode/settings.json (1946, 2023-12-11)
code/code_cpu/ (0, 2023-12-11)
code/code_cpu/.vscode/ (0, 2023-12-11)
code/code_cpu/.vscode/settings.json (1946, 2023-12-11)
code/code_cpu/CMakeLists.txt (986, 2023-12-11)
code/code_cpu/Makefile (7346, 2023-12-11)
code/code_cpu/cmake_install.cmake (1650, 2023-12-11)
code/code_cpu/download_mnist.py (2635, 2023-12-11)
code/code_cpu/job.sh (3689, 2023-12-11)
code/code_cpu/main.cpp (4736, 2023-12-11)
code/code_cpu/msa.cpp (4109, 2023-12-11)
code/code_cpu/msa.h (411, 2023-12-11)
code/code_cpu/output.txt (76811, 2023-12-11)
code/code_cpu/parallel_mm.cpp (1102, 2023-12-11)
code/code_cpu/vit.cpp (8502, 2023-12-11)
code/code_cpu/vit.h (660, 2023-12-11)
code/code_cpu/vit_block.cpp (1137, 2023-12-11)
code/code_cpu/vit_block.h (553, 2023-12-11)
code/code_fpga/ (0, 2023-12-11)
code/code_fpga/.vscode/ (0, 2023-12-11)
code/code_fpga/.vscode/settings.json (2024, 2023-12-11)
code/code_fpga/CMakeLists.txt (1482, 2023-12-11)
code/code_fpga/Makefile (7366, 2023-12-11)
code/code_fpga/accel_driver/ (0, 2023-12-11)
code/code_fpga/accel_driver/CMakeLists.txt (625, 2023-12-11)
code/code_fpga/accel_driver/include/ (0, 2023-12-11)
code/code_fpga/accel_driver/include/accelerator.h (1611, 2023-12-11)
code/code_fpga/accel_driver/include/dma_utils.h (3848, 2023-12-11)
code/code_fpga/accel_driver/include/matrix_utils.h (2533, 2023-12-11)
code/code_fpga/accel_driver/src/ (0, 2023-12-11)
code/code_fpga/accel_driver/src/main.cpp (19285, 2023-12-11)
code/code_fpga/cmake_install.cmake (1658, 2023-12-11)
code/code_fpga/download_mnist.py (2635, 2023-12-11)
code/code_fpga/fpga.h (20053, 2023-12-11)
code/code_fpga/fpga/ (0, 2023-12-11)
... ...
# ViT-FPGA-TPU
FPGA based Vision Transformer accelerator (Harvard CS205)
# To Run ViT in libTorch (PyTorch C++ API)
#### Go into the `code/code_cpu` directory
- `cd code/code_cpu`
- Everything below is done in the `code/code_cpu` directory!!!
#### First we download the LibTorch library if it's not in the folder yet:
- `wget https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip`
- `unzip libtorch-shared-with-deps-latest.zip`
#### Then build the project
- `mkdir build`
- `cd build`
- `cmake -DCMAKE_PREFIX_PATH=/absolute/path/to/libtorch_folder/that/you/just/downloaded ..`
- `cmake --build . --config Release`
- `cd ..` (we exit the `build` directory)
#### Then download the MNIST dataset if it's not in the folder yet:
- `python download_mnist.py`
- `mkdir mnist`
- Put all the downloaded `.ubyte` files into `./mnist` folder
#### Run the model :)
- `cd build`
- run `./main` with either `test` (test model with loaded weights) or `train`
- You can move job.sh into build directory and run `./job.sh` to get time benchmarks in output.txt
- Pretrained weights can be downloaded here: [Drive](https://github.com/gnodipac886/ViT-FPGA-TPU/blob/master/https://drive.google.com/drive/folders/1A7SSotTqGF_dax9yL8jX7Qt5W7MeOm-j?usp=share_link) (90%+ acc) or use the one in the folder (~86% accuracy)
## FPGA Testbench
Please checkout `code/fpga/test_accel/test_accel.srcs/sim_1/imports/src/` for the testbenches. Please also import the files in `code/fpga/test_accel/test_accel.srcs/sources_1/imports/src/` as the rtl sources. The testbench demonstrates the controller working in parallel with the DMA engine "spad_arbiter". Please feel free to email Eric for further questions.
## Contribution:
|Name |Contribution|
|-----------|--------------------------------------------------------------------------|
| Eric | FPGA Matrix Multiplication Design + Code, SW/HW Co-Design |
| Hongyi | ViT on Pytorch C++, SW/HW Co-Design and Integration |
| Wenyun | ViT on Pytorch C++, Test and Benchmark (and corresponding shell scripts) |
| Sebastian | ViT and FPGA literature research, FPGA demo + testing |
近期下载者：
相关文件：
收藏者：