文件列表:

.editorconfig (822, 2023-05-30)
LICENSE (1060, 2023-05-30)
api.dockerignore (80, 2023-05-30)
converter (0, 2023-05-30)
converter\Dockerfile (1962, 2023-05-30)
converter\codegen_gptj_convert.py (3813, 2023-05-30)
converter\config_template.pbtxt (3705, 2023-05-30)
converter\download_and_convert_model.sh (383, 2023-05-30)
converter\huggingface_gptj_convert.py (8239, 2023-05-30)
converter\models (0, 2023-05-30)
converter\models\codegen-16B-mono-1gpu (0, 2023-05-30)
converter\models\codegen-16B-mono-1gpu\fastertransformer (0, 2023-05-30)
converter\models\codegen-16B-mono-1gpu\fastertransformer\1 (0, 2023-05-30)
converter\models\codegen-16B-mono-1gpu\fastertransformer\1\.tmp (0, 2023-05-30)
converter\models\codegen-16B-mono-1gpu\fastertransformer\config.pbtxt (3620, 2023-05-30)
converter\models\codegen-16B-mono-2gpu (0, 2023-05-30)
converter\models\codegen-16B-mono-2gpu\fastertransformer (0, 2023-05-30)
converter\models\codegen-16B-mono-2gpu\fastertransformer\1 (0, 2023-05-30)
converter\models\codegen-16B-mono-2gpu\fastertransformer\1\.tmp (0, 2023-05-30)
converter\models\codegen-16B-mono-2gpu\fastertransformer\config.pbtxt (3620, 2023-05-30)
converter\models\codegen-16B-mono-4gpu (0, 2023-05-30)
converter\models\codegen-16B-mono-4gpu\fastertransformer (0, 2023-05-30)
converter\models\codegen-16B-mono-4gpu\fastertransformer\config.pbtxt (3620, 2023-05-30)
converter\models\codegen-16B-multi-1gpu (0, 2023-05-30)
converter\models\codegen-16B-multi-1gpu\fastertransformer (0, 2023-05-30)
... ...

# FauxPilot This is an attempt to build a locally hosted alternative to [GitHub Copilot](https://copilot.github.com/). It uses the [SalesForce CodeGen](https://github.com/salesforce/CodeGen) models inside of NVIDIA's [Triton Inference Server](https://developer.nvidia.com/nvidia-triton-inference-server) with the [FasterTransformer backend](https://github.com/triton-inference-server/fastertransformer_backend/).

## Prerequisites You'll need: * Docker * `docker compose` >= 1.28 * An NVIDIA GPU with Compute Capability >= 6.0 and enough VRAM to run the model you want. * [`nvidia-docker`](https://github.com/NVIDIA/nvidia-docker) * `curl` and `zstd` for downloading and unpacking the models. Note that the VRAM requirements listed by `setup.sh` are *total* -- if you have multiple GPUs, you can split the model across them. So, if you have two NVIDIA RTX 3080 GPUs, you *should* be able to run the 6B model by putting half on each GPU. ## Support and Warranty lmao Okay, fine, we now have some minimal information on [the wiki](https://github.com/moyix/fauxpilot/wiki) and a [discussion forum](https://github.com/moyix/fauxpilot/discussions) where you can ask questions. Still no formal support or warranty though! ## Setup This section describes how to install a Fauxpilot server and clients. ### Setting up a FauxPilot Server Run the setup script to choose a model to use. This will download the model from [Huggingface/Moyix](https://huggingface.co/Moyix) in GPT-J format and then convert it for use with FasterTransformer. Please refer to [How to set-up a FauxPilot server](documentation/server.md). ### Client configuration for FauxPilot We offer some ways to connect to FauxPilot Server. For example, you can create a client by how to open the Openai API, Copilot Plugin, REST API. Please refer to [How to set-up a client](documentation/client.md). ## Terminology * API: Application Programming Interface * CC: Compute Capability * CUDA: Compute Unified Device Architecture * FT: Faster Transformer * JSON: JavaScript Object Notation * gRPC: Remote Procedure call by Google * GPT-J: A transformer model trained using Ben Wang's Mesh Transformer JAX * REST: REpresentational State Transfer

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：