Search
Results
abi/secret-llama: Fully private LLM chatbot that runs entirely with a browser with no server needed. Supports Mistral and LLama 3.
LLaMA 7B GPU Memory Requirement - Transformers - Hugging Face Forums
[https://discuss.huggingface.co/t/llama-7b-gpu-memory-requirement/34323/6] - - public:mzimmerm
With the optimizers of bitsandbytes (like 8 bit AdamW), you would need 2 bytes per parameter, or 14 GB of GPU memory.