Train and use my model

High level how to train a model

With the optimizers of bitsandbytes (like 8 bit AdamW), you would need 2 bytes per parameter, or 14 GB of GPU memory.

training a model like Llama with 2.7 billion parameters outperformed a larger model like Vicuna with 13 billion parameters. Especially when considering resource consumption, this might be a good alternative to using a 7B Foundation model instead of a full-blown ChatGPT. The best price-to-performance base model for our use case turned out to be Mistral 7b. The model is compact enough to fit into an affordable GPU with 24GB VRAM and outperforms the other models with 7B parameters.

Simpliest start with ai. Use the Github code linked in

Use the Bert model to train on Yelp dataset

BigCode is an open scientific collaboration working on the responsible development and use of large language models for code

Hi level only talk about training for a language

Describes how to train a new language (desperanto) model.