This is an automated archive made by the Lemmit Bot.

The original was posted on /r/selfhosted by /u/yoracale on 2025-02-21 21:28:41+00:00.

Hey amazing people! Thanks so much for the support on our GRPO release 2 weeks ago! Today, we’re excited to announce that you can now train your own reasoning model with just 5GB VRAM for Qwen2.5 (1.5B) - down from 7GB in the previous Unsloth release! GRPO is the algorithm behind DeepSeek-R1 and how it was trained.

The best part about GRPO is it doesn’t matter if you train a small model as you just need more training time. You can also leave it running in the background of your PC while you do other things.

1. Due to our newly added Efficient GRPO algorithm, this enables 10x longer context lengths while using 90% less VRAM vs. every other GRPO LoRA/QLoRA implementations.
2. With a GRPO setup using TRL + FA2, Llama 3.1 (8B) training at 20K context length demands 510.8GB of VRAM. However, Unsloth’s 90% VRAM reduction brings the requirement down to just 54.3GB in the same setup.
3. We leverage our gradient checkpointing algorithm which we released a while ago. It smartly offloads intermediate activations to system RAM asynchronously whilst being only 1% slower. This shaves a whopping 372GB VRAM since we need num_generations = 8. We can reduce this memory usage even further through intermediate gradient accumulation.
4. Try our free GRPO notebook with 10x longer context: Llama 3.1 (8B) on Colab

Blog for more details on the algorithm, the Maths behind GRPO, issues we found and more:

GRPO VRAM Breakdown:

• Also we spent a lot of time on our Guide for everything on GRPO + reward functions/verifiers so would highly recommend you guys to read it: docs.unsloth.ai/basics/reasoning

Thank you guys once again for all the support it truly means so much to us! 🦥