Model builder

Build a task-specific model on top of an open base: pick a base model and a governed dataset, set the LoRA adapter and training config, then export as adapters, a merged model or GGUF.

Unsloth 2.1× faster

Training speed

2.1×

vs. stock HF + PEFT

VRAM used

−68%

11.4 GB peak on one A100

Tokens / sec

4,820

packed sequences

Context length

4× longer

at the same VRAM

Base model

4-bit QLoRA, loaded through Unsloth.

Llama 3.1 8B Instruct

8B · ctx 128k · Llama 3.1

11 GB

Qwen2.5 7B Instruct

7B · ctx 32k · Apache 2.0

10 GB

Mistral 7B v0.3

7B · ctx 32k · Apache 2.0

10 GB

Phi-4 14B

14B · ctx 16k · MIT

18 GB

Gemma 2 9B

9B · ctx 8k · Gemma

13 GB

Dataset

Governed training data from the catalog.

kyc-policy-qa

18,420 rows · ShareGPT · 6.1M

90 / 5 / 5

onboarding-conversations

42,130 rows · ShareGPT · 14.8M

95 / 5 / 0

credit-memo-summaries

7,860 rows · Alpaca · 3.4M

90 / 10 / 0

nl2sql-warehouse

12,040 rows · Alpaca · 2.2M

90 / 5 / 5

Training curve

teal = train loss · violet = eval loss · onboarding-conversations on Llama 3.1 8B Instruct

complete

step 480 / 480loss 0.840eval 1.050

11.4 GB VRAM

Unsloth loader

Generated from the settings on this page.

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = "unsloth/Meta-Llama-3.1-8B-Instruct",
    max_seq_length = 4096,
    load_in_4bit   = True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r = 16, lora_alpha = 32, lora_dropout = 0,
    target_modules = ["q_proj","k_proj","v_proj","o_proj",
                      "gate_proj","up_proj","down_proj"],
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
)

Recent runs

Adapters produced by this workspace.

Run	Base model	Dataset	Loss	Time	Adapter	Status
ft-2314	Llama 3.1 8B	onboarding-conversations	0.84	38m	onb-lora-v4	Running
ft-2309	Qwen2.5 7B	kyc-policy-qa	0.71	52m	kyc-lora-v2	Complete
ft-2301	Mistral 7B	credit-memo-summaries	0.93	31m	memo-lora-v1	Complete
ft-2288	Llama 3.1 8B	nl2sql-warehouse	—	4m	—	Failed

LoRA adapter

Rank and alpha drive capacity vs. overfitting.

Rank r · 16

lora_alpha32

lora_dropout0

max_seq_length4096

load_in_4bittrue

checkpointingunsloth

q_projk_projv_projo_projgate_projup_projdown_proj

Training

Effective batch 8.

Epochs · 3

batch_size2

grad_accum4

learning_rate2e-4

schedulerlinear

optimizeradamw_8bit

seed3407

Resource use

One A100 80GB, packed sequences.

VRAM14%

GPU utilisation91%

Dataset consumed100%

Export

Ship the adapter wherever it runs.

LoRA adapters
safetensors · smallest

Merged 16-bit
drop-in replacement

GGUF q4_k_m
llama.cpp / Ollama

vLLM runtime
served from RaboAurora Core