A user demonstrates fine-tuning a 7B instruction model on Apple Silicon using MLX to shift its style to high-fantasy literature. The experiment shows that a small, curated dataset can significantly alter a model's register and diction with minimal computational resources.

  • Hardware: Apple M2 with 64 GB unified memory running macOS Sonoma and Python 3.12.4.
  • Model: Mistral-7B-Instruct-v0.3 quantized to 4-bit using QLoRA, training only 0.145% of weights.
  • Dataset: Approximately 1,200 examples extracted from Tolkien’s works and Gene Wolfe’s The Book of the New Sun.
  • Training: Less than 2 epochs with loss computed only on assistant completions via `--mask-prompt`.
  • Results: Perplexity decreased by 35%, achieving a measurable shift from generic responses to specific literary prose.

This process proves that local fine-tuning is a fast, offline project at near-zero marginal cost, validating research that small, high-quality datasets can powerfully change model outputs.