User extends Gemma4-31B to 44B via layer duplication

A Reddit user has expanded Google's Gemma4-31B model to approximately 47 billion parameters by increasing the layer count from 60 to 88. The process involved identity-init expansion following the LLaMA Pro approach, followed by fine-tuning on Korean legal and STEM data.

• Expanded Gemma4-31B from 60 to 80 layers using identity-init with a specific layer_scalar fix.
• Performed a second block duplication expansion from 80 to 88 layers on the already fine-tuned model.
• Fine-tuned the resulting ~47B parameter model on Korean legal and STEM datasets.
• Verified that duplicated full-attention layers actively contributed to training rather than remaining inactive.

The author shares the architecture details and model card on Hugging Face, noting early promise for legal and STEM use cases while seeking community help to improve coding and tool-calling capabilities.