Reddit user recalls early struggles running BigScience's Bloom model

A Reddit user reflects on the significant progress in local large language models by recalling their experience with BigScience's Bloom. They remember purchasing 768GB Optane drives to run the full model with swap, which took approximately 20 minutes to generate a single token.

The post links to the BLOOMZ-P3 checkpoint on Hugging Face as a reference point for these early models.

The author expresses a desire to revisit these older models to appreciate the advancements made in local inference capabilities.