CFAgentBench introduces a reproducible, self-hostable environment with 1,014 machine-gradeable tasks across eight domains, grounded in real-world sources. It features 40 oracle-validated tasks with executable evaluators that assess functional correctness via state diffs and output regexes, including a money-movement guard requiring human approval for payments. A key finding is that top agents lose 43% of successes when repeating tasks under temperature-0 decoding, indicating single-attempt performance does not reflect real-world deployability.