Fixed-Size Neural Networks Achieve Arbitrary Sobolev Approximation
A new activation function enables fixed-size neural networks to approximate any function in Sobolev spaces $W^{s,\infty}((a,b)^d)$ with arbitrary accuracy in the $W^{s-1,\infty}$-norm. The results use elementary activations like EUAF and DUAF$_\infty$, with explicit width and depth bounds, and extend to sigmoidal variants $\widetilde{\mathrm{DUAF}}_n$ preserving accuracy for all $1\leq s\leq n$.