Intermittent 500s on Inference Endpoint - requests not reaching the container

A user reports experiencing intermittent HTTP 500 errors on a custom-handler Inference Endpoint for text embeddings using Llama-3.1-8B, with approximately 70% of requests failing since July 14, 2026.

The failures occur at the gateway or router layer before reaching the container, as failed requests do not appear in container logs while successful ones do. The issue persists across different GPU types (L40S and A100) and new endpoint instances, ruling out local code or hardware-specific causes. The user notes that the status page shows no incident and seeks to determine if there is a known issue with the us-east-1 inference gateway.