Bonus points if your static site sends a 503 with a retry after header.

Usually that’s just for their version. Arxiv the version before it was accepted.

Exactly.

The general approach is to use interpretable models where you can understand how the model works and what features it uses to discriminate, but that doesn’t work for all ML approaches (and even when it does our understanding is incomplete.)

Maybe not the hardest, but still challenging. Unknown biases in training data are a challenge in any experimental design. Opaque ML frequently makes them more challenging to discover.