My idempotent claim was too broad and wrong, up to aliasing that is reversible.
I think I’ve been talking about this differently than you, when you get 8bit into, 255 in/out is correct.
When generating data I do it in the natural [0,1] space, then map to whatever domain I’m quantizing for.
In that case, for N bins you divide by N+1 (and round however you want via correction) so 256. I was probably wrong earlier around this, 256 is correct for quantizing that way.
My idempotent claim was too broad and wrong, up to aliasing that is reversible.
I think I’ve been talking about this differently than you, when you get 8bit into, 255 in/out is correct.
When generating data I do it in the natural [0,1] space, then map to whatever domain I’m quantizing for.
In that case, for N bins you divide by N+1 (and round however you want via correction) so 256. I was probably wrong earlier around this, 256 is correct for quantizing that way.