I’d recommend to start with a mini-pc, or something like Odroid H3. I love my Odroids and can highly recommend them. It has two SATA, NVME, supports up to 64 GB RAM and dual 2.5 GBe - I couldn’t imagine a better machine to start with at that price.
Just to add my two cents: When I started out I thought I’d need a datacenter, with 10 Gig connectivity and a lot of storage. Turns out, a Raspberry Pi 4 8GB would’ve been sufficient for the things I actually use.
My recommendation would be therefore to start minimalistic and build up according to your needs from there. Start with a Raspberry PI and Docker or use a used Micro SFF and go up from there, this advice would’ve saved me a lot of money and electricity.
I love the Odroids. Perfect also for clustering setup, as they are pretty cheap in terms of buying price but also operating costs.