I'm confused why you would even need AWS then (what's running on the VMs)?
My impression is the standard compute (as in CPUs+RAM) isn't expensive, it's the storage (1 PB is less than half a rack physically now, comparing with the yearly prices listed), and so if you don't have much data, the value of on-prem isn't there.
For smaller shops I'd argue storage is the hardest part. I've done several OpenStack and baremetal K8s deployments on prem and the part that always stressed me out the most was storage. I'd happily pay a markup for that vs just about anything else that would be more economical to do on prem for smaller simpler workloads.
My impression is the standard compute (as in CPUs+RAM) isn't expensive, it's the storage (1 PB is less than half a rack physically now, comparing with the yearly prices listed), and so if you don't have much data, the value of on-prem isn't there.