Isn’t there quite a significant latency problem if you’re going across the internet for db instead of say, the same switch?

No experience with Mongo Atlas but other managed DB providers will IME be transparent about where they host and you can often request resources in an appropriate DC, sometimes even the same. Businesses providing this in Hetzner, OVH etc too. If you plan accordingly you can eat your cake and have it too.