Do you run traceroutes and pings in both directions?
In the case of a ping you might think it shouldn't matter but I can imagine a world where a VPN provider configures a server in London to route traffic via Somalia only when a user establishes a connection to the "Somalia" address of the server. You could only test this if you did a traceroute/ping through the VPN.
And I'm not saying this is what's happening but if you just ping the IP from your infra, couldn't stuff like anycast potentially mess you up?
In the case of traceroutes, you only see the route your traffic takes to the VPN, you don't see the route it takes to get back to you, which I think is really important.
We run traceroutes and latency measurements from many different locations, so we are looking at aggregate behavior rather than any single path. When you combine data from hundreds of ProbeNet PoPs over time, asymmetric routing mostly shows up as noise. When that happens, latency based hints lose weight and we lean more on other signals.
We have seen this in practice. For example, when we deployed servers in Gambia, even traffic between local networks often left the country and came back due to limited peering and little use of the national IXP. Stil, the overall routing patterns were still learnable once you look at enough paths.
For VPNs, we are measuring the location of the endpoint IP itself, not user traffic inside a tunnel. If routing only changes after a tunnel is established, that is a service level behavior, not the network location of the IP.
Anycast and tunneling are things we explicitly detect. They tend to create clear patterns like latency clustering or unstable paths, and when we see those and flag them as anycast IPs by defaulting to their geofeed location.
In the case of a ping you might think it shouldn't matter but I can imagine a world where a VPN provider configures a server in London to route traffic via Somalia only when a user establishes a connection to the "Somalia" address of the server. You could only test this if you did a traceroute/ping through the VPN.
And I'm not saying this is what's happening but if you just ping the IP from your infra, couldn't stuff like anycast potentially mess you up?
In the case of traceroutes, you only see the route your traffic takes to the VPN, you don't see the route it takes to get back to you, which I think is really important.