My Siri-initiated timers are always done with my phone, probably 50 or more each week (work stuff). The only time I get a failure is when I release the side button too quickly. I've made certain the spoken feedback is enabled to reduce the risk of me making that mistake. (Settings > Siri > Siri Responses > Prefer Spoken Responses)
As for, "What time is it?"... Try activating Siri and only saying, "Time."
I suspect that's the main difference; if you're trying to use hands-free voice activation via "hey Siri" you get a much different experience than if you can touch the watch/phone to trigger Siri first.
And thinking back over it, more than half the failures are complete - e.g., it likely never activated at all. Very few are "it set a timer, but for the wrong time".
Good chance that's what captures our different Siri experiences. The few times I've done it spoken was always with AirPods and I always waited for the Siri reply (been a while; is it, "Uh-huh"?) after I said, "Hey, Siri." But my experience activating Siri with speech is so minimal as to be untrustworthy of anything broader.
As for, "What time is it?"... Try activating Siri and only saying, "Time."