I'd say the time to switch was about 5 years ago when WinXP x64 came That's when I switched.
That release was in the same league as the Win32 subsystem on Windows 3.11: Total crap, with near to none support from applications or drivers.
It's sister-release Windows Server 2003 x64 edition was a little better (read: less worst) as there was actual hardware-driver support from the larger (server) vendors, but generic consumer-stuff was, as to be expected, a total disaster.
Starting with Vista x64/Server 2008 x64 this has dramatically changed, even though the first year after it's release there where a lot of issues with hardware-drivers, but ever since then it only has become a lot better. Windows 7/Server 2008 R2 is near default in x64 release on new hardware, even with the local discounters, IMHO meaning Windows x64 is ready for prime-time.
Don't know, and have never seen, the issues vlastimil
is talking about, but AFAIK that's a configurable item, but deeply hidden in the registry (don't have a link to MSDN available where I am now), and it's not stopping me from pushing everybody to x64 where possible. If Microsoft wants to go there it's most likely the (unavoidable) future