My vote goes to the PSU - it sounds like the electrolytics are failing.
Other than that, usually when a piece of equipment won't start at switch on but will after it's had current flowing for a little while it indicates a hardware fault, (bad soldering, components gone marginal, etc), because the equipment has had a little time to warm up.
As per SJ, change the PSU first if that doesn't work then take it back to PSU, MB, RAM and CPU, see if that works then add things one at a time until it fails.
The only problem with temperature related faults is they can take a while to isolate since about the only time you can guarantee it'll happen is first thing on a cold morning, (and even then it won't if you're watching for it
EDIT: Pipped at the POST by Bamse
In one case, the reason for the manufacture of faulty electrolytic capacitors was industrial espionage gone wrong: several Taiwanese electrolyte manufacturers began using a stolen formula that was incomplete, and lacked ingredients needed to produce a stable capacitor.
More likely than exploding electrolytics is that they have gone high ESRw
due to aging and/or electrical ripple/spikes. This will normally not be visible and you'd need to pull the equipment boards out and test each electrolytic capacitor with an ESR meter - not something to do if you're not electronically inclined.