Idea is simple, .obj files no longer have machine code but rather the intermediate "p-code" the compiler is generating anyway, and optimizing is deferred until the link stage.
-f0dder
Which to me sounds as if it's not too far away from just-in-time compilation. I think in the future we will have more abstract machines which optimize themselves all the time; total virtualization so to say. I will have to pick up the proper technical terms so I can better explain what I mean.
The trend seems to be to generate the machine code as late as possible in the uhm "runtime/lifetime cycle" of The Code.
-housetier
Perhaps - we're not there yet, though. Suns JVM has abysmal performance (MS JVM beat it by far), and while dotNET is pretty okay, it's still easy to beat with native code (code speed as well as memory use) - it looks like it's hard to beat dotNET for development time in a bunch of situations, though.
There's a lot of interesting techniques that can be applied with JIT optimization, some of which is hard to do with normal code optimization. For instance, taking note of branch prediction misses and rearranging the code to minimize this. However, the profiling overhead needed for doing this is not free - and it can be done with profile-guided "static" code generation as well (Iirc supported in vc6, vc2005, intel compilers).
Then there's more or less specialized pieces of code, where (on x86 anyway) a decent programmer can beat compilers by pretty large amounts. Compilers still don't have extremely good intrinsics support for new instruction sets (SSE/2/3, MMX) - besides those intrinsics makes code less portable, so it's safer and more portable to write an external assembly module. Even for code using the plain old instruction set, it's still possible to beat compilers - but of course this rarely makes sense (hence "specialized pieces of code").
As long as we stay on x86 (which will probably be pretty much forever, after AMD introduced x86-64 and since Apple moved to intel hardware, curse them both) I don't see JIT'ing replace normal code generation... unless there's enough SHEEP who blindly swallow VISTA.