TO me the problem of installing any config data with the prog files is if/when the files are updated, backup restored, that sort of thing, user data can be lost. Shouldn't be a big problem, but alas with software problems occurring all too often, stuff gets deleted in the process of fixing it & that can be a minor pain.
Installing in one of the user folders (i.e. XP's Doc & Settings) can solve that, but there's usually too many location variables... And try to find some of this stuff in Vista. Like the start menu, file-type associations, and booting with Windows, total developer anarchy.
Using the registry works, but it's inaccessible to most users, and certainly can't be used as portably with a flash drive. Plus with Vista, the few Vista (vs XP) examples I've seen manhandled the registry in the most nasty ways.
If I had to design something I think I'd try the user folders with an option to sync them to a flash drive or similar so that the experience could be seamless regardless the hardware I was using, as long as the software was installed on that machine. That sort of setup could make everyone happy I'd think.