There is probably a way to turn off this "feature" in the various Office programs.
I haven't installed Office for a while, but I seem to remember some stupid "Office Startup" program starting with my comp afterwards. I think it's suppose to help with searches or something....
Maybe that stupid office startup thing is putting those items in your context menu, stop it from starting with your comp (there's usually a link in Start > All Programs > Startup, delete that). If there isn't a link in your start menu, it might be lurking somewhere in you startup items, you can edit your startup items by clicking Start > Run (or hitting Winkey + R) and typing "msconfig" <enter> and unchecking stuff in the Startup tab (look for items with the word "office" or "mso" etc in them).