If you alter a frame in any way, it have to be reprocessed/recompressed; and not only it, since modern lossy movie codec relying on differences from frames to frames. The only thing that can be simply "direct streamed", without any reprocessing, is the audio track.
Bye!