Other Software > Developer's Corner

Multithreading tutorials

<< < (3/4) > >>

f0dder:
Then those multi-threaded encoders must be coded pretty inefficiently - forking separate processes has more overhead. Surprises me, really - also when taking file I/O into account, encoders are usually pretty disk-heavy as well as CPU heavy, a single multithreaded process would have a better chance of being smart about disk I/O and avoiding disk thrashing than running multiple instances.

Sounds like bad programming to me :)

MilesAhead:
I'd suggest trying a multitude of multi-threaded converters(is there a pun there or just a reasonable facsimile?) written by various programmers, then a gui supervisor app that launches multiple instances. Look at Core Temp to see which method maximizes the machine.  Although the multi-thread apps will use more than one core's worth of CPU, the average tends to be around 68% of available CPU power.  Whereas mult-processing apps like DVD-Rebuilder and FAVC the video encoding has the cores pegged at a nearly constant 100%.  Of course that's the part of the overall process that lends itself to multiprocessing.  The rest of the job you pretty much only run one muxer, one authoring app etc.. but even so it sure cuts down on the wait.

f0dder:
CPU usage isn't necessarily the best indicator of efficiency, I'd rather look at what get's the job done quickest - but it is true that some (lots of) algorithms are hard to parallelize. I'd still say it would be more efficient to do the processing from a single multi-threaded application, though; if for nothing else, just for the ability to do I/O more efficiently.

I'm not familiar with the various video compression algorithms, but since many of them have the concept of keyframes, perhaps there's no dependency on the previous data stream once there's a keyframe - if that's the case, it might be possible to thread at keyframe granularity. Dunno if it'd be worth the possible code complexity, but it would definitely be nice to do all the encoding and muxing in one step.

MilesAhead:
afa indication of efficiency I'd say the time to produce the video at the same or better quality is the measure.  The CPU usage indicator only measures CPU usage. The fact that the multiprocessing approaches seem to have in common that they maximize the use of the CPU power and get done quickest is surely coincidental then according to your theory?

f0dder:
afa indication of efficiency I'd say the time to produce the video at the same or better quality is the measure.
--- End quote ---
Yep :)

The CPU usage indicator only measures CPU usage.
--- End quote ---
Yep :)

The fact that the multiprocessing approaches seem to have in common that they maximize the use of the CPU power and get done quickest is surely coincidental then according to your theory?
--- End quote ---
Huh?

What I'm saying is that if a multiprocess approach gets the job done faster than a multithreaded approach, the threaded version is coded inefficiently, since MP means more overhead than MT.

Navigation

[0] Message Index

[#] Next page

[*] Previous page