Have you tried "Make your own DVD" and just using the normal video file produced by PS3, without bothering to use TMPGEnc. Nero should automatically recode anything as required.
Just tried this and it works fine ... here is a sequence you can try ...
(1) Build you PS3 project and out put the slideshow as a .WMV file.
(2) In Nero go to "Make your own DVD-Video" and add the WMV file
(3) Do what is necessary to generate the DVD menus etc on the following screens.
(4) On the Burn screen select "Burn to Image Recorder" and choose ISO output when it asks for the file name to save.
(5) Use WinRAR to extract the folders from the ISO image on your hard disc or use Nero ImageDrive to mount the image and copy the folders.
(6) Add a folder with your original photos.
(7) Go into Nero Burning Rom and select the DVD Video Template. Copy the folders you have created (AUDIO_TS, VIDEO_TS and you original photos folder) to the root directory of the DVD-Video template
Burn the DVD to disc
You should now have a working DVD video with the slideshows on a menu screen, and also have folders on the DVD so that the original files can be accessed on a computer (like DVD extras on commercial discs).
Let me know how it goes.