Author Topic: Who wants to experience C# depths of hell and evilness? (directx code) (Read 27470 times)

mouser · « **on:** February 07, 2012, 11:01 PM »

So i spent a good portion of today trying to get this code to compile and work:
http://spazzarama.wo...-11-using-api-hooks/

I did eventually manage to get it to COMPILE AND RUN (but not work) on windows xp; trying to do so on win7 was an epic adventure in frustration and anger.

I swear I will never ever ever be able to comprehend how Microsoft managed to make .net building so insanely prone to such an evil convoluted hell of incompatible versions, insane dependency strangeness, UAC pain, and just a general all around insane mess of deployment and execution. They took some very nice language design and made it a cesspool of evilness when you actually want to run the stuff. I have said it a million times -- nothing that Microsoft does makes sense unless you assume that half the employees are saboteurs absolutely devoted like samurai to destroying the company.

Hell is other people's code.

ANYWAY.. I beg you not to embark on trying to get that code to actually build and work (that is capture directx screenshots) -- because i don't want to inflict the pain on you. And I am warning you it requires dlls and installing an additional "assembly" (slimdx).

BUT..

If you do manage to get it to work please let me know, i'd love to modify this to be a standalone commandline c# directx window grabber that I could invoke from my Screenshot Captor tool. (or even better, convert the code to c++).

Renegade · « **Reply #1 on:** February 08, 2012, 01:08 AM »

I've done a fair amount with P/Invoke, but it still is a kind of black science. I most certainly wouldn't want to venture into unknown territory...

If you can remember, make sure to set things to x86. You'll likely have a lot better luck that way. I know it sounds retarded, but...

Ath · « **Reply #2 on:** February 08, 2012, 01:30 AM »

(or even better, convert the code to c++).
-mouser (February 07, 2012, 11:01 PM)

But there are also a lot of examples in C++, why not start/try with those

?
Like this on CodeProject or this on StackOverflow

mouser · « **Reply #3 on:** February 08, 2012, 02:16 AM »

None of those methods do what needs to be done to capture direct x.
It's quite confusing -- but the DIRECTX methods of screen capture (which the codeproject tutorial does) is not the same as capturing from a running directx app.

To do the later you need to inject a dll into the target process and reroute directx rendering functions. And it has to be done differently for different directx versions..

The first link I posted has everything that *should* be needed to make it work.. If it could be made to work..

Don't get me wrong, I'd *MUCH* rather have C++ code than C# for this, but i haven't found any good code for c++. Converting this code to C++ might still be the best way to go..

Ath · « **Reply #4 on:** February 08, 2012, 05:33 AM »

Or sneakily integrate Gadwin Printscreen Free and call that from SSC

mouser · « **Reply #5 on:** February 09, 2012, 02:11 PM »

Update:

I spent all day on directX capture stuff and basically got my a** kicked by it. I thought that I had come up with a clever technique to grab directX windows, but it turns out that it just won't work for full screen directX apps, which is the only one thing that's important since SC will capture windowed directx fine. SO in short we are back to this hook injection code I linked to above. It should theoretically work to do the job, but is a bit nasty and requires registered a global assembly (like registering a system DLL); What would be really ideal is for someone to convert that C# code to C++..

Anyway, what I ended up doing is adding a kind of protocol to interface a 3rd party commandline screenshot grabbing tool, which SC will invoke and then grab the resulting screenshot and treat it like one of it's own (put it in right directory, name it properly, do post processing on it, show post-cap dialog if appropriate, handle hotkey stuff for it, etc.). SO.. basically what I have now is a placeholder for a fullscreen directx commandline capture tool, ready to drop one in as soon as someone codes it. [As an aside this functionality could also be used for some different 3rd party "capture" tool if someone wanted to write one to do something like capture from a remote device, etc.).

Renegade · « **Reply #6 on:** February 09, 2012, 07:00 PM »

Update:

I spent all day on directX capture stuff and basically got my a** kicked by it. I thought that I had come up with a clever technique to grab directX windows, but it turns out that it just won't work for full screen directX apps, which is the only one thing that's important since SC will capture windowed directx fine. SO in short we are back to this hook injection code I linked to above. It should theoretically work to do the job, but is a bit nasty and requires registered a global assembly (like registering a system DLL); What would be really ideal is for someone to convert that C# code to C++..

Anyway, what I ended up doing is adding a kind of protocol to interface a 3rd party commandline screenshot grabbing tool, which SC will invoke and then grab the resulting screenshot and treat it like one of it's own (put it in right directory, name it properly, do post processing on it, show post-cap dialog if appropriate, handle hotkey stuff for it, etc.). SO.. basically what I have now is a placeholder for a fullscreen directx commandline capture tool, ready to drop one in as soon as someone codes it. [As an aside this functionality could also be used for some different 3rd party "capture" tool if someone wanted to write one to do something like capture from a remote device, etc.).
-mouser (February 09, 2012, 02:11 PM)

Been there a few times myself. The process for me usually goes something like this:

1) See a fantastic way to do something.
2) Bang head for unspecified period that is way too long
3) Repeat step #2 several times until knocked unconscious
4) Wake up with massive headache
5) Look for alternative way that isn't nearly as cool, but gets the job done

You have my deepest sympathy as I can totally relate!

mouser · « **Reply #7 on:** February 09, 2012, 10:00 PM »

Update:

It looks like the c# code actually did work after a reboot after the gac global assembly cache registration.

SO although it's an ugly process to register the assemblies and reboot, I am going to see if I can't hook this into Screenshot Captor as a first test of DirectX capturing.

Stay tuned.

mouser · « **Reply #8 on:** February 10, 2012, 02:56 AM »

Full screen directx capture from SC is now working.. barely. Stay tuned.

cranioscopical · « **Reply #9 on:** February 10, 2012, 10:27 AM »

now working.. barely.
-mouser (February 10, 2012, 02:56 AM)

And here is your award

The bare minibum.

Oh, wait...

mouser · « **Reply #10 on:** February 10, 2012, 02:29 PM »

I have to say this is really painful to do without a windows7 machine at my desk.. Vmware is not good for directx testing which means i have to keep walking usb sticks back and forth to the other room where i have a windows 7 directx11 machine.

This doesn't justify buying even a cheap laptop, but if someone happens to have an older but reasonably powered laptop that is capable of running windows 7 and directx11 (that part may not be trivial -- i don't know how common that is) that is sitting on a shelf gathering dust, i'd be very happy to receive it.

Ath · « **Reply #11 on:** February 10, 2012, 02:56 PM »

i have to keep walking usb sticks back and forth to the other room
-mouser (February 10, 2012, 02:29 PM)

No network connection possible between those machines? Wireless perhaps? Or a NAS in your network?

Got no spare/left-over laptop/PC, sorry, but transport-cost from here would probably be higher then buying a cheap-ass new one anyway

spazzarama · « **Reply #12 on:** February 12, 2012, 11:20 PM »

Hehehe, I'm glad to see you got it working in the end mouser, and thanks for the invite to the forums. I'm sorry you had to go through so much pain to get it up and running

For those that don't know me (I'm brand new to these forums) I am the author of the blog @ http://spazzarama.wordpress.com/ and have felt the significant amount of pain and frustration that I'm sure mouser has endured when dealing with capturing images from DirectX applications. I maintain that site in the hope that what I learn from difficult problems I run across in my travels as a programmer can in turn help others. I experience great joy when people find it useful.

The problem with the technique I use in the code mouser has been using is that it uses process injection. Unfortunately this complicated approach was the only reliable way I could determine to capture at a reasonable speed from the DirectX application, and especially from a full screen one. I'm happy to elaborate on this, so if someone wants to know more about it just let me know.

Good luck with the next release of the screencapture app mouser - I hope that you've found that DirectX project helpful

As discussed separately there are a few enhancements I have planned that will hopefully improve things a little - but it is still a complicated process, and there's no real way around it

. If you are keen to exclude the GAC from the equation it might be possible to use the PATH environment variable but I haven't tried it to date...

I'll post about another opensource project currently in progress in the near future once bare bones are in place. I hope to find people that will find it useful and perhaps want to join in with the development fun

- a hint is it involves pretty flashing lights (come on, everyone loves those!!)

Cheers,
Justin

mouser · « **Reply #13 on:** February 12, 2012, 11:33 PM »

Fantastic to see you posting, Justin -- welcome! It's an honor to have you stop by.

My hat is off to you with this process injection/hook stuff -- it's way over my head and *very* impressive work.

And I apologize for the over-the-top ranting on my part -- It's well known I have a very low tolerance for frustration and no one is as good at pushing my buttons as microsoft is

I'm really looking forward to adapting your code to provide a directx capture option for Screenshot Captor -- so i'll be following your work closely, and look forward to hearing about your other projects.

I'm hoping in the end i'll be able to build a little installer that will take care of the GAC stuff for the directx capture, and make that an optional install for users who need directx capture.

Once again, welcome

Renegade · « **Reply #14 on:** February 12, 2012, 11:40 PM »

The problem with the technique I use in the code mouser has been using is that it uses process injection. Unfortunately this complicated approach was the only reliable way I could determine to capture at a reasonable speed from the DirectX application, and especially from a full screen one. I'm happy to elaborate on this, so if someone wants to know more about it just let me know.
-spazzarama (February 12, 2012, 11:20 PM)

I'm certainly interested in the dark arts!

Feel free to elaborate! (Here at DC or a blog post -- whatever works for you -- just post here if you do a blog post so I and others know.)

mouser · « **Reply #15 on:** February 12, 2012, 11:43 PM »

Read his blog Renegade, he goes into it quite a bit -- it's very interesting reading.

spazzarama · « **Reply #16 on:** February 13, 2012, 12:02 AM »

The problem with the technique I use in the code mouser has been using is that it uses process injection. Unfortunately this complicated approach was the only reliable way I could determine to capture at a reasonable speed from the DirectX application, and especially from a full screen one. I'm happy to elaborate on this, so if someone wants to know more about it just let me know.
-spazzarama (February 12, 2012, 11:20 PM)

I'm certainly interested in the dark arts!

Feel free to elaborate! (Here at DC or a blog post -- whatever works for you -- just post here if you do a blog post so I and others know.)

-Renegade (February 12, 2012, 11:40 PM)

There's basically three techniques that I'm familiar with Renegade:

1) Proxy DLL (don't think this works on Vista up anymore) where you create a DLL that implements the same interfaces as the library you are attempting to intercept, and rename the original - with yours doing what you need to before passing the call on to the original - then all applications using the library will call your DLL instead
2) Hooking + Interface Wrapping (see the link below)
3) Hook the virtual function addresses of the COM object in question (this is the one I use as its less work, and I also describe this in the link below)

You will find a good description of the last two and how I learnt about them here: http://spazzarama.wo...-direct3d-api-hooks/

Cheers,
J

mouser · « **Reply #17 on:** February 13, 2012, 12:08 AM »

And what I have done is shamelessly taken Justin's code and modified it slightly to make it a commandline non-interactive directx screenshot taker, and then added code to Screenshot Captor that will invoke it on a target window and grab the resulting screenshot as if it was taken from within the app itself. I'm just waiting for Justin to hopefully succeed in the hard work of getting it to work on full screen dx10 and dx11, and it will be ready for testing.

spazzarama · « **Reply #18 on:** February 13, 2012, 12:10 AM »

Fantastic to see you posting, Justin -- welcome! It's an honor to have you stop by.

My hat is off to you with this process injection/hook stuff -- it's way over my head and *very* impressive work.

And I apologize for the over-the-top ranting on my part -- It's well known I have a very low tolerance for frustration and no one is as good at pushing my buttons as microsoft is

I'm really looking forward to adapting your code to provide a directx capture option for Screenshot Captor -- so i'll be following your work closely, and look forward to hearing about your other projects.

I'm hoping in the end i'll be able to build a little installer that will take care of the GAC stuff for the directx capture, and make that an optional install for users who need directx capture.

Once again, welcome
-mouser (February 12, 2012, 11:33 PM)

Thanks mouser, I know the pain don't worry!! Will definitely keep you posted on changes - I'll probably move the project source so that it is in code.google or github which will make it easier for it to be worked upon by others too - at the moment the project itself is really only a demonstration of the technique and requires a bit of tweaking for any real world usage.

Happy to be here

Cheers,
J

spazzarama · « **Reply #19 on:** February 13, 2012, 12:11 AM »

And what I have done is shamelessly taken Justin's code and modified it slightly to make it a commandline non-interactive directx screenshot taker, and then added code to Screenshot Captor that will invoke it on a target window and grab the resulting screenshot as if it was taken from within the app itself. I'm just waiting for Justin to hopefully succeed in the hard work of getting it to work on full screen dx10 and dx11, and it will be ready for testing.
-mouser (February 13, 2012, 12:08 AM)

That's what it is there for

(he did donate tho guys which was much appreciated but not necessary

mouser · « **Reply #20 on:** February 13, 2012, 12:15 AM »

when the directx capture addon is ready i'll definitely have a link sending users to your donate page

spazzarama · « **Reply #21 on:** February 18, 2012, 01:06 AM »

Well good news mouser.

The errors with capturing in D3D10/11 were a different problem, you should leave both D3D10/11 as DriverType.Hardware in the device creation, and instead fix a different problem as outlined in the code below. D3D9 needs to still have the DriverType.None for it to work in fullscreen.

The problem is actually within the PresentHook where SlimDX is giving us some grief over disposing of its reference to the SwapChain - which cannot be done while in Fullscreen window mode - therefore we just have to keep a reference to the same SwapChain.

The bad news (for me) was that I didn't bother testing the original code until after half a day of stuffing around with it LOL!!! I just assumed it was a problem with the device creation

... It's times like these I wish I followed in my fathers footsteps and became a carpenter or maybe an electrician

. Anyway I think SlimDX might have something to do with it - and I'm in the process of switching to SharpDX, but that is a ways off for the moment.

Here is a snippet that will get it working for you, enjoy!

DXHookD3D10.cs:

[Select]

private SwapChain _swapChain;
   private IntPtr _swapChainPointer;

   /// <summary>
   /// Our present hook that will grab a copy of the backbuffer when requested. Note: this supports multi-sampling (anti-aliasing)
   /// </summary>
   /// <param name="swapChainPtr"></param>
   /// <param name="syncInterval"></param>
   /// <param name="flags"></param>
   /// <returns>The HRESULT of the original method</returns>
   int PresentHook(IntPtr swapChainPtr, int syncInterval, SlimDX.DXGI.PresentFlags flags)
   {
   if (swapChainPtr != _swapChainPointer)
   {
   _swapChain = SlimDX.DXGI.SwapChain.FromPointer(swapChainPtr);
   }
   SwapChain swapChain = _swapChain;
   //using (SlimDX.DXGI.SwapChain swapChain = SlimDX.DXGI.SwapChain.FromPointer(swapChainPtr))
   {

DXHookD3D11.cs:

[Select]

private SwapChain _swapChain;
   private IntPtr _swapChainPointer;

   /// <summary>
   /// Our present hook that will grab a copy of the backbuffer when requested. Note: this supports multi-sampling (anti-aliasing)
   /// </summary>
   /// <param name="swapChainPtr"></param>
   /// <param name="syncInterval"></param>
   /// <param name="flags"></param>
   /// <returns>The HRESULT of the original method</returns>
   int PresentHook(IntPtr swapChainPtr, int syncInterval, SlimDX.DXGI.PresentFlags flags)
   {
   if (swapChainPtr != _swapChainPointer)
   {
   _swapChain = SlimDX.DXGI.SwapChain.FromPointer(swapChainPtr);
   }
   SwapChain swapChain = _swapChain;

   //using (SlimDX.DXGI.SwapChain swapChain = SlimDX.DXGI.SwapChain.FromPointer(swapChainPtr))
   {

mouser · « **Reply #22 on:** February 18, 2012, 01:11 AM »

ooh this is exciting. going to try it now and report.

spazzarama · « **Reply #23 on:** February 18, 2012, 01:18 AM »

Good luck - I also switched to a non-GAC setup for testing purposes, for this I would run the D3D10/11 sample direct from my application directory.

The "EmptyProject10" and "EmptyProject11" are perfect for this as they have no dependencies (although on my machine getting them to fullscreen and then capturing was a pain at times - one of my screens would sometimes go black so would have to Alt+Enter a few times before I was ready to try the capture).

This might help you out in future for debugging:
To debug I set the ScreenshotInject project as my startup project in VS2010, and edited the build settings so that it started the appropriate D3D sample app on Debug. Then externally I would run the screenshottest to do the injection etc... All your breakpoints will be hit nicely (although fullscreen apps are not so easy to debug, well especially with only one screen hehe)

Cheers,
J

mouser · « **Reply #24 on:** February 18, 2012, 02:05 AM »

DX11 fullscreen capture now seems to work!
(DX10 not having any luck so far)