Hmmm, you could try and use the instructions from this
link to get things running on Windows OS's higher than XP.
You could try to set things manually in the registry. Make a backup of the registry before starting any of this. If you don't know, ask someone who does to help you out with this. Seriously!
Ok, after that obligatory registry-editing warning, lets go to key:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\SubSystems\Windows
There should be a line with the following or very similar text:
ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16
The part 'SharedSection=1024,20480,768' is the most important section in your case. Adjusting these values can have serious implications for the well-being of your system, so tread carefully!
First some explanation
After the Microsoft® Windows® operating system has started, various areas of memory for resource tracking are reserved. Memory allocation of the heap is controlled by the number of open applications, windows, services and other resources. When a large number of processes are running, this heap may run out of memory.
How this memory is allocated has implications in your system. Changes in the default environment will affect overall performance considering your system architecture. The numeric values following SharedSection= control how desktop heap is allocated and are specified in kilobytes. Windows uses desktops classified as interactive and non-interactive.
The first SharedSection value (1024) is the shared heap size common to all desktops and changing this value is not (likely) necessary as this part of the heap contains the handles to windows, menus, icons, cursors, etc. and shared system settings. The second SharedSection value (20480) is associated with the "interactive" window station WinSta0. User objects like hooks, menus, strings, and windows consume memory in this desktop heap. Changing this value is not (likely) necessary as well.
The third SharedSection value (768) is associated with the "non-interactive" window station. If this value is not present, the size of the desktop heap for non-interactive window stations will be same as the size specified for interactive window stations (the second SharedSection value) and is used to keep track of processes without graphical user interfaces. Changes are likely necessary here.
Prior to Windows NT v3.5, Microsoft recommended a heap size of 3000 Kbytes (allowing for 460 processes). For later versions of NT and for Windows 2000 they recommend a size of only 512 Kbytes (allowing for 79 processes). Increasing the heap size beyond 512 Kbytes can cause instability or "blue-screening" of the system but the factors involved are sometimes obscure. Possibly, increasing the heap size depletes some other vital resource.
The size of the heap is the limiting factor controlling how many software packages can be active on any given system. Setting these values comes with a trade-off, your system will be able to support more running processes but overall performance will decrease. If performance is degraded too much for your liking, returning the third number to the original value should fix that.
The info above is gathered from several internet sources. However I tried the mentioned suggestions myself to initiate 80 Excel 2010 instances (on a WinXP SPIII with 2GByte of RAM) and that did work. You should never want to try that though, as your system will become really slow. An hour delay between mouse-click and computer responding to the mouse-click was not uncommon, when all instances were activated. Without that load, but with the changes to the heap that XP PC felt only a little bit slower than usual.
Please, don't take these Desktop Heap changes lightheartedly. During that time I did cross the limit (>80 Excel instances) and the BSOD monster did rear its ugly head almost immediately.
Sorry for the long post, I hope it can help you out though.