How to bundle a game's source file and its interpreter into a single EXE

[kyrathaba]

I'm writing an Interactive Fiction compiler, and then will write the corresponding interpreter (long term project, yes I know).  The compiler will take a game source file (plain text file) and convert it into a binary game file that the interpreter can interpret.

Okay, all fine and dandy.

But I've noticed that some interactive fiction authors, notably Adam Cadre, offer Windows executables of their games.  I'm wondering how one would bundle both the source file AND its interpreter into one EXE file.

[tinjaw]

Self-extracting compressed file. For example you can use WinZIP to turn a ZIP file into an EXE, or use WinRAR to turn a RAR file into an EXE.

For more information search on SFX.

[mouser]

one thing you will want to decide is: do you want user to be able to easily extract the original files, or do you want to hide them and make it behave like one exe and not have any sign of original files.  if the former, than sfx is good approach.  if the latter, then you might want to look for more specialized tools.

[kyrathaba]

I want to bundle the EXE interpreter and the binary game source file.  These two files would be contained by the EXE and when the user launched the EXE, it would, in effect, cause the interpreter to load the game file and run it.  To the end user, it would appear that the EXE is itself the game.

[tinjaw]

Sorry, I misunderstood what you wanted.  I do not know of any tools to do what you want. My guess is that you will have to write your own custom code. If you do find an existing product that does what you want, I would be interested in knowing about it.

[mouser]

i'm not sure why tinjaw is retracting, i think sfx tools will do what you want, though they are mainly designed to be used with installers.  they should still do the job though i think.  there are also specific programs designed to for this task as well (for example see this page).

[Eóin]

Embedding raw data into an exe isn't too difficult. Using a resource script and compiler which packages the data into an object file ready for linking into an exe is probably the easiest way to go.

Basically you would write the interpreter and get the compiler to link the different resources for the different games into final exes. Inside the interpreter would use the FindResource and co. family of API calls to access the binary source file.

So in reality the exe IS the interpreter, but since for end-users the behaviour of the exe is completely different depending on the packaged binary source file that it appears that the exe is in fact the game.

What language are you writing the interpreter in? That could have a big bearing on the options available to you.

[kyrathaba]

I'm writing the game compiler in C#.  The game compiler will take game source files, which are plain text files, and read their data into game objects that will then be serialized to disk during compilation.  This is the binary game file the interpreter (a separate program) will run for the user.  I'll also be writing the interpreter in C#.

The reason for asking about this is that I noticed on Adam Cadre's homepage (he's a well-known Interactive Fiction game author) that he had some of his Inform games available for download as standalone EXE files, even though normally you'd have to run an Inform game with an interpreter such as Glulx.

[mouser]

I think people are over-complicating the matter.  There are plenty of tools out there designed to bundle up a set of files to make a single exe than can be run, for exactly the purpose you want.  Just find one that you like the best.

[Eóin]

Unfortunately I don't have any real .NET programming experience myself. But as I understand it resource file do still exist so linking the binary source file into the exe should still be viable, though you'll need to look into the C# or .NET equivalent APIs for manipulating resources.

mouser, I get the impression that kyrathaba wants a transparent solution which the end-user will not be aware of. Many tools out for bundling files will probably need to unbundle the files onto the harddisk before they can be used.

[mouser]

Re-reading kyrathaba's posts, it is true that he is writing the programs, rather than needing to bundle an executable interpreter written by someone else.  In this case, it's true that the most elegant solution is to write your own code that can build a single exe with the interpreter and data combined.  It's not as hard as you might think, all installer tools do this, and there are a variety of ways.

[kyrathaba]

So, in effect, what I would need to do is write a program that

(1) can compile plain-text game source files into binary game files ready for interpretation

(2) can interpret the binary game file produced by compilation

(3) can itself produce an EXE housing the binary game file and interpreter rolled together.

Sounds challenging.  I've never considered how C# might be capable of producing a program that can, in turn, produce its own EXEs.

I'm interested in doing this mostly for the learning involved.  In the world of Interactive Fiction games, generally one uses an interpreter to open and play compiled game files.  There's nothing wrong with that.  And, in fact, many end-users would probably be more comfortable with that arrangement than with running an EXE, even if their virus-scanner gave a thumbs-up.

[kyrathaba]

Here's one option I found, while searching:

http://support.microsoft.com/kb/304655

One could have a C# program that contains all interpreter methods, and then use the ICodeCompiler compiler execution interface of the .NET framework to create an EXE, based on game source text read into the program.

[Eóin]

Hi kyrathaba, do you want to give others the tools to do step (3) themselves? If not then for yourself you won't need to write a program to do that bit. You'll already have the necessary tools as part of your build system.

[kyrathaba]

Yes, I was thinking of a game compiler that would take the text source file, convert it to a serialized binary file of game objects (derived from the source file information), and bind this game file and the interpreter into one EXE.

My idea was for game authors to have the option of releasing their games as EXEs, or as binary game files that come with a separate interpreter.  Magnus Olsson wrote this sort of thing for z-code games using something called Jzip (I think that's what it was called).

[Eóin]

Ah ok, well you're right then when you say it sounds challenging, but you needn't lose heart either. Off hand I wouldn't have a solution for you, as I mentioned I don't use C# and/or .NET myself properly. But I do know it is possible.

I would suggest working away at steps (1) and (2) because they are the real challenge. You'll need your interpreter to not be fussy about where it gets the binary source data from. So it should be able to read it from a file, and also quite happily accept being given a raw pointer to the data already in memory. With prototypes of the interpreter existing as a single exe as well as a prototype binary source file which is self-contained also as one file then the final step of putting them together shouldn't prove too difficult.

Someone here may be ale to help you with step (3). I'm intrigued enough by it myself that I might hack away if I've the time (which honestly isn't too likely ). But if not here I can pretty much guarantee you'll get an answer at say programmingforums.org or codeproject.com. The key though will be going to those people with basic working prototypes of part (1) and (2) so that they can narrow in on the best solution for your problem.

I'm slightly wary about mentioning this because I thought the poor OP got a hard doing but at the same time more knowledge is rarely a bad thing. You might like to read this thread where someone with similar (I use that word VERY lightly, in truth the OP was being wildly naive and unrealistic whereas I do not think you are being so) goals was frustrated by not getting an answer for the equivalent of your step (3) when he didn't have a prototype therefore making it very difficult for people to help. Don't dwell on the negative bits of the thread but take heart that there are very knowledgible and helpful people out there if you can just meet them halfway

On a final note, I suspect that if you write the interpreter in a native language like C++ it might make the step (3) much easier. But if doing so would make step (2) much more difficult for you then don't switch, stick with what you know.

[hollowlife1987]

If you want I can write a program that will take an interpreter program and a compiled script file.
Then execute the script using the interpreter and give you the source code.

[mouser]

You'll need your interpreter to not be fussy about where it gets the binary source data from. So it should be able to read it from a file, and also quite happily accept being given a raw pointer to the data already in memory.

yep, that's a good place to start.

[kyrathaba]

Thank you all for the input, it's much appreciated.

Obviously, I'm only doing this project in my spare time, as a learning exercise (an exercise made more interesting/fun because I like text-adventures).  I have no aspirations to one-up mature authoring tools like Inform 7 or TADS 3.  However, I am going to incorporate such elements in my game language that -- to me -- are easier to use.

HollowLife, thank you for your offer.  It's much appreciated.  If I understand you correctly, you're staying that if I can come up with a working interpreter EXE, I can supply that, plus a binary game file produced by my game compiler, and you can give me source code that would incorporate those two files into an EXE.  What language would you supply this in?  I am only familiar with C# and (to a lesser extent) Visual Basic.  However, I do have a couple of different C/C++ compilers.

Eoin, thank you for the link, and the advice.

You'll need your interpreter to not be fussy about where it gets the binary source data from. So it should be able to read it from a file, and also quite happily accept being given a raw pointer to the data already in memory.

I was planning on using .NET's object serialization capabilities, so that my compiler would read in the source file, syntax check it, and if it doesn't fail any syntax checks it would incorporate the data read in from file into game objects (rooms, items, actors, etc.).  The resultant compiled game file would be a master game object (containing all other objects), serialized to disk.

Then, the interpreter would deserialize the file and cast it to the master game object, then proceed to use that master game object's properties and methods to produce the interactive game that the end-user experiences.

[kyrathaba]

I've discovered that .NET 2.0 contains a compiler class that permits runtime compilation of sourcecode.  So it's possible to have a C# program that can itself produce a physical executable.  This opens up more possibilities.

[hollowlife1987]

HollowLife, thank you for your offer.  It's much appreciated.  If I understand you correctly, you're staying that if I can come up with a working interpreter EXE, I can supply that, plus a binary game file produced by my game compiler, and you can give me source code that would incorporate those two files into an EXE.  What language would you supply this in?  I am only familiar with C# and (to a lesser extent) Visual Basic.  However, I do have a couple of different C/C++ compilers.

The language I can do it would be C# as that would be the best your interpreter is in C#

If you let me use your interpreter and a sample source file it would be alot easier for you to understand when I give you the source code.

[kyrathaba]

Excellent.  I don't have the compiler anywhere near a functional level, yet.  I'm still working on methods to extract, from the source file, data for rooms, items, actors, etc.  I'll work on it.  If I can just get enough done to compile a simple game (say, two connected rooms), and can provide an interpreter to run the compiled game file, I can probably learn enough by seeing how you implement that for me to extend your source code as I add further functionality.

[kyrathaba]

Thinking about it further, if I supplied you with a text file containing the string "I am a string", and an EXE interpreter that when executed would write that file's string to the console, could you in turn show me how to include those two files into an EXE wrapper such that executing the EXE caused the string to be written to the console?

[hollowlife1987]

Sure that would work just fine.

[kyrathaba]

I did a little experimentation and came up with a C# program that can, itself, compile C# source code.  The downloadable zip contains a folder with two files: (1) temp_app.exe, and (2) myfile.txt.  The first file is the physical executable I created with the Visual C# Express IDE.  The second file contains a brief C# program source code listing.  

Make sure you have the .NET Framework 2.0 installed, then open the folder and double-click on the EXE.  Read the program's console output and wait for it to exit, and you'll see that the text source file has been compiled into "myfile.exe" :> 

Here's the source code:

Code: C# [Select]

1. using System;
2. using System.IO;
3. using System.CodeDom;
4. using System.CodeDom.Compiler;
5. using Microsoft.CSharp;
6.  
7. namespace CompilationTest {
8.  
9.  
10.  
11.     class Comp {
12.  
13.         public static int Main(string[] args) {
14.  
15.            
16.             CodeDomProvider cdp = new CSharpCodeProvider();
17.  
18.             bool result = Comp.CompileCode(cdp, "myfile.txt", "myexe.exe");
19.  
20.             Console.WriteLine("\tProgram will exit on its own in 7 seconds...");
21.             System.Threading.Thread.Sleep(7000);
22.  
23.             return 1;
24.  
25.         }
26.  
27.  
28.         public static bool CompileCode(CodeDomProvider provider, string sourceFile, string exeFile) {
29.  
30.             string PersonalFolder = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData);
31.             CompilerParameters cp = new CompilerParameters();
32.             CompilerResults cr = null;
33.  
34.             // Generate an executable instead of
35.             // a class library.
36.             cp.GenerateExecutable = true;
37.  
38.             // Set the assembly file name to generate.
39.             cp.OutputAssembly = exeFile;
40.  
41.             // Generate debug information.
42.             cp.IncludeDebugInformation = true;
43.  
44.             cp.ReferencedAssemblies.Add("mscorlib.dll");
45.  
46.             // Save the assembly as a physical file.
47.             cp.GenerateInMemory = false;
48.  
49.             // Set the level at which the compiler
50.             // should start displaying warnings.
51.             cp.WarningLevel = 3;
52.  
53.             // Set whether to treat all warnings as errors.
54.             cp.TreatWarningsAsErrors = false;
55.  
56.             // Set compiler argument to optimize output.
57.             cp.CompilerOptions = "/optimize";
58.  
59.             // Set a temporary files collection.
60.             // The TempFileCollection stores the temporary files
61.             // generated during a build in the current directory,
62.             // and does not delete them after compilation.
63.             cp.TempFiles = new TempFileCollection(PersonalFolder, true);
64.  
65.             Console.WriteLine();
66.  
67.             if (provider.Supports(GeneratorSupport.EntryPointMethod)) {
68.                 // Specify the class that contains
69.                 // the main method of the executable.
70.                 //cp.MainClass = "Comp";
71.  
72.  
73.                 // Invoke compilation.
74.                 cr = provider.CompileAssemblyFromFile(cp, sourceFile);
75.  
76.                 if (cr.Errors.Count > 0) {
77.                     // Display compilation errors.
78.                     Console.WriteLine("\tErrors building {0} into {1}",
79.                         sourceFile, cr.PathToAssembly);
80.                     foreach (CompilerError ce in cr.Errors) {
81.                         Console.WriteLine("  {0}", ce.ToString());
82.                         Console.WriteLine();
83.                     }
84.                 } else {
85.                     Console.WriteLine("\tSource {0} built into {1} successfully.",
86.                         sourceFile, cr.PathToAssembly);
87.                     Console.WriteLine("\t{0} temporary files created during the compilation.",
88.                         cp.TempFiles.Count.ToString());
89.                 }
90.             }//end if
91.  
92.             // Return the results of compilation.
93.             if (cr.Errors.Count > 0) {
94.  
95.                 return false;
96.  
97.             } else {
98.  
99.                 return true;
100.             }
101.  
102.  
103.         }//end method CompileCode()
104.  
105.     }//end class Comp
106.  
107. }//end namespace CompilationTest