Best choice for server-side game programming

[kyrathaba]

Reposted as separate post per Tinjaw's suggestion (also slightly modified the question):

Question: can applets be downloaded from just any old website, and run in a browser?  Do they actually just sit on the server, inert, and not execute until downloaded (if that were the case, I'd think you could host them on even geocities or some other free host)?  There's a game idea I'd like to eventually implement, where the game would just be textual.  I'm wondering if Java or wxWidgets would give me the capability of authoring such a game that would run on my website and allow users to connect to it.

Traditionally, server-side programming has been performed using
Perl, Python, C++, or some other language to create CGI programs, but more sophisticated
systems have since appeared. These include Java-based Web servers that allow you to perform all
your server-side programming in Java by writing what are called servlets. Servlets and their
offspring, JSPs, are two of the most compelling reasons that companies that develop Web sites
are moving to Java, especially because they eliminate the problems of dealing with differently
abled browsers

[f0dder]

Applets are downloaded and run by the client, and can thus be hosted more or less anywhere (some free hosts have limitations on the types of files they let you store).

If you want to do something server-side, you need a decent host. Things like the non-realtime space conquest games can be implemented with some server-side scripting language and a database (usually php+mysql), and while you might not be able to find a free host that supports this, you can certainly find some cheap ones that do.

If you want to do something interesting, you need to run real, live code on the server... and then it comes down to which programming language(s) you're most comfortable with. Usually things like that are programmed in C/C++, but you could do java, dotNET, whatever. You're not very likely to find free hosting that lets you run code, though, and for paid hosting it's probably not going to be cheap.

[kyrathaba]

some free hosts have limitations on the types of files they let you store

Yes.  For example, I've never encountered a free web host that would allow .EXE files (unless they were zipped), and I suppose that's quite understandable.

In particular, the "game" my friends and I are hoping to someday develop would be a first-cousin to multi-user dimensions.  It would have quite low server load (probably no more than a handful of people connected at any one time).  However, it sounds to me that, at this point, multiplayer game programming isn't very accessible to the hobbyist programmer on a tight budget.

Of course, I could always serve such a program off my home machine via Filezilla or some equivalent, but that of course comes with the well-known security threats and reliability problems.  Still, these concerns aside (and I'm not saying I'd actually do this), I suppose if I did serve off my machine (just for a handful of friends, say), I could use .NET to write a server that web browsers can connect to.  That would effectively make it multiplatform, since everyone has an internet browser, regardless of their OS.

[f0dder]

Multi-User Dimensions?

If you mean MUD, then you'll need to write a real game server. You probably could fudge around and make something PHP-based, but I don't think it would be pretty.

As for running a MUD on your home machine, this shouldn't be a problem at all. Even a small ADSL connection should be able to serve "quite a few" people, and just be a bit careful when coding (ie., use safe datatypes and avoid buffer overflows) and you'll be fine.

[kyrathaba]

Thanks, Fodder.  Yes, I mean something similar to MUDs, as defined by your link.  I actually had a working vb6 program to which several people successfully connected simultaneously and were able to chat and issue commands to effect the virtual environment.  I abandoned that project, however, because vb6 did not natively, without the purchase of 3rd party components, offer robust socket programming.  I couldn't get asynchronous connections going, and had to use a queue (might have been more programmer ignorance than language weakness, though ;>)  I know, however, that .NET offers fairly robust support for network programming.  Also, I'm sure that .NET's OOP is more suitable to such a project than vb6.

[f0dder]

Traditional MUDs were done in C, which doesn't have OOP support; however, I wouldn't be surprised if their programming model was still OOPish, even without language support

[mouser]

If you were going to build a large-scale multi-user project, i would certainly not rule out any language like .net even if it required a dedicated server.

The larger the scale of the project, the more speed might matter and the more important code organization matters.  You could develop it on your pc and only after it gets to a stable state then raise money to pay for a virtual or dedicated server where you can run whatever software you need to run.

[kyrathaba]

That's true, Mouser.  Also, there are quite a few linux server packages out there in the $4 to $10 per month range that offer more than adequate server capabilities for the sort of application I'm envisioning (assuming that I develop in .NET and that Mono or whatever equivalent fully materializes).  And, it's more or less a pipedream at this stage:  my skill level is not yet quite adequate, I don't think (though I'm closing the gap, gradually).

BTW, what is the distinction between a virtual and a dedicated server?

[mouser]

A dedicated server is a real computer.  DonationCoder.com website for example runs on a dedicated server.  Essentially it means your operating system is running on the pc, and ONLY your stuff is running on the computer.

A *virtual* private server means your have a complete operating system running on the computer, but in a virtual environment along with other people's virtual environments.  So, you have complete control over your operating system and all files, and it is very secure against other virtual users touching your files, BUTall the resources are shared between all virtual users - so it's significantly slower (and cheaper).

A VPS (virtual private server) could cost you $25-$100 a month.
A Dedicated server could cost you $100 to $500 or so.

[tinjaw]

There are a couple of main issues bundled in there. Let me attempt to separate them out.

• Where the code is executed.
• Where the data is stored.
• What technology is used.
• Build from scratch or use an existing game engine.

Where the code is executed.
In order for Stuff & Things (TM) to happen, code needs to be executed. Code can run on the server, the client, or both.

• On the server: This is the traditional manner. This includes Server Side Includes (SSI) and Common Gateway Interface (CGI). This is commonly used when you want to use dynamically generated web pages instead of static web pages. The classic example is a phone directory. A web browser (client) contacts the web server (HTTP server) and requests a web page identified by a Uniform Resource Locator (URL). (Actually a URI, if you want to be technically correct.) With a static page, the names and telephone numbers are "hard coded" into the page and only change when the file is edited by the author. If SSI or CGI is used, the HTML file is processed by the HTTP server and the names and telephone numbers are dynamically added to the web page before it is sent to the browser. In this manner, the web (D/X/HTML) page does not need to be edited every time the list of names and telephone numbers is changed. In these cases all the web browser (client) does is use the HTML rendering engine to display the web (D/X/HTML) page. No code is executed on client computer. This also includes standard X/HTML forms. The client doesn't execute any code outside of the HTML engine. It just take the data entered into the form by the user and sends it back to the HTTP server.
• On the client: The classic example is a static web page that contains Javascript/Actionscript/ECMAscript. The HTML file on the server is static - it does not change, every time the page is requested by a client the same exact page is sent. (This make it a good candidate for a highly requested web page as it can be cached by the HTTP server in memory instead of it constantly accessing the slower hard drive and/or requiring processing.) With a static web page containing script, the HTML engine of the web browser hands of the scripting parts of the page to a separate engine.
  Script should not be confused with (embedded) binary attachments like images, video, audio, or, more feature rich items like Flash. The former is (ascii/text, i.e. human-readable) computer code that is interpreted and executed in a manner similar to a standard computer application, while the later is the equivalent of a DVD, VHS or audio cassette tape being played by a player. This later form, the binary attachment, also include Java applets. Java applets are binary files containing "byte-code". You can think of byte code and something in between human-readable source code and the zeros and ones understood by the CPU. In this case the web page is just an envelope and the web browser is just a canvas. The applet is run in a virtual machine. It is oversimplifying to a large degree, but it might be helpful to think of virtual machines as tiny computers inside another computer.
• On both: By combining the two you can get something that often looks no different that a stand-alone computer software application. The most recent use of code executing on both the server and the client is Asynchronous JavaScript and XML (AJAX). AJAX uses a constantly open connection between the client and the server and code is executing on both sides. (For standard or "legacy" web pages, one or more connections are made to the server, the pages are downloaded, and all of the connections opened to the server are closed.)
  One of the main advantages of this later use of technology allows the "software" to be updated constantly without any need for intervention by the user/client. This is because, although code is executed on the client computer, the code it executes is requested from, and delivered by, the web server. This means that the "software" can be updated on the server and the next time the client attempts to run the "software" it downloads the latest version from the server seamlessly. As computers get faster and as broadband Internet connections get fatter, it may soon become nearly impossible to the the difference between these web-based applications and traditional stand-alone software.

This is the end of part 1. Later today I will post more on : Where the data is stored. What technology is used. Build from scratch or use an existing game engine.

[tinjaw]

Where the data is stored.
This includes not only on which physical computer the data is stored, but in what format it is stored.

• "Flat files" on the server: In the example given above of the telephone directory, the information may be stored in a text/ASCII - possibly in Comma Separated Values (CSV) format. These "flat files" are more likely than not in some human-readable format. In this case the files are processed on the server and the resulting web page is delivered to the client. Other examples of the use of flat files are Javascript and CSS files. In both cases these files are delivered to the client alongside the web page requested by the client. The client then uses these text-based files to determine how to render the HTML and what code to execute.
• Binary files on the server: The classic examples of this type are Macromedia Flash files and video/audio files. The data is stored on the server and each time it is sent to the client and the client "plays" the file(s).
• Databases: When large amounts of data needs to be accessed and/or updated a database (server) is often employed. Databases are just large bundles of data. However, the term is overloaded and often refers to, not only the data itself, but the software applications that are used to create, update, retrieve and delete (CRUD) that data. Of the software applications that are fine-tuned to handle that data, the most widely used type of databases are relational database management systems (RDMS) that use Structured Query Language (SQL) as the computer language, the means to interact between humans and the databases. In almost all cases there is code on the web (HTTP) server that executes when requested by a client that connects to the database and CRUDs the data as necessary and, ultimately, dynamically creates the web page that will be returned to the client.
  Databases are invaluable when creating applications larger than a simple telephone directory. They are also required when doing any kind of online game. The data needs to be maintained, securely, by the application provider - be that a hobbyist run MUD or the likes of World of Warcraft. The data will surely be accessed countless times and heavily CRUDed as the application runs.
• Data on Clients: Although this is unlikely to be used by an application developer, it is reasonable and has some advantages. The most common form of this type of use are cookies - those files stored by your web browser when you visit most websites. Another viable use might be data files for online office applications. You would not want to store data files on the client computer for an online game. Even though you could take measures to conceal and encrypt the data, it is just too risky. Imagine if you will, storing the information about your World of Warcraft characters on your local hard drive. How long do you think it would take for the Lazy Web to come up with a crack to modify those files and turn freshly minted characters into maxed-out super characters?

Coming Soon: What technology is used. Build from scratch or use an existing game engine.

[kyrathaba]

Very nice, Tinjaw.  Thanks for the in-depth explanation.  It clears up a couple of misconceptions that I'd had.

As computers get faster and as broadband Internet connections get fatter, it may soon become nearly impossible to tell the difference between these web-based applications and traditional stand-alone software.

And won't that be sweet    I'm looking forward to video-phone conversations via computers that won't be choppy like today's webcam/chat-client combos.

Databases are invaluable when creating applications larger than a simple telephone directory. They are also required when doing any kind of online game. The data needs to be maintained, securely, by the application provider - be that a hobbyist run MUD or the likes of World of Warcraft. The data will surely be accessed countless times and heavily CRUDed as the application runs...You would not want to store data files on the client computer for an online game. Even though you could take measures to conceal and encrypt the data, it is just too risky. Imagine if you will, storing the information about your World of Warcraft characters on your local hard drive. How long do you think it would take for the Lazy Web to come up with a crack to modify those files and turn freshly minted characters into maxed-out super characters?

Not very long at all, I'm sure.  Just like all the hacks that came out after Diablo was released.  It seems that there are those that simply thrive on "beating" the system, so to speak.  The upside is that such "hackers" usually enjoy letting the programmer know about their exploits.  I suppose it's a form of bragging rights   Sort of like all the experts who hack Windows, then delightedly report the vulnerabilities to Microsoft 

But yes, I definitely see the need for database storage instead of flat files.  From my reading, I know that flat files have been the historical standard in MUDs, but more and more MUDs are popping up that use a database instead.  Also, there are beginning to be more and more MUDs coded in Java or .NET instead of C/C++.

[tinjaw]

What technology is used.
The ultimate answer to "What technology should I use?" is "It depends." The traditional metaphor of a tradesman and his toolbox full of tools applies here. Many technologies overlap and many factors need to be considered. If the application in question is a full-featured multiplayer online game, then you will most likely decide on a beefy server-side application that talks to a database server and passes on various types of data to a client application that will both render the data and do some local computation/execution of code. The results will then be sent back to the server and the cycle will be repeated. So, what flavors are available?

On the server side your options are vast. Nearly all (contemporary mainstream) computer languages can interact with HTTP servers and database servers. In close to 100% of the cases the web server hands a chunk of data off to a computer program that gets more data from a database server. It then massages the data and sends a new or modified chunk of data back to the web server, and possibly to the database server as well. These computer programs can be written in Perl, Python, PHP, ASP, C/C++, Java, C#, or a variety of other languages.

On the client side you will need to execute code. Your first choice is to decide between three technology types: 1) (X)HTML + CSS + client-side scripting via Javascript/Actionscript/ECMAscript rendered by a web browser and its plug-ins/extensions, 2) web pages similar to #1, but also containing binary data that is executed by a player, like Flash, also in a web browser, or 3) a custom built stand-alone application that may, or may not, employ some, or none, of the technologies of #1 and #2 and is written by any of the same computer programming languages used on the server side (albeit unlikely PHP, ASP, or Perl will be used client-side, it is technically possible).

Weaved ever so tightly into these two choices are a myriad of other technologies: What operating system will be used on the server? What operating system will be required of the client? What web browsers should be targeted? Will the web server be Microsoft's IIS, Apache, or some other HTTP server? What database will be used? Will it be a RDMS or an OO database? What intermediate means will be used to communicate between your applications and the database? A proprietary means or something like ODBC, JDBC or the dozens of other common database libraries and standards? These decisions will determine whether you have Sun hardware running Linux with Apache, JBoss, MySQL, using Java for JSPs, EJBs, and other code, or, just as likely, AMD hardware running Microsoft Windows with IIS, Tomcat, Oracle, using C#, .NET, and ASP.

In either of these two scenarios you will need a computer you can fully control and upon which you may execute any code you choose. This may be a physical computer, or computers, you own and collocate at an local facility, a leased dedicated server, or Virtual Private Server (VPS) you rent monthly. However, you may choose to go with an architecture that is more lightweight and use IIS/Apache + PHP/Python/Perl + MySQL/Postgress/SQL Server on a traditional web hosting account. Both would allow you to host files that can be downloaded and executed on the client. These include multimedia files (avi, mpg, jpg, etc.), Flash/Shockwave files (.swf), Javascript/Actionscript/ECMAscript (.js, etc), or even stand-alone applications, that are possibly installed and configured, and run by the client (.exe, or other binary executables for Linux, MacOS, etc.)

Some groups of technologies are a more natural fit together than others. For example, if you are already familiar with Java you might choose to match Java, JSP, Java Faces, EJBs, Java applets, JDBC, and Apache + Tomcat. Or, equally likely, Microsoft shops will use Microsoft servers with Visual Studio development tools and .NET and ActiveX technologies. Either is perfectly viable and more a choice between left and right or up and down than right or wrong.

The key point to get from this is that you need to spend some time surveying all of the various tools available to you, determining where they are tuned to be used, and deciding if you have a personal affinity for any of them over others.

Next: Roll your own or just make a movie?

[urlwolf]

Very nice job, tinjaw.
I have been thinking about this for a while.
For MUD and realtime stuff XMLsockets seems to be the way to go, no matter which language you use on the client.

It seems that all the server-client talk can take two forms:
CGI (post and get methods): cumbersome to parse, data are not self-describing necessarily, and limited in size (theoretically)
XML: most apps send XML back and forth, as a way of serializing more complex data structures.

This gets the programmer in a position where he needs to parse XML on both sides (what is known as XML push-ups). Not the most rewarding work.

However, there seems to be an emerging third option. It seemst that flex/flash can serialize objects in a binary format (not XML) and send them back and forth that way. It is faster, and it saves coding time in that you don't have to deal with the XML intermediate version.

Looks like for this kind of serialization needs some server-side libs, and they are J2EE or some other 'enterprise' stuff. They cost money. But I could be wrong. Of course, this is worth thinking only if you want your client side to be actionScript/flash/flex.

Current AJAXY stuff could be more than enough for a MUD-like app. However, it looks like a hack, and in fact it is: javascript + whatever on the server + Mysql... three sources of trouble. And then, you have to test it against several browsers.

I'm just debating all this myself. Not that I know much about it, mind you, just reading around.

In any case, if you decide to take the AJAX route, consider using a framework, not just a language. I have started playing around with Rails and I love it. Things will be developed fasts and code will be maintainable.

My 2 cents.

[tinjaw]

Build from scratch or use an existing game engine.
For a moment think about the entertainment industry. They have a similar choice. Sometimes they build from scratch, an example would be the move from celluloid motion picture film and movie theaters to DVDs and home theaters. Think about the time just before the transition. In the case of "use an existing game engine" you could have used the existing technologies and simply worried about content. IOW you create the script, provide the actors, provide the stage or shooting location, etc. You may even "customize" the existing technology by creating news ways of using it. For example, you may invent some new means of special effect like adding audio effects and music after the film is shot, or go even further and invent something like "bullet time" as seen in The Matrix. The "build from scratch" parallel would be deciding that you wanted to be able to allow viewers to watch the movie in their own home. You could use the existing tools (soldering irons, servo motors, etc.) and technologies (laser light, magnetic recording, analog to digital to analog conversions, etc.) and invented DVDs, Blu-ray disc players, and HDTVs to make your movie. In this case it would take a whole lot of work and some might consider the choice of theater versus home viewing not an issue. In which case much of your work outside of content creation was wasted. And there are various degrees of gray between these two extremes.

So, when it comes to computer games, whether that be massively multiplayer online games or solo-play console games or something in between, you need to make similar choices. On the one extreme you could write everything from scratch using nothing more than a text editor and a compiler, thereby creating both the content and the delivery medium. On the other, you could use one of the existing game engines like Garage Games Torque Engine or Torque Game Builder (see this thread for other ideas.) and simply focus on content. For example many well selling and praised games are built on other game engines like the Unreal Tournament engine.

There are a vast quantity of game engines using different technologies and focused on various niche uses from FPS-based MMOG, to modern versions of the old BBS door games. Some of them have been recently mentioned in DonationCoder threads and searching the forum archives will provide you with plenty of suggestions.

A very small list of game engines that have caught my eye includes:

• http://www.byond.com
• http://sourceware.org/xconq
• http://www.fifengine.de
• http://www.mitorahga...ah-Games-Engine.html
• http://www.vassaleng.../community/index.php
• http://www.nowhere-else.org/rpg

These game engines are ones where the vast majority of your work in simply creating the content. (Equivalent to writing the script and shooting a convention movie).

The next step up is the equivalent of writing a script and shooting a movie, but adding something new, like the CGI used in LoTR battle scenes or special software to rotoscope A Scanner Darkly. And you can do this to varying degrees.

At the base level you have a programming language like C++ or Python, but you can choose to utilize the work others have done to various degrees. Some code libraries exist to handle things like drawing graphics on the screen (OpenGL) or playing sounds (OpenAL). There are also larger libraries that handle multiple jobs like DirectX, SxDL, or Simple DirectMedia Layer (SDL). Some handle AI aspects, like AI.implant, OpenAI, and Soar. Above that are various language wrappers, like pygame which allows Python coders to use SDL and PyOgre which allows Pythonistas to use Ogre without knowing C/C++. And many other combinations exist like Delta3D, Irrlicht, or Allegro. Some are off the beaten path like Croquet.

The next level up are the game engines and SDKs Unreal Tournament and Garage Game's TGE and TGB. Theses require coding, but are generally handled by (most often proprietary) scripting languages instead of C++ or Java. These allow for greater flexibility, but at the cost of greater time spent on delivery instead of content. Others include:

• http://www.phelios.com/ptk
• http://www.adventuregamestudio.co.uk

Any of the items mentioned above could be used, to some degree, in even a "server-side" game.

If somebody asked me to suggest a path through all of this, I would suggest the following strategy or crawl-walk-run.

• Create a "paper and pencil" prototype of your game.
• Use on of the high-level "game builder" applications that allow you to concentrate on content. Make due with the limitations this imposes on your game for now.
• Move to a "scriptable" game engine like Torque Game Builder
• Give up any notion of "building games" for a year or more and learn the basics of a computer language. (My personal suggestion is to start with Python for its "ease of use" over "anal control of every bit".)
• Using your year+ of study start using game libraries and SDKs like pygame and pyOgre to build simple games.
• Give up any notion of "programming games" for a year or more and revist what it takes to make a good game. Read Will Wright, Chris Crawford, and Raph Koster, et al.
• Leverage your knowledge of Python, or learn PHP, and develop a client-server (aka web-based or server-side) game. Start with something like the source code version of the Nowhere engine and begin to heavily modify it.
• Now, with all that under your belt, you can think about using Java, C++ or Python to create your own World of Warcraft or other best-selling MMOG of your chosen flavor.
• Profit!

Well, enough for now. I hope this helps some of you properly orient the map. "Digg"s and/or Donations would be appreciated ("D&D appreciated") if you found this helpful. It will also determine whether I expand on this further or eat and sleep. 
[/list]

[kyrathaba]

Thanks for the references to exising game engines, Tinjaw.  I'm currently investigating the BYOND system, and it seems like it will do everything I'd like to do in developing a multiuser online gaming experience.  I think I can ascend the learning curve in a few weeks of reading/practice, thanks to my previous exposure to OOP via C#.

Thanks for the good, in-depth coverage of this topic.

[tinjaw]

kyrathaba,

I am very pleased that you found my rambling at all useful. I'd be happy to answer any follow up questions you (or any other reader) has. Good luck on your project.

[Deozaan]

I am very pleased that you found my rambling at all useful. I'd be happy to answer any follow up questions you (or any other reader) has. Good luck on your project.

-tinjaw (February 22, 2007, 02:41 PM)

I found it useful, too!