NANY 2011: Techniques, Tributes and Tribulations

[timns]

Ok it's just me, but I have a great interest in such things. So would the NANY crew be willing to share:

• Why were the NANY applications coded? (I mean why decide to code this particular app)
• What IDE did you use, if any?
• What language(s) is the application written in?
• Does it rely on any 3rd party libraries / code / graphics?
• Were any clever design principles used?
• Or any really hairy algorithms that you'd like to boast about?
• What was the trickiest part?
• Would you like to make a mention of any other DC members who helped out?

Edit: Actually, we all have an interest in such things, so in lieu of the regular survey maybe the NANY Entrants would like to fill this? Thanks Tims for posting the questions! 

[skwire]

• Why were the NANY applications coded?  List Numberer and Ten Timer were made upon request.  Snap DB was an app I had written some time ago for my personal use that I decided to flesh out and pretty up for a NANY release.
• What IDE did you use, if any?  No IDE per se.  I use PSPad (which I have set up much like an IDE).
• What language(s) is the application written in?  AutoHotkey
• Does it rely on any 3rd party libraries / code / graphics?  A couple.  One for form/control resizing and another for the toolbar.  Neither are built-in to AHK.  The toolbar icons are taken from the Fugue set by Yusuke Kamiyamane
• Were any clever design principles used?  Ehhh...a few?  Ten Timer is written in such a way that it could easily be "Hundred Timer" with the change of one value.  Snap DB uses some pretty in-depth listview manipulations.
• Or any really hairy algorithms that you'd like to boast about?  Nope.
• What was the trickiest part?  N/A
• Would you like to make a mention of any other DC members who helped out?  Mouser, Perry and all the DC members that give feedback.

[timns]

• How were the NANY applications coded?  For great justice?  With tender loving care?  I'm not sure what you're asking here.

Yes, sorry typo - I meant "why" as in "why did you decide to do it?" - fixed the original post.

[skwire]

Yes, sorry typo - I meant "why" as in "why did you decide to do it?" - fixed the original post.

I adjusted my first answer. 

[timns]

Yes, sorry typo - I meant "why" as in "why did you decide to do it?" - fixed the original post.

I adjusted my first answer. 

And a very good answer it is too 

[worstje]

Seeing how you ask so nicely, I'll oblige for both my applications.

JottiQ

How Why was the NANY application coded?
In my free time. But seriously - I edited this a few times now since what I thought it meant ended coming up in later questions. I think you might need to clarify what you mean. Nevermind, long posts mean others beat me to that horse.

I always loved Jotti's malware scan and felt it didn't get the praise or attention it deserves. To boot, it's a pain in the arse to use to the point where I often don't bother and just take a risk when I'm in a crabby enough mood where otherwise I might Jotti-19-check it to be safe.

What IDE did you use, if any?
Microsoft Visual Studio 2010.

What language(s) is the application written in?
The main application was developed in C#. It is based on the Windows Presentation Foundation (WPF) framework because I wanted to learn how it works. The end result targets the .NET framework v4 and can work fine with just the Client Profile installation (which is basically a Lite version with less features that's featureful enough for these purposes.)

The shell extensions for 32-bit and 64-bit were developed in C++.

Does it rely on any 3rd party libraries / code / graphics?
I used a lot of sources on the internet for my development. I took some public domain code as bases for things I knew how to generally do but knew I would mess up if I fully reinvented the wheel. However, the only libraries/assemblies it uses are all standard Microsoft related stuff. The only images used are vlastimil's custom-made lady bug, and a free-to-use magnifying glass by a dude whose name I can't possibly remember, so if you want that look in the Readme or About box. And there's the Silk icon set by FamFamFam, which are also great and yet so easy to forget.

Were any clever design principles used?
There aren't any particularly clever design principles. As I never really use threading in other apps, I had to reacquaint myself with all the proper knowledge, and C# just has so many classes for so many different situations that I ended up getting lost in them. Sure, there's tutorials on what classes are good for what situation and simple examples, none of them really matched my situation. In the end, I ended up bastardizing some example, rewriting it and generally making a mess of it for as far most C# programmers would be concerned. But it functions well enough.

Or any really hairy algorithms that you'd like to boast about?
Plenty.

• One might notice how JottiQ tends to remain stuck on 'Awaiting further processing' for some items. This is because at one point in time, it seemed useful to optimize in speed and in the general case one would expect most of the time spent processing an item to be inside the uploading or result-requesting process. Yet the caching mechanism requires the calculation of hashes and those can take several (gasp) seconds, which is a disk IO and CPU bound process as opposed to networkbound. So, in order to minimize wasted seconds, right as the more time-consuming tasks take place, an extra thread is kicked off to pre-emptively calculate the hashes of the first fully unprocessed object. That way, once some item is completed, it can move right-up with the rest of the network-related activities, ensuring most efficient ('maximum') use out of your internet connection. Of course, in the end it turned out JottiQ became so popular that most time is now spent waiting for Jotti's malware scan to give you permission to upload... A future version is going to need tweaking for the sort of situation where JMS is overloaded - I want it to process cached items first in those cases without canceling the non-cached ones (as it does now). I wasn't able to hack that in thus far as I need the time... and as it is now, stuff is functional and usable enough that I'm just biding my time, collecting feature requests and studying other general issues that come up.
• Displaying an icon in a popup menu is hell. Thankfully, I am not supporting anything pre-XP, but even as it is, I had to take care of several different approaches. First of all, one has to convert an icon to an image. Second, the actual displaying is different for each OS, with different gotchas and ugly glitches. The very old, original method of adding an image to a menu item is from the Win3.1 era and W95.
  
  For example, the XP method will turn any Vista+ machine into having the most ugly horrifying XP-style menus (some sort of compatibility downgrading involving Aero), or has a tendency to make it an ugly aliased pain (assuming you get the background to be transparent at all, which was a frustration on its own. 'Yeeaah I did it... wait, no, crap, it is a slightly different bg color than all the other icons!' ). For those who know a bit more about the details, this method involves custom-drawing the icon, and apparently Windows 98 (or maybe 2000) had an ugly bug involving that. But as I said above, that is something I could thankfully ignore. Oh, and I need to make sure the icon fits on the line, too. And some really short sentence involving the IContextMenu3 interface got overlooked, which is why TotalCommander and other programs had a habit of crashing when JottiQ was in the menu.
  
  The Vista+ method goes back to the roots, and uses the old original method of adding an image to a menu item. But of course it can't be simple. It takes a MSDN page (full with oodles and oodles of code) with custom processing of the image to pre-multiply alpha levels and do some other magic-fu in order to get the icon drawn properly with its proper alpha transparency. Then throw in that for good performance it needs to make some calls to the theming engine, but said calls don't exist on XP, so I have to load those dynamically, and.... well, it's a pain.
  
  I'm half tempted to throw up an example project at some point in time, since even with the example project I used as a base (and it was pretty good!) I had to do so much tinkering just to get an icon to show up properly that I think it could save people a lot of time in the future. But I'd need time for that which I do not have at present.
• Why code something custom for the context menu in the first place rather than just dumping an entry in the registry? One reason, and one reason alone: placement. Windows puts stuff 'fully defined' in the registry at the far top of the context menu, in the same sort of spot as the usual Open, Explore, etc commands. The stuff that is really specific to this sort of file. But JottiQ is an entry you'd find in all files, and not a specific action for that file. Over the years, people (or at least I have) come to expect that the more general stuff that applies not specifically to this sort of file comes lower in the popup menu where it is out of the way. And I simply could not meet that simple requirement through the registry alone, since if I could I'd never have written the context menu shell extension in the first place.

What was the trickiest part?
Windows XP and the .NET WPF framework. And cranioscopical by proxy.

WPF and .NET have so many bloody annoying bugs or visual glitches on XP, it makes it a real pain. Let's see what I can remember off the bat...

• .NET on XP simply crashes hard upon accessing a high-resolution, Vista-era icon of I think 128x128 dimensions or higher. This is why the faded lady bug is missing on XP.
• Thus, since WPF could not properly determine the right icon to load for my aboutbox, I figured I would do the looping myself based on meta-properties, which should be safe enough. But it did not work. Why? Because the meta-properties are also BUGGED on _all_ platforms and always saying one of those png-encoded icon sizes has all the bitdepth and colors in the world. Thus, my code would spit out ugly 16 color dithered icons, or some other variety.
• In the end, I ended up hardcoding the internal frame index for the proper icon size with the proper bit depth and alpha channels for those places that used a small icon. I'm sad if it looks ugly for people on crappy displays, but .NET forced my hand.
• WPF as a whole is a nightmare. Just once you think you figure some of that XAML stuff out, and need to make one last tweak, you find out it isn't possible and end up reading a few hours to try and make an attempt for a different method of doing things. (That bloody lady bug put me through so much pain, as did that toolbar, as did that bloody UAC shield, as did that ugly white background on XP...)
• WPF is meant to deal with different DPI settings. But it doesn't deal well with specially changed font sizes on its own. In the end, I figured out the general trick to playing nice with that sort of stuff, but it was by trial and error as opposed to documentation pointing me to it. (And trust me I looked.)
• Oh, and a golden oldie that I almost forgot about as it was one of my first frustrations. Of course there's the matter of images being fuzzy and fonts being fuzzy. The second tends to show itself as being an easy fix with the SnapToPixels property, but the other takes a fix that is new to .NET v4 (yes, WPF could not display fonts nice and crisp until now) and was hell to find. In general: avoid WPF if you want your UI to blend in real well with the rest of your apps.

Would you like to make a mention of any other DC members who helped out?
Ath for the installer and relentless testing on different OSes, cranioscopical for crashing JottiQ on XP and throwing it through the wrangler with his odd fonts, vlastimil for the awesome icon.


I'll do Cautomaton when I feel like it - I spent way more time on this one than expected.

Edit: Holy turd that's a far longer post than I thought. I guess I wanted to vent about .NET and WPF a bit more than I thought.

[timns]

Wow! Thank you worstje - a very enjoyable read, and quite insightful I thought.

[cranioscopical]

Yes, a very interesting post… apart from one or two outlandish comments 

[mouser]

There is another thing to note about Worstje which i think represents the best of DC.
Let's travel back in time to Worstje's first post on the forum about a year and a half ago:
http://www.donationcoder.com/forum/index.php?topic=18277.msg163724#msg163724

• In his first post he states that he is a long time user of the Dina font (created by DC member Jibz and shared on DC in 2005), and then makes a request for a new coding snack.
• The coding snack is then implemented by DC member skwire: CovertOps.

And then 1 and a half years later he is participating in NANY and releasing a great new program for all of us to use.  I find that very cool 

[timns]

My first entry for NANY 2011 is: WhirlyWord

This is a simple puzzle game based loosely around mating Scrabble with a slot machine. Spin the reels and try to make the words shown in the list.

• Why were the NANY applications coded? (I mean why decide to code this particular app)

I fancied doing a simple game again this year after I had quite a lot of fun coding up Twigatelle for last year's NANY. I wondered if I would be able to re-create the effect of spinning reels in a nice way, from scratch. No doubt I "discovered" a very standard way of doing that trick, but I get some satisfaction of trying to solve these things myself.

• What IDE did you use, if any?

I used IntelliJ IDEA from JetBrains. It's a Jave IDE I've used for years now: I'm very comfortable with it, and I find it about the nicest IDE I've personally tried.

• What language(s) is the application written in?

Java, and is launched using Java Web Start from my site http://head-in-the-clouds.com

• Does it rely on any 3rd party libraries / code / graphics?

All of the code is my own.

The graphics are all generated on-the-fly, using procedural textures that I developed last year for Twigatelle, or in the case of the reels, simple text with filters on top.

Didn't even borrow any icons for this one

• Were any clever design principles used?

Not really. Java 2D was not too bad to get going in the first place, and I lifted a lot of the Twigatelle code for the basic animation of the reels. Being Java, the code is reasonably object-oriented, a bit verbose but easy to maintain.

It's probably about 20,000 lines of code, although the main game loop itself is probably only about 10% of that. It's all the faffing about with loading resources, creating textures, beziers, particles etc. that add up.

• Or any really hairy algorithms that you'd like to boast about?

The only "clever" thing that I think is worth mentioning is the motion-blur on the reels when they are spinning. I was pretty pleased with the effect, as I may have mentioned

• What was the trickiest part?

Getting the dictionary in place. In the end I wrote a little sub-program to find seven letter words that had at least 120 valid anagrams each, pulled from a public domain word list.

• Would you like to make a mention of any other DC members who helped out?

There were some nice messages from a few folks, and, in alphabetical order, I would like to particularly thank Cranioscopical, Deozaan, Mouser, and Stephen66515 who all made suggestions that I feel improved the game.

I also want to thank Perry one more time, since he worked so hard on keeping all us "nanyteers" organised.

[nharding]

DCDisplay

• Why were the NANY applications coded? (I mean why decide to code this particular app)

I like CDisplay, it is easy to use but it has not been updated for a few years. I suggested a few improvements years ago to the author, but they were never implemented. I decided to write a replacement, that would add the things I wanted plus a database and some other features that are still in the pipeline.

• What IDE did you use, if any?

Visual Studio Express.

• What language(s) is the application written in?

C#

• Does it rely on any 3rd party libraries / code / graphics?

Yes, I am using a few 3rd party libraries, Nunrar to extract files from rar files, ZipStorer to extract files from zip files, 7ZipSharp to handle a problem with Nunrar on certain archives. Speech Balloon code, and a BTree database implementation.

• Were any clever design principles used?

No, I wanted to learn C# so I wanted a reasonably complex project to get used to the language.

• Or any really hairy algorithms that you'd like to boast about?

The file sort took a few months to get working, since it does English language sorting. So A Tale of Two Cities sorts as Tale,2,Cities. It also extracts dates in a variety of formats, so Mar 2001 and 2001-03 are the same, when comparing it works on word by word basis, but some can be ignored. So Computer & Video Games (2001-11) 12.cbr and C&VG 13.cbr would sort correctly (it has abbreviations expansion before it generates the tokens for the name sort).

• What was the trickiest part?

Handling all the possible naming of files I came across. Since I can't rely on names being a standard format it makes extracting the name and issue number awkward especially with a title like Babylon 5, I added a list of numeric titles that it uses to get the actual issue number.

• Would you like to make a mention of any other DC members who helped out?

Mouser for some comments and a small donation

Neil

[mouser]

I plan to do a longer writeup of the tribulations and lessons learned from one of my NANY projects (userquotes) but thought i'd get a start by writing some quick thoughts here.

   * Why were the NANY applications coded? (I mean why decide to code this particular app)
   * What IDE did you use, if any?
   * What language(s) is the application written in?
   * Does it rely on any 3rd party libraries / code / graphics?

I did two NANY apps this year, one was Webcam Video Diary, coded using C++ Builder (which is basically C++ with lots of visual component libraries); the other was userquotes.com, a website service coded in php on top of the open source Drupal CMS.

Both entries made HEAVY use of existing 3rd party components and code.

UserQuotes relies on Drupal to do all user/group management, combined with a custom set of modules for handling the quote items and sets, and then some drupal-free helper classes that were written to be drupal-independent so that they could be run without needing drupal.  Using Drupal no doubt saved considerable time in building an infrastructure for users and groups, and it's the second project i've done using drupal in this way.  I have *very* conflicted mixed opinions about Drupal, which i have vented about here.  My bottom line at this point, however, is that Drupal is probably a bad choice for projects like this that aren't really CMS-like at their core.  I find drupal completely unintuitive and extremely frustrating.  Their efforts at building a developer-friendly codebase is commendable, and Drupal does in fact have one of the very best interfaces to let developers write custom clean independent modules.  But trying to use Drupal as a kind of foundational framework for building something that isnt mostly a CMS is just too frustrating I think.

Webcam Video Diary grows out of existing code i've been writing to add video recording features to my Flipbook Printer application.  It relies heavily on a commercial video capture library to do the recording.  The main struggle I had with this program was dealing with the video capture libraries and the idiosyncrasies and options involved in video capture and compression.  There are just so many different things to tweak and configure, and options that affect audio sync, compression rates, etc.  I find such things, and working with any kind of hardware, exhausting and frequently frustrating.  Really makes me miss the days of straightforward platform-neutral code.

   * Were any clever design principles used?
   * Or any really hairy algorithms that you'd like to boast about?
   * What was the trickiest part?

I ended up implementing a bunch of code that does some hairy local proxy caching (actually on multiple levels) of data on UserQuotes, so that people could embed quotes on their pages in an efficient manner that wouldn't overload the main userquotes server even with huge numbers of page requests.  Unfortunately i have proven to myself once again that the more complex things get the harder they are to maintain and explain and configure, and the more things can go wrong.. so i'm currently trying to work out in my mind how much effort should be spend on this project.

   * Would you like to make a mention of any other DC members who helped out?

JavaJones worked closely with me on UserQuotes and it was a real pleasure to work with him.  The automatic thumbnail capturing functions are only working because dc server admin Gothi[c] was nice enough to spend some real time getting the code modified to run on the new server.