Author Topic: DonationCoder Major Upgrades - Progress Report Thread (Read 126862 times)

mouser · « **on:** July 10, 2015, 12:00 PM »

Ok I'm going to use this thread to document progress on some major DonationCoder upgrades that are long overdue.
If you're not interested in reading insider useless progress report notes for the next month, please ignore this thread, otherwise read on.

Inspired by the success of the last fundraiser, I've decided that the time has finally come to bite the bullet and spend a couple of months modernizing the donationcoder website and forum.

At first I wrote a giant manifesto on a revolutionary new approach to DonationCoder, including a long harsh critique of the current system, and showed it to the DonationCoder moderators. Everyone who read it told me it was a depressing read and a grossly impractical proposal. So that idea was dumped.

SO.. Instead, we are going to simply focus for now on sustainability, by finally moving DonationCoder to a proper cms and upgrading our forum system from an older and *highly* customized/kludged version of smf1.x which has become hard to keep updated.

This first stage of upgrades involve a large amount of behind the scenes work, will take many weeks, and will result in little or no outward improvements -- in fact it's virtually guaranteed that things will get occasionally worse and buggy during the process until the kinks are all worked out. Some of our non-critical features may be removed from the forum during the process -- probably just temporarily.

So the current goal is not to bring new features or fancy looks to the site -- but rather to clean up cruft and make the site much easier to maintain, moderate, and update. After we achieve that we can talk about adding stuff.

IMPORTANT:
No changes are being made on the CURRENT LIVE site -- so you're not going to see any changes here for a while!
Instead I will be creating a mirror site and working on that. When the time comes to move everything -- we will simply shut down the current forum and move all forum messages and then turn on the new one.
No forum posts, attachments, or user accounts will be lost.

rgdot · « **Reply #1 on:** July 10, 2015, 12:08 PM »

If there is something I can test, help, etc let me know

mouser · « **Reply #2 on:** July 10, 2015, 12:14 PM »

Some additional details on the upgrades:

Those who know me know that I tend to spend inordinate amounts of time weighing options before i pull the trigger on large projects. This vacillation and hesitation can be extremely unproductive. Part of what has hung up the modernization of DC has been a struggle to decide what direction to go in.

In the end I/we decided that what was best for the community that hangs out here, at least at this point, is to move to the Drupal CMS, and upgrade from our highly customized SMF 1.x to the latest version of the SMF 2.0 forum (and minimizing any customization that makes maintenance/upgrading too difficult).

There was quite a bit of discussion regarding moving to a more modern and actively developed forum system -- but in the end several things seemed to weigh against that: First, this forum community seems to be comfortable with a more traditional system; secondly, if we did eventually want to move to a more modern forum like Discourse, it would still be helpful for us to first go from a highly-customized version of smf1 to a more stock smf2 before doing that.

And as for using Drupal as the CMS.. I have a long love/hate relationship with Drupal. I had really hoped to move DC to a completely customized new kind of cms system that I have been working on part time (Mewlo), but waiting for Mewlo to reach a usable stage is not practical. Drupal represents a conservative stable choice.

mouser · « **Reply #3 on:** July 10, 2015, 12:17 PM »

If there is something I can test, help, etc let me know

A major additional motivation for finally moving the website to a proper cms, is to be able to have additional moderators and content creators who can help maintain pages and add new content. So I am hoping that when it's all set up we can finally revisit the idea of having some regular DC folks add more new content to the website on a regular basis. This might take the forum of having a bunch of dc regulars writing weekly columns like we tried in the early days of DC, etc.

Also: Converting the hundreds of static html pages into CMS pages is going to be a huge amount of work. After I get the basic infrastructure and process set up, I would love to have volunteers help in the process. It won't be particularly difficult work, mostly a matter of copy and paste then fixing links and formatting.

rgdot · « **Reply #4 on:** July 10, 2015, 12:51 PM »

A bit offtopic: I have experience in moving from a cms to WordPress, non static of course and good amount of headaches due to db collation among other issues.

Unless something comes up, I should be available for some copy paste as well.

mouser · « **Reply #5 on:** July 10, 2015, 03:52 PM »

If we have any skilled web dev people here, there is one job that could be "volunteer-outsourced" right now -- creating a custom smf2 theme to approximate the current donationcoder forum look..
It could be done totally independently from what I'm working on -- you would just install the current version of smf2 on a local machine and then make a new theme based off the default on, but trying to capture the basic feel of the current donationcoder theme.
Any takers?

mouser · « **Reply #6 on:** July 13, 2015, 09:53 AM »

Update report:

I have been working hard on porting over and cleaning up the forum customizations and tweaks to smf2.. My hope had been to be able to jettison them all in favor of new core features in smf2, but that has not turned out to be feasible -- we still have some custom stuff that is needed (custom spam handling/reporting/etc).

However, the cleaning process should pay dividends eventually -- after 10 years there is a lot of cruft to get rid of.

Now, originally my thinking was that we might change hosts and move to the a new cms and upgrade to the new smf2 forum all at once, but i'm thinking now we might try to reduce the confusion by first upgrading our forum in place, without messing with CMS or host changing. That would reduce the possible sources of problems and let me work out the kinks on the new forum more easily.

So, I'm hoping we might do that sometime in August if all goes well...

ewemoa · « **Reply #7 on:** July 13, 2015, 10:24 PM »

Is starting with something pretty vanilla and gradually adding features when enough clamor is raised infeasible? Or alternatively, classifying things to be added into two groups -- one group for must-have up-front and the other for things to possibly added down the line...

May be we'd find we could live with much less -- so less work for you all at once and perhaps less maintenance burden going forward?

Perhaps this is what you had in mind already...

mouser · « **Reply #8 on:** July 13, 2015, 10:39 PM »

I'm working my way through the must-have stuff.. Have been having some long days plodding through slowly, but also having some occasional fun.

I'm making good steady progress each day.

I don't want us to take too big a step backwards, so I'm going to try to get 95% of the forum features working as before, and then go live -- then we can work out the kinks and see if that 5% is still needed, and see what other new stuff we might want.

There are a couple of things that were really messy in code which we would probably be smart to jettison: The programming School and the "Personal Area" where people could post drafts. The spaghetti code that was necessary to implement those is probably not worth the maintenance hassle.

cranioscopical · « **Reply #9 on:** July 13, 2015, 10:58 PM »

It's a pretty safe bet that, going forward, not everybody will find everything to their liking as things change. We must be prepared to let go of some current conveniences if and when they become the biggest obstacles to progress. If maintaining 5% of something is accounting for 45% of your effort that will make a persuasive argument for dropping it. A new system can't hope to be a winner in every category, it just has to offer a set of benefits that significantly improves on today's set.

app103 · « **Reply #10 on:** July 14, 2015, 06:09 AM »

From what I gather, the loss of the Programming School may only be temporary, until everything has been upgraded and CMS put in place, then it could be added back on (later) in a much better way, that could make it even more useful, perhaps even more structured, with actual lessons and not just assignments?

mouser · « **Reply #11 on:** July 14, 2015, 09:00 AM »

The programming school pre-dates DonationCoder by several years -- it was the first forum I made, and I have a very soft spot in my heart for it.
It was created at a time where there were no other real alternatives -- no good places to get self-teaching assignments for a variety of languages.

Now of course there are tons of such options, massively funded dedicated online learning websites with large support staffs, etc.

So, I think the programming school is probably best handled by others, with us maintaining a great page of information on what sites we would recommend and why, etc. The new cms will make it easier for us to have such pages.

Alternatively, we could create a new self-teaching programming school of our own, on a sister website using a cms designed for online learning, or on a separate area of our new cms.

But the bottom line is that i think it's probably time to stop having the programming school exist as dozens of private little sub-forums of the dc forum.

mouser · « **Reply #12 on:** July 14, 2015, 09:13 PM »

Forum modification progress so far:

3RD PARTY MODS INSTALLED:

ILA attachment (customized)
http://custom.simple...s/index.php?mod=3770

HS4SMF (highslide image viewer) (customized and integrated with ILA)
https://github.com/Spuds/SMF-HS4SMF
http://www.simplemac...x.php?topic=379200.0

Installed Yet Another Youtube mod -- takes care of our youtube stuff
http://custom.simple...s/index.php?mod=3982

nobbc button (replace with our own?); good example of standalone hook mod
http://custom.simple...s/index.php?mod=3755

spoiler:
http://custom.simple...s/index.php?mod=2940
GOOD example of hook no-modification mod

Mouser mods:

+mouser's suspicious user post reporting
+mouser's profile change reporting
+mouser's emailing of all posts and changes
+mouser's email admin on forum error
+mouser's ability to delete posts while banning, and banning improvements
+mouser's show website url and text on profile summary page
+mouser's hide untrusted user spammable profile fields from profile and message display page (signature, website, personal text)
+mouser's preset text mod
+mouser's yearbook quote trap -- handled with new profile fields builtin feature
+mouser's blogging system

+mouser's simple comment hiding
+mouser's member search in dropdown (added to search dropdown mod)

+mouser's custom bbcode buttons with js functionality
+mouser's custom LI button
+mouser's custom IMG button
+mouser's custom URL button

+mouser's custom image/smileys popup

JavaJones · « **Reply #13 on:** July 15, 2015, 06:47 PM »

I guess it's probably too late now but for whatever it's worth I have a couple of anti-spam mods installed on 2 fairly active forums I run and they take care of 99% of spam. We very rarely have a problem with an actual spam post getting through. The most I have to do is check and approve (or delete) a couple of questionable registrations once a week. On DC it'd probably be more like every couple days with the volume, but still the actual work is minimal and done mostly by the plugins.

- Oshyan

mouser · « **Reply #14 on:** July 15, 2015, 07:32 PM »

It's not too late at all. I have implemented my own custom spam reporting and related modifications (which are mainly focused on DETECTING AND ALERTING human moderators about suspicious posts and profile changes rather than trying to stop it outright) but I'll be looking into additional 3rd party spam mods.

Let me know which ones you use.

I spent today porting the badges mod, so everyone will be able to keep their forum "flare".

mouser · « **Reply #15 on:** July 16, 2015, 02:00 AM »

Another full day of work, from wake to sleep, working on the upgrade. I'm in the groove

Everything is progressing steadily, and I can't imagine anything getting in the way of the new forum going live in August.

It bears repeating that people shouldn't expect dramatically wonderful new features -- it's going to look and feel mostly the same.

The main point of all this work is to make everything easier to maintain and upgrade going forward for the next 10 years.

Having said that, it's clear that one benefit of all this work is that I will feel much better about adding some new features, now that I have a better hang of what is easy vs hard to maintain and what changes make upgrading more difficult.

I'm getting excited about unveiling the new forum.

tomos · « **Reply #16 on:** July 16, 2015, 02:17 AM »

Another full day of work, from wake to sleep, working on the upgrade. I'm in the groove
Everything is progressing steadily, and I can't imagine anything getting in the way of the new forum going live in August.

It bears repeating that people shouldn't expect dramatically wonderful new features -- it's going to look and feel mostly the same.

The main point of all this work is to make everything easier to maintain and upgrade going forward for the next 10 years.

Having said that, it's clear that one benefit of all this work is that I will feel much better about adding some new features, now that I have a better hang of what is easy vs hard to maintain and what changes make upgrading more difficult.

I'm getting excited about unveiling the new forum.
-mouser (July 16, 2015, 02:00 AM)

great

mouser · « **Reply #17 on:** July 16, 2015, 08:52 PM »

I'm coding a few slick new features.. How about a (sortable) page where you can see all the topics started by (or alternatively, participated in by) a given member? I'm having fun!
The icing on the cake is that you can also filter by whether the topic is locked and/or sticky, and by # of replies (useful for seeing unreplied or ultra-popular topics, etc.)
Screenshot - 7_16_2015 , 10_44_18 PM.png

Screenshot - 7_16_2015 , 10_44_18 PM.png

Deozaan · « **Reply #18 on:** July 16, 2015, 10:47 PM »

Very cool!

tomos · « **Reply #19 on:** July 17, 2015, 02:31 AM »

ge where you can see all the topics started by (or alternatively, participated in by) a given member?
-mouser (July 16, 2015, 08:52 PM)

that'll be very helpful

(visuals look nice too)

Tuxman · « **Reply #20 on:** July 17, 2015, 03:40 AM »

Drupal sucks. From a security perspective as well as from a UX perspective. But that might be just me.

Regarding the choice of the "new" forum system: I - as a user - am glad that DC sticks to the "old" Curved theme and SMF. There's not much that sucks more with forum upgrades than having a completely different system afterwards. I am afraid that SMF2 (I hope the final relaunch will use the much improved v2.1 which is about to leave the beta phase soon) is - more or less - a dead cow, with the initial release of version 2.0 facing many forks like ElkArte (they even provide a SMF fallback theme) and the heavily customized Wedge and many original developers parting their ways.

Admittedly, it could be worse. Discourse or NodeBB, for example, have really awful user interfaces. Or just another XenForo.

mouser · « **Reply #21 on:** July 17, 2015, 03:45 AM »

I'm having second thoughts about Drupal again too, but I've decided to punt on that for now.

I've got my eye on 2.1 and hopefully the upgrade from 2.0 to 2.1 will be relatively painless when it is suitable for production use. I agree with your concerns about smf but judge it to be the best way forward for us.

You seem to be following these trends pretty seriously -- perhaps i can chew your ear over email occasionally about the issues? ([email protected])

Tuxman · « **Reply #22 on:** July 17, 2015, 03:51 AM »

I'm not really deep into SMF (currently I'm working on some ElkArte modding though), but I keep at least three eyes on board systems.

You got mail.

TaoPhoenix · « **Reply #23 on:** July 17, 2015, 12:36 PM »

I'm coding a few slick new features.. How about a (sortable) page where you can see all the topics ... (or alternatively, participated in by) a given member? I'm having fun!
-mouser (July 16, 2015, 08:52 PM)

For me, I have no interest in who started a topic - topics just appear.

So for me, "participated in by" is the power feature ...

- You can find (and cringe!) at all the things you were involved in

- You can find things other people were involved in, that slipped past that "10 topics per day" dashboard.

JavaJones · « **Reply #24 on:** July 19, 2015, 01:48 PM »

Tuxman, I'm curious to hear your thoughts about Discourse. I "discovered" it some months ago (or perhaps re-discovered) and I have kind of a love/hate relationship with it. I'm not actively running it on any sites mind you, so my experiences are limited. I absolutely *love* some aspects of it (the default design is not one of them), and I really like what their goals are, what they're trying to do. But as I said I do have issues with the design, and there are some other concerns too. So I'd be interested in hearing your perspective, especially since you said you're tinkering with some SMF forks (don't know why I didn't think to look for some before now!). Perhaps this should be split off into its own thread?

- Oshyan