New article posted about the DC Remodeling in 2018 -- server changes, cms, etc.

[mouser]

Here it is, as promised.  You can read my new article on the major DC server remodeling here: https://www.donation...frastructure-remodel

I'm not sure it's of much interest to anyone but I thought I would write everything down before I forgot it.

It has a little bit of everything: Some DC history, discussion of the options we struggled with, why certain decisions were made, some lessons learned, etc.

https://www.donation...frastructure-remodel

[rgdot]

Starting, be back in a few days 

Seriously though, thanks, will read with interest 

[wraith808]

I'm currently with LiquidWeb, and have been thinking that though I like them, I should make sure there's not a better option.  Why did you go with HiVelocity rather than LiquidWeb?  Was it just the specs for the price?

And I will say that LiquidWeb has crept up in price (though they've given incrementally more for the price).  But their new cloud servers are a very good value as I'm not going with dedicated.  The fact that you can provision them, set them up, do what you need to do, then tear them down has worked very well for me.  It's a lot like EC2, but with someone backing you up (though I still use EC2 as it has better front end tools for if I know what I'm doing at the time).

[mouser]

Why did you go with HiVelocity rather than LiquidWeb?  Was it just the specs for the price?

yes.

Note that HiVelocity does not do shared hosting, only dedicated servers (and higher).

[wraith808]

Why did you go with HiVelocity rather than LiquidWeb?  Was it just the specs for the price?

yes.

Note that HiVelocity does not do shared hosting, only dedicated servers (and higher).

-mouser (March 28, 2018, 04:14 PM)

Yeah, I noticed that after the fact... I used to have a dedicated server.  But after I got hit by ransomware in December because of a flaw in the RDP implementation, I dropped down to my one linux server, getting rid of both of my Windows servers.

[IainB]

@mouser: Thanks for a very interesting article.   
The potted history brought things into context and focus for me, as I was unaware of that background.
Good point about "practicing a migration" - it has to work, so there's every good reason for taking a rigorous and risk-averse approach, which ultimately could improve one's knowledge/understanding of the assets and their configuration/performance, in the acts of planning for and migrating the installation. Migration has never been an "easy" project.
Also, the experiential advice to take notes.

[Nod5]

Interesting read, thanks mouser. The Wordpress system at dcmembers is very easy to use. Much quicker to manage now from a users POV compared to the old way of .html edits and FTP uploading.

[mouser]

Thanks for saying so, Nod5   

[TWmailrec]

To: Mouser
From: TWmailrec, Charter Member, joined in 2005:
I have been away for the last year and just noticed that DC has changed server!
I used to use my own web page, address was http://twm.dcmembers.com/ which has now changed.

In order to add recent programs/update old ones, is it possible to email me with Filezilla settings which were:
Host:        ftp.dcwing.com
Protocol:    FTP - File Transfer Protocol
Encryption: Require explicit FTP over TLS
Logon Type: normal               -  user: twm
   Password:   presume this the same?
Server type: Unix
Default Remot Directory: /twm.dcmembers.com

I still have same email address
Many thanks, Tim

[mouser]

I've sent you a pm, contact me and we can get your old stuff set up.

[Chessnia]

Ok, the following are just my honest thoughts. At the end of the day what really matters is that your vision is becoming reality the way you want. I love what you do here and your programs are amazing and so are the people who contribute. I'm only telling you what I think not because I want people to think less of you, but because I wish those financial resources were spent on making donationcoder even better than it already is in a different way.

I was totally shocked to learn that you're paying 400$ a month for hosting services (I had to read it several times, even 400$  a year would look very expensive to me!).
Total overkill. The way I see it, . License key scripts take only a few seconds of cpu power (And I'm guessing you're not getting a new one every 5 minutes),  a forum with less than 10.000 active members + CMS and a few extras here and there, shouldn't cost more than 300 a year. Sure, shared hosting, it works fine. I don't think fpt hosting is needed on your site, the coders don't mind hosting their files anywhere else (all the links would live here).
I ran a forum with 9500 members, online chat (which eats lots and lots of cpu) and online arcade running fine for years, paying 200 a year. 
In my opinion, you've created a paralell universe in your mind where you think you need a dedicated server + maintenance, etc. but to an ordinary admin, it looks totally unnecessary given your current needs. At the end of the day, members want to see : life, movement, activity. If the page loads in 0.10 instead of 0.03, cool, but it is meaningless.

By the way, I think donationcoder's skin looks terribly out of date. It feels like visiting a 199x forum. I love it just the same, but I'm just saying...

[mouser]

Your points are well taken.  It's fair to say we are spending more than we have to.

A big part of the expense comes from running two servers instead of one.  All of DonationCoder.com runs on one dedicated server, and so really the cost for that is $200 a month not $400.  We have two servers so that we can run the member server on an isolated machine to reduce security risks associated with that.  And the member server is way overpowered for what we use it for currently; one hope is that it allows us to try other experiments in the near future.

I do disagree with you about the need to pay for server management service and cpanel license;  I've tried doing the server management before or having a volunteer do it -- it's just not practical for us.  We need to have a professional person we can call on to update server software regularly.

As for a shared hosting -- we tried that as well way back in the day, and it was an uncomfortable experience where we would have mysterious slowdowns of the server and forum.

Having said that, we are certainly on a server that is more powerful than our current needs, even as we do 600 gb of bandwidth per month. 

In selecting servers for this major move, and moving to a proper CMS, I was focused on reducing the number of ways things could go wrong to a bare minimum.  So yes, I erred on the side of having an over-powered server rather than an under-powered one, especially since it was going to be hard to predict the additional load of running the website on Joomla.  After a year or two of experience running the new server system, we may re-evaluate our needs and try to downscale to a less powerful server -- especially now that we have spent so much time ensuring that moving servers will not be so painful.

I should also note that everyone has different areas where they experience stress more acutely than others.  For me, server hardware slowdowns was an area that caused real grief and anxiety, and was something I never had the skills to solve, and which caused an extreme amount of frustration.  So my inclination ever since the early days of DC was to put enough money into server costs to comfortably eliminate this problem.  Spending the extra money on a superfast, managed dedicated server and offsite backup system is not fun, but it does help me sleep better at night.  And it's a great warm sense of comfort to me to know that whatever we throw at the new server it can handle.

[wraith808]

It is a lot of money, and it is possible to cut costs, but I think the current setup takes into account the future of the site more than cutting costs would.  My reasoning from my own experience is below.

• Shared hosting puts you at the mercy of the other clients on the machine, and overselling by the host.
• It is better to have too much power and too much memory than too little.  Mine is less by quite a bit (~$1200 a year at this point), and I run into problems if I attempt to use simple modern technologies they tend to bog down.  An example is when I attempted to install Ghost, the server bogged down from the use of memory by node.js.
• The policy of the site is to try to host as much of the site as possible- including developer downloads and images.  This is so that parts of the site do not become inaccessible based on outside problems, i.e. the other sites being down or the developer becoming unable to keep the files available.  This increases not only storage requirements, but bandwidth requirements.

Because of my own experiences with clients and my own sites and the frustrations associated with server administration, I totally get where mouser's head is, and where he's coming from.  "You get what you pay for" is definitely true in regards to hosts, and though something may not go wrong with your servers, when it does, it's a real pain and having that support from people that know what they're doing and you can actually talk to rather than just e-mailing is a great boon. 

If the money isn't there, that's a different problem, and loss of extra business is one of the reasons I'm on a VM now instead of a dedicated server.  But if I had the extra funds, you can bet I'd be upgrading to a dedicated server again (though in all honesty, the VM isn't bad).

[rgdot]

Unlike my home setting experience disk space is more expensive than bandwidth and other things in the hosting world. Of course hard drives used on servers are not the same as those for personal use but over my time I have shopped for hosting countless times and one can get the best processors, bandwidth but suddenly over 100GB disk space (or 500GB for example for dedicated servers) the price climbs rapidly. I suspect hosting all software, images, videos here on DC servers is itself requiring more cash. Several examples I have come across are in hosting forum sites. The 'not expensive' processor and RAM options can easily handle the number of connections and users who are simultaneously viewing the site and demanding resources...but as the forum accumulates GBs of posts and content the costs rise faster than one might think.

[wraith808]

That's a good point.  I know that there was some similar kerfluffle over the ChattyPics image hosting service, which is the image hosting site for a lot of people that use shacknews.com.  As the usage of the site increased, it changed from on person donating hosting to untenable really quickly.  They're up to 100GB and with dreamhost, the hosting bill was $45 per month- just to store images using a simple front end.  This came about also, because DreamHost realized that even though it wasn't commercial, it was being used for commercial purposes, so there is that also.

From the original post there:

Chattypics.com News: My host, Dreamhost, is trying to get me to use their cloud service, called DreamObjects, because chattypics.com violates their Acceptable Use policy. Unsurprisingly, unlimited storage is not actually unlimited. At the moment, chattypics.com has 98GB worth of photos on disk. I calcualte that, if I were to use DreamObjects, based on their pricing structure, that I'd had to pay approximately an additional $8/mo for the service (this takes into account other domains that I have going and takes up storage) and which will slowly increase in price as more images are uploaded over time. Chattypics.com grows by 20GB each year, at its current rate. DreamObjects costs 2.5c per GB for storage, and 5c per GB for downloads.

Unfortunately, I can't afford the extra expense or the figuring out of how CDN/cloud storage works.

More info and a snippet of the email Dreamhost sent me in reply.