Welcome Guest.   Make a donation to an author on the site April 24, 2014, 07:13:04 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Check out and download the GOE 2007 Freeware Challenge productivity tools.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1]   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: August 26 had some slight downtime -- thanks to gothic for fixing it  (Read 11378 times)
mouser
First Author
Administrator
*****
Posts: 32,701



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: August 26, 2008, 01:39:38 PM »

Just a quick note that we had a few hours of downtime this morning, and a big thanks to Gothi[c] for waking up out of his sleep just to fix the problem.  It reminds me how lucky we are to have him taking care of, looking out for, and fixing our server when there are problems.

Thank you Gothi[c] (John).

ps.
I think i will try to set up a very basic backup forum on our dcmembers.com server so we can post when such problems occur and just serve as a place for people to chat if the main server goes down.
Logged
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 855



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #1 on: August 26, 2008, 01:41:33 PM »

 smiley thanks mousie.
Logged
mouser
First Author
Administrator
*****
Posts: 32,701



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: August 26, 2008, 01:42:57 PM »

I mean it, one of the best things that happend to this site is when you came around and volunteered to take care of these issues.  I know it can be really frustrating work and I want you to know how much we all appreciate it.
Logged
TucknDar
Charter Member
***
Posts: 1,093


Advanced coder of Nowt

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: August 26, 2008, 01:46:47 PM »

So wasn't just me then smiley

Great work, Gothi[c]
Logged
Armando
Charter Member
***
Posts: 2,666



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #4 on: August 26, 2008, 01:54:34 PM »

Thanks for your hard work guys.  Kiss
Logged

"I suppose it can be said that I'm an absent-minded driver. It's true that I've driven through a number of red lights on occasion, but on the other hand, I've stopped at a lot of green ones but never gotten credit for it."
Glenn Gould
nudone
Cody's Creator
Columnist
***
Posts: 4,116



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: August 26, 2008, 03:01:33 PM »

yes, many thanks again to Gothic.

a backup server/forum sounds like a nice idea when troubles happen.
Logged
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,158



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #6 on: August 26, 2008, 04:32:23 PM »

a big thanks to Gothi[c] for waking up out of his sleep just to fix the problem

That's some dedication there, Gothic, thank you very much!


Logged

Chris
Shades
Member
**
Posts: 1,553


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #7 on: August 26, 2008, 07:17:20 PM »

a big thanks to Gothi[c] for waking up out of his sleep just to fix the problem

That's some dedication there, Gothic, thank you very much!


How so?  I woke up too.... Wink


All jokes aside, cudos to Gothi[c]!!!! Thmbsup
Logged
jgpaiva
Global Moderator
*****
Posts: 4,710



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #8 on: August 26, 2008, 08:07:39 PM »

It's good to know that the DC server is on very good hands!
Thank you for your efforts, Gothi[c]!

Also, how do you know the DC server is well treated? The DC server goes down, and mouser doesn't hiperventilate on the irc channel.
The peace of mind warranted by having a great sysadmin is definitelly priceless cheesy
Logged

PhilB66
Supporting Member
**
Posts: 1,510


View Profile Give some DonationCredits to this forum member
« Reply #9 on: August 26, 2008, 08:29:49 PM »

How so?  I woke up too....

All jokes aside, cudos to Gothi[c]!!!! Thmbsup

Wake up man, it's kudos  cheesy
Logged
Shades
Member
**
Posts: 1,553


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #10 on: August 26, 2008, 09:06:31 PM »

Sorry, am from the "Give that man a cigar!" generation... Wink

Hmmm, is Cudos not a brand of cigars?  Grin
Logged
mouser
First Author
Administrator
*****
Posts: 32,701



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #11 on: August 26, 2008, 09:39:19 PM »

Quote
Also, how do you know the DC server is well treated? The DC server goes down, and mouser doesn't hiperventilate on the irc channel.

only the people who were online on irc a couple of years ago when the server had it's first serious outage can know how big a deal this is.  i felt like i was going to have a heart attack i was so panicked and frustrated by the server being down and rushing around trying to figure out what to do.

a little part of my new calmness is just getting used to the fact that servers sometimes have downtime and you don't need to get so worked up, but mostly its the fact that i know John is on the case and will make sure the server is fixed after it has a problem.  it has saved my sanity.
Logged
Josh
Charter Honorary Member
***
Posts: 3,298



View Profile Give some DonationCredits to this forum member
« Reply #12 on: August 27, 2008, 01:14:02 AM »

OK Folks, we now have an official explanation for the problem this morning. If you are a regular of the donationcoder.com IRC channel, you will know that our very own mouser has a problem with air conditioners. Well, today, we have discovered that after a night out on the town, mousey came back to his apartment and mistook his air conditioner for a comode. As such, the air conditioner shorted out. Since mouser runs donationcoder.com off of his 386 running windows nt 3.51 and an aol 2.0 connection, the system overheated due to improper ventilation. This has been resolved, but who knows when we will have another issue with air conditioning. Given mouser's extensive problematic history with air conditioners, this reporter says "Nope".

</comedic satire>
Logged

Strength in Knowledge
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 855



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #13 on: August 27, 2008, 07:08:30 AM »

might still be bumpy today, i'm installing a new kernel today.
Logged
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,158



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: August 27, 2008, 07:37:30 AM »

the air conditioner shorted out.

What's passed is past, it's water under the southbridge.
Logged

Chris
Darwin
Charter Member
***
Posts: 6,979



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: August 27, 2008, 08:27:18 AM »

Many, many thanks, Gothi[c]  Thmbsup I was going a bit crazy yesterday morning wondering why DC wouldn't load!
Logged

"Some people have a way with words, other people,... oh... have not way" - Steve Martin
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: August 27, 2008, 06:05:25 PM »

So, what caused the glitch, pray tell?
Logged

- carpe noctem
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 855



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #17 on: August 27, 2008, 06:45:07 PM »

Quote
So, what caused the glitch, pray tell?

Ok, here goes a quick one-paragraph un-formatted attempt of explaining:

Well, the conntrack table was getting full, which is normal and common since it's max is rather retardedly low in centos. But increasing the max didn't seem to help. Upon further inspection it seems that some never cleared out of the table. Lowering the conntrack timeout settings didn't help either. I found some references online to an old kernel bug causing connections to not be cleared from there, though I'm not sure if that was the actual cause. It's likely though, since I was heavily monitoring incoming connections and at no point did we get excessively many. So at first to resolve the situation, I figured I'd just unload the ip_conntrack modules and it's depending modules. The only problem is, once loaded, lots of modules depend on it, so I ended up disabling too much of iptable's functionality, causing some weird fail weirdness in the firewall. Then today I actually broke down and recompiled the kernel without connection tracking support, which seems to have fixed it for now. I was hoping for a quick fix by unloading the module, but in the end recompiling the kernel without connection tracking was a quicker fix - typical. Annoyingly, the server went down, like 3 days in a row, every time I go to bed, and sleep for like 1 hour or so. It's like the server waits for the perfect time, stalks me, and then goes down at the worst moment just to bug me cheesy -- So when I recompiled the kernel, I also just went and grabbed the latest vanilla from kernel.org while I was at it, so to fix any old lurking bugs. So far so good, lets see if it borks out again tomorrow or not smiley
« Last Edit: August 27, 2008, 06:46:46 PM by Gothi[c] » Logged
lanux128
Global Moderator
*****
Posts: 6,049



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: August 27, 2008, 09:24:51 PM »

just caught this thread. good work, Gothi[c]! i actually didn't notice the downtime, i guess that's advantage of living in a different time-zone. smiley
Logged

f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #19 on: August 28, 2008, 12:22:59 AM »

Hm, connection tracking... that's a iptables/firewall feature, right? Sounds nasty if such a crucial part of the system is bugged :/
Logged

- carpe noctem
Deozaan
Charter Member
***
Posts: 6,090



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: August 31, 2008, 07:00:18 PM »

Thanks Gothic!
Logged

Pages: [1]   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.057s | Server load: 0.27 ]