Welcome Guest.   Make a donation to an author on the site September 23, 2014, 11:27:11 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the full one-year retrospective report on DonationCoder.com.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1]   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Server Slow/Load Issue - Jan 22, 2011 - search temporarily offline  (Read 9445 times)
mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: January 22, 2011, 04:33:05 PM »

search is temporarily offline while we investigate.
Logged
mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #1 on: January 23, 2011, 04:49:05 PM »

So far it's looking like Gothi[c] and I may have identified and remedied the problem:

Looking through that access logs (shout out to my Web Log Filter utility for making it a little easier), it seems like the server slowdown may have been caused by a web spider from russia making huge numbers of calls to the search function, which puts pretty heavy demands on the cpu and memory.

In fairness, the fault may have been mostly ours -- when we moved servers, we changed the forum urls from /Forum/bb to the simpler /forum but we didn't update the robots.txt file which tells robot spiders not to try to spider things like the search page.  It's *possible* that this is why the spider was trying to do such massive spidering of search result pages.

This is our working hypothesis -- and we will reopen the search functionality to the public on monday.
Logged
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,170



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: January 24, 2011, 07:12:27 AM »

Search seems quite fast today. I searched on "align" which produced 7 pages of results in a timely manner.
Logged

Chris
kyrathaba
N.A.N.Y. Organizer
Honorary Member
**
Posts: 3,010



while(! dead_horse){beat}

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: January 24, 2011, 11:07:10 AM »

I've been unable to connect to my DC FTP account since 1/21.  Tried it again a few minutes ago.  Is this related to the server problem?  Not a problem, since I can back up my files to my other site.  Just curious.
Logged

Win 7 Home Premium 64bit-SP1 AMD Athlon II X2 220 Socket AM3 (938) @ 2.1GHz 6GB RAM Firefox 26.0
_________________________________________________________________________________________

I'm fighting against patent trolls. Join me and tell your representative to support the #SHIELDAct: https://eff.org/r.b6JJ /via @EFF

My DC page: http://kyrathaba.dcmembers.com | My blog: http://williambryanmiller.com/ | Proofreading Service: http://bit.ly/1fQSqQP

worstje
Honorary Member
**
Posts: 555



The Gent with the White Hat

View Profile Give some DonationCredits to this forum member
« Reply #4 on: January 24, 2011, 11:19:57 AM »

I've been unable to connect to my DC FTP account since 1/21.  Tried it again a few minutes ago.  Is this related to the server problem?  Not a problem, since I can back up my files to my other site.  Just curious.

See: moving DCMembers.com to the new server
Logged
kyrathaba
N.A.N.Y. Organizer
Honorary Member
**
Posts: 3,010



while(! dead_horse){beat}

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: January 24, 2011, 11:21:56 AM »

Sorry, the link gives me "An Error Has Occurred!"  I take it we're in the process of migrating user accounts to the new server, and may have a few days unavailability?
Logged

Win 7 Home Premium 64bit-SP1 AMD Athlon II X2 220 Socket AM3 (938) @ 2.1GHz 6GB RAM Firefox 26.0
_________________________________________________________________________________________

I'm fighting against patent trolls. Join me and tell your representative to support the #SHIELDAct: https://eff.org/r.b6JJ /via @EFF

My DC page: http://kyrathaba.dcmembers.com | My blog: http://williambryanmiller.com/ | Proofreading Service: http://bit.ly/1fQSqQP

worstje
Honorary Member
**
Posts: 555



The Gent with the White Hat

View Profile Give some DonationCredits to this forum member
« Reply #6 on: January 24, 2011, 11:34:28 AM »

Yeah. Website and such will stay up, but to my knowledge FTP is turned off and SQL if you use that might or might not end up losing some data incurred between the time of backups and the sorts. Or so I understood from Gothi[c] in irc when I asked.
Logged
kyrathaba
N.A.N.Y. Organizer
Honorary Member
**
Posts: 3,010



while(! dead_horse){beat}

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #7 on: January 24, 2011, 11:38:31 AM »

No problem.  Thanks for the info, worstje!
Logged

Win 7 Home Premium 64bit-SP1 AMD Athlon II X2 220 Socket AM3 (938) @ 2.1GHz 6GB RAM Firefox 26.0
_________________________________________________________________________________________

I'm fighting against patent trolls. Join me and tell your representative to support the #SHIELDAct: https://eff.org/r.b6JJ /via @EFF

My DC page: http://kyrathaba.dcmembers.com | My blog: http://williambryanmiller.com/ | Proofreading Service: http://bit.ly/1fQSqQP

mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #8 on: January 25, 2011, 07:45:00 PM »

Just a heads up, server may be down briefly in the next 24 hours as we install a cache library and activate some additional code to adapt to server load; should make forum more robust and peppier smiley
Logged
mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: January 26, 2011, 03:53:58 PM »

ok server is going to go down briefly today while we put in some new code.
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: January 27, 2011, 11:41:43 AM »

Just a heads up, server may be down briefly in the next 24 hours as we install a cache library and activate some additional code to adapt to server load; should make forum more robust and peppier smiley
memcached?
Logged

- carpe noctem
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 855



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #11 on: January 27, 2011, 12:02:03 PM »

Quote
memcached?

APC
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: January 27, 2011, 12:05:18 PM »

Ah, PHP precompilation - I thought database result caching would result in larger performance wins? (Harder to implement, though smiley)
Logged

- carpe noctem
mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: January 27, 2011, 01:48:05 PM »

Quote
I thought database result caching would result in larger performance wins

In addition to PHP opcode cahcing, smf also has some routines for data content caching, which is also happening through APC.
Logged
Gothi[c]
DC Server Admin
Charter Honorary Member
***
Posts: 855



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #14 on: January 27, 2011, 08:26:34 PM »

And, on top of that mysql also caches query results.
Logged
mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: January 27, 2011, 08:29:46 PM »

And on top of that Gothic built a little desk and console terminal for Cody to sit at and monitor raw tcp traffic to the server -- when he sees a request come in that he recognizes, he can intercept it and peck out a memorized reply himself.  I don't know if that will lead to faster results, but it keeps Cody busy.
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: January 27, 2011, 08:30:27 PM »

Oh, the Cody detail is especially wonderful smiley
Logged

- carpe noctem
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,170



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: January 27, 2011, 09:37:31 PM »

And on top of that Gothic built a little desk and console terminal for Cody to sit at

And on top of that, I hope, you will place the miniature mouser mug that you ordered for him!
Logged

Chris
Ath
Supporting Member
**
Posts: 2,217



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #18 on: January 30, 2011, 09:44:23 AM »

Does this new caching stuff have impact on the refresh/update of the main forum page, at the bottom where the last 10 updated threads are shown? Now if I make a post, it takes a little while and a few screen refreshes for it to show up there ohmy
Logged

Ath
Supporting Member
**
Posts: 2,217



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #19 on: February 03, 2011, 01:57:27 PM »

And somewhere about 1/2 hour ago, SMF couldn't reach the database, so here I'm back a few minutes later... huh
Logged

mouser
First Author
Administrator
*****
Posts: 33,386



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: February 03, 2011, 01:58:48 PM »

yep, was offline for a few minutes while we were backing things up.  should be ok now.
Logged
Ath
Supporting Member
**
Posts: 2,217



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #21 on: February 03, 2011, 02:23:55 PM »

Ok, is there a time-window where backups are being made, so I can avoid hitting the barriers? Wink
Logged

Pages: [1]   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.039s | Server load: 0.07 ]