|
52
|
News and Reviews / Official Announcements / Re: DonationCoder.com New Server Migration Coming Soon
|
on: December 16, 2010, 04:54:05 PM
|
Fixed the MySQL slowness... This was a really weird one to debug. When the MySQL sort resorted to a filesort it was actually running FASTER than in memory. (It's supposed to be the other way around). All MySQL and MyISAM buffers etc were set correctly so it was quite a mystery in which nothing much was making sense. It was as if the world was turned upside down, and queries that were supposed to be slow in theory, ran faster than those that were supposed to be fast. It turned out that the problem wasn't with the MySQL configuration at all, but rather an option in the hardened kernel we were running. ( We use a security hardened kernel which prevents many buffer overflow exploits from working through the well known PaX and grsecurity kernel patches )The culprit turned out to be the "prevent invalid userland pointer dereference" PaX option. When running a google query for this issue there's only two results, one of which, an ubuntu hardened guide (pdf), mentions the following: Restrict mprotect → YES if you are running a physical server → NO if it is a virtual one
Prevent invalid userland pointer dereference * If you are running on a physical machine → YES * If it is a virtual Machine it will slow it down a lot → NO
Glad to see that confirmed. It doesn't really explain the theory as to why unfortunately. Too bad google only returned this result after we had already found which option caused the problem after lots of debugging  The MySQL query that was taking over 14 seconds to complete now runs in less than a second.
|
|
|
|
|
54
|
Main Area and Open Discussion / Living Room / Re: Wi-Fi Makes Trees Sick!
|
on: November 24, 2010, 09:58:45 AM
|
The talk here about sponsored research would imply to a reader that the linked article is sponsored research but I see no indication of that. (unless i missed something?) While I agree that PCWorld is not a good source, and they don't link to the original, it seems to me that there wouldn't be a valid reason to simply dismiss research done by a university as 'not scientific'. (again, unless I'm missing something really obvious, in that case, correct me  ) Under the comments for the original pcworld article someone linked a bunch of links to other research done on the subject that may be enlightening: Enough material, right?  I'm not saying wifi kills trees, by the way PCWorld may well be misquoting the research or jumping to conclusions as it's not clear what type of tests they performed, at what frequencies and what power. But dismissing it as non-scientific or sponsored when it's not may upset a few researchers that put a lot of work in the study
|
|
|
|
|
56
|
News and Reviews / Official Announcements / Re: Preparing for a new master DonationCoder server
|
on: November 20, 2010, 12:22:21 AM
|
|
To clarify, our two servers currently have two separate functions (main server and backup server) it's not like one is a backup of the other, though we do have a limited functionality backup forum ready on the member server in case the main server goes down. Our budget is currently not big enough to actually do real HA on two physical servers, or we'd have to get rid of the member server, which wouldn't be very nice either.
|
|
|
|
|
58
|
News and Reviews / Official Announcements / Re: Preparing for a new master DonationCoder server
|
on: November 19, 2010, 07:24:12 PM
|
So, there's a few ways we can go about this.... One suggestion: * main dc server http on a vm (minus mail) * dcwing member server on a vm (minus mail) * vm for main dc email * vm for member server email * vm for newsletter sending * (optional - vm for sql ) * vm for commercial project hosting The main reasoning behind putting the newsletter on it's own vm (and more importantly IP) is so that should we ever unjustly be considered spammers due to the high volume mail of the newsletter and end up on some blacklist, the mail forum registration mails won't stop working (ie end up in people's junk mailboxes). This vm wouldn't have to run all the time, only during newsletter sending time. The reasoning for seperating SQL on a separate server is security and scalability. A compromised sql server won't touch the main server's files. (Same reason why email would be separate), it also allows us to tune the OS setup towards the specific task. (eg, mail would have a larger /var partition for the mail queue, and perhaps a filesystem that performs better with lots of smaller files) Currently we already have MySQL and Apache isolated on the member server using BSD jails. It works but I feel there is some performance hit due to the need for NAT in pf. Having separate vm's would be a much cleaner solution. There's many other ways we could go at this... The mail vm's could all be diskless OS'es with a master server so that 1 OS upgrade migrates to all the vm's automatically for example. (It would also save a lot of drive space). But then the problem of course is that if the master server goes down, all the nodes will too. (We could put them on a NAS if we can afford it, though that's up in the air right now) If any one has good ideas on what services to separate in which vm's lets hear them. It's more fun if we can involve everyone in building our new home
|
|
|
|
|
59
|
Main Area and Open Discussion / General Software Discussion / Re: SMF or phpBB... that is the question?
|
on: November 17, 2010, 12:12:21 PM
|
SMF is pretty great right out of the box and has proved very stable, and has a wonderful feature set and organization that just makes sense.
However the one thing I will say, and i think i've said this in another thread, is that SMF is part of the past-generation of web applications, where "add-ons and extensions" are not handled through hooks and signals, but are instead handled by actually *modifying* the code.
What this means is that addons and mods are not easy to maintain, and can be a bit hairy to deal with when applying updates to the core forum.
So SMF is a great forum system for someone who wants a powerful and easy to maintain forum system, but who isn't planning on installing lots of addons. Although there are some great addons for SMF, if you are the kind of person who will want to install lots of addons and modifications for your forum, you should look elsewhere, in my humble opinion, because of the difficulty in maintaining and upgrading in the presence of addons.
That's an important point, and I'd like to add to it from a ServerAdmin point of view. Keeping up to date is one of the single most important things you can do to keep a server secure. With the design of smf making this HARDER when you have modifications, you REALLY have to stay on top of things. This means, watching all the regular sites for posted smf exploits (fulldisclosure etc) and patching & testing them MANUALLY when they occur. (thus you must be skilled in php to even consider running smf with custom mods if you're planning to maintain the site a long time) In the past Mouser and I have had to do this a few times, but luckly not too much, which brings me to the second point: I see a lot more vulnerabilities posted for phpbb than smf. Now that doesn't mean that it's more secure, it could just be that it's more popular/targeted Honestly I think we (especially mouser  ) dread the day we have to finally go for a major version upgrade of smf 
|
|
|
|
|
60
|
DonationCoder.com Software / Mircryption / Re: mircryption-compatible script for irssi
|
on: November 05, 2010, 11:01:19 PM
|
|
Hi Mayti,
I just tested blowssi with mouser in both cbc and ecb, using +OK prefix AND mcps prefix and we could not duplicate the problem no matter what we did. Perhaps you have some script that messes with the format of messages? That's the only thing I can think of right now...
|
|
|
|
|
62
|
DonationCoder.com Software / Mircryption / Re: mircryption-compatible script for irssi
|
on: November 05, 2010, 07:43:34 PM
|
Blowssi should recognise both +OK and mcps. # default prefix my @prefixes = ('+OK ','mcps ');
And the last time I tested it cbc was working both ways ... But that's a while ago. In the mean time, someone also contributed some code for key sharing etc... I've been meaning to merge that (It's a bit messy so it needs some adjustments). Whenever I get to it, I'll make sure I re-test everything else. I wouldn't mind implementing encryption for the keys in the config file, but I'm not sure what the point is. If it needs to be decrypted, then the encryption key should be stored somewhere (in the code or in a separate file) which would allow anyone to decrypt it easily anyway, no? Unless you store the keyfile on a separate volume maybe... I've been busy with work and <insert random excuse here>, but i still plan to get around to all of this some day Please use the redmine issue tracker for all of this, and create an issue ticket for each feature or problem. That at least will serve as a better reminder and help me keep track of all of this. http://redmine.dcmembers.com/projects/blowssi
|
|
|
|
|
63
|
Main Area and Open Discussion / Living Room / Re: What's Your Internet Speed/Reliability SATISFACTION?
|
on: November 05, 2010, 07:33:19 PM
|
Please don't just say Google, because google's a mess. How would you explore the alternatives? Do any of you know of lesser-known companies that are good?
Back then I actually did find localisp via google, but not easily. It took about a week of searching using non-obvious queries... That was quite a while ago. It may be even harder now, especially since the smaller companies are slowly vanishing or getting swallowed by bigger ones. The one major tip I could give you is to not confine your search within your area. The company I went with was actually from out-of-state.
|
|
|
|
|
64
|
News and Reviews / Mini-Reviews by Members / Re: LogMeIn Hamachi²
|
on: October 16, 2010, 06:15:20 AM
|
<tinfoil hat> Does the complete lack of needing to do any port forwarding configuration mean that all traffic between the user networks uses hamachi servers as intermediate party? If so, wouldn't that put them in a perfect position to mitm snoop your traffic? :S </tinfoil hat> 
|
|
|
|
|
67
|
Main Area and Open Discussion / Living Room / Re: What's Your Internet Speed/Reliability SATISFACTION?
|
on: September 07, 2010, 05:22:24 AM
|
Ok, I'll bite. I've had DSL and Cable in many places, and my findings are that DSL can get you some packet loss under really high load (eg: torrents), but the normal uptime has always been extremely stable. The advantage of DSL is that you have unbundled service in most countries (at least in the US and Belgium). Meaning, the company providing your physical line has to, by law, allow you to chose a different ISP. This is VERY important. Many people think they are stuck with something crappy like verizon, when in fact they can pick any ISP in the country offering dsl, including many good smaller ISP's. When we lived in PA, we were with localisp ( http://localisp.com/) - the advantage being that most of these smaller isp's don't have the TOS from hell. Verizon's TOS for example, last I checked, does not allow you to run any type of server (including game servers) or use third party voip. Not that they actually apply this in practice, but they are bastards for it nontheless. With localisp I had 100% uptime. I ran a monitoring server from my connection and received alerts+logs when something was down, and in the few years we lived there it never happened. They also offered a redundant line which could serve as a backup if one goes down, for relatively cheap (If i want the same from comcast here i'll be broke in an instant). The only times things became unstable (read: packet loss) is when the upload bandwidth was consumed 100%. I solved that using QoS. DSL was a lot slower, but it sure was magnitudes more stable. I've had cable in the US from RCN and now Comcast Bussiness, both randomly cut out. When it's up, the connection is stable. Unlike dsl, using 100% upload does not cause packet loss, however, one day you may wake up to find your internet not working at all, when you call them they claim to be working on something or other,... I've experienced downtimes of >3 hours in the middle of the day on a monday. Bad mojo when you're in the middle of work, and rely on a connection to do your job. If I were given the option again, I'd pick the uptime stability over the speed. Interestingly, when I was in Belgium, the same story was repeated. I had DSL (belgacom) there and cable (telenet). The DSL service had very stable uptime, though gets funky when you max things out (need a good QoS setup in your router). Cable is stable when up, but more overall downtime. - That leads me to believe the phenomenon is inherit in the technologies used. I haven't tried FIOS yet, it would be interesting to see how it compares in uptime versus speed. Also, with cable your speed will vary up and down depending on where you live and the time of the day. (Everything is 'shared' with your neighbors so to speak) - This can work to your advantage when you live in the middle of nowhere, or disadvantage when you're in the city ; while dsl lines are dedicated. [attach=1] [edit] PS: It's interesting to compare my speed and mouser's - same provider, same area, except he is more in the city than I am - (It's also a different plan, though not sure how much different) [/edit]
|
|
|
|
|