Thread about the DonationCoder.com server Shutdown on March 2nd, 2008

[mouser]

I'm just going to start this thread now so that it's up, and then i'll add more in the coming days.

NOTE: A separate thread celebrating our coming back online after the outage is here.


Ct+paste of the message that was up on our server for the last 4 days for everyone that tried to access any page on the server:

What happened:

On Sunday morning, March 2nd, around 10:30am(EST), the server was hacked into by someone who used an exploit on a piece of older software to get root access.
Thankfully the attacker was only in the machine for 1-2 hours when the intrusion was discovered, and we immediately locked down access to all services.

There doesn't appear to be any data loss, but the attacker did manage to put up some sort of activex code on the homepage of the site which attempted to infect visitors of our homepage using older versions of Internet Explorer. If you visited the home page of the site on Sunday morning EST using Internet Explorer and noticed anything strange please make sure you run a virus scan on your computer. If the activex is allowed to run, it attempts to install a version of the ntos.exe virus on the users PC. To make sure you have not been infected, please go to the (C:\)Windows\System32\ directory on your PC and look for a file called "ntos.exe". If you do find a Windows\System32\ntos.exe file on your pc, then you need help removing the infection. Here is one page with some instructions. The virus is also detected by the free antivirus programs AVG and AntiVir.

Please note that none of our file downloads were ever compromised in any way.

We have decided that the best thing for us to do in order to be absolutely certain that the attack cannot be repeated is to reinstall new server software from scratch, with tighter security restrictions, and then restore the site content from known good backups.

We can't apologize enough for the downtime and inconvenience. It's heartbreaking to us that someone would do this to the site. The only thing we can do is re-dedicate ourselves to security and take the time to fix it properly so it never happens again.

Thank you for your understanding and patience. And thank you so much for your support while we work to bring the site back up.

-mouser, gothic, wordzilla, and rest of the DC team

[mouser]

Let me quickly add a few words about the code that the hacker put on the homepage (actually it turns out it was also on a few more index.html pages on the site, like the Reviews/ start page).

We were fortunate to have a few white hack hacker types and some malware professionals take a look at what the code placed on the page was trying to do (thanks everyone on our irc channel -- #donationcoder on efnet).

Basically the code was some obfuscated javascript that simply opened a page on a far away site, which attempted to trigger some exploits in older versions of Internet Explorer.  It looks like it was some version of something called icepack/mpack, which believe it or not is a product that people SELL AND BUY for the explicit purpose of hacking computers.

The code was designed to try various tricks on people who were using Internet Explorer.  I actually tried in a virtual machine to let it infect me and it was a bit difficult since by default, the latter versions of Internet Explorer (v7 and on) have some pretty reasonable steps that try to warn you that something strange is happening and ask you if you want to install activeX components, etc.  But if you had an old version of Internet Explorer you may have been at risk -- please run an antivirus check to be sure.

I cannot tell you how distraught and angry i was when i found out that someone had put this code on our homepage.  I felt like I had let down the visitors to this site.  If i don't seem contrite enough at the moment, it's only because in the last 4 days since the server was down i have gradually calmed down from a state of hyperventilation.  The only thing now to do is go forward and work at making the site more secure.  Thanks to everyone on our IRC channel who put up with me freaking out, and who helped analyze the attack, and especially to DC member Jazper who alerted us about the initial intrusion so quickly so that we had the site wasn't exposed for more than an hour or so.

A few things to note about the code they added to the page that should give you some pause while surfing:

• The only thing they did to the page was add 1 line of javascript.
• That is enough to open a page on another site which can begin delivering you attempted exploit code designed to trick your browser into downloading and installing a virus.
• There is nothing special about it being on our server -- anyone who owns any site could put this code on their page without having to hack anything.
• In other words, the owner of any site on any page you ever visit could put code like this on their page to try to infect you.  It's just plain simple javascript.  No one needs to hack a site to put this code on their own created pages.
• What this means is that you should expect that if you do even a little bit of regular surfing, you need to be aware how important it is to have up-to-date software installed -- keep your browsers updated to latest versions, be on the lookout for announcements about possible security risks, have a good antivirus.
• Listen to your browser -- all new versions of IE and firefox will alert you if a site is trying to open and run some executable or active X.  If you get an unexpected pop-up question on a site asking if you want to run some addon or something, say no unless you know exactly what it's for.

[f0dder]

It's worth to note that malware package is designed to do really nasty things, like stealing your banking information - so it's very important that you check whether you've been infected or not. The quick way is checking whether you have a file called ntos.exe in %SystemRoot\system32, like mouser mentioned above.

The scumbags that do this are obviously interested in getting as many people infected as possible, and while I don't know how many visitors we have per day, my guess is it's a fair amount of people.

This wasn't the typical defacing hack just to say i ownz j00, it was done by people with monetary crime in mind!

[mouser]

A couple of good free antivirus tools:

• AVG
• AntiVir

By the way if anyone confirms that they were infected by this from our site (remember that the exploit was only in place for an hour or so on Sunday morning between 9a-11am ) please email me at mouser@donationcoder and let me know.

[iphigenie]

Do you know how they got access to the pages in the first place?

When I had the problem 1.5 years ago they gained access to the tinyportal/smf uploads directory (cant remember which) and from this created some folders to put a warez server on. They didnt manage to change the pages, and couldnt do very much (thank goodness for BSD!) but still the bandwidth bill for 36 hours was.... about a year's hosting!

Do i need to worry and go check my site or can I just stay here and post some of the things I wanted to post last weekend?

[mouser]

It looks like they got in using an exploit in an older version of the Subversion Version Control System (SVN) that i had installed on the server a while ago.  It's a good lesson that the moment you install a service on your server, you need to forever after keep it updated, or disable it.  You cannot just install something on a web-accessible server and forget about it.

[Rover]

I felt like I had let down the visitors to this site.  If i don't seem contrite enough at the moment, it's only because in the last 4 days since the server was down i have gradually calmed down from a state of hyperventilation.

-Mouser

Mouser, while I appreciate your humility, I do not hold you or anyone on the DC team responsible for this attack.  To me, that'd be like blaming a rape victim* for their attack.  It's not your fault, we're happy DC is back. 

THANKS FOR ALL YOU DO   

*I am not attempting to equate the two, relax.

[Gothi[c]]

It looks like they got in using an exploit in an older version of the Subversion Version Control System (SVN) that i had installed on the server a while ago.

It does look like that, but there is no way to be 100% sure.
The facts are:

• Someone logged into the svn user account (which for some reason had a bash shell bound to it instead of being pointed to /sbin/nologin or something) before logging in as root (timestamps show svn was first)
• The svn user account had " nano /etc/passwd " in it's .bash_history.   It is safe to assume that they erased the .bash_history on every log-in, so it will only show the commands they ran on last login, nothing before that.
• About one hour and a half later, they logged in as root through the front door. According to the ssh logs, it seems they used a valid password. Then went straight to installing their trojan code on the webpage. As far as we can tell anyway, the .bash_history does show that it wasn't erased because it had commands in there we ran before the attack. However, they could easily manipulate it and only delete the lines they were responsible for.
• They also killed the log daemons upon login. Thus adding more uncertainty since we only have partial information.
• The attackers came from at least 3 different IP addresses:
  
  24.39.219.73
  82.201.163.136
  62.13.171.41
  
  It's most probably safe to assume that these are also hacked computers.
• The way they infected the pages was by running a script called fr.sh which traversed the directories looking for index.html/htm pages (It also got 2 PHP pages that were not accessible to the public). It seems like it grabbed the code to inject from a file they created (filename was Script).
  

[Deozaan]

The quick way is checking whether you have a file called ntos.exe in %SystemRoot\system32, like mouser mentioned above.

-f0dder (March 07, 2008, 08:52 AM)

Is it only ntos.exe? I checked and I have an ntoskrnl.exe in the system32 folder. Is that safe?

[tomos]

The quick way is checking whether you have a file called ntos.exe in %SystemRoot\system32, like mouser mentioned above.

-f0dder (March 07, 2008, 08:52 AM)

Is it only ntos.exe? I checked and I have an ntoskrnl.exe in the system32 folder. Is that safe?

-Deozaan (March 07, 2008, 02:13 PM)

a google search gives the impression it's more of a problem not to have that (ntoskrnl.exe) -
I had checked it out myself earlier
http://www.google.co....exe&btnG=Search

[Lashiec]

Is it only ntos.exe? I checked and I have an ntoskrnl.exe in the system32 folder. Is that safe?

-Deozaan (March 07, 2008, 02:13 PM)

Check up the link to the Symantec information page about the virus that mouser included in the first post for more details, but that file comes with Windows by default.

[mouser]

only ntos.exe is evil. the ntosekrnl.exe is a normal file that you have nothing to worry about.

[f0dder]

only ntos.exe is evil. the ntosekrnl.exe is a normal file that you have nothing to worry about.

-mouser (March 07, 2008, 04:09 PM)

Yeah, and please do not delete that file

ntos.exe was obviously chosen to try and camouflage it next to ntoskrnl.exe - the trojan code even grabs filetime from ntoskrnl.exe and sets the downloaded ntos.exe filetimes based on that!

[Stoic Joker]

Automated attacks are (unfortunately) quite common these days. The attack model used to be pick a target, probe it for weaknesses, and then try to exploit one of them (this actually required knowledge & skill). But now exploits are picked ahead of time in an almost shopping cart manner and are then launched against (completely) random servers using service/port scans in the hopes of finding a "soft" target that (via scripted exploit) can just be popped open, and be prepped and for ravaging when some lazy assed "attacker" gets back from lunch.

One of the biggest problems with this (or any) type of attack is that most sites/companies try to conceal the security breach and make every effort to hide the fact that it happened ... Which only server to assist it in propagating further. Foolishly prideful admins not wanting to admit "something went wrong" trying to hide "the mess", which only serves to spread this kind of exploit farther and faster.

In that regard I would like to sincerely applaud Mouser in his handling of this event because he actually made the effort to use common sense, and not only inform visitors that something had happened, but what had happened, and what they could do to clean their systems and prevent it from spreading it further. If more admins had the stones to do that ... The Internet would be a lot safer.

So I would just like to say Thank You Mouser, for doing the right thing!

Trust & Respect,
Stoic Joker

[f0dder]

I agree with Joker, while it's never a fun thing to admit you've been hacked & it might be a bit unnerving to the end-users, it's the proper thing to do.

Btw it doesn't seem like it was one of those fully automated drive-by hacks in this case, too much fumbling around showing in the log files.

[mouser]

I appreciate your message so much Stoic.

I can definitely see why there is such an incentive for companies to cover up when this happens to them -- it's incredibly embarassing, and the public relations damage could be severe, and put jobs on the line, etc.  You can definitely imagine why a company would just want to brush it under the rug and actively deny it happened if asked.  I suspect this happens a LOT.

But like you say -- the problem is that this just makes the situation for the end users worse.

We have tried from it's inception to just be totally up front about everything that happens on this site.  When the server goes down we try to post why and what we are doing, etc.  There was never really a question that we would post about exactly what happened and what we were doing about it.

I will be posting a much longer thread and gothic will chime in too about general lessons learned and strategies to avoid such things in the future.

[nowshining]

hi, um... those IPs are suspicious: (i just got ur eletter a few mins ago):

First of all if ur using linux/etc.. best thing is to disable the timeout in sudo privs. in Ubuntu it's 15m To do so in ur sudoers insert the following (note also logfile for passwords and last time accessed is sent to secure.log which should secure u even more):

[Select]

Defaults !lecture,tty_tickets,!fqdn
Defaults:ALL  !syslog
Defaults:ALL logfile=/var/log/secure.log
Defaults:ALL timestamp_timeout=0

Also are u using useragent blocking in ur .httpaccess files? U could also get and put up an ipblocker that are primeraly used by p2p users to block media ips, etc.. This should also keep ur site secure. not to mention keep gov., mil, and even RIAA and MIAA, out of this site, etc..?

if i rem. correctly ipblocker is for linux. I just forgot the exact one for windows. Atho I had trouble with it with arno-iptables-firewall.

oh well..

Also u should have the latest linux kernel updates. There were an exploit that could easily gain root access and that is fixed.

I have the test code on my website if anyone wants to use it for testing. just cd to ur desktop or wherever u saved it, sh exploit.sh or whatever and if u get root then ur affected..

I got it on my site for if others want to test their kernel.

Also many security sites have the test exploit code too..

http://www.botnetgod...ale.dnsdojo.org:8000

is my shared files site (dir). without the 8000 port is my wiki, which I gotta set-up and fine a use for. as it's public..



by the way:

[Select]

24.39.219.73: This seems suspicious because HoldCO I think is from RR internal. RIAA? MIAA? Some employee from RR, I read on forums of blocking RR IPS with HoldCO in them, of course those were p2p forums. One Forum said that it looks to be an internal IP..

OrgName:    Road Runner HoldCo LLC
OrgID:      RCNY
Address:    13241 Woodland Park Road
City:       Herndon
StateProv:  VA
PostalCode: 20171
Country:    US

ReferralServer: rwhois://ipmt.rr.com:4321

NetRange:   24.39.0.0 - 24.39.255.255
CIDR:       24.39.0.0/16
NetName:    RR-COMMERCIAL-NYC-4
NetHandle:  NET-24-39-0-0-1
Parent:     NET-24-0-0-0-0
NetType:    Direct Allocation
NameServer: NS1.BIZ.RR.COM
NameServer: NS2.BIZ.RR.COM
NameServer: DNS4.RR.COM
Comment:
RegDate:    2004-02-19
Updated:    2004-06-09

OrgAbuseHandle: ABUSE10-ARIN
OrgAbuseName:   Abuse
OrgAbusePhone:  +1-703-345-3416
OrgAbuseEmail:  [email protected]

OrgTechHandle: IPTEC-ARIN
OrgTechName:   IP Tech
OrgTechPhone:  +1-703-345-3416
OrgTechEmail:  [email protected]

# ARIN WHOIS database, last updated 2008-03-26 19:10
# Enter ? for additional hints on searching ARIN's WHOIS database.


Found a referral to ipmt.rr.com:4321.

%rwhois V-1.5:003fff:00 ipmt-02.rr.com (by Network Solutions, Inc. V-1.5.7.3)
network:Class-Name:network
network:ID:NETBLK-ISRC-24.39.128.0/17
network:Auth-Area:24.39.216.0/21
network:Network-Name:HAEFELE-TV-INC.-24.39.216.0
network:IP-Network:24.39.216.0/21
network:IP-Network-Block:24.39.216.0 - 24.39.223.255
network:Organization;I:HAEFELE-TV-INC.
network:Tech-Contact;I:[email protected]
network:Admin-Contact;I:IPADD-ARIN
network:AbuseEmail:[email protected]
network:Created:20080326
network:Updated:20080326
network:Updated-By:[email protected]

network:Class-Name:network
network:ID:NETBLK-ISRC-24.39.128.0/17
network:Auth-Area:24.39.128.0/17
network:Network-Name:ISRC-24.39.128.0
network:IP-Network:24.39.128.0/17
network:IP-Network-Block:24.39.128.0 - 24.39.255.255
network:Organization;I:Road Runner Commercial
network:Tech-Contact;I:[email protected]
network:Admin-Contact;I:IPADD-ARIN
network:Created:20080326
network:Updated:20080326
network:Updated-By:[email protected]

%ok
.............................................
62.13.171.41:suspicous? IT DEPT. ?? H3G?? Seems to be a  hosting Company? http://builtwith.com/?Tre.it = notice the "Who is Hosting This" at the bottom right.

inetnum:      62.13.171.0 - 62.13.171.255
netname:      H3GIT
descr:        H3G IT department
country:      IT
admin-c:      VO175-RIPE
tech-c:       RC497-RIPE
tech-c:       EMF4-RIPE
tech-c:       GB1450-RIPE
status:       ASSIGNED PA
mnt-by:       H3G-CN-MNT
source:       RIPE # Filtered

person:       Vittorio Orsini
address:      H3G Italia S.p.A.
address:      Via Cristoforo Colombo, 416 - 420
address:      I 00145 Roma RM
address:      Italy
phone:        +39 06 59551
fax-no:       +39 06 54602123
e-mail:       [email protected]
nic-hdl:      VO175-RIPE
source:       RIPE # Filtered

person:       Raffaele Celentano
address:      H3G Italia S.p.A.
address:      Via Cristoforo Colombo 416
address:      I-00145 Roma RM
address:      Italy
phone:        +39 06 59556068
fax-no:       +39 06 54602123
e-mail:       [email protected]
nic-hdl:      RC497-RIPE
source:       RIPE # Filtered

person:       Giuliano Biondi
address:      H3G Italia S.p.A.
address:      Via Cristoforo Colombo, 416 - 420
address:      I 00145 Roma RM
address:      Italy
phone:        +39 06 59551
fax-no:       +39 06 54602123
e-mail:       [email protected]
nic-hdl:      GB1450-RIPE
source:       RIPE # Filtered

person:       Enrico Maria Fondi
address:      H3G Italia S.p.A.
address:      Via Cristoforo Colombo, 416 - 420
address:      I 00145 Roma RM
address:      Italy
phone:        +39 06 59556066
fax-no:       +39 06 54602123
e-mail:       [email protected]
nic-hdl:      EMF4-RIPE
source:       RIPE # Filtered

% Information related to '62.13.160.0/19AS24608'

route:        62.13.160.0/19
descr:        H3G Italy SpA
descr:        UMTS operator and ISP
origin:       AS24608
mnt-by:       H3G-CN-MNT
mnt-routes:   H3G-CN-MNT
source:       RIPE # Filtered

......................................................................
82.201.163.136:suspicious due to "African Internet Numbers Registry".

% This is the RIPE Whois query server #3.
% The objects are in RPSL format.
%
% Rights restricted by copyright.
% See http://www.ripe.net/db/copyright.html

% Note: This output has been filtered.
%       To receive output for a database update, use the "-B" flag.

% Information related to '82.201.128.0 - 82.201.255.255'

inetnum:      82.201.128.0 - 82.201.255.255
org:          ORG-AFNC1-RIPE
netname:      AFRINIC-NET-TRANSFERRED-20050223
descr:        This network has been transferred to AFRINIC
remarks:      These IP addresses are assigned in the AFRINIC region.
remarks:      Authoritative registration information for this network
remarks:      is available for query and modification in
remarks:      the AFRINIC whois database: whois.afrinic.net or
remarks:      web site: http://www.afrinic.net
remarks:      The routing registry information (route(6) objects)
remarks:      may be published in any Routing Registry, including
remarks:      RIPE Whois Database
country:      EU # country is really somewhere in African Region
admin-c:      AFRI-RIPE
tech-c:       AFRI-RIPE
status:       ALLOCATED PA
mnt-by:       RIPE-NCC-HM-MNT
mnt-routes:   RIPE-NCC-RPSL-MNT
source:       RIPE # Filtered

organisation: ORG-AFNC1-RIPE
org-name:     African Internet Numbers Registry
org-type:     RIR
address:      see http://www.afrinic.net
e-mail:       [email protected]
admin-c:      AFRI-RIPE
tech-c:       AFRI-RIPE
remarks:      For more information on AFRINIC assigned blocks, use
remarks:      AFRINIC's whois database, whois.afrinic.net.
mnt-ref:      RIPE-NCC-HM-MNT
mnt-by:       RIPE-NCC-HM-MNT
source:       RIPE # Filtered

role:         The African Internet Numbers Registry
org:          ORG-AFNC1-RIPE
address:      AFRINIC, see http://www.afrinic.net
admin-c:      AFRI-RIPE
tech-c:       AFRI-RIPE
nic-hdl:      AFRI-RIPE
e-mail:       [email protected]
remarks:      For more information on AFRINIC assigned blocks, connect
remarks:      to AFRINIC's whois database, whois.afrinic.net.
mnt-by:       RIPE-NCC-HM-MNT
source:       RIPE # Filtered

% Information related to '82.201.128.0/18AS24863'

route:        82.201.128.0/18
descr:        LINKdotNET Route
origin:       AS24863
mnt-by:       MAINT-LINK
source:       RIPE # Filtered

% Information related to '82.201.128.0/17AS24863'

route:        82.201.128.0/17
descr:        LINKdotNET route
origin:       AS24863
mnt-by:       MAINT-LINK
source:       RIPE # Filtered

% Information related to '82.201.160.0/22AS24863'

route:        82.201.160.0/22
descr:        LINKdotNET route
origin:       AS24863
mnt-by:       MAINT-LINK
source:       RIPE # Filtered

% Information related to '82.201.160.0/21AS24863'

route:        82.201.160.0/21
descr:        LINKdotNET route
origin:       AS24863
mnt-by:       MAINT-LINK
source:       RIPE # Filtered

% Information related to '82.201.162.0/23AS24863'

route:          82.201.162.0/23
descr:          LINKdotNET route
origin:         AS24863
mnt-by:         MAINT-LINK
source:         RIPE # Filtered

My conclusion, u were hacked either by the RIAA, MIAA, or someone in media company affiliated with these companies as these IPs point to what many p2p users see pointing to the end result the RIAA, MIAA, etc..

It could of been also the MIL, GOV. Helping out the RIAA and MIAA. My suggestion get the ips of thes orginazations and block them from ever connecting to this website with ipblocker or some other p2p blocking program and have them updated once per week (they ged upset if u do it more )...

Again i wouldn't be suprised if u received a court order to take down this site in the near frute to to copyright issues or them claiming it.

[Josh]

I can honestly say that I know for a fact this wasnt a .mil/.gov based attack. For one, the military does not do these types of attacks. There is no reason for them to do so. As a US Military member, I can tell you that most military installations have a hard enough time maintaining the endless problems and issues with their current networks and would not waste their time with an RIAA/MPAA issue. I am sorry, but just because the RIAA/MPAA have used an ip range in the past, doesnt mean they use it everytime. There is no reason for DC to be attacked by either of these organizations. We have no content that they would even care about

But anyways, I browse this site from work so if the site did block .mil, a stupid idea, I would be shut out and I would not like that too much. I am sorry, but this post sounds like paranoia.

[f0dder]

Oooooookay, please lay off the crack pipe and conspiracy theories