DOS Attack?

[Moonsword]

Uh, wow.  Wonder what's going on with that.  Looking forward to your post-attack analysis.

[cray]

DOS attack? I'd be curious WHY. Bored hacker with a grudge against BT or some extortion scheme?

[Von Jankmon]

Allowing for Hotmails temporary ban of CBT it sounds to me like the hacker tried to turn CBT servers into a super-sized zombie for further DDOS efforts.

Pure speculation though, I would like DS to tell us more of what went on, or at least what he thinks went on.

[deathshadow]

Basically we got hardcore hammered by a spammer sending literally hundreds of e-mails a SECOND to the server to random addresses. This turned into a denial of service because ClamAV was scanning and rejecting the attachments on the 'catchall' account -- the one where rejected letters get stored.

First I banned the IP address region using iptables, then I set up the mail server to discard e-mail attachments on mails not sent to actual accounts on the server. Combined it has reduced the server load to.. uhm... a total joke.

Code: [Select]

top - 07:14:36 up 10 days,  9:29,  1 user,  load average: 0.04, 0.08, 0.22
Tasks: 147 total,   1 running, 146 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.3%us,  0.2%sy,  0.0%ni, 96.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1995264k total,  1715444k used,   279820k free,    85604k buffers
Swap:  5847620k total,   105276k used,  5742344k free,  1072408k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
19628 www-data  20   0  359m  23m 9012 S    2  1.2   0:00.19 apache2
19637 www-data  20   0  358m  23m 9124 S    1  1.2   0:00.26 apache2
19654 www-data  20   0  356m  22m 8476 S    1  1.2   0:00.25 apache2
10313 mysql     20   0  459m 200m 7500 S    1 10.3  14:29.69 mysqld
 1843 ntp       20   0 38332  696  568 S    0  0.0   0:56.88 ntpd
11245 root      20   0 19196 1420  980 R    0  0.1   2:58.93 top
    1 root      20   0  8352  300  268 S    0  0.0   0:12.72 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
    3 root      RT   0     0    0    0 S    0  0.0   0:01.45 migration/0
    4 root      20   0     0    0    0 S    0  0.0   0:05.39 ksoftirqd/0
    5 root      RT   0     0    0    0 S    0  0.0   0:00.02 watchdog/0
    6 root      RT   0     0    0    0 S    0  0.0   0:01.34 migration/1
    7 root      20   0     0    0    0 S    0  0.0   0:02.63 ksoftirqd/1
    8 root      RT   0     0    0    0 S    0  0.0   0:00.02 watchdog/1
    9 root      20   0     0    0    0 S    0  0.0   0:14.74 events/0
   10 root      20   0     0    0    0 S    0  0.0   0:11.64 events/1
   11 root      20   0     0    0    0 S    0  0.0   0:00.00 cpuset

MUCH better, and what it should be on a sunday morning with ~70 or so people online at once. (though I still laugh at how half that much traffic used to bury us when we were on a dual Xeon with twice the RAM)

We were looking at 100% IOWAIT during the attack with ClamAV scanning stuff and postfix trying to deal with the volume of input. For some reason I cannot fathom they just kept hammering us hard with inbound mails to made-up addresses. They were all bouncing into the black-hole "catchall" account, so I have no clue what they were trying to accomplish APART from shutting us down.

It was cute how many of the accounts it was trying to send to were ... infamous FORMER forum account names with @classicbattletech.com tacked onto the end... as in the Permabanned, no longer welcome or fled for not fitting in, and people who in the past have been problem cases in terms of hacking/circumventing or just being royal pains in my backside. It's also VERY suspect as... well, I can't say more until I glean more proof out of the server logs and consult with.. uhm... a certain organization. (here's a hint, some people call it the party van)

... and apparently they all need debt relief, problems with their paypal accounts that require sending your credit card number to a website that has nothing to do with paypal, and of course Viagra. Everybody needs Viagra... NOT.

For the past... six to eight months it's like we've had this giant bullseye painted on us -- I'm spending a good two hours a day now just trying to keep things afloat on the back-end... though it LOOKS like I might be able to take a day off now.

... and remember, if a SPAM attack lasts for more than 8 hours, consult a professional.

[Peter Smith]

...and consult with.. uhm... a certain organization. (here's a hint, some people call it the party van)

Scooby-Doo and his crew?

[Von Jankmon]

First I banned the IP address region using iptables,

Is that as in geogrpahical reason. Let me guess, Nigeria?

It's also VERY suspect as... well, I can't say more until I glean more proof out of the server logs and consult with.. uhm... a certain organization. (here's a hint, some people call it the party van)

Sounds like you have personal contacts.

[Moonsword]

Man, what a mess.  I wonder why people would be targeting us...

[Neufeld]

Man, what a mess.  I wonder why people would be targeting us...

Mech haters?

[Havock]

Giant bullseye indeed, no idea what or why. Hard to imagine it is just a bunch of griefers by now, the attention span of that type normally dies after they wreck something. Must be someone (or someones) with a chip on their soldier.

[Von Jankmon]

This is why I asked if they came from Nigeria, its quite likely the people who shut down the website did. Or at least its the only solid clue to the location of the perp who used the front address in Germany Deathshadow found while autopsying the carcass of our old forum.

[trboturtle]

Is it possible that the two events (Forum meltdown and this DOS attack) are connected?

Craig

[Moonsword]

Is it possible that the two events (Forum meltdown and this DOS attack) are connected?

Craig

Could be, but that still leaves the question of what this joker's hoping to accomplish.  Another one is the interesting question of whether or not the guy behind this is connected to the community somehow given the selection of names Deathshadow mentioned.  While it seems likely the answer is yes, the details are probably interesting.

[Von Jankmon]

I draw my conclusions fro human behaviour and a healthy understanding of the unhealthy human mind.  The actual science geekery is 100% deathshadow, if he points to an internet address and says "j'accuse" I just go with it, because I dont know any better.

Is it possible that the two events (Forum meltdown and this DOS attack) are connected?

Craig

I think so.  Though not necessarily directly.  For a hacker who takes down a website there are bragging rights, and also the mental needs to kick someone when down customary to all scum.  Now an internet community could have global bragging rights but most are in actuality local because it involves the word of mouth message that accompanies the online.

Also information about victims, zombies etc is much more likely to be kept for solo use or shared locally.

Now I have no direct evidence that our attackers comes from Nigeria, but when deathshadow lamented  "Leave it to some German to goose-step through Belgium on me."  I took a second look at the link provided.

Its obviously a form of mask or zombie or whatnot, I very much doubt the owner of the German computer used to open CBT was in the slightest bit aware of all that went on.  Looking at the report page the computer is used a lot.  Here is the only report with a geographical clue.

It is a fraud, do not trust them. I have advertised an item on a local Malaysian website (similar to eBay), and received an email offering to purchase my product. Very prompt in replying his emails. Insisted on only doing bank to bank transfer instead of using PayPal. Also requested for the item to be mailed via DHL courier service. He claims to live in Portugal and wish for the item to be posted to Nigeria.
I thought the man sounded suspicious, decided to google to details he provided and sure enough, found a lot of scam alert on this man/woman/institution using several different names but similar stories.

Co-oincidence, very possibly. An unsecure computer could be more than one persons toy, and the Nigerian user was after something other than a bit of mischief.  But it could equally be a connected activity.  DS has blanket banned Nigeria from access to CBT forums, so a scum may well have taken umbridge to earlier viagra or porn links being intercepted and decided to download a new hacking script from a misguided geek (probably not a Nigerian) got a zombie, targeted us and pressed the OK.

Its laughable that these scum, 99.9% of whom could not actually write/understand the malicious software they use run these programs in order to brag and appear intelligent by taking down websites. They are deluding themselves.

[deathshadow]

Its laughable that these scum, 99.9% of whom could not actually write/understand the malicious software they use run these programs in order to brag and appear intelligent by taking down websites. They are deluding themselves.

Well it's like the recent fights between teeny-bopper anons and various security agencies -- the feds are breaking down the doors of teenagers and the IT industry is spending most of their time pimping the kids as geniuses, when most of them just downloaded software off a torrent, ran it, and typed in "Koch Brothers" or "HBGary".

Missing that when all you have to do to hack HBGary, an alleged computer security firm is download some software and type in a URL... let's just say it doesn't do much for their credibility as security experts.

There's a movie/documentary called "hackers wanted" where one of the white hats talks about how he will often blind-call sites he finds vulnerabilities in -- and how some companies then try to go after him for daring to reveal how vulnerable they are even though he contacts them first about it. This stems from many companies trying to do "security by prosecution"... It often seems a lot of the IT directors and 'security experts' out there actually aren't qualified to even do their jobs -- and having that pointed out to them or worse, their Boss immediately puts them on the defensive; which is NOT a constructive way to solve the problem. IF they are trying to help; take the help. If they are using those vulnerabilities to shut you down, THEN you go after them.

That said, EVERY computer system can be hacked no matter how secure. As the old joke goes "the only secure system is one with zero access"... all it takes is time and determination (and usually who has more free time than school-kids still having life paid for by mommy and daddy)

It's really no shame to the server or the software we got hacked back in January... since we fend off hundreds of brute force attacks every day, and the server weathered no less than five major break-in attempts without failing and one attempt that did manage to append code to the settings.php file through a vulnerability in the avatar system.

Though right now it's reaching the point of every time I secure it a new approach is tried. I secure the forums with a new copy, we immediately get 2K+ spam login attempts a day for the first month. (we've had over 36K known spammers try to join and blocked by the "stop spammer" forum mod that ties into stopforumspam.com since we went live)... They obviously can't get in there so they tried brute-forcing the FTP server; so I switched that to SFTP/TLS -- and within a day they switch to trying to brute force the SSH root login. (funny since 'root' is blocked from remote login and I sudo everything)... So many attempts are tried there it starts to chew CPU, so I add 'fail2ban' to kill IP addresses that fail more than three times (whitelisting my own)... and what happens within a day of that change? Massive DoS against the e-mail servers... So I ban that address region (one ISP in nigeria that says it's a data center) with IPTables, and the next day it's a data center in Pakistan that was what actually hit us hard enough to shut us down. (the most recent attack).

We SEEM to be stable for now... but I'm just sitting here waiting for the next shoe to drop. EVENTUALLY they have to realize there are easier targets and move on.

In that way computer security is often like door-locks on a car or your home. They're there to keep the honest people out -- as a hardened crim or a teen who doesn't care will laugh in your face; a slim-jim down the window frame or a brick through the glass and they're in... that doesn't mean you leave it unlocked. It ends up being about making yourself LESS of a target than someone else.

Though if someone decides they want you -- you're in for one hell of a fight.

[Moonsword]

So, in your professional opinion, does someone want us?

[Von Jankmon]

So, in your professional opinion, does someone want us?

In my unprofessional opinion, yes.

I cannot speak for deathshadow but I get the feeling our findings are pretty close this time.  He sees the tech. I am seeing a growing pattern of scum who have tried to take down CBT for nothing more than bragging rights, and now one has made it others in the local community feel the need to match his 'prowess'.  The culture is similar to that of 419 in Nigeria, criminals get bragging rights and recognition in their local community, normally the takings are slim, and in the event of destructive attacks literally non existant.  419 is underhandedly encouraged, the local authorities knows who the perpetrators are and consider them folk heroes as do much of the populace. Hacking is very similar, and in fact a lot easier. Over here we call people who do this 'script kiddies', and the real operators 'crackers'.  The phenomenon only moved to Nigeria relatively recently, which says something about how advanced (or not) they actually are that they couldnt get past the level of sending poorly worded emails to vulnerable people as their means of proving their intellectual superiority.

Personally I don't think anyone in their group has any likelihood of being a true computer techie. If they were they would have left years ago and got a real job, that is where the money is. Napoleons of crime they are not.

[Moonsword]

I'm seeing it myself, but I'm wondering if there's something else going on given how heavily we're being targeted.

[Daemonknight228]

fact of life: somewhere, someone hates you for a reason you arn't fully aware of

[Von Jankmon]

Yeah, we call them Mods.  #P

[trboturtle]

The question is, are we being singled out for some reason. For Example, is anyone trying to hack Games Workshop or WoTC servers? (And I'm not accusing anyone from those forums being behind the attack -- I am just wondering....)

Let's face it, there are more visable forums out there. If it's an ego thing, why not attack them? Why us?

Craig

[Daemonknight228]

Even if someone were attacking WotC, or GW, or PP, theres no way to tell if they're connected without DS linking up with his equals.

I highly doubt it's some mass, anti-gaming thing(though it would be an interesting change of tactics from the 'Violence games are bad!' groups). I'd assume its more likely that the perps move in gaming circles with their friends, and thus bragging about taking down CBT is fun, because they can point and go, "oh you can't get in? HA! that was me!".

admittedly, the only people I know who do cracking are white hats(so they say), but i've yet to meet one of them that is the type to go through the trouble DS was describing without something specifically against CBT. Most people would get annoyed and move on to easier targets I should think.

[Peter Smith]

Frontbridge.