DDOS in a Nutshell. How it works and why it stinks.

Trka

Posted by Trka in Systems on Aug 09, 2017


DDOS, by nature, is a particularly tricky attack strategy. It's important for anyone who comes close to a server - tech. or no - to understand what it is and how it works, because it operates so differently from "regular" malware.


Things to understand

  1. A web server is really just a computer.

It runs programs, and when it tries to run too much, things get strange. your desktop works fine when you open Word or Photoshop or something. But if you open 600 Word or Photoshop windows side-by-side... it doesn't run so great.

  1. Web servers run server software as applications, like your computer runs Word

Server software is a server application, just like Word is a desktop program. Different combinations of software and setup work differently, but generally: A user hits a URL on your site, the software starts, it finds what they want and sends it to them.

In a nutshell, a server is just like your regular computer - it just has different programs that do different things.


How does DDOS stack up against "classic" malware?

  1. It doesn't 'live' on your site. Usual malware runs on your site, under your control. It's completely possible - and usually pretty easy - to get rid of it. DDOS, on the other hand, does not. It's outside of your zone-of-control.
  2. It's not even malware. It's more like mal-activity, because from your site's perspective, it's just traffic - albeit, malicious traffic.
  3. DDOS is smart. Infosec is smart, but DDOS networks are very good at keeping up.
    • We made server rules that say "This IP is hitting quicker than a person would be able to, it's blocked for a few minutes". They started rotating IPs - using 1 machine to hit a site 6 times, then switching to another one. They still hit 80k times a minute, but the server thinks it's different people.
    • So we tighten it up. We add monitoring and deeper behavior checks. They started rotating their sites. It's dead-simple to find out what other sites are hosted on the same machine. If they target awesomesite.com, it's easy to find out that awesomesite.com is hosted alongside coolsite.com, badsite.com, and awfulsite.com. So they spread the traffic over everything else on that server, which makes it a blip on the radar.
  4. Traditional malware is easy enough to control. It's annoying, but it's controllable. DDOS, on the other hand - is completely out of your zone-of-control. It's an outside thing that's killing key internal processes.

Breaking-Down a simple attack

Getting into DDOS, let's look at a simple case. Imagine you have a server that runs a single site. People visit it - maybe a few hundred at a time if you're doing good - they see your stuff, and all is well.
Then it gets targeted for a DDOS attack.

With a couple hundred people at once on your site, your server was running at about 50% load if your sysadmin has done a good job with it. But the DDOS network uses possibly MILLIONS of things (clients, from here on) to send an obscene amount of traffic to your site. These clients can be anything hooked up to the internet. In the old days, it was infected PCs. Now, with IOT, it's a lot of things: cable boxes, security cameras, coffee makers. Your at-one-time users (concurrent users, from here on) jumps from a couple hundred to thousands or millions. your server load jumps from 50% to 20,000% (real number) and things start going bad.

  • At about 500% load, site speed CRAWLS. Nothing's shut down yet, but site hits are queued on a first-come-first-serve, so real traffic has to wait.
  • At 1000%, your server shifts into focus mode and starts killing background things that aren't a priority for it.
  • From there up, it goes into self-preservation mode and shuts down more and more things to make room for the things that are most important to its job - serving web sites.
  • If the attack is bad enough, it'll eventually flat-out stop doing its job of serving web sites. Your database goes to sleep, your server software stops serving files, and at this point the computer is just keeping its power on and not doing much else.

Looking at an attack in progress

Seeing a DDOS in action might help the conversation, moving forward. We have a pretty intense real-time monitoring system,and we'll use that to deconstruct a past attack.

  1. Server Load: This is a view from one of our monitoring dashboards that shows 4 servers' loads over time. The yellow-orange one is under attack, and the other 3 are at normal afternoon use. Over about 6 hours, the load rose to 8000% before going back down to its usual +-50%.

  2. What Caused It: This view shows the number of incoming requests. I've zoomed in a bit to show only the attack (not so much of its wake). During the attack, the incoming requests spiked from about 40\minute to 80,000\minute - obscenely beyond normal traffic.

  3. Where's it coming from? This screen shows the map locations of where the hits came from. Except for a couple of hotspots, it shows the distributed nature of the attack. It's basically global - at least, everywhere that has an internet connection


That's what it is, so how to stop it?

There's not too much that can be done to prevent it. And that stinks. It's just traffic, really, and that's what your site is supposed to do. It just happens to be a lot of traffic. Actually, there's no such thing as prevention for DDOS. There's mitigation, but that's the best technology can do.

Mitigation doesn't prevent, but it does take the edge off. Some keys to mitigation:

  • Put your site's DNS behind a vendor like Cloudflare. This offers a couple of prime benefits.
    • Your co-hosted sites' IP addresses are masked behind Cloudflare's. The strategy of distributing the attack across sites on the same server is severely crippled by this.
    • If an attack is underway, Cloudflare has a safe mode that stalls a request for several seconds. This can be mildly annoying to your users, but it will keep your site online. Alternatively, if the attack is extreme enough that you need to temporarily take your site offline, this is much easier in Cloudflare than your main DNS. Both of the DNS-cutoff measures are - usually - incredibly effective at choking an in-progress attack. The attack is orchestrated by an automated program. When it senses that the target is inaccessible it moves onto the next; most often, this only takes a minute or two.
  • Load-balancing. Again, this doesn't prevent. It doesn't even slow down the attack once it's started. Load balancing is basically hosting identical copies of your site in 2 or more places, then configuring a central web server to route incoming traffic to different places. This effectively spreads the impact of your site's traffic across a number of servers. If you have a DDOS that would have put you over 2000% load, and you had 10 servers in your cluster, then your peak on any one of them would be more-or-less 200%. That's still high, but very manageable.
    Load-balancing, though, is not a simple topic.

Cover\Featured image from Digital Attack Map showing realtime DDOS activity.