Blocking Download Managers and Accelerators

Some users like to use Download Managers & Accelerators in an attempt to complete file downloads faster.

These download managers/accelerators work by creating dozens to hundreds (and sometimes thousands) of independent concurrent and sequential connections, with each connection downloading a different part (byte range) of the same file.

The client sends a request with a “Range” Header specifying the part of the file it wants, and the server returns that part of the file back to the client using the HTTP 206 (“Partial Content”) Response.

This type of download abuse can easily overload your server’s connection limits and resources, and also get around any per-connection bandwidth restrictions you might have set.

Here is how to stop these Download Managers dead in their tracks by using mod_headers and mod_rewrite under Apache (or a WAMP Server such as WampDeveloper Pro).

This example will abort all partial requests for content located within URL:

http://www.example.com/files/

Unset Accept-Ranges Header

Indicate to the clients that the server will not attempt to honor Range requests (partial content requests), by changing Response Header “Accept-Ranges” from “bytes” to “none“.

<IfModule !mod_headers.c>
    LoadModule headers_module modules/mod_headers.so
</IfModule>

<Location /files/>
    <IfModule mod_headers.c>
        Header set Accept-Ranges none
    </IfModule>
</Location>

But this is only a superficial message to the client, that the download manager/accelerator software can easily ignore…

The client is still able to send “Range” requests (partial content requests), and Apache will still return the requested byte range of the file. So let’s remove that option once and for all…

Abort All Range (Partial Content) Requests

<IfModule !mod_rewrite.c>
    LoadModule rewrite_module modules/mod_rewrite.so
</IfModule>

<IfModule mod_rewrite.c>
    RewriteEngine On

    # Detect URL /files/...
    RewriteCond %{REQUEST_URI} ^files/
    # Detect "Range" request header
    RewriteCond %{HTTP:Range} !^$
    # Stop and Return HTTP FORBIDDEN (403) response header
    RewriteRule .* - [F,L]
</IfModule>

* If instead of placing this inside a VirtualHost block, you place it in an .htaccess file, then “AllowOverride FileInfo” (or “AllowOverride All“) and “Options +FollowSymLinks” (or “Options All“) have to be set (in the VirtualHost) for the directory the .htaccess file is in (otherwise neither mod_rewrite, nor working with the Header data, will work).

* Don’t use “RequestHeader unset Range” as this will get around the mod_rewrite configuration while turning all partial content downloads into full size downloads.

Problems

Incomplete downloads are not resumable… A client will not be able to pause or stop a download, and later resume it.

Downloads started with download accelerators will stop at whatever % of full file size the first connection retrieves.

May also break some -

  • Clients that do streaming of video and audio
  • Clients that do reading/loading of meta-data from large files
  • e-Readers
  • Client-side bandwidth throttling

Notes on using mod_rewrite

Per-directory Rewrites

http://httpd.apache.org/docs/current/mod/mod_rewrite.html#rewriterule

For mod_rewrite, “Options FollowSymLinks” must be enabled for anything related to directories to work…

To enable the rewrite engine in this context, you need to set “RewriteEngine On” and “Options FollowSymLinks” must be enabled. If your administrator has disabled override of FollowSymLinks for a user’s directory, then you cannot use the rewrite engine. This restriction is required for security reasons.

Behavior of mod_rewrite in <Location> sections can be unpredictable…

Although rewrite rules are syntactically permitted in <Location> and <Files> sections, this should never be necessary and is unsupported.

2 thoughts on “Blocking Download Managers and Accelerators

  1. Monte

    This article describes how to “eliminate” partial downloads and resuming with ranges which is the fundaments of a download manager. It however, doesn’t share how to provide quality downloads to the majority of my customers while preventing a few cheaters…

    So let’s say I have a fairly large file 10GB and I want to let clients download it. Based on your previous article, I could throttle in Apache. But if I setup the information in this article, and the connection expires, gets refused, or disconnected, any partial downloads is a waste to the customer and to me.

    So is this the best engineered solution? And if so, what header changes would I need to make to guarantee the slow-download connection doesn’t expire? Can I monitor how many simulataneous connections a user has? What suggestions do you have for this scenario?

    Reply
    1. admin Post author

      Monte, if you are serving a 10GB file via HTTP (via Apache) you are already making a mistake. HTTP is a stateless protocol, it was never designed to transfer large files, and everything that attempts to work around this is just patchwork.

      The above solutions are for typical cases, and for typical user-bases, that just click on a regular sized zip file to download it, instead of installing and using download managers to create 100s of connections. They are also the most simple and straight forward to implement.

      Only a few percentage points of some user-bases use download managers/accelerators, and the majority of them have it set up in a way that causes trouble to the server. Generally, you won’t miss much killing partial downloads unless you are hosting movie downloads (e.g., many of your users might be using download managers/accelerators).

      If you want to manage and session-track downloads via Apache and HTTP, I suppose the next best solution will be for you to write or find a PHP script that does so (i.e., a script that tracks the number of sessions and refuses more). * Just be careful when transferring the download through PHP, as it will grow the PHP process or thread to the size of the data you are sending.

      …And since these same broken download managers can’t seem to handle the refusal of partial requests (i.e., no fallback to 1 request), they also might not be able to handle the refusal of subsequent requests – and that PHP script might be a pointless exercise as half of those managers will still break.

      Anything more advanced than this, will most likely require a setup that goes outside of Apache… I’m sure there are FTP servers that have a solution you are looking for. Or if you are streaming, then streaming servers.

      Reply

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>