Build your own CDN (on a Raspberry Pi)

Utilize Cloudflare’s CDN to serve files

Utilize Cloudflare’s CDN to serve files

PDF Version of this post

I’m a huge fan of Cloudflare Workers (serverless compute, don’t fight me on the term “serverless” XD) because it allows me to add end-layer logic so I don’t have to go back and change my app’s logic and build and ship it – all I have to do is configure Cloudflare Workers and it’ll deploy on any and all domains and be on Cloudflare’s expansive edge network to quickly run whatever I need.

Cloudflare a while back made a blog post about how to make your CDN, but with Google Cloud Buckets. This is a post that shows you: How to build a CDN just with a simple webserver you may already have (not tied to any storage platform). You will need a Cloudflare account to make these workers.

Essentially what we’re doing is: You serve requests from your webserver –> Cloudlfare Workers caches it in Cloudflare’s CDN –> Serve to visitors

Step 1: Create a simple web server

As I’ve said, you’ll need a regular web server to serve content. If you haven’t already I’d reccomend just spinning up an NGINX web server (very basic). I run my web server on a Raspberry Pi and it works perfectly fine (more details on how I set it up soon). If you want to see how basic I’m talking, here’s the NGINX Conf file I have running: My CDN NGINX config.

So for example, I have a cool gradient background served on a basic NGINX web server I setup. You can see here: picture

cool multi color gradient background

Step 2: Make a Cloudflare Worker

Now that picture is a couple of megabytes that I can’t be constantly serving from my web server (both because my ISP would get interested in why I am serving possibly terabytes of data over my regular plan and because my CDN is hosted on a Raspberry Pi which could slow down other services)

So in the Cloudflare Worker browser editor, add:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event))
})

This basically says: Whenever the worker is requested, handle that request via the handleRequest method.

So now lets make the handleRequest method:

async function handleRequest(event) {
  if (event.request.method === 'GET') {
    if((new URL(event.request.url)).pathname == ("/" || ""))
    {
      return Response.redirect("https://cdn.sdan.cc/", 301)
    }
    let response = await serveAsset(event)
    if (response.status > 399) {
      if(response.status = 404){
        return Response.redirect("https://cdn.sdan.cc/404", 301)
      }
      response = new Response(response.statusText, { status: response.status })
    }
    return response
  } else {
    return new Response('Method not allowed', { status: 405 })
  }
}

Here’s whats going on:

  1. Event is passed to the function as a parameter
  2. Only if the event is a ‘GET’ request (like the visitor is trying to ‘GET’ that sweet multi color gradient background) go ahead
  3. If the visitor is trying to hit the landing page (like sdan.cc/) then just forward them to the landing page of the web server… you shouldn’t be serving anything/anything heavy on your landing page)
  4. If they are trying to reach a real asset (like the colorful background) lets setup a Promise in the form of variable response to the function serveAsset which caches the asset upon request
  5. If the asset isn’t there, serve a 404 page/appropriate error page

Basically: If the request is to a legit file (like the background) then asynchronously make a request to serveAsset to cache it and then eventually serve it to the visitor.

Here’s where the caching magic happens:

async function serveAsset(event) {
  const url = new URL(event.request.url)
  const cache = caches.default
  let response = await cache.match(event.request)

  if (!response) {
    response = await fetch(`${BUCKET_URL}${url.pathname}`)
    const headers = { 'Access-Control-Allow-Origin': '*', 'cache-control': 'public, max-age=14400' }
    response = new Response(response.body, { ...response, headers })
    event.waitUntil(cache.put(event.request, response.clone()))
  }
  return response
}

My BUCKET_URL is equal to https://cdn.sdan.cc

This asynchronous function does the following:

  1. If the file has already been cached on Cloudflare CDN’s edge network, return that response
  2. If the file has never been cached by Cloudflare or it has gone stale (if Cloudflare hasn’t gotten a refreshed version of it within the next hour or so)
    1. Cache the file on Cloudflare CDN’s edge network
    2. Set the headers on that file such that it caches on the visitor’s browser for 4 hours
    3. Return that response

So once serveAsset finishes up, we go back up this recursive chain:

  1. serveAsset resolves the response Promise in handleRequest
  2. Now that response is resolved, handleRequest can resolve the initial request of the visitor

Step 3: Clearing things up

My web server is at cdn.sdan.cc and my CDN is at sdan.cc which is… confusing, but I did it because a ton of legacy links are pointed to sdan.cc.

So cdn.sdan.cc serves requests straight from my Raspberry Pi and sdan.cc serves it via Cloudflare’s super fast edge network.

In DNS terms: cdn.sdan.cc is pointed to my servers and sdan.cc is pointed to nothing (technically its pointed to my servers, but that is because Cloudflare worker routing don’t work unless its pointed at something)

Cloudflare Worker configuration:

My worker name is cdncdncdn.surya.workers.dev and you can name it whatever you want.

To setup custom routing (which you will need to do) go under your domain (which presumably is hosted on Cloudflare) and under the “Workers tab” add a route as such:

https://sdan.cc/*

And point it to the worker you just created. Pretty soon you’ll have your own CDN!

Final Thoughts

I use Cloudflare workers as end-layer logic and its really amazing once you get to know what you can do with it. Honestly the marketing for both Workers and Lambda is a bit weird IMO and you’ll need to get your hands dirty with “serverless” for a while until you can do something really cool.

I for one made Rapid Analytics which is the 1st fastest website analytics platform after Google Analytics and I say this with extensive testing (more details coming soon). This is possible only with Cloudflare Workers since its more expansive than any other platform (172 compared to AWS lamba’s 92… as far as I remember).

Code

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event))
})

const BUCKET_URL = `https://cdn.sdan.cc`

async function serveAsset(event) {
  const url = new URL(event.request.url)
  const cache = caches.default
  let response = await cache.match(event.request)

  if (!response) {
    response = await fetch(`${BUCKET_URL}${url.pathname}`)
    const headers = { 'Access-Control-Allow-Origin': '*', 'cache-control': 'public, max-age=14400' }
    response = new Response(response.body, { ...response, headers })
    event.waitUntil(cache.put(event.request, response.clone()))
  }
  return response
}

async function handleRequest(event) {
  if (event.request.method === 'GET') {
    if((new URL(event.request.url)).pathname == ("/" || ""))
    {
      return Response.redirect("https://cdn.sdan.cc/", 301)
    }
    let response = await serveAsset(event)
    if (response.status > 399) {
      if(response.status = 404){
        return Response.redirect("https://cdn.sdan.cc/404", 301)
      }
      response = new Response(response.statusText, { status: response.status })
    }
    return response
  } else {
    return new Response('Method not allowed', { status: 405 })
  }
}