Note: This article was published a long time ago. I no longer host or maintain cors-escape.

The Scenario

While exploring the source code and network requests of a certain popular blogging site (we’ll call it, er… Maximum), you discovered that this site actually has an API which you can call to retrieve a user’s posts (more on that in a subsequent article). So you decide to add a section on your website where you can display your latest posts, gotten from their API via AJAX, like the boss you are.

The Problem

You set up everything:

let loadPosts = function () {
  let xhr = new XMLHttpRequest();
  xhr.onreadystatechange = function() {
    if (this.readyState === 4 && this.status === 200) {
      let response = JSON.parse(this.responseText);
      renderPosts(response);    }
  }
  xhr.open("GET", "https://maximum.blog/@shalvah/posts");
  xhr.setRequestHeader("Accept", 'application/json');
  xhr.send();
}

And this is what you end up with:

😭😭😭

Debugging

In case you’re not familiar with what’s going on here, here’s a quick run-through. This is a result of something known as “same-origin policy”. This means that an AJAX request made from one domain (for instance, google.com) can only access other endpoints hosted on google.com. Here’s a snippet from MDN:

For security reasons, browsers restrict cross-origin HTTP requests initiated from within scripts. For example, XMLHttpRequest and Fetch follow the same-origin policy. So, a web application using XMLHttpRequest or Fetch could only make HTTP requests to its own domain. To improve web applications, developers asked browser vendors to allow cross-domain requests.

To get around this, the concept of CORS (Cross-Origin Resource Sharing) was introduced. This allows requests to be made from one domain to another.

A simplified explanation of CORS (for GET requests) is that the resource owner (the guy you’re asking for stuff) can add the header Access-Control-Allow-Origin: google.com to their API responses if they wish to allow an AJAX request at google.com to pass. They could also make it Access-Control-Allow-Origin: * to allow requests from all domains. That’s CORS in a nutshell. (It’s a bit more complicated than that, especially when you’re using a different HTTP method.)

If you were the developer of the API, this wouldn’t be a problem. You could do that with just one line of code. But in this case, Maximum’s tech team isn’t so friendly, so you can’t tell them to do this juuuuust for you.

The Solution

What can we do?

• We could spoof (fake) the “Origin” header when making our request — something like this:

xhr.setRequestHeader("Origin", 'maximum.blog');

But because your browser is such a friendly policeman, this is what we get:

We still have another option:

• We can get someone else make the request for us and send us the response.

And, this, my friends, is what we’re going to do.

A bit of hunting on the web leads us to a solution that’s perfect for us (or almost) — CORS-Anywhere. CORS-Anywhere provides a proxy that passes on our request along with its headers. And it’s the source code is on Github, so you can host your own.

So if we modify our request URL like this:

xhr.open("GET", "https://cors-anywhere.herokuapp.com/https://maximum.blog/@shalvah/posts");

it should work. There’s actually one tiny cinch you might encounter, though.

You see, Maximum’s API only allows requests from the maximum.blog domain. But CORS-Anywhere passes along ALL our headers, including our Origin (which we can’t change, remember?). So we need a way of telling CORS-Anywhere to use the target url (maximum.blog) as the value of the Origin header.

In my case, I forked the repo and added a configuration option that allows you to spoof the Origin header, which defaults to true. You can check it out here. I also hosted this on Heroku as https://cors-escape.herokuapp.com. [Update: hosting proxies is against Heroku’s ToS, so it’s no longer available there.]

Now, if we run

xhr.open("GET", "https://cors-escape.herokuapp.com/https://maximum.blog/@shalvah/posts");

everything should run fine, and we get the expected response.

{  
    "status": "success",
    "payload": {
        ...
    }
}

Yay, Maximum!!

Alternative approach

CORS-Anywhere (or CORS-Escape, as I named my fork) uses a proxy server to send our requests. (You can read more about proxying here.)

Proxying is kinda like “passing on" your request, exactly as you sent it. We could solve this in an alternative way that still involves someone else making the request for you, but this time, instead of using passing on your request, the server makes its own request, but with whatever parameters you specified.

I built a small library that does that. Here’s the core of its code (NodeJS):

getPostData(req)
   .then(body => {
      console.log(body);
      let options = {
         uri: query.url,
         headers: body.headers,
         method: body.method || 'GET
     };

      let proxyCallback = (proxyErr, proxyRes, proxyBody) => {
        res.writeHead(proxyRes.statusCode, proxyRes.headers);
        res.write(proxyBody);
        res.end();
};
      request(options, proxyCallback);
   })

The request function dispatches the API call for us, and executes the proxyCallback when it’s done, in which it sends us the response.

The way this library works, all you need to do is make a POST request to the server and pass a url query parameter containing the url you want to access. You can also pass a request body specifying different options for the request (the headers you want and the method).

xhr.open("POST",
  "https://localhost:2000/?url=https://maximum.blog/@shalvah/posts"); // assuming you’re hosting it locally
xhr.setRequestHeader("Content-type", 'application/json’);
let data = {
  headers: {
    Accept: "application/json",
    Origin: "http://maximum.blog"
 },
  method: 'GET'
};
xhr.send(JSON.stringify(data));

The library returns all the response body and headers to you too.

You can check out the code here.

Phew!

Solved. 💃💃

Thanks for reading! Hit the Clap button if you feel clap-py (😊 Akapo Damilola Francis ). Also, if you have another solution to this problem, as always, I’d love to hear it. I’m always lurking in the comments section , and I’m on Twitter at @theshalvah.

Have a great day!

---

I write about my software engineering learnings and experiments. Stay updated with Tentacle: tntcl.app/blog.shalvah.me.