Ensuring browser compatibility of apps is an ongoing concern among browser developers. It is common for web apps to be required to run on multiple browsers.

Understanding the Problem

It is very common for businesses to require that their app works on certain browsers. Facebook, for example, requires that a version of their web app should still work on Internet Explorer, a browser known for its gaping holes in browser API support. Enforcing this requirement is met by testing the app on different browsers but doing so usually introduces development overhead because it requires tests to be written and time for the tests to be run.

In place of tests, it is common for developers to reference compatibility tables, which are tables which list all the browsers a given API is supported in. However many of these tables are either incomplete or non-existent for certain APIs because they are manually filled in by developers, which is not easy. There are roughly 10K APIs and 131 variants of browsers, which amounts to 1,310,000 API compatibility entries that must be filled in. Compatibility records are maintained by two projects: Mozilla Developer Network (MDN) and caniuse.

MDN

Examples of unknown API records:

caniuse

caniuse is the most complete collection of these API compatibility records. But this solution is difficult to scale. With a limited amount of developers with limited time and resources, it is difficult to maintain a table with thousands of records.

As you can see, caniuse has over 1K issues and only one active maintainer to address them.

Furthermore, developers manually checking a compatibility table isn't a scalable solution for handling browser compatibility inconsistencies. caniuse provides no scalable solution for integrating with existing static analysis (i.e. code analysis) tools.

proposed solution: compat-db

compat-db is a project that automates the generation of compatibility records of APIs. It takes APIs as input and return compatibility records as output. To generate all the browser compatibility records, it is given all the standard browser APIs as input and it returns all the corresponding compatibility records.

Input: WebIDL

APIs are defined with a special language called WebIDL. API authors write the specification for APIs in the form of WebIDL, a language which specifies the interface of the API. Here's an example of WebIDL corresponding to the TextEncoder.encodeInto API:

dictionary TextEncoderEncodeIntoResult {
  unsigned long long read;
  unsigned long long written;
};

[Constructor]
interface TextEncoder {
  TextEncoderEncodeIntoResult encodeInto(USVString source, Uint8Array destination);
};

JavaScript tests are generated from the WebIDL.

Here is a simplified version of the corresponding compatibility tests that are generated from the TextEncoder.encodeInto API WebIDL example above:

if (typeof TextEncoder === 'undefined') {
  return false;
}

if (typeof TextEncoder.encodeInto === 'undefined') {
  return false;
}

if (typeof TextEncoder.encodeInto === 'function') {
  return false;
}

return true;

Determining Compatibility: Browser VMs

The next step is to run these generated compatibility tests in each browser. This is done by dispatching the tests to run in remote browsers where it is then evaluated. These browsers are hosted by a number of services, including browserstack and saucelabs. The results of the evaluated tests are then returned as responses to compat-db.

Performance Considerations

Naive implementation

The naive implementation would need to run tests for every version of every browser for every API in order to build a complete compatibility table.

for (const API of APIs) {
  for (const browser of browsers) {
    for (const version of browser.versions) {
      dispatchCompatTest(API, version);
    }
  }
}

This algorithm is an O(m * n) algorithm, where m is the number of APIs that are tested and n is the total number of browser versions that are tested.

There are 10K API's, and 131 versions of all the browsers:

Summing the total number of browser versions, we get 131 total versions.

The naive method would require $$10,000*131=1,310,000$$ tests 😱.

Optimizations

We can observe some certain browser heuristics to reduce the number of tests required to build a complete compatibility table. Once a browser implements an API, it cannot be taken out. This is a general principle web browsers follow because they can't "break the web". If a browser yanks a previously supported API then it will break pages that depended on that API existing in that browser. APIs are deprecated but never removed (only in some extremely rare scenarios i.e. SharedArrayBuffer).

Using this heuristic, we run a modification that finds the first version of a browser that supports a given API. The algorithm almost identical to binary search: look at the sorted versions of any given browser.

Consider the example sorted versions of a browser:

[12, 14, 16, 18, 22]

If we execute compatibility tests against the "middle" version (16 in this example) of the browser and they return false then we know all the previous versions of that browser must not have had that API implemented before. So we look at the versions "right" of the "middle" version: [18, 22] and then recursively perform the previous operation.

This can be done in O(m * log n) time for one browser where n is the number of versions of a browser and m is the number of APIs that need to be tested.

When determining the runtime for all the browers we need to make some changes. First we'll create a list of all the browser versions and then sum over it:

$$ \sum_{i=0}^{|x|} m * log x_i = m * \sum_{i=0}^{|x|} log x_i = 64,393$$

This brings down our initial number of tests from 1,310,000 to 64,393, a 95% drop in the number of tests needed to determine the browser compatibility of all APIs.

Use cases for compat-db

The future of compat-db includes integration with static analysis tools, like the following projects.

Static Analysis

eslint-plugin-compat, an ESLint plugin which lints the compatibility of JavaScript code. Here's an example of it in use:

compat-db will serve as the source which contains the compatibility records which eslint-plugin-compat refers to for API compatibility lookups.

Automatically Including Polyfills and Shims

Polyfills and shims are pieces of code which implement an API if it is not supported by the runtime. For example, if chrome does not support the fetch API then a polyfill for it would implement that function using native JavaScript.

At the time of writing, there is no way of automatically including the necessary polyfills or shims for a codebase given a list of target browsers. Library authors include polyfills for their libraries with the intention of simplifying the process of using libraries. This way, library consumers do not have to worry about including polyfills themselves. However, this comes at the cost of bloating sizes of apps and libraries which consume these libraries.

An ideal solution, now made possible with compat-db, is iterating through every API used, check compat-db to see if it is supported by the browsers the user is targeting, and then append the polyfills to the user's build. This solution, which should be used only in the context of apps as opposed to libraries, ensures that only the necessary polyfills and shims are included in the compiled output of apps.

The Project

compat-db is hosted on GitHub. I'm actively looking for contributors so let me know if you're interested in contributing! There's some work that needs to be done to make compat-db production ready.