On dev prod parity

Posted on Dec. 6, 2017, 10:07 p.m.

A piece of required reading for all developers is the Twelve-Factor App. Mastering this guide is a critical step towards understanding modern application development. All of it matters, but nothing compares to dev/prod parity. This is a post illustrating just how surprising differences between your environments can be. And how much trouble they can cause.

At its core, the dev/prod parity section emphasizes that your development environment should mirror the production environment as closely as possible. That much is clear. Ideally, it's identical: same tech, same content, same networking layout.

In the real world, this isn't always possible. At work we connect to trading systems: it's impossible to guarantee the same content in a lower environment. Or worse yet, you may need to rely on integration systems that can never be run in a lower environment. Need to connect to the live Edgar Filing feed in production? Good luck getting access to a local version.

In these case, you do the best you can: mock services, cache them, etc. There'll be more risk, but there's little you can do.

But there are other areas where disparity can creep up on you. Are your lower environments properly encrypted? Make sure they are.

What about your URL structures? Does your staging environment exactly mirror the production's URL in format/convention?

Guess what? If your production environment is at https://www.silviogutierrez.com and your staging environment is at https://stage.www.silviogutierrez.com, you might just run into problems.

In my case, on a work project, we had a similar setup to the above URLs. Production was acting fine, but lower environments did not correctly display custom fonts in Internet Explorer. How was that possible?

After hours and hours of packet sniffing and debugging, we noticed the lower environments injected a Vary: * header. None of our code was doing it. It took critical thinking to realize that our CORS library was injecting it. But why in lower environments but not in production?

Turns out, said library let you whitelist a few URLs. In a moment of weakness, the regex I provided skipped only accounted for two sub sections after the TLD. On the lower envs, this introduced a special case in the library that injected the header.

In production, it didn't.

Fortunately, this was a bug that affected only dev environments, but not the real thing. But it illustrates just how sneaky dev/prod disparity can be. Mirror production as closely as possible. Then mirror it some more.

Tags: