The Ultimate Guide to API Versioning

Successful software always gets changed.
– Frederick P. Brooks

What is true for software in general, also holds for APIs: successful APIs get changed. The reason is simple: successful APIs are used by various API consumers who demand new features, extensions, bug fixes and optimizations. From this perspective, changes in APIs are inevitable.

But this is only half the story. Successful APIs are used by various API consumers, who rely on the API to remain stable to avoid breaking their existing integrations. Their software clients have dependencies on the API. Even a small change in the API is enough to break some of the clients consuming the API. From this perspective, changes in APIs are undesirable.

Obviously, there is a dilemma. And it is the task of the API provider to resolve this dilemma. In this article, I want to give you some tips for dealing with it. To deal with change and versioning effectively, we need to take a life cycle approach. From the perspective of the API life cycle, we look at the API from its inception and design all the way to its retirement. And I hope it does not come as a surprise to you – but dealing with change is also about being prepared. So you need to start thinking about changes to the API way before the concrete change proposal is appearing on the radar.

We can divide the life cycle of an API into two big phases: the design-time phase before publishing the API and the phase after publishing the API. In these two phases, you need to deal with changes in a very different manner; in fact, when transitioning from one phase to the next, your mindset about change needs to switch 180 degrees.

Before publishing the API

When you initially design an API, you might not even think about future changes. But they will come. So here are three tips on how to support the evolution and versioning of your API already during design time.

Before publishing the API: welcome feedback in iterative API design

Before the API is published, change is great - you need to welcome it and seek it out. Use an iterative development methodology, elicit feedback from beta customers, adapt the API spec based on the feedback you have received on API mocks and beta-versions of the API. This iterative development should ensure that the API you create is actually quite stable, and should reduce the likelihood of immediate future changes. But don’t stop with your API design here and publish the API.

Before publishing the API: prepare an API design pattern for versioning

Create an API design pattern for versioning right from the start when you publish the API for the first time. It might seem superfluous at the time of initial design, but it allows you to roll out consistent versions in the future and sets the right expectation for API consumers. Ideally, the API design pattern for versioning chosen for your API is documented in your API design style guide.

While the most commonly observed pattern is realizing API versioning as the URL path parameter, there are various alternative API design patterns for making major versions accessible:

  • Realize API versioning in the Accept header: the client can use the HTTP Accept header to explicitly indicate the version of the API. The Accept header still contains the MIME-types as usual, but in addition, the version is appended. The URL of the API does not contain any version information. For new versions, the URL does not change, but the request header.
  • Realize API versioning as URL path parameter: the most common technique for versioning uses a URL parameter with the version number
  • Realize API versioning in a custom HTTP header: a custom HTTP header could be defined for the version
  • Realize API versioning as a query parameter: a query parameter could be defined for the version
  • Realize API versioning as a new subdomain: a subdomain could be defined for the new version

Before publishing the API: anticipate future change

Anticipate future changes to your API and accommodate for it in your API design. Everyone knows that this is difficult as there are no hard facts about the future. But it is important preparation for the evolution and versioning of your API. Base your assessment on experience and domain knowledge. Maybe you know there is an upcoming industry standard in the domain of the API that needs to be supported. Maybe you know that some scalar data fields will likely need to become an array in the future. The point is to prepare the API spec so that evolution in the anticipated direction is possible without breaking changes.

After publishing the API

After the API has been published, your attitude to change in the API needs to be much more conservative in the interest of your existing API consumers. Change needs to be managed much more rigorously. The basic rule is that the externally observable behavior of an API (from the perspective of the clients) cannot be changed, once the API has been published – and the exception to the rule is: creating a new version of the API.

Managing change – step 0 – is the change really necessary?

First, assess if the proposed change is really necessary. If you can avoid the change: great. One less problem.

Managing change – step 1 – is the change backward compatible?

If the change is really necessary, determine what the change means for the API contract — typically your OpenAPI spec — and for the API consumers relying on the API contract. Ideally, you have an OpenAPI spec for the published API and a new API spec that has all the proposed changes in it. Assess each of the changes and determine whether they are backward compatible or incompatible.

Backward compatible changes:

  • Adding query parameters (they should always be optional)
  • Adding header or form parameters as long as they are optional
  • Adding new fields in JSON or XML data structures as long as they are optional
  • Adding endpoints, e.g. a new REST resource
  • Adding operations to an existing endpoint, e.g. when using SOAP
  • Adding optional fields to the request interfaces
  • Changing mandatory fields to optional fields in an existing API

Incompatible changes (a.k.a. breaking changes):

  • Removing or changing data structures, i.e. by changing, removing or redefining fields in the data structure
  • Removing fields from the request or response (as opposed to making it optional)
  • Changing a previously optional request field in the body or parameter into a mandatory field
  • Changing a previously required response field in the body or parameter into an optional field
  • Changing the URI of the API, such as hostname, port or path
  • Changing the structure or relationship between request or response fields, e.g. making an existing field a child of some other field
  • Adding a new mandatory field to the data structure

Managing change – step 2 – dealing with backward compatible changes

If you have found no incompatible changes, only backward compatible ones, you should be able to implement them without breaking clients. Apply semantic versioning and give the API release that covers the changes a new minor version. This new minor version will replace the published API already available. Old clients should not be affected by the changes while new clients can take advantage of the new functionality.

There is one general issue with changes even if they are backward compatible. Backward compatibility is based on the API spec, the OpenAPI specification, which serves as the "official contract” with the API consumer. But Hyrums law tells us that, given enough API consumers, there will be an API consumer relying on any observable behavior of your API, not only on the behavior described in the Open API specification.

Managing change – step 3 – dealing with breaking changes

If you have found at least one incompatible change, it will break clients that rely on the changed feature. A new major version of this API is required, and this new version needs to be maintained in parallel with the previous version. Moreover, major versions need to be visible to the consumers, and they need to be accessible by the consumers. If you have followed this guide from the start, you already have a mechanism in place for picking different versions. See section (Prepare an API design pattern for versioning).

(TODO: alternative - remove: incompatible changes need to be implemented in such a way that existing clients are not affected. This can be accomplished by creating a new major version of the API and maintaining the unchanged API alongside the changed API. Creating and publishing a new version of the API is overhead and thus slows down any changes to the API.)

Managing change – step 4 – sunsetting

Operating and maintaining multiple versions of the API requires a lot of effort on the side of the API provider. Sooner or later, old versions of the API need to get turned off. But this should not be a sudden event, rather a gradual one – like a sunset. That’s why we call this phase the API sunset.

You need to set clear expectations for the API sunset and communicate it as early as possible to all affected API consumers – giving them as much time as possible to deal with the sunset and lift their API clients to the new version of the API. You need to know your API consumers and have a way to contact them, which is typically in place when API consumers need to be onboarded. Get API consumers on a mailing list – not for marketing purposes but for product changes. Send out a targeted message on the mailing list to all API consumers when a new version of the API is published, including a timeline for switching off the old API. Motivate your API consumers to switch to the new version in time and explain the benefits of using the new version.

Conclusion

We have seen that dealing with versioning in APIs starts way before the first breaking change is introduced in an API, and even before the API is initially published. A properly designed API will likely be able to live without breaking changes because it is iteratively designed, future changes have been anticipated and a versioning pattern is defined. And if those changes eventually come, you can categorize the changes in backward compatible and breaking change and react accordingly with minor and major versions and eventual sunsetting of API versions.

About the Author Dr. Matthias Biehl

image

Matthias empowers customers to discover their opportunities for innovation with APIs & ecosystems and turn them into actionable digital strategies. Based on his experience in leading large-scale API initiatives in both business and technology roles, he shares best practices and provides strategic guidance. Matthias works as an API strategist at Software AG, publishes a blog at API-University, is the author of several books on APIs, and regularly speaks at technology conferences.

:twitter: :linkedin:

One option that is missing from this (very informative) guide is – don’t version the API.

One can find various articles about the downsides of versioning, whichever option is chosen. It should not be foregone conclusion that your API must have versioning. One counter-example to consider is at API Versioning: What Is It and Why Is It So Hard?

Most APIs don’t need versioning; they need the ability to support compatible changes over time (not as flashy a term, right?).

The most effective change-compatible APIs can continue to add features without ever breaking already existing API consumers—even ones created years ago. The least effective API changes require all API producers and consumers to rewrite and re-release their code each time the API design is updated.The key to success is to design-in support for compatible changes from the very start.

And sometimes these API consumer teams don’t want or need the new features anyway.

If you’re committed to releasing breaking changes, you also need to commit to side-by-side release, documentation, online support, and so on.

Make every effort to avoid making breaking changes. Follow the 3 rules advocated in the above blog. API clients will appreciate it.

Thanks for the comment! Not versioning is covered by Step 0 Is the change really necessary. The challenge of building APIs without forseeing the capability for versioning from the start, is that you cannot fix it later. Psychology typically plays a trick on API designers here - when you publish the first version of your API, you cannot imagine that anyone would ever want to change the API. It is easy to skip over potential future changes. Two years later, the world might have changed, and you may anyway need to create a new version. Better be prepared from the start, you still may use it or not use it.

Sort of. Not versioning is not the same as not changing. We agree that considering whether a change is really needed is good – but as you’ve rightly noted, change is inevitable.

We agree that one needs to plan for change. Where we may differ is on the focus on versioning. Change is inevitable. But don’t push the cost and effort of all changes to API clients – which is what most change and versioning schemes seem to do.

The real intent of the “don’t version” notion is to plan for changes without breaking existing clients. Noted in the “API Versioning” blog and in your post is to commit to non-breaking changes. And if that cannot be done, fork the API and support both/all. From the blog I referenced:

If you know you need to introduce a breaking change, fork your API and release it in a new URL space. Call that a “version” if you like. You’re still not James Bond though, so be aware that releasing new versions does not give you license to kill the old ones.

The versioning schemes that API clients will object to are those that force a change to something (URL, header) simply because the version number changed. Indeed, I’m working with one right now that is going to force a URL change every 3 years, even if the API itself offers nothing new for what we need.

Edit: An example of an API that changes without breaking existing stuff yet adding new functions, there is an API (XML over HTTP) that has been in use for 10+ years. It does not use a version number anywhere and have 30+ sets of revisions in that time. We’ve never had calls we’ve been using break due to changes nor had to make “administrative” changes to keep up with a version number. The only time we’ve changed is to use new objects/fields/functions. They always add, never take away. It is possible to do. :slight_smile: