Improving Your API Conversations with Hypermedia, ETags, and Real-Time APIs

There is considerable source material on great API design. We can read discussions on how to use HTTP properly, from selecting the proper HTTP verb to which response code(s) are appropriate to return based on the success or failure of the request. These are all important things to consider when designing a great API.

As API providers and designers, we can forget what it is like to consume APIs as we get so busy building them. So, let’s revisit our API design from the client view of the API conversation to see how we can make it even better.

How API clients see our API design today

APIs tell a story. Part of that story is the conversation that is exchanged between the API client and server. It often goes something like this:

The problem is that we often add in update and delete support, then stop the design effort. Our API is not much of a conversationalist, is it? We need to look for ways to extend the conversation our API clients can have with our API. Let’s examine a few ways we can do this.

Extending the API client conversation with hypermedia

One consideration may be to add hypermedia constraints to your API. This moves the focus from a content-only design to a context-centric design. Our API can now tell our client what is possible (and what isn’t) based on the current state of the system, perhaps combined with the caller’s permissions of what their role allows them to perform (sometimes called entitlements).

The conversation starts to get a lot more interesting when we add hypermedia. For example:

So, now our client is interacting with the server using a hypermedia API. It is able to better understand both the state and actions available, along with links to related resources. The client can now present its user interface, showing or hiding links and buttons as appropriate based on the hypermedia links present (or absent).

Another advantage of hypermedia is that it can prevent the need to push out new versions of the app to our app stores when minor changes to business logic are made on the server. Our application is now a little more resilient to changes in server-side logic and our client will immediately reflect those changes if behavior is driven from the hypermedia links. Perhaps the client is a universal client that knows how to talk to any kind of hypermedia API (we can dream, can we?).

Extending the API client conversation with concurrency control

But what about when something changes on the server side? How will we know and ensure that we are kept in sync?

A common solution to this problem is to add API support for ETags and the If-None-Match header, pushing the business rules that determine when a resource has changed to the server rather than the client.

For those not familiar, an ETag is an opaque identifer for a specific version of a resource at a specific URL. They are often, but not always, represented as one of the following: the hash of the resource state, the last modified timestamp, or a version number. An ETag is sent back to the client when a resource representation is sent to the client. The client may then use the ETag to perform actions on the resource, but only if the ETag matches (or doesn’t match if that is the desired behavior).

Now when we are retrieving a resource representation, the API client will receive an ETag as well:

The client can then ask to update a resource only if the resource hasn’t changed since it was last retrieved:

It also prevents overwriting resources that have changed underneath us by forcing our requests to match a specific resource ETag/version:

This will save our clients lots of problems and ensure our server-side state remains consistent.

Extending the API client conversation with cache support

Sometimes our clients need to know if a resource representation has changed on the server side to prevent displaying or working with stale data. Currently, our API design would require our client to retrieve the most recent representation every time, whether it has changed or not. The client is responsible for trying to figure out if the resource has changed by comparing it to a previously stored representation, or by completely replacing any previously stored version. However, the client doesn’t know the business rules that indicate a resource has changed (and shouldn’t have to know) . We need a way to allow the API server to inform the client if a resource has changed since it was last retrieved.

To achieve this, most API clients will occasionally check for resource changes using the ETag approach described above. The conversation may go something like this:

 

This kind of loop has to be executed for every resource you want to monitor, which may be in the hundreds of resources. That’s a lot of conversations and work for the API client.

Instead, some APIs may offer an optimized solution, providing a specific endpoint to poll and receive any changes across a resource collection:

Now our API client can be informed when one or more resource representations have changed and refresh its local data to reflect those changes.

You may be thinking that polling an API seems like a clunky approach to keep client data up-to-date – you would be correct. We can solve this design problem by adding support for server-originated conversations.

Extending the API client conversation with real-time API messaging

Real-time API messaging changes the conversation dynamic from one-way (client-initiated) to two-way. The client doesn’t have to be the initiating party in this conversation. Instead, the server can start the conversation when it becomes aware of an internal state change. This means that we can start to have conversations like the following:

Now, clients can respond to change by allowing servers to initiate communication. Whether you decide to use Webhooks or WebSockets, opening up your API to two-way communication enables the creation of collaborative API integrations.

Moving the API design discussion forward

Many of our API design discussions today are focused on applying proper HTTP principles. There is nothing wrong with this and it is an important step in the API design process. However, by focusing only on a data-centric request-response design style, we are missing a great opportunity.

Instead, we should strive to design our APIs to change the entire conversation that is taking place with our API clients. These new conversations afford clients the opportunity to fully collaborate in conversations between the server. This includes support for real-time API messaging to allow APIs to engage in rich conversations with our API clients.