Importance of idempotency in your backend

Importance of idempotency in your backend

According to Idempotency’s definition on Wikipedia:

Idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application.

This is a crucial attribute of a sensible API and can prove to be a must-have depending on the use case. Taking an example of an online payment processing service, you wouldn't want to charge your users twice if they happen to be impatient & click the button more than once.

Understanding the importance

Let's take an example of a URL shortener with idempotency in place:

url-shortener-lock-cycle.jpeg

  1. It gets two exact same requests at the same time, and one of them successfully acquires the lock. The other doesn't get processed further.
  2. Once the operation succeeds, the lock is released.

But what if the locking mechanism was not in place for this URL shortener? We would've got multiple short links for the same long URL:

multiple short links for the same URL

Approaches

There are two ways to implement idempotency in an API:

  • Attach a request ID to each incoming request and map out every API response to a unique request ID. Return the same response if a request with the same request ID comes again. Or,
  • Simply reject the request (with a message "Already processed!"?) if the request ID is already seen before.

Again, the idea is to avoid spending resources processing what's already processed once.

Testing

You can play around and see the results for yourself by cloning this repo.

Here are the steps:

  1. Install the packages.
  2. Make sure you're on the main branch.
  3. Run the project locally.
  4. Run this script on another terminal window.
  5. Observe the short links created.

You should see different short links for the same original long URL. To see how idempotency fixes this issue,

  1. Checkout to lock branch.
  2. Run the project again.
  3. Run the same script.
  4. Observe the short links created now.

You would see how idempotency helps you avoid the scenario of multiple short links for the same original URL.

What next?

  1. Check out Redlock npm package to understand how you can leverage the Redlock algorithm to acquire & release locks.
  2. Read their official blog post on distributed locking.