206. Design Retry Handler for API Calls
Asked in
Amazon LLD
Design Retry Handler for API Calls

Design a retry handler for API calls that may fail due to transient errors such as timeout, rate limiting, temporary server error, or connection failure.

The retry handler should support fixed delay, exponential backoff, and jitter backoff. The design should allow new backoff strategies to be added later without changing the retry handler logic.

Requirements

  • Register an API call with its retry policy.
  • Retry only when the error is retryable.
  • Do not allow more retries once the API call succeeds.
  • Do not allow retry attempts after the maximum retry attempt limit is reached.
  • Support fixed, exponential, and jitter backoff strategies.
  • Store retryable errors inside the retry handler.
  • Track retry attempts separately for each API call name.
  • All method calls should receive current time in milliseconds.

Constructor

RetryHandler(List<String> retryableErrors)

  • retryableErrors contains error types that can be retried.
  • Error matching is case-sensitive.

Method Signatures

String registerApiCall(String apiCallName, String backoffStrategy, int maxRetryAttempts, long baseDelayMillis, long currentMillisecondsNow)

  • Registers retry configuration for apiCallName.
  • This method should be called before updating execution results for that API call.
  • backoffStrategy is one of FIXED, EXPONENTIAL, or JITTER.
  • maxRetryAttempts is the maximum number of retry attempts allowed after the first API attempt.
  • Total API executions allowed for an API call are 1 + maxRetryAttempts, including attempt 0.
  • baseDelayMillis is used to calculate retry delay.
  • The first actual API attempt is considered attempt 0.
  • Attempt 0 is not counted in retryAttempts.
  • If apiCallName is not already registered, register it and return registered=<apiCallName>, attempt=0, retryAttempts=0.
  • If apiCallName is already registered, ignore the new values and return already_registered=<apiCallName>.

String updateExecutionResult(String apiCallName, String callResult, long currentMillisecondsNow)

  • Updates the latest execution result for apiCallName.
  • callResult is either SUCCESS,response or ERROR,errorType.
  • SUCCESS,response means the API call succeeded with response value response.
  • ERROR,errorType means the API call failed with error type errorType.
  • The first updateExecutionResult call for an API call records attempt 0.
  • Each later updateExecutionResult call records one retry attempt.
  • Therefore, retryAttempts is 0 after attempt 0, 1 after the first retry attempt, and so on.
  • If the result is success, mark the API call as successful.
  • If the result is a non-retryable error, mark the API call as non-retryable.
  • If the result is a retryable error and retry attempts are still available, mark it as retryable failure.
  • If the result is a retryable error but the maximum retry attempt limit has been reached, mark it as retry limit reached.
  • Returns a string in this format: status=<status>, result=<result>, retryAttempts=<retryAttempts>
  • status is SUCCESS, FAILED_RETRYABLE, FAILED_NON_RETRYABLE, or FAILED_RETRY_LIMIT_REACHED.
  • result is the successful response or the error type.
  • retryAttempts is the number of retry attempts made so far, excluding attempt 0.
  • For a non-retryable error, retryAttempts should be -1.
  • For FAILED_NON_RETRYABLE, return retryAttempts=-1 as a sentinel value, even if some retry attempts were made before the non-retryable error occurred.

long getDelayMillis(String apiCallName, long currentMillisecondsNow)

  • Returns the time left before the next retry attempt for apiCallName.
  • Returns -1 if the last error for this API call is non-retryable.
  • Returns -2 if the API call has succeeded.
  • Returns -3 if the maximum retry attempt limit has been reached.
  • Returns 0 if the next retry can be attempted immediately.
  • Otherwise, returns the remaining wait time in milliseconds.
  • The retry delay countdown starts from the currentMillisecondsNow value passed to the latest failed updateExecutionResult call.

int attemptsTillNow(String apiCallName, long currentMillisecondsNow)

  • Returns the number of retry attempts already made for apiCallName.
  • The first API attempt is attempt 0, so it is not counted.
  • Returns -1 if the last error for this API call is non-retryable.
  • For a non-retryable final state, this method returns -1 as a sentinel value instead of the actual retry attempt count.
  • If the API call has succeeded, returns the number of retry attempts made before success.
  • If the maximum retry attempt limit has been reached, returns the maximum retry attempt count reached for that API call.
  • The currentMillisecondsNow value in attemptsTillNow is used only for monotonic time validation and does not affect the returned count.

Time Rules

  • currentMillisecondsNow must be monotonically non-decreasing across all method calls.
  • For any two consecutive method calls, the later call must have currentMillisecondsNow greater than or equal to the previous call.
  • If the same timestamp is passed to multiple methods, it is valid.
  • The currentMillisecondsNow value in registerApiCall is used only for monotonic time validation and does not affect retry delay.

Retry State Rules

  • registerApiCall will be called before updateExecutionResult, getDelayMillis, or attemptsTillNow for the same API call name.
  • For the same apiCallName, only the first registerApiCall call should be used.
  • Later registerApiCall calls for the same apiCallName should be ignored.
  • Test cases will call updateExecutionResult only after the corresponding API attempt has happened.
  • Test cases will call updateExecutionResult for a retry attempt only when getDelayMillis would return 0 for that API call.
  • After an API call reaches SUCCESS, FAILED_NON_RETRYABLE, or FAILED_RETRY_LIMIT_REACHED, no further retry result will be added for that API call name.

Backoff Rules

  • For FIXED, delay is baseDelayMillis.
  • For EXPONENTIAL, delay is baseDelayMillis * 2^(retryAttemptNumber - 1).
  • For JITTER, delay is baseDelayMillis * 2^(retryAttemptNumber - 1) + retryAttemptNumber.
  • The jitter formula is deterministic to keep outputs simple.
  • retryAttemptNumber is the retry attempt that is going to happen next.
  • After attempt 0 fails, the next retry has retryAttemptNumber = 1.
  • After retry attempt 1 fails, the next retry has retryAttemptNumber = 2.
  • In general, when calculating delay after a failed call, use retryAttemptNumber = retryAttempts + 1, where retryAttempts is the number of retry attempts already made.
  • Test cases will ensure calculated delay fits in a signed 64-bit integer.

Output Format

The registerApiCall method should return one of these formats:

registered=<apiCallName>, attempt=0, retryAttempts=0

already_registered=<apiCallName>

The updateExecutionResult method should return:

status=<status>, result=<result>, retryAttempts=<retryAttempts>

  • status should be SUCCESS, FAILED_RETRYABLE, FAILED_NON_RETRYABLE, or FAILED_RETRY_LIMIT_REACHED.
  • SUCCESS means the API call succeeded.
  • FAILED_RETRYABLE means the API call failed with a retryable error and at least one retry attempt is still available.
  • FAILED_NON_RETRYABLE means the API call failed with a non-retryable error.
  • FAILED_RETRY_LIMIT_REACHED means the API call failed with a retryable error, but the maximum retry attempt limit has been reached.
  • result should be the successful response or error type.
  • retryAttempts should not include attempt 0.
  • For FAILED_NON_RETRYABLE, retryAttempts should be -1.

The getDelayMillis method should return:

  • -1 if the last error is non-retryable.
  • -2 if the API call has succeeded.
  • -3 if the maximum retry attempt limit has been reached.
  • 0 if the next retry can be attempted immediately.
  • Otherwise, the remaining wait time in milliseconds.

Constraints

  • 1 ≤ apiCallName.length() ≤ 100
  • 1 ≤ maxRetryAttempts ≤ 1,000
  • 0 ≤ baseDelayMillis ≤ 1,000,000,000
  • 0 ≤ retryableErrors.size() ≤ 1,000
  • 1 ≤ currentMillisecondsNow ≤ 1,000,000,000,000
  • currentMillisecondsNow will be monotonically non-decreasing across all method calls.
  • backoffStrategy will be one of FIXED, EXPONENTIAL, or JITTER.
  • callResult will contain exactly one comma.
  • The first part of callResult will be either SUCCESS or ERROR.
  • The second part of callResult will be non-empty.
  • The response value and error type will not contain commas.
  • No parameter value will be null.

Examples

Example 1

Constructor call:

RetryHandler(retryableErrors = List.of("TIMEOUT"))

Method calls and outputs:

registerApiCall(apiCallName = "OrderApi", backoffStrategy = "FIXED", maxRetryAttempts = 3, baseDelayMillis = 200, currentMillisecondsNow = 1000)

"registered=OrderApi, attempt=0, retryAttempts=0"

updateExecutionResult(apiCallName = "OrderApi", callResult = "ERROR,TIMEOUT", currentMillisecondsNow = 1000)

"status=FAILED_RETRYABLE, result=TIMEOUT, retryAttempts=0"

getDelayMillis(apiCallName = "OrderApi", currentMillisecondsNow = 1100)

100

getDelayMillis(apiCallName = "OrderApi", currentMillisecondsNow = 1200)

0

updateExecutionResult(apiCallName = "OrderApi", callResult = "SUCCESS,ORDER_CREATED", currentMillisecondsNow = 1200)

"status=SUCCESS, result=ORDER_CREATED, retryAttempts=1"

getDelayMillis(apiCallName = "OrderApi", currentMillisecondsNow = 1300)

-2

Explanation: The API call has already succeeded, so getDelayMillis returns -2.

Example 2

Constructor call:

RetryHandler(retryableErrors = List.of("TIMEOUT", "RATE_LIMIT"))

Method calls and outputs:

registerApiCall(apiCallName = "ProfileApi", backoffStrategy = "EXPONENTIAL", maxRetryAttempts = 3, baseDelayMillis = 100, currentMillisecondsNow = 5000)

"registered=ProfileApi, attempt=0, retryAttempts=0"

updateExecutionResult(apiCallName = "ProfileApi", callResult = "ERROR,AUTH_ERROR", currentMillisecondsNow = 5000)

"status=FAILED_NON_RETRYABLE, result=AUTH_ERROR, retryAttempts=-1"

attemptsTillNow(apiCallName = "ProfileApi", currentMillisecondsNow = 5100)

-1

getDelayMillis(apiCallName = "ProfileApi", currentMillisecondsNow = 5100)

-1

Explanation: The last error is non-retryable, so getDelayMillis returns -1.

Example 3

Constructor call:

RetryHandler(retryableErrors = List.of("RATE_LIMIT"))

Method calls and outputs:

registerApiCall(apiCallName = "InventoryApi", backoffStrategy = "EXPONENTIAL", maxRetryAttempts = 2, baseDelayMillis = 100, currentMillisecondsNow = 1000)

"registered=InventoryApi, attempt=0, retryAttempts=0"

updateExecutionResult(apiCallName = "InventoryApi", callResult = "ERROR,RATE_LIMIT", currentMillisecondsNow = 1000)

"status=FAILED_RETRYABLE, result=RATE_LIMIT, retryAttempts=0"

getDelayMillis(apiCallName = "InventoryApi", currentMillisecondsNow = 1100)

0

updateExecutionResult(apiCallName = "InventoryApi", callResult = "ERROR,RATE_LIMIT", currentMillisecondsNow = 1100)

"status=FAILED_RETRYABLE, result=RATE_LIMIT, retryAttempts=1"

getDelayMillis(apiCallName = "InventoryApi", currentMillisecondsNow = 1200)

100

getDelayMillis(apiCallName = "InventoryApi", currentMillisecondsNow = 1300)

0

updateExecutionResult(apiCallName = "InventoryApi", callResult = "ERROR,RATE_LIMIT", currentMillisecondsNow = 1300)

"status=FAILED_RETRY_LIMIT_REACHED, result=RATE_LIMIT, retryAttempts=2"

attemptsTillNow(apiCallName = "InventoryApi", currentMillisecondsNow = 1300)

2

getDelayMillis(apiCallName = "InventoryApi", currentMillisecondsNow = 1400)

-3

Explanation: The error type is retryable, but the maximum retry attempt limit has been reached. Therefore, updateExecutionResult returns FAILED_RETRY_LIMIT_REACHED and getDelayMillis returns -3.

Example 4

Constructor call:

RetryHandler(retryableErrors = List.of("TIMEOUT"))

Method calls and outputs:

registerApiCall(apiCallName = "PaymentApi", backoffStrategy = "JITTER", maxRetryAttempts = 3, baseDelayMillis = 100, currentMillisecondsNow = 2000)

"registered=PaymentApi, attempt=0, retryAttempts=0"

updateExecutionResult(apiCallName = "PaymentApi", callResult = "ERROR,TIMEOUT", currentMillisecondsNow = 2000)

"status=FAILED_RETRYABLE, result=TIMEOUT, retryAttempts=0"

getDelayMillis(apiCallName = "PaymentApi", currentMillisecondsNow = 2050)

51

Explanation: After attempt 0 fails, the next retry is retry attempt number 1. For retry attempt number 1, jitter delay is 100 * 2^0 + 1 = 101. Since current time is 50 milliseconds after the failed call, time left is 51.

Example 5

Constructor call:

RetryHandler(retryableErrors = List.of("TIMEOUT"))

Method calls and outputs:

registerApiCall(apiCallName = "SearchApi", backoffStrategy = "FIXED", maxRetryAttempts = 3, baseDelayMillis = 100, currentMillisecondsNow = 3000)

"registered=SearchApi, attempt=0, retryAttempts=0"

registerApiCall(apiCallName = "SearchApi", backoffStrategy = "EXPONENTIAL", maxRetryAttempts = 10, baseDelayMillis = 500, currentMillisecondsNow = 3000)

"already_registered=SearchApi"

Explanation: The second registration is ignored because SearchApi is already registered.



Please use Laptop/Desktop or any other large screen to add/edit code.