Sitecore Send: API Rate Limits

Background

Sitecore Send provides an API to interact and integrate with your custom system. You can check my articles about various APIs and how to use them: here and here.

If the API is used to subscribe and unsubscribe users, you can easily face API rate limits.

API Rate Limits

Current Limits

Up-to-date limits can be checked in the official documentation.

Currently, the limits are the following - number of requests per 10 seconds:

Note: All resource limits are counted separately, so you can execute 2 x Add multiple subscribers, 10 x Add subscribers, and 20 x Unsubscribe a subscriber from an account simultaneously within a timespan of 10 seconds without being rate limited.

Rate Limited Response

If you reach the rate limit, you'll get the following response:

{
  "Code": 429,
  "Error": "RATE_LIMITING",
  "Context": null
}

Additionally, information about rate limits is included in response headers, e.g.:

x-ratelimit-expires:    07/10/2024 14:12:52
x-ratelimit-firstcall:  07/10/2024 14:12:42

These headers can be handled to understand when the next call can be executed.

Handling Rate Limits in C# with Polly

The source for Polly policies can be checked here.

Rate Limiting Using Polly

This policy will limit number of executions withing specified timespan. If it's executed more often, then RateLimitRejectedException is thrown.

public static AsyncPolicy<T> ConfigureRateLimit<T>(int numberOfExecutions,
    TimeSpan perTimeSpan)
{
    return Policy
            .RateLimitAsync<T>(numberOfExecutions, perTimeSpan)
        ;
}

Retry if Polly Rate Limit Failed

If the request is executed more often than configured in the policy above, retry it 5 times with some delay.

public static AsyncPolicy<T> RetryRateLimitException<T>(int numberOfExecutions, TimeSpan perTimeSpan)
{
    var delay = Backoff.DecorrelatedJitterBackoffV2(medianFirstRetryDelay: perTimeSpan.Divide(numberOfExecutions) * 1.5,
        retryCount: 5);
    return Policy<T>.Handle<RateLimitRejectedException>()
        .WaitAndRetryAsync(delay);
}

Retry if HTTP Response is RATE_LIMITING

If the request is executed but the HTTP response contains a RATE_LIMITING error, retry it after the x-ratelimit-expires header date.

public static AsyncPolicy<T> RetryRateLimit<T>() where T : SendResponse?
{
    return Policy
        .HandleResult<T>((response) => response?.RateLimitDetails != null)
        .WaitAndRetryAsync(5, (i, result, _) =>
        {
            if (result.Result?.RateLimitDetails?.Expires != null)
            {
                var utcNow = DateTimeOffset.UtcNow;
                return result.Result.RateLimitDetails.Expires.Value.Subtract(utcNow) +
                        TimeSpan.FromMilliseconds(i * 50);
            }

            return TimeSpan.FromMilliseconds(i * 50);
        }, (_, _, _, _) => Task.CompletedTask);
}

Note: This code is based on the NuGet package SitecoreSend.SDK.

Result Policy

All previously mentioned policies can be combined together:

public class SendRateLimiterPolicy<T> where T : SendResponse?
{
    public AsyncPolicy<T> Instance { get; private set; }

    public SendRateLimiterPolicy(int numberOfExecutions,
        TimeSpan perTimeSpan)
    {
        Instance = Policy.WrapAsync(PoliciesHelper.RetryRateLimit<T>(),
            PoliciesHelper.RetryRateLimitException<T>(numberOfExecutions, perTimeSpan),
            PoliciesHelper.ConfigureRateLimit<T>(numberOfExecutions, perTimeSpan));
    }
}

And then configured for each endpoint separately:

public static class SendRateLimits
{
    public static readonly AsyncPolicy<SendResponse<IList<Subscriber>>?> AddMultipleSubscribers =
        new SendRateLimiterPolicy<SendResponse<IList<Subscriber>>?>(2, TimeSpan.FromSeconds(10)).Instance;

    public static readonly AsyncPolicy<SendResponse<Subscriber>?> AddSubscriber =
        new SendRateLimiterPolicy<SendResponse<Subscriber>?>(10, TimeSpan.FromSeconds(10)).Instance;

    public static readonly AsyncPolicy<SendResponse?> UnsubscribeFromAllLists =
        new SendRateLimiterPolicy<SendResponse?>(20, TimeSpan.FromSeconds(10)).Instance;

    public static readonly AsyncPolicy<SendResponse?> UnsubscribeFromList =
        new SendRateLimiterPolicy<SendResponse?>(20, TimeSpan.FromSeconds(10)).Instance;

    public static readonly AsyncPolicy<SendResponse?> UnsubscribeFromListAndCampaign =
        new SendRateLimiterPolicy<SendResponse?>(20, TimeSpan.FromSeconds(10)).Instance;
}

You can automatically use it together with the SitecoreSend.SDK NuGet package:

var client = new SendClient(
    apiConfiguration,
    httpClientFactory, 
    new RateLimiterConfiguration()
    {
        Subscribers = new SubscribersWrapper()
        {
            AddSubscriber = SendRateLimits.AddSubscriber.ExecuteAsync,
            AddMultipleSubscribers = SendRateLimits.AddMultipleSubscribers.ExecuteAsync,
            UnsubscribeFromAllLists = SendRateLimits.UnsubscribeFromAllLists.ExecuteAsync,
            UnsubscribeFromList = SendRateLimits.UnsubscribeFromList.ExecuteAsync,
            UnsubscribeFromListAndCampaign = SendRateLimits.UnsubscribeFromListAndCampaign.ExecuteAsync,
        },
    }
);

For more information, check the README of the NuGet package.

Suggestions to Minimize Rate Limits Errors

1. Use Bulk Actions

Sitecore Send comes with bulk actions by default:

• Import Members into List
• Archive List Members
• Unsubscribe List Members
• Delete List Members
• Copy Members from Another Email List

Maybe you don't need to use the API? Perhaps you can export subscribers into a CSV/XLSX file and perform a bulk operation in the Admin Panel?

2. Use Add Subscribers endpoint only to add NEW, not to update EXISTING

You may know that using the /v3/subscribers/{MailingListID}/subscribe.{Format} endpoint, you can not only create a new subscriber but also update information about an existing subscriber. Don't do it or do it very carefully!

This may cause API rate limits errors. The better approach is to use two requests:

• GET /v3/subscribers/{MailingListID}/view.{Format}?apikey={API_KEY}&Email={Email} - to get a subscriber by email (retrieve {SubscriberId} from the response)
• POST /v3/subscribers/{MailingListID}/update/{SubscriberId}.{Format}?apikey={API_KEY} - to update a subscriber by ID

3. Handle RATE_LIMITING Error Code Response

Always handle RATE_LIMITING error code with retries: using Polly as mentioned above or using any other mechanism.

4. Ensure subscribe/unsubscribe endpoints are used correctly and limited

On one of our projects, we had a webhook that sends requests to Sitecore Send with updated subscriber information. However, we did not filter anonymous requests, so many requests were sent incorrectly, leading to numerous RATE_LIMITING errors. After excluding anonymous requests, we resolved this issue.

I hope all the explanations and suggestions will help you avoid issues when using the API.