Loop API Calls for Historical Data

This article is related to the PurpleAir API. If you are unfamiliar with it, check out our API Landing Page to get started.

The historical API can be used to download PurpleAir data from any date range. However, there is a limit to the amount of data that can be collected per request. Multiple requests will need to be made to collect data past that limit.

The time limits for each average are listed below. Averages are on the left, and the time limit is on the right.

Average Time Limit
Real-time 30 days
10-minute 60 days
30-minute 90 days
1-hour 180 days
6-hour 1 year
1-day 2 years
1-week 5 years
1-month 20 years
1-year 100 years

Example

Say you were collecting historical PM2.5 data using a real-time resolution from January 1 to February 4. Only 30 days of real-time data can be collected per request, so two requests need to be made; the first request for January 1 - January 30 and the second request for January 31 - February 4. These dates will be in UTC.

The request URLs may look like this:

Request 1
https://api.purpleair.com/v1/sensors/123456/history?start_timestamp=2023-01-01T00:00:00Z&end_timestamp=2023-01-31T00:00:00Z&fields=pm2.5_atm

Request 2
https://api.purpleair.com/v1/sensors/123456/history?start_timestamp=2023-01-31T00:00:00Z&end_timestamp=2023-02-05T00:00:00Z&fields=pm2.5_atm

Notice how the end timestamp for the first request is January 31. This is because this date is exclusive and will not appear in the data. The start timestamp is inclusive, meaning data on that date will be collected. The first example request will get data from January 1 at 12:00 a.m. to January 30 at 11:59 p.m.

API Data is Not Sorted by Time

Historical data downloaded from the PurpleAir API is not sorted by default and must be done on the client side. This client-side sorting is automatically done when using the PurpleAir Data Download Tool.

Looping with Code

Programming languages will allow you to loop through a date range and automatically make the required number of API requests. The PurpleAir Data Download Tool does this in the background without the user needing programming experience.

Wait Between Requests

Rate Limiting

The PurpleAir API uses a rate limit to protect the functionality of the service. If you send too many API calls in quick succession, you will receive an error from the API.

For historical endpoints, you run the risk of rate limiting if your requests are sent less than 1 second apart.

Some real-time endpoints have different rate limits than historical endpoints and can be called more frequently.

Avoiding Rate Limiting

One way to avoid rate-limiting is to insert a delay between every API call. This isn’t perfectly efficient, but is the easiest way and may only require a single line of code.

Some suggested delays are included in the table below:

Delay Behavior
1-second A delay of 1 second between each call will ensure you never encounter rate-limiting.
500 milliseconds This will typically be sufficient to avoid rate-limiting in most cases. However, it can still occur, especially when sending many requests in a row.
250 milliseconds For smaller amounts of API calls, you might not encounter rate-limiting. When sending large numbers of requests, it’s more likely that you will encounter rate-limiting errors.
2 Likes