Historical Data Bulk Downloads

Data
1

Hello all,

I’m reaching out to request raw data over a five-year period for the state of California (2017-2022). My team is conducting research and had been working with the AirSensor package, but unfortunately with recent changes, are having some difficulties.
We have tried the following:

  • Requesting using data using the API key with historical access caps removed
  • Scripting, but can only access two days’ worth at a time
  • Sensor data download tool

Is there any advice for this situation?

3 Likes
2

I’m having same problem. Can’t figure out how to download long term historical data now that the old Sensor Download Data Tool has been retired. The most data I am able to download is 2 weeks, when in past I have been able to download multiple years. Appreciate any help on how to download large intervals of historical data.

3 Likes
3

I am also having the same problem. I need two years of data from 6 sensors and am trying to figure out a way to get the data via the API with limited success. Have any of you tried to email purpleair support?

2 Likes
4

If you wanted hourly data, you’d need to break your individual requests into 2-week increments (the max request period for hourly data) per sensor, going back 2 years. You’d likely wanted to loop your input variables (sensor index, start time, end time) over the request URL in a program like R, and compile those requests over time.

2 Likes
5

PurpleAir Guys we all need our data back online as soon as possible. The server change is by no means a believable excuse.

6

Att Purple Air, I ask …Where is your heart

Note you at PurpleAir stated from the start.
PurpleAir is built on the ideal of sharing data and by doing this, empowering individuals to effect change. All public PurpleAir data is available to download under various licenses

Now your sensor users find that their data is NOT freely available for download due to the very limited (crippled) tools that are now provided for users. The very limited tools of today now make the data for practical purposes for many uses unavailable.
So again I ask the staff at Purple Air…
Where is your heart…

1 Like
7

Any chance you have some example code you’d be willing to share??

8

This thread has a link to a generalized version of the script I use to pull the last two weeks of hourly data for the sensors we have deployed.

Also, here is a snippet of code that loops the request period (2 weeks of hourly data) over the request URL, instead of looping the sensor index over the URL, like I show in the script I linked:

#API keys assigned by PurpleAir support
read_key <- "your key here"
write_key <- "your key here"

#end date is today, start date is the beginning of last year
start_date <- as.POSIXct("2022-01-01")
end_date <- as.POSIXct("2023-02-14")

while(start_date <= end_date) {
  #define end of 2 week period, max request period for 1hr data
  end_period <- start_date + days(14)
  
  #format start and end times as UNIX timestamps
  start_timestamp <- as.numeric(start_date)
  end_timestamp <- as.numeric(end_period)
  
  #substitute start_timestamp and end_timestamp into request URL
  #replace "sensorindex" with your actual sensor index
  url <- paste0("https://api.purpleair.com/v1/sensors/sensorindex/history/csv?start_timestamp=", start_timestamp, "&end_timestamp=", end_timestamp, "&average=60&fields=pm2.5_cf_1_a%2C%20pm2.5_cf_1_b%2Cpm2.5_atm_a%2C%20pm2.5_atm_b%2Cpm2.5_alt_a%2Cpm2.5_alt_b%2Chumidity%2Ctemperature%2Cpressure%2Cuptime%2Crssi%2Cpa_latency%2Cmemory")
  
  #send API request and write to text file
  data <- GET(url,add_headers('X-API-Key'=read_key))
  data <- content(data, "raw")
  writeBin(data, paste("sensorindex",start_timestamp,end_timestamp,".txt", sep="_"))

start_date <- end_period + 1
flush.console() #this makes sys.sleep work in a loop
Sys.sleep(60)
}
2 Likes
9

Thank you so much Chloe!!

1 Like
10

I just uploaded a python script to download and compile historical data from the API here. Hope it helps.

1 Like
11

Yesterday we released a temporary download tool for users to download PurpleAir data. We have a community article about it here: PurpleAir Data Download Tool.

2 Likes
12

Hello Ethan. I just used the temporary download tool to attempt to get PM2.5 data from May 2022 to September 2024. I followed all instructions and have more than enough points, however, I only got data for 1st January 2022 and 1st january, 2023. What can I do to correct this, please? Thank you.