The challenge
The shipment tracking API limited how much data a single key could request. Pulling a full history in one go was not possible, and stitching partial downloads together by hand created gaps and duplicates.
The solution
The work is split into parallel fetch scripts and one consolidation step:
SplitEach fetch script uses its own set of API keys and date range
FetchEvery script saves its batch as JSON and Excel
MergeA consolidation script loads and combines all batches
Deduplicate & filterOne record per tracking number; targeted extracts such as partial deliveries
What the solution includes
Rate-limit handlingLoad spread across scripts and keys
No overlapsDate ranges segmented to avoid double counting
Two formatsJSON for systems, Excel for people
Data integrityDeduplication on the tracking number
Targeted extractsFiltered files appended without duplicates
Error handlingMissing files and invalid JSON handled gracefully
Results
- Full datasets retrieved despite API limits
- A single consolidated dataset with no duplicate shipments
- Output ready for Excel and Power BI without further cleaning
- Easy to scale by adding scripts or keys
Stack
Python
REST API
JSON
Pandas
Excel
Client names and confidential figures have been omitted. The images on this page contain no client data.