API Call Required for Project Stupidly Slow

[kitkat9933]

I have a project for work where it relies on heavy use of API calls to a 3rd party service. However, these calls are painfully slow and when adding some PHP back end or JS processing on this data-set can take up to 30-40 seconds (with small amount of objects which will grow exponentially) to return the result to the user. Now while I can shave a tiny bit of this I'm sure with improved code, it's not going to make the world of difference.

The call gets a list of objects and then basically loops through and gets a list of attributes for those objects, so essentially looping through the original returned objects the API offers no other way to get this required result.

Now, I looked into running background API calls and caching the result and even caching the result every set run through, however the problem is the resulting information needs to be completely accurate as important decisions could be made on the result of it (not that they aren't reversible just a pain in the arse). The alternative is to load the page and then give a spinning wheel of death for 30+ seconds.

Now I know I'm pretty stuck as poor user experience is the lesser of two evils compared to accurate data but I'm wondering if anyone knows of any better tricks/trips to get around this?

 

[kitkat9933]

Which part is slow, the API call returning data or the calculation loop?

Can the loop be parallel? Can it be batched in any way?

So first API query is sent and takes <1s to return. Then processing makes an API call for each element of the array returned in the original API query. The subsequent queries onwards are what take the time and as their are always multiple it all stacks up.

Perhaps running the subsequent could be ran parallel but potential their could be an infinite number of these queries, so potentially that could be difficult to manage.

 

[kitkat9933]

Are the follow up queries made to the same 3rd party?

If so, could you speak to them about changing what is return with the 1st call so you get everything you need in one hit.

I raised a request but they are a multi billion $ company, so probably not worth their time for just one user's requirements.