API Call Required for Project Stupidly Slow

kitkat9933 · 27 Jun 2017 at 20:07

I have a project for work where it relies on heavy use of API calls to a 3rd party service. However, these calls are painfully slow and when adding some PHP back end or JS processing on this data-set can take up to 30-40 seconds (with small amount of objects which will grow exponentially) to return the result to the user. Now while I can shave a tiny bit of this I'm sure with improved code, it's not going to make the world of difference.

The call gets a list of objects and then basically loops through and gets a list of attributes for those objects, so essentially looping through the original returned objects the API offers no other way to get this required result.

Now, I looked into running background API calls and caching the result and even caching the result every set run through, however the problem is the resulting information needs to be completely accurate as important decisions could be made on the result of it (not that they aren't reversible just a pain in the arse). The alternative is to load the page and then give a spinning wheel of death for 30+ seconds.

Now I know I'm pretty stuck as poor user experience is the lesser of two evils compared to accurate data but I'm wondering if anyone knows of any better tricks/trips to get around this?

rexehuk · 28 Jun 2017 at 09:01

Pre-calculate it on a job?

I.e every 20 minutes build the cache, serve only cache to the users? Make clear it's delayed by 20 minutes information if crucial to do so.

chroniclard · 28 Jun 2017 at 09:08

Which part is slow, the API call returning data or the calculation loop?

Can the loop be parallel? Can it be batched in any way?

mid_gen · 28 Jun 2017 at 12:37

If your problem is as stated :

1) API call is slow
2) API is black box and can't be altered
3) Caching is not an option because results cannot be stale

Then the only solution is to have users wait for the API call to complete.

You need to be able to negotiate on point 3) to have any kind of solution....and depending on the usage patterns, even that may not make much difference. i.e. if you set a short lifespan of 20 minutes or something on the cache, and multiple users aren't requesting the same data in that 20 minute period usually then you'd get no benefit.

kitkat9933 · 28 Jun 2017 at 16:58

chroniclard said:
Which part is slow, the API call returning data or the calculation loop?

Can the loop be parallel? Can it be batched in any way?

So first API query is sent and takes <1s to return. Then processing makes an API call for each element of the array returned in the original API query. The subsequent queries onwards are what take the time and as their are always multiple it all stacks up.

Perhaps running the subsequent could be ran parallel but potential their could be an infinite number of these queries, so potentially that could be difficult to manage.

chroniclard · 28 Jun 2017 at 18:08

Doesnt sound like there is much you can do if the information needs to be upto date/accurate.

Could possibly create a semi accurate progress bar to indicate its generating the data.

Ergates · 29 Jun 2017 at 12:15

Are the follow up queries made to the same 3rd party?

If so, could you speak to them about changing what is return with the 1st call so you get everything you need in one hit.

kitkat9933 · 29 Jun 2017 at 16:33

Ergates said:
Are the follow up queries made to the same 3rd party?

If so, could you speak to them about changing what is return with the 1st call so you get everything you need in one hit.

I raised a request but they are a multi billion $ company, so probably not worth their time for just one user's requirements.

ZombieFan · 29 Jun 2017 at 22:36

kitkat9933 said:
Perhaps running the subsequent could be ran parallel but potential their could be an infinite number of these queries, so potentially that could be difficult to manage.

This would be the way I would approach it. Do the initial call and then run the resulting calls in parallel. You'll have to write something to cap how many threads you've got going at once (so you could have 10 calls running in parallel, for example).

kaku · 30 Jun 2017 at 00:10

kitkat9933 said:
Then processing makes an API call for each element of the array returned in the original API query.

This is the red flag for me. Is there really no way to query batch data from the API? How many elements are we talking here? Tens, hundreds, thousands? Assuming the data for each of the items you're querying is related in some way, it seems bizarre there's not a better way than querying one by one.

Without any cooperation from the API provider I'm really struggling to see how you could remedy this without some sort of compromise.

Hades · 1 Jul 2017 at 19:32

Have you discussed the issue with the third party providing the API? They may be able to either tune it their side or advise on ways to influence the processing by amending the data in the call.