"503 Service Unavailable" hiding other issue

I use the REST API to sync up NFT sales data to a local database. Every once in a while when doing the syncing, one of the queries gets stuck in the following issue:

Requesting https://deep-index.moralis.io/api/v2/nft/0x15a2d6c2b4b9903c27f50cb8b32160ab17f186e2/transfers?chain=eth&format=decimal&limit=500&offset=0
Service unavailable - trying again after 5s - https://deep-index.moralis.io/api/v2/nft/0x15a2d6c2b4b9903c27f50cb8b32160ab17f186e2/transfers?chain=eth&format=decimal&limit=500&offset=0
Full body: <html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html>

(this is from my logs)

The error keeps happening until either:

  • I Ctrl-C the process and start it again; or
  • I let it continue until my code decides to move on to the next project to sync

Before I added code that stops trying after a number of tries, I once let this run for hundreds of attempts overnight and it did not manage to get a useful response.

But as soon as I do either of those things, the service works again immediately. This suggests, to me, there’s some other issue happening that’s not a case of “Service Unavailable” (which would suggest the server being down or some similar issue), but instead an error happening on your side when the query comes in.

I’ve been asked to post this up by Ivan on the Discord.

Thanks for looking into it :pray:

Daniel

1 Like

Did you try with different intervals of delay between requests?
It always happens on same request with the same parameters?

What tool you use to make those requests?

I’ve tried with 0.1s, 2s, and 5s.

It happens on different requests. If I do enough requests (and when I was syncing up several dozen highly active projects there were a lot of requests), eventually this happens. At first I thought it was when I reached a certain offset - but it turns out it’s quite random and sometimes happens on the first request.

I’m using Ruby’s standard Net::HTTP to make the requests.

I know that you can get that error randomly, but it should not get stuck on that error at the same request.

Can you try with bigger delays? It should work fine with 1s, but I don’t know what is the problem of getting stuck.

Sure, I can try setting it to 10 seconds, but then I won’t keep it looping for too long because I do need to get on with syncing the data!..

My guess is when I do this there’s a pile of errors accumulating in your logs - are those recorded somewhere? Can you link them up to e.g. the request above?

Can you skip on error, do the next request and try later again the request that gave the error?

I could but I’d have to write rather awkward code to do so :laughing:

yes we will fix it! :slight_smile:
we are just clarifying the behavior you see

The way I’ve been solving this is just to run the sync a few times - or, on my server, run it every hour, so if a project gets stuck it should be picked up again later. Though that depends how early it gets stuck, annoyingly. It’d be nice to have this solved properly. I might do a resync on that occasion.