NFT Endpoints Temporary Offset Rate Limit

Moralis General
#1

Calling:
moralis.Web3API.token.getAllTokenIds
original path: /nft/{address}
every 1 second and it failed after 51-53 requests (that is almost a minute).
I tried to have a 2-3 sec timeout between the calls but it still eventually failed with a rate limit error.

According to documentation, it’s a 5 weight call that means I should be able to do around 11\sec.

an example of the code running:

const collAddressSlug = '0xf1268733c6fb05ef6be9cf23d24436dcd6e0b35e',
  limit = 50;

let offset = 0;

while (true) {
    const nfts = (await moralis.Web3API.token.getAllTokenIds({ address: collAddress, limit, offset })).result;

   // do something....

    if (nfts.length < limit) {
      break;
    }

  offset += limit;
  await new Promise(f => setTimeout(f, 3000));
}
#2

For pagination there is a different rate limit formula depending on the page number. Higher page numbers have a bigger impact

#3

thanks a lot for the fast reply

  1. what is the rate limit?
  2. it’s seems to be a bit too low. a few minutes to fetch 10K assets.
#4

10k assets should work with a delay of 1 second between requests

#5

It doesn’t, please checkout the code on the original post

#6

You can use a limit of 500 instead of 50

#7

We have just released the possibility to set rate limits for web3-api calls.
See https://docs.moralis.io/moralis-server/web3-sdk/rate-limit on how to configure this.

By default it is set to 50 requests per minute for authenticated users.

#8

I want to do as much as call as I can.
limiting myself is actually the opposite.
I must mention that this is a backend call (node.js).

#9

These rate-limit checks are handled on the moralis server, so it’s the same for node.js
If you don’t want to limit yourself, then you can increase authenticated and/or decrease windowMs

Moralis.settings.setAPIRateLimit({
  anonymous:30, authenticated:500, windowMs:1000
})

Will give you 500 requests per second for example.

Error ''Rate limit exceeded''
#10

I can’t find “settings” inside Moralis package, is it out yet? any specific version?
(I see the documentation was added few hours ago)

  • I’m using version 0.0.157 now
#11

You may need to also update your Moralis server instance. I think that code is specific for cloud code and not for front end.

#12

I mentioned using Web3API.token.getAllTokenIds specifically.

so what’s next?

#13

did you try with a limit of 500 and with a delay of 1 second between requests?

#14

yes, but why do I even need a 1 sec delay?
I need a clear understanding of the mechanism and what the limitations are?

Thanks

#15

it looks like currently the weight is multiplied by offset / 500

I think that we plan on doing some improvements soon on pagination and we may lower that weight after that

#16

At the moment we temporary restriction on some endpoints (like NFT endpoints) pagination while we scale the systems:

The weight of the NFT endpoint that uses offset is calculated like this (offset/500)*5

if the result of the above calculation plus the weight of other requests you made the same second is above 60 - wait 1 second

if more requests are sent before 1 second i over - wait 5 seconds

these are temporary restrictions we had to put in place as we scale our systems and will be removed in the coming week approximately

2 Likes
"getAllTokenIds" rate limit exceeded (429 Too Many Requests)
Error 141 - Getting rate limited when I shouldn't be
getAllTokenIds: Rate Limit Error - Error Code 141
Moralis API request error code 429
Rate limit math for Web3 nft endpoint with offsets
Error 141 - Getting rate limited when I shouldn't be
#17

Is there any ETA on when this weighting will be removed?

#18

some NFT endpoints now have cursor based pagination (from changelog in discord):

  NFT transfer endpoints are automatically order by timestamp descending.
  Cursor based pagination is now supported for those endpoints
       - getNFTTransfersByBlock /block/{block_number_or_hash}/nft/transfers
       - getNftTransfersFromToBlock /nft/transfers
       - getNFTTransfers /{address}/nft/transfers
       - getContractNFTTransfers /nft/{address}/transfers
       - getWalletTokenIdTransfers /nft/{address}/{token_id}/transfers
1 Like
#19

Is there any ETA on when the getAllTokenIds endpoint will have the offset weighting for request throttling removed?

#20

I don’t have an ETA now for that