Help to understand rate limiting headers

Moralis General
#1

When I run this request:

curl -X 'GET' \
  'https://deep-index.moralis.io/api/v2/nft/0x65c5493e6d4d7bf2da414571eb87ed547eb0abed?chain=eth&format=decimal' \
  -H 'accept: application/json' \
  -H 'X-API-Key: KEYHERE'

I gest these headers:

 x-rate-limit-ip-used: 5 
 x-rate-limit-limit: 3500 
 x-rate-limit-remaining-ip-ttl: 60 
 x-rate-limit-remaining-ttl: 60 
 x-rate-limit-throttle-ip-used: 5 
 x-rate-limit-throttle-limit: 88 
 x-rate-limit-throttle-remaining-ip-ttl: 1 
 x-rate-limit-throttle-remaining-ttl: 1 
 x-rate-limit-throttle-used: 5 
 x-rate-limit-used: 5 
 x-request-weight: 5

This is what I understand:

  • Cost of request is 5
  • I should be able to do 700 per minute ( x-rate-limit-limit / x-request-weight) = (3500 / 5) = 700?
  • I can do 88 requests per second? ( x-rate-limit-throttle-limit)

Questions:

  • why does it say remaning is 1? ( x-rate-limit-throttle-remaining-ttl) I was expecting it to be 83? (since I had 88 and 5 were used)?
  • I want to make all requests from one single IP, a server? Can the server use all the quota?
  • is it possible to see the x-rate via the SDK? e.g. when running Moralis.Web3API.token.getAllTokenIds(options)?
#2

ttl may mean the time interval of 1 second in this case

Yes

In this case I think that header information is not propagated to the final returned result.

#3

I see, so I should have 83 left in this case?

#4

you have 88 - 5 left in a second

#5

Okay, I am still struggling to get the math to add up with the limits I am getting. Do we still have a penalty on using offset?

#6

that may still be, what math problem you have?

#7

Sure, can you help me decipher this:

{"x-rate-limit-limit":"3500",
"x-rate-limit-remaining":"39",
"x-rate-limit-remaining-ttl":"39",
"x-rate-limit-throttle-limit":"88",
"x-rate-limit-throttle-remaining":"5",
"x-rate-limit-throttle-remaining-ttl":"5",
"x-rate-limit-throttle-used":"260"
,"x-rate-limit-used":"155"}

PER SECOND
I can do 88 requests per second (x-rate-limit-throttle-limit)
I have used 260 (x-rate-limit-throttle-used)
I have 5 left (x-rate-limit-throttle-remaining)

How does this add up?

88 - 260 = 5?

#8

can you paste the entire response header info that you got there?

#9

I am somewhat confused about how to proceed. The headers i shared with you was via an error message received in the SDK.

  1. I can get response headers by using the REST API via deep-index.moralis.io. However, in that case I can’t increase the api rate limit for users like this:
Moralis.settings.setAPIRateLimit({
  anonymous:9007199254740991, authenticated:9007199254740991, windowMs:60000
})
  1. And the opposite is true. Via the SDK I can maximize the API Rate that a user can spend, but not get headers?

So how do I proceed with this? Even if I use the REST API to get the headers it will not correlate to the limits when I use the SDK?

#10

what I mean, is that you had there an example with some values from those headers that seemed a little strange with that 260 throttle-used with 5 ttl for throttle

you can make REST requests to api in a cloud function directly if you want the headers back there

those two limits are different limits, what you set with Moralis.settings.setAPIRateLimit and what you get in the response headers

#11

OK. Running the REST API provides the headers I expect…

Is there a way to adjust the limits per user for the REST API, so that a server from one IP can consume as much as possible?

#12

When calling this:

https://deep-index.moralis.io/api/v2/nft/0x2ad72ecc13738f50662ea0af44210c6f7d42ddef/1/metadata/resync?chain=eth&flag=uri&mode=sync

I get these headers:

{
  date: 'Mon, 28 Feb 2022 22:48:15 GMT',
  'content-type': 'application/json; charset=utf-8',
  'content-length': '22',
  connection: 'close',
  'x-powered-by': 'Express',
  'access-control-allow-origin': '*',
  'x-rate-limit-remaining-ttl': '60',
  'x-rate-limit-remaining-ip-ttl': '60',
  'x-rate-limit-used': '5',
  'x-rate-limit-ip-used': '5',
  'x-rate-limit-limit': '3500',
  'x-rate-limit-throttle-remaining-ttl': '1',
  'x-rate-limit-throttle-remaining-ip-ttl': '1',
  'x-rate-limit-throttle-used': '60',
  'x-rate-limit-throttle-ip-used': '60',
  'x-rate-limit-throttle-limit': '88',
  'x-request-weight': '5',
  etag: 'W/"16-PU857yijBg2HpA4WN1nXLGUZt3Q"',
  vary: 'Accept-Encoding',
  'cf-cache-status': 'DYNAMIC',
  'expect-ct': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
  server: 'cloudflare',
  'cf-ray': '6e4d3302fe40f13a-ARN'
}
  • Cost is 5 (x-request-weight)
  • Limit per second is 88 (x-rate-limit-throttle-limit)
  • Used is 60 (x-rate-limit-throttle-used)

Doesn’t make sense? Used should be 83?

#13

There isn’t a way to add restrictions for making direct requests to web3api besides the normal rate limit for the entire account.

#14

Ok, got it. What about my question on the headers and math not adding up?

Server >> web api rate limits
#15

found out what it happens there: /metadata/resync endpoint has an impact of 5 compute units, but the impact on the throttle limit for that particular endpoint is 60, that means that you can only call that endpoint once per second

#16

Aha. So where are the throttles documented per endpoint, and how can we see that in the headers? Without trial and error?

  • Throttle per user
  • Compute unit costs
  • Throttle limit per minute
  • Throttle limit per second

Seems like everything is quite under-documented, and un-user friendly. :-1:

1 Like
#17

now there is more information about that particular resync endpoint in documentation: https://docs.moralis.io/misc/rate-limit

1 Like
#18

Thanks, is it correct to assume that the resync endpoint is the only one that detracts more from the throttle than its weight?

#19

it seems that there are 3 special cases:
syncNFTContract => 60
reSyncMetadata => 60
getTokenIdOwners => 30

2 Likes