Server >> web api rate limits

How can I manage the rate limits between the server and the web3 api? We have server-driven functions that call the API. I’d like to give the server significantly more latitude to make API calls than a client (if the server uses up 2000 calls/minute out of the 5000 we have, that’s fine, but I only want a client to be able to make 5 or 10 calls/minute

We have no authenticated users in our system (and we won’t), so if we can find a way to treat the server as an authenticated user, could that help?

what do you mean by server?

you can limit the clients now in the authenticated or unauthenticated form

Cloud code, running on an upgraded server. I can’t tell whether the cloud code is treated as authenticated or not, and what the various limits are. I think I’m setting the limits in an ok manner:

  Moralis.settings.setAPIRateLimit({
    anonymous:10, authenticated:4000, windowMs:60000
  })

ok, so you run some code in cloud code that makes requests to web3api, and you get rate limited in that cloud code with 10 requests per minute?

Yep, I think that’s what’s happening.

There’s a relevant log, which might confirm that:

2022-03-04T20:50:03.463Z - Ran cloud function coreservices_getWeb3ApiToken for user undefined with:
  Input: {}
  Result: {"status":200,"data":{"success":true,"result":"XXXtokenXXX"}}

that output is for any web3api call

anyway, you can also call web3api directly as a http request, without using the SDK, and by using the API key, in that case you don’t even need a server

Ok, that’s what we need (we’re using direct http requests anyway). Right now, we appear to be sending the “Application ID:” in the X-API-Key header. What should we change it to (there’s lots of similar keys)? The MasterKey or the CLI API key/secret?

a different key, the web3api key

an example on how to do it from cloud code:

Moralis.Cloud.define("test_web3api", async (request) => {
  
  return Moralis.Cloud.httpRequest({
   url: 'https://deep-index.moralis.io/api/v2/web3/version',
   headers: {
    'accept': 'application/json',
    'X-API-Key': 'WEB3API_KEY_HERE'
   }
 }).then(function(httpResponse) {
    return httpResponse;
 }, function(httpResponse) {
   logger.info('Request failed with response code ' + httpResponse.status);
 });

})

Hrm. That appears to be the key we’ve been using, not the app ID. So the limits are coming from some other cause?

you can look in the response header to see, there are two types of limits

Here’s a response header. It’s incredibly hard to figure out what’s needed to change to make this work properly. I’m going to echo the sentiment here. These limits are badly-defined, badly reported, and therefore impossible to understand

{
   "status":429,
   "headers":{
      "date":"Fri, 04 Mar 2022 21:17:12 GMT",
      "content-type":"application/json; charset=utf-8",
      "content-length":"34",
      "connection":"close",
      "x-powered-by":"Express",
      "access-control-allow-origin":"*",
      "x-rate-limit-remaining-ttl":"49",
      "x-rate-limit-remaining-ip-ttl":"56",
      "x-rate-limit-used":"429",
      "x-rate-limit-ip-used":"425",
      "x-rate-limit-limit":"5000",
      "x-rate-limit-throttle-remaining-ttl":"1",
      "x-rate-limit-throttle-remaining-ip-ttl":"1",
      "x-rate-limit-throttle-used":"130",
      "x-rate-limit-throttle-ip-used":"130",
      "x-rate-limit-throttle-limit":"125",
      "x-rate-limit-remaining":"49",
      "x-rate-limit-throttle-remaining":"1",
      "etag":"W/\"22-A17CvmppE17nAhS6Eu3AUBz7zmU\"",
      "vary":"Accept-Encoding",
      "cf-cache-status":"DYNAMIC",
      "expect-ct":"max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"",
      "server":"cloudflare",
      "cf-ray":"6e6da32668508cbf-EWR"
   },
   "text":"{\"message\":\"Rate limit exceeded.\"}",
   "data":{
      "message":"Rate limit exceeded."
   }
}

it looks like too many requests were done in a second and that throttle limit comes in

now it can depend on what type of requests you make, if you try to resync metadata, you will be able to do only 2 requests per second maybe

One idea is that you build in a cue in the SDK to automatically handle these headers and adapt OR make a far easier system. I already spent 6-8 hours trying to adapt to all your limits, and still don’t feel confident that I have a good solution. Feels like time wasted.

1 Like

Explain the throttle limit, please, and any ways to prevent it applying to the cloud code on the server. Should we be using the Moralis SDK rather than http calls? Should we be doing something completely different?

Using http calls is fine. Based on what requests you make could be a different reason for getting that throttle limit that means that too many requests were done in a second. Usually adding a delay between requests removes that error.

That’s… not helpful. I’m not sure you understand how these systems need to work. We’re not talking about a single client with a single set of calls. We’re in a world with multiple clients, multiple servers, with spiky loads. There’s no such thing as “put a delay in”. Or, rather, there could be, but it should be your dev team doing those things so your system is fit for purpose.

We can do our best to deduplicate requests (but even that’s a hard problem with the upgraded server, as they’re multi-instance).

But that still doesn’t help the basic problem that the limits are set up badly, not communicated well, and not configurable for the uses people need. And the only way to figure all this out is spending a ton of engineering time and doing detailed trial-and-error.

You can also increase the limits with a custom plan if you want. The delay you can add it also when you get a rate limit error and try same request after delay.

+1. Moralis team, please be more clear about limits: The pricing page says the “Pro” package supports 3500 requests per minute…

…But in practice, it is more like 100 per minute. Try to do more and you run into “throttle” or some other undocumented limit.

My program analyzes <100 wallets using Node.js + Moralis SDK. I hit the rate limit error after about 20 wallets. I waited ~10 min to start over, but the rate limit is still there for even 1 request.

error {
  code: 141,
  error: 'This Moralis Server is rate-limited because of the plan restrictions. See the details about the current rate and throttle limits: {"x-rate-limit-limit":"3500","x-rate-limit-remaining":"51","x-rate-limit-remaining-ttl":"51","x-rate-limit-throttle-limit":"88","x-rate-limit-throttle-remaining":"1","x-rate-limit-throttle-remaining-ttl":"1","x-rate-limit-throttle-used":"108","x-rate-limit-used":"40"}'
}

This error message is incredibly poorly formatted. Is this really the best naming scheme we could come up with, x-rate-limit-throttle-remaining-ttl?

The documentation here covers exactly two of the dozen or so elements in the error message. What we need is a reference that defines all of them with precision. Here’s another customer asking for the same thing.

If I’m reading this correctly, it says I’ve used only 40 requests over the past minute (“x-rate-limit-used”:“40”) so if I’m paying for 3500/min, why on earth am I getting rate-limited?

At this point, I would expect a response from support that says something like “some requests count double” and “check your cloud limits” but these are not the point. The pricing page says 3,500 requests per minute. I would challenge the Moralis team to show us some code that gets anything close to that. Using any request, cloud limit or other parameters that are available to paying customers like us.

it seems like some extra throttle limit is counted for those requests.

it also looks like we have a problem now with the headers that are returned when using web3api with a server, it looks like some fields are missing.