504's and and cloudflare blocking requests

cryptokid · November 3, 2021, 4:12pm

can you share more information, as an example of a request that gets that error?
you can share it in DM if you don’t want to post it publicly.

coder · August 29, 2022, 5:50am

Apologies for waking a “sleeping” thread, but I’m having this same issue randomly as well.

Been testing an app in beta. It runs fine for hours at a time (even overnight), but will randomly start getting blocked by CloudFlare. No amount of rebooting or waiting works. I end up having to acquire a different IP (via VPN or otherwise), and everything starts working again.

While I recognize the need to protect endpoints, an issue like this is absolutely not tolerable on a paid plan.

cryptokid · August 29, 2022, 5:52am

In case that you are using walletconnect then there is a rate limit per IP for the public RPC url that is used. If you try it too many times then you will get that rate limit error.

coder · August 29, 2022, 5:58am

I’m making use of direct Mongo connections and V1 Web3API calls. The application is a NodeJS app. I generate a “singleton” Moralis instance and use it throughout the entire application. It does generate a decent number of calls, and while I might expect one or two calls to get bounced, I definitely don’t expect CloudFlare to get involved. 🫠

cryptokid · August 29, 2022, 5:57am

For what url in particular you get errors?

coder · August 29, 2022, 6:04am

Most recently Web3API.token.getTokenPrice. But it seems fairly random. We make use of a substantial portion of the Web3API calls.

But the bigger issue is that the IP gets blocked. Not sure if it’s permanent, but long enough that we must change it, and of course manually change the IP for Mongo access.

cryptokid · August 29, 2022, 6:06am

Can you check the http headers for when you get those errors? How many requests do you make?

You shouldn’t get easily blocked for that request unless you make thousands per second

coder · August 29, 2022, 6:12am

I’ve already cleared the most recent series of errors and logs. Letting it run again overnight and I’ll check it in the morning.

Not making thousands per second, but some bursts between 50-200 per second. We’re making various calls regarding Swap “actors”. Token prices, holder information, etc. – for multiple token swaps at a time.

I’m offloading as much as possible to direct contract calls, but we can’t always rely on a contract to be verified or the source to be accessible, so we rely on Moralis in those instances.

Regardless, it would be nice to know what the explicit limitations were in all circumstances so as to code around it (or drop the feature(s)).

cryptokid · August 29, 2022, 6:21am

You don’t get rate limited for 200 requests per second?

coder · August 29, 2022, 3:32pm

FYI, this is my second time posting this – from the same test IP. The first reply apparently got blocked due to, well, CF/Moralis blocking traffic. Swapped my IP and now this works.

Yes, I will occasionally get transactions rejected during bursts, but CF doesn’t usually step in and completely block my IP in those situations. But, for some reason, at some point, CF/Moralis decides it doesn’t want to hear from us anymore and simply blocks the IP.

Here is the most recent error output. Not sure if it pertains to the issue, but it happened hours ago while I was asleep:

2022-08-29T06:44:20.375Z - Error: {}
    at fetch (/moralis-server/lib/cloud-code/plugins/helpers/apiUtils.js:218:26)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-08-29T06:44:20.368Z - Failed running cloud function getTokenBalances for user undefined with:
  Input: {"chain":"bsc","address":"0xD4825ad5B1A863587e7116f3DE1387C1238e4319","to_block":"20858573"}
  Error: {"message":"{}","code":141}
2022-08-29T04:57:32.391Z - Error: Invalid function: "getPluginSpecs"
    at handleCloudFunction (/moralis-server/lib/Routers/FunctionsRouter.js:201:13)
    at /moralis-server/lib/PromiseRouter.js:85:20
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

For what it’s worth, Get all logs in your terminal: Learn about Moralis CLI is a dead link and needs to be fixed.

cryptokid · August 29, 2022, 3:37pm

you are using a Moralis Server to make web3api requests from a backend?
asking because that looks like server log

in that case what IP was blocked, your local IP, server IP?

coder · August 29, 2022, 3:57pm

It’s a node application. Yes, just using the existing Web3API collection (rather than axios/fetch with endpoints on the backend). Is the EVM object available in V1? What do you recommend?

There are a few devices (hardware and docker) behind the same VPN’ed firewall so they’re all sharing the same external IP. But only one VM is interacting with Moralis. It’s blocking that shared VPN IP

cryptokid · August 29, 2022, 3:58pm

I recommend to use web3api directly without a Moralis Server, either using moralis v1 sdk with moralisSecret parameter, or REST API, or moralis v2 SDK

coder · August 29, 2022, 4:07pm

But what about the various API-limited calls? Don’t I need server credentials to increase limits?

Is the REST API not “limited”?

This is basically what I’m using:

const Moralis = require('moralis-v1/node');
const options = {
  serverUrl: dAppServerUrl,
  appId: appId,
  masterKey: masterKey,
};

await Moralis.start(options);

I’m using that object to then make the majority of calls. What’s the problem with that specifically?

cryptokid · August 29, 2022, 4:10pm

you should use only moralisSecret for those options if you only use web3api

you don’t need to increase those limits, those limits can not be increased more than the limits that you have for your account, increasing them more will make it hit the account rate limit

coder · August 29, 2022, 4:17pm

These are the relevant calls we make to the Moralis object from the app:

...Web3API.token.getTokenMetadata

...Web3API.token.getTokenPrice

...Web3API.native.getTransaction

...Web3API.account.getTokenBalances

...Web3API.native.getLogsByAddress

...Plugins.oneInch.getSupportedTokens

...Web3API.defi.getPairAddress

...Web3API.defi.getPairReserves

...Cloud.run

You mind providing a specific example of one that you’d swap out for a different SDK or call?

cryptokid · August 29, 2022, 4:21pm

it looks like you don’t use only web3api, there is also 1Inch and running cloud functions

coder · August 29, 2022, 4:24pm

The 1Inch call is a raaaaaare call. Like weekly. The only cloud functions involve boot-up coreservices_addEventSync just to ensure that sync actions are running for retrieval later via Mongo.

The native/token/account calls are what generate the traffic.

cryptokid · August 29, 2022, 4:26pm

I mean that you can not completely stop using the server now, you are still using it for other things besides web3api

for web3api you should get a specific error message when you hit the rate limit

coder · August 29, 2022, 4:29pm

I think we might be conflating two issues:

Moralis rate-limit
CloudFlare IP banning

My primary issue isn’t rate-limiting . . . I expect this to happen. The primary issue is that CF randomly decides to ban the IP, and I can no longer access Moralis things. I have to change my IP to regain access then everything works fine again.