Need help about structure of our (not small) project

[kosta]

Hello, friends!

Recently I posted some questions about problems that I am facing on this my project, but answers are not fully adjusted on our needs and it can’t fully fix our problems.

I decided to go through details, explain my current state and get some more help from you guys.

So, we are making Dapp where you have crystals, fanatics, weapons.

All information, normally, is parsed though events. We are parsing those event tables and making another table from it (using afterSave).

Recently, we had to update code on moralis and for 1 minute server was down. In that 1 minute, our testers got new weapons and fanatics.

When server turned on, the information within 1 minute was in sync table, but afterSave did not work for last events and our main collection information was incorrect.

I posted similar question and answer was to use custom flag somewhere so I can continue from that flag.

Tried this solution, but it produced even more problems, as after start, maybe newer transaction will appair and my custom sync will downgrade information instead of upgrade (before it will run). So then I must check blocks, compare to current elements and decide what to do. This will be overkill.

So I decided to thow afterSave away and parse all information by myself. Then, it turns out that I can’t sort some columns in collection, as they are strings (and converting from string to int in query will have problems in future, it’s not resource friendly). I needed to convert it to int32 in order to use it properly. On forum, you answered me that I should create new column with same value but with int32, for example. (and I don’t like this approach, but, anyway let it be like that)

Then next problem appaired - we have option to transfer ownership of weapons. We can transfer either one or many items. It’s 2 separate collections, handling same data. So I need to select from both of them, sort it by certain column, and then do my stuff. It’s really uncomfortable to make queries like that and also keep track that new events will not make mess.

On my own server, I’d buld separate collection, that will hold both table data in same structure, by increment. But here, it’s unachievable as afterSave may not work for any reason.

This is my current state. I need your opinion, how I can handle this situation? Is it correct way for moralis? Should I use another approach?

Thanks!

[cryptokid]

for the afterSave problem: if you create a separate table, with only one column that it saves the IDs of the rows that you need to process (something like a flag), and insert that an ID after that row is processed, be it in afterSave, or in a separate script that can check if anything wasn’t processed, will it work for you?

I didn’t understand that part with:

@kosta

Tried this solution, but it produced even more problems, as after start, maybe newer transaction will appair and my custom sync will downgrade information instead of upgrade (before it will run).

I don’t know what you mean by downgrading information

This part isn’t clear for me either:

@kosta

Then next problem appaired - we have option to transfer ownership of weapons. We can transfer either one or many items. It’s 2 separate collections, handling same data. So I need to select from both of them, sort it by certain column, and then do my stuff. It’s really uncomfortable to make queries like that and also keep track that new events will not make mess.

how do you keep the data for those collections now?

[kosta]

@cryptokid

for the afterSave problem: if you create a separate table, with only one column that it saves the IDs of the rows that you need to process (something like a flag), and insert that an ID after that row is processed, be it in afterSave, or in a separate script that can check if anything wasn’t processed, will it work for you?

So, before inserting data into my table, on each event I must check if there are older transactions, insert them and check for new ones. But what if aftersave will run within 1 second more than once, with 2 or more users, when first aftersave is not finished yet? It will run twice and modify data incorrectly. This situation may appear. So attaching this task is incorrect for this task. Don’t you think it’s problem?

@cryptokid

I don’t know what you mean by downgrading information

I mean, it may happen that new event can run (aftersave) and after server is on, my custom function will run later. It will be incorrect order of data parsing.

@cryptokid

how do you keep the data for those collections now?

Dapp is in beta, not released yet. I am only using afterSave function for now and need to fix my problem with best possible way.

I am trying to find out what is the is the best workaround.

[cryptokid]

I don’t completely understand first part with:

@kosta

So, before inserting data into my table, on each event I must check if there are older transactions, insert them and check for new ones.

where do you check for older transactions?
I don’t understand what you do there with older transactions, or on what type of events you listen.

regarding the order of processing, you can set specific flags, so that you are sure of the order, but I don’t completely understand what is happening there.

About collections, I mean you listen for an event for a smart contract, or how you collect data for those collections, are two tables now and you want one table, or is one table and you want two tables?