Stream duplicate transactions

Moralis General
#1

I have streams setup to a webhooks endpoint which takes Transfer events and each object in nftTransfers from the stream data is inserted into the database as a new row.

However, the stream is frequently returning duplicate streams that are confirmed = true. Below is an example, got this twice in a row. I map each transfer item into a new object like this. But I can see in the incoming data that the there is duplicated data at least every 1/10 txns.

newTransfer - Handled response: /api/v2/transactions/streams-webhook. 
    confirmed: true
    txn_hash: "0x89fb6e9b8cec60423c1351854cc69886457ff6c16fd661803f3307dafcb6e257"
    chain_id: 137
    stream_id: "bdc........."
    tag: "nft-transfers-dev"
    block_number: "39983562"
    block_hash: "0xbc74c3153faa05858a8d4a519ba28c937e02beb6a426b8953a38c3bd36d26c6d"
    block_timestamp: "1678000015"
    event_type: "Transfer(address,address,uint256)"
    nft_transfer: {
      "operator": null,
      "from": "0x0000000000000000000000000000000000000000",
      "to": "0xd98b68c52f3aec26381b1d942d5ddcf80e55c9a7",
      "tokenId": "583349",
      "amount": "1",
      "transactionHash": "0x89fb6e9b8cec60423c1351854cc69886457ff6c16fd661803f3307dafcb6e257",
      "logIndex": "135",
      "contract": "0x67f4732266c7300cca593c814d46bee72e40659f",
    }
    from_address: "0x0000000000000000000000000000000000000000"
    to_address: "0xd98b68c52f3aec26381b1d942d5ddcf80e55c9a7"
    token_id: "583349"
    contract_address: "0x67f4732266c7300cca593c814d46bee72e40659f"

I set a table constraint on the database to block duplicate inserts but you can see here how often its happening:

Screenshot 2023-03-05 at 2.18.38 AM
Screenshot 2023-03-05 at 2.18.38 AM650×670 47.9 KB

#2

Can you paste the original webhook request for 2 duplicated events?

Everything is identical on those 2 webhook requests?

#3

These are incoming streams for the same txn. It looks like all 4 of these are in my failed deliveries page.

My API endpoint is setup to return a 400 error if the database returns an error when attempting to insert a duplicate.

Would this 400 error cause the stream attempt to fail and then it retries the same txn? Should I just set the response to 200 either way?

#4

Yes, if you don’t return a 200 status code then it will be redelivered.

#5

Okay changing errors to 200 helped but I still get some duplicates.

I checked this block number 39984962 and it’s not in my failed deliveries

[03:01:02.490] INFO: new Data - Handled response: /api/v2/transactions/streams-webhook. Logged with pino.
    env: "development"
    confirmed: true
    chainId: "0x89"
    abi: [
      {
        "anonymous": false,
        "inputs": [
          {
            "indexed": true,
            "internalType": "address",
            "name": "from",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "to",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "uint256",
            "name": "tokenId",
            "type": "uint256"
          }
        ],
        "name": "Transfer",
        "type": "event"
      },
      {
        "anonymous": false,
        "inputs": [
          {
            "indexed": true,
            "internalType": "address",
            "name": "operator",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "from",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "to",
            "type": "address"
          },
          {
            "indexed": false,
            "internalType": "uint256[]",
            "name": "ids",
            "type": "uint256[]"
          },
          {
            "indexed": false,
            "internalType": "uint256[]",
            "name": "values",
            "type": "uint256[]"
          }
        ],
        "name": "TransferBatch",
        "type": "event"
      }
    ]
    streamId: "bdcf9e...."
    tag: "nft-transfers-dev"
    retries: 0
    block: {
      "number": "39984962",
      "hash": "0x3b1c4cb88e60f47e0a2e12c62b248f23363e525aee37109636a161f83e528ed3",
      "timestamp": "1678003044"
    }
    logs: [
      {
        "logIndex": "65",
        "transactionHash": "0x87eab3bf97c2551797e8cbe65818d0e2b1a0c3c6efc4698d1e9d413ae3e440bc",
        "address": "0xe91bd2fb48fe10786e29776023acb782ba8fc0f1",
        "data": "0x",
        "topic0": "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "topic1": "0x000000000000000000000000eb857ccebeb347749c50bcddb2cb43511617dd7e",
        "topic2": "0x00000000000000000000000067f4732266c7300cca593c814d46bee72e40659f",
        "topic3": "0x0000000000000000000000000000000000000000000000000000000000002c42"
      }
    ]
    txs: []
    txsInternal: []
    erc20Transfers: []
    erc20Approvals: []
    nftTokenApprovals: []
    nftApprovals: {
      "ERC721": [],
      "ERC1155": []
    }
    nftTransfers: [
      {
        "operator": null,
        "from": "0xeb857ccebeb347749c50bcddb2cb43511617dd7e",
        "to": "0x67f4732266c7300cca593c814d46bee72e40659f",
        "tokenId": "11330",
        "amount": "1",
        "transactionHash": "0x87eab3bf97c2551797e8cbe65818d0e2b1a0c3c6efc4698d1e9d413ae3e440bc",
        "logIndex": "65",
        "contract": "0xe91bd2fb48fe10786e29776023acb782ba8fc0f1",
        "tokenName": "ZED Racehorse Skin",
        "tokenSymbol": "ZEDSKIN",
        "tokenContractType": "ERC721"
      }
    ]
    nativeBalances: []

Second:

[03:01:09.324] INFO: new Data - Handled response: /api/v2/transactions/streams-webhook. Logged with pino.
    env: "development"
    confirmed: true
    chainId: "0x89"
    abi: [
      {
        "anonymous": false,
        "inputs": [
          {
            "indexed": true,
            "internalType": "address",
            "name": "from",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "to",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "uint256",
            "name": "tokenId",
            "type": "uint256"
          }
        ],
        "name": "Transfer",
        "type": "event"
      },
      {
        "anonymous": false,
        "inputs": [
          {
            "indexed": true,
            "internalType": "address",
            "name": "operator",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "from",
            "type": "address"
          },
          {
            "indexed": true,
            "internalType": "address",
            "name": "to",
            "type": "address"
          },
          {
            "indexed": false,
            "internalType": "uint256[]",
            "name": "ids",
            "type": "uint256[]"
          },
          {
            "indexed": false,
            "internalType": "uint256[]",
            "name": "values",
            "type": "uint256[]"
          }
        ],
        "name": "TransferBatch",
        "type": "event"
      }
    ]
    streamId: "bdcf9ea...."
    tag: "nft-transfers-dev"
    retries: 0
    block: {
      "number": "39984962",
      "hash": "0x3b1c4cb88e60f47e0a2e12c62b248f23363e525aee37109636a161f83e528ed3",
      "timestamp": "1678003044"
    }
    logs: [
      {
        "logIndex": "65",
        "transactionHash": "0x87eab3bf97c2551797e8cbe65818d0e2b1a0c3c6efc4698d1e9d413ae3e440bc",
        "address": "0xe91bd2fb48fe10786e29776023acb782ba8fc0f1",
        "data": "0x",
        "topic0": "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
        "topic1": "0x000000000000000000000000eb857ccebeb347749c50bcddb2cb43511617dd7e",
        "topic2": "0x00000000000000000000000067f4732266c7300cca593c814d46bee72e40659f",
        "topic3": "0x0000000000000000000000000000000000000000000000000000000000002c42"
      }
    ]
    txs: []
    txsInternal: []
    erc20Transfers: []
    erc20Approvals: []
    nftTokenApprovals: []
    nftApprovals: {
      "ERC721": [],
      "ERC1155": []
    }
    nftTransfers: [
      {
        "operator": null,
        "from": "0xeb857ccebeb347749c50bcddb2cb43511617dd7e",
        "to": "0x67f4732266c7300cca593c814d46bee72e40659f",
        "tokenId": "11330",
        "amount": "1",
        "transactionHash": "0x87eab3bf97c2551797e8cbe65818d0e2b1a0c3c6efc4698d1e9d413ae3e440bc",
        "logIndex": "65",
        "contract": "0xe91bd2fb48fe10786e29776023acb782ba8fc0f1",
        "tokenName": "ZED Racehorse Skin",
        "tokenSymbol": "ZEDSKIN",
        "tokenContractType": "ERC721"
      }
    ]
    nativeBalances: []
#6

How I can identify the account that has that stream ID?

#7

there is a delay of 6 seconds between them?

#8

Should I DM you the account email?

#9

Looks like it, I’ll wait for others to see the time.

They aren’t happening as often now that I changed the endpoint to return 200 status but its still every few minutes I get a duplicate

#10

can you DM me on discord or here on forum?

#11

I sent a friend request on Discord because it says I can’t message unless I do that. I don’t see any other way to private message

1 Like