I’ve seen this mentioned somewhere on Discord and not sure if it’s already in your backlog, but I needed it so made my own version.
What does it do?
It stores NFT metadata in the database!
I need NFT metadata that is contained in the token_uri json file in my database, so I can use it in queries for filtering and sorting, for example sorting on Name or even searching in the Description.
At the same time it will speed up my app because I get the data from the query and don’t have to wait client side to load the token_uri data after the query.
My approach
I prefer this data in its own Class/table and call it ‘EthNFTMetadata’. I’d like to keep EthNFTOwners as clean as possible since this is auto generated by Moralis. In my query I can easily reference and sort to this data using a ‘lookup’ in an aggregate. I can also easily create BscNFTMetadata and the likes.
I’ve made 2 things:
- a Job to check all EthNFTOwners items and scrape the token_uri data into the the EthNFTMetadata Class. You need this if you already have data in EthNFTOwners and you’re not starting from scratch. Or maybe when you migrate to a new fresh server and EthNFTOwners data is repopulated from blockchain. Don’t forget to first create the Class ‘EthNFTMetadata’ in the Moralis db via the Dashboard. This Job code can be added to you Cloud Functions and you can add and run a new Job under Jobs > Schedule a job. You just need to run it once if you also use the BeforeSave hook mentioned in Step 2.
// add token_uri metadata to database for caching and filtering
Moralis.Cloud.job('addMetadata', async (request) => {
logger.info('Running addMetadata job')
const query = new Moralis.Query('EthNFTOwners')
query.equalTo('token_address', request.params.input.token_address) // only scrape own tokens
query.doesNotExist('isScraped') // only check those that have not been scraped before
const items = await query.find({useMasterKey:true})
for (let i = 0; i < items.length; ++i) {
const token_uri = items[i].get('token_uri')
if (token_uri && token_uri.length) {
// get token_uri data
const result = await Moralis.Cloud.httpRequest({
url: token_uri,
headers: {
'Content-Type': 'application/json;charset=utf-8'
}
})
// add new EthNFTMetadata item
const EthNFTMetadata = Moralis.Object.extend('EthNFTMetadata')
const metadata = new EthNFTMetadata();
metadata.set('token_address', items[i].get('token_address'))
metadata.set('token_id', items[i].get('token_id'))
metadata.set('name', result.data.name)
if (result.data.description) metadata.set('description', result.data.description)
if (result.data.image) metadata.set('image', result.data.image)
if (result.data.external_url) metadata.set('external_url', result.data.external_url)
if (result.data.animation_url) metadata.set('animation_url', result.data.animation_url)
if (result.data.background_color) metadata.set('background_color', result.data.background_color)
if (result.data.traits) metadata.set('traits', result.data.traits)
if (result.data.unlockable) metadata.set('unlockable', result.data.unlockable)
await metadata.save(null, {useMasterKey:true})
// flag item was scraped
items[i].set('isScraped', true)
await items[i].save(null, {useMasterKey:true})
}
}
})
- a beforeSave hook to scrape and save token_uri data when a new NFT has been minted:
// set Metadata after NFT is created
Moralis.Cloud.beforeSave('EthNFTOwners', async (request) => {
// add new Metadata item if not scraped already
if (!request.object.get('isScraped')) {
const token_uri = request.object.get('token_uri')
if (token_uri && token_uri.length) {
// get token_uri data
const result = await Moralis.Cloud.httpRequest({
url: token_uri,
headers: {
'Content-Type': 'application/json;charset=utf-8'
}
})
if (result && result.data) {
const EthNFTMetadata = Moralis.Object.extend('EthNFTMetadata')
const metadata = new EthNFTMetadata();
metadata.set('token_address', request.object.get('token_address'))
metadata.set('token_id', request.object.get('token_id'))
metadata.set('name', result.data.name)
if (result.data.description) metadata.set('description', result.data.description)
if (result.data.image) metadata.set('image', result.data.image)
if (result.data.external_url) metadata.set('external_url', result.data.external_url)
if (result.data.animation_url) metadata.set('animation_url', result.data.animation_url)
if (result.data.background_color) metadata.set('background_color', result.data.background_color)
if (result.data.traits) metadata.set('traits', result.data.traits)
if (result.data.unlockable) metadata.set('unlockable', result.data.unlockable)
await metadata.save(null, {useMasterKey:true})
// flag item was scraped
request.object.set('isScraped', true)
}
}
}
If you then want to get the NFT with metadata, you can use aggregate and pipeline like so:
const query = new Moralis.Query('EthNFTOwners')
const pipeline = [
{
lookup: {
from: request.params.network+'NFTMetadata',
let: { token_id: '$token_id', token_address: '$token_address' },
pipeline: [
{ $match:
{ $expr:
{ $and:
[
{ $eq: [ '$token_id', '$$token_id' ] },
{ $eq: [ '$token_address', '$$token_address' ] }
]
}
}
},
{ $project: { _updated_at: 0, _created_at: 0, ACL: 0, _id: 0 } }
],
as: 'metadata'
}
},
{
project: {
token_id: 1,
token_address: 1,
token_uri: 1,
owner_of: 1,
metadata: { $first: '$metadata' },
}
},
{ sort : { metadata.name: 1 } },
]
const queryResults = await query.aggregate(pipeline)
Now you can also sort your query on metadata.name for example!
If you want to see more in depth sorting, for example on askingPrice, see my post at Collation for aggregate([<pipeline>]) not supported by Moralis?
