[SOLVED] Moralis.Cloud.httpRequest handling of binary responses

Moralis General
#1

Hi there,

I’ve been scratching my head for a whole day trying to understand this. I’m testing using Moralis.Cloud.httpRequest function in a cloud function to retrieve a jpeg file from a website using the following code:

Moralis.Cloud.define("fetchData", async (request) => {
    const logger = Moralis.Cloud.getLogger();
    return Moralis.Cloud.httpRequest({
        method: "GET",
        url:  request.params.theUrl,
    }).then(function(httpResponse) {
        logger.info(httpResponse);
        return httpResponse;
    }, function(httpResponse) {
        logger.error('Request failed with response code ' + httpResponse.status);
    });
});

I’ve feed it with just a link of the random jpeg online (so i know the data works). but the data in the httpResponse is all weird and such and don’t look like binary at all:

Result: {“status”:200,“headers”:{“content-type”:“image/jpeg”,“content-length”:“125966”,“connection”:“close”,“date”:“Wed, 20 Jul 2022 00:19:56 GMT”,“last-modified”:“Tue, 14 Nov 2017 18:45:22 GMT”,“etag”:"“13f01d01b5f356709254f36925145a5d”",“x-amz-version-id”:"_C042mLKEmLupO2ENS4kINrcphlKKDdy",“accept-ranges”:“bytes”,“server”:“AmazonS3”,“x-cache”:“Hit from cloudfront”,“via”:“1.1 baaa01540e8048678da317f40119ee06.cloudfront.net (CloudFront)”,“x-amz-cf-pop”:“MRS52-P4”,“x-amz-cf-id”:“SK7kbyAwRxVeOLbEiFH4KDGushTzsiLowwaNECqpuZMqdYf-QziyDQ==”,“age”:“211735”},“buffer”:{“type”:“Buffer”,“data”:[]},“text”:"����\u0000\u0010JFIF\u0000\u0001\u0001\u0001\u0001,\u0001,\u0000\u0000��\fXICC_PROFILE\u0000\u0001\u0001\u0000\u0000\fHLino\u0002\u0010\u0000\u0000mntrRGB XYZ \u0007�\u0000\u0002\u0000\t\u0000\u0006\u00001\u0000\u0000acspMSFT\u0000\u0000\u0000\u0000IEC sRGB\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000��\u0000\u0001\u0000\u0000\u0000\u0000�-HP… (truncated)

Notice the weird characters in the “text” field. It looks like the pure jpeg binary get converted to utf8 and thereby corrupting it.

So my question is that if there’s a way to preserve the binary data in the httpRequest call?

Sorry if the question is bad I’m still learning

Cheers!

#2

this may be only how it is showed here, when you print/log it it could try to convert it to utf8.

I would expect it to return binary data without issues.

is the length of the response different than this number?

#3

they are different, the browser console said that weird text is around 300 kB long. I’ve still yet to be able to convert the data back to anything that resemble an image lol

#4

try to convert it to base64

#5

Ok I’ll try it when I get back to the office. Thanks for the help so far!

#6

Hello I’ve manage to convert the the data to base64 and for some reason the result is slightly different than expected, the code I use is as follows:

const params = { theUrl : imageURL };
const metadata = await Moralis.Cloud.run("fetchImage", params);
console.log(metadata); 
const base64Tex =  Buffer.from(metadata.text, 'binary').toString('base64');
console.log(base64Tex);

the result base64 string is:
/VBORw0KGgoAAAANSUhEUgAAA/0AAAP9CAYAAABN/f39AAAABmJLR0QA/QD9AP… (truncated)

However. I ran the same image in a online image to base64 and the result is as follows:
iVBORw0KGgoAAAANSUhEUgAAA+gAAAPoCAYAAABNo9TkAAAABmJLR0QA/wD/AP… (truncated)

The data is really close but some of the data seems to be replaced by “/” and such. I ran both base64 strings to a base64 to image converter and the second one works. so I’m just trying to find out what went wrong with the conversion. Any help would be greatly appreciated

Thank you in advance!

#7

Try to return base64 directly from this cloud function

#8

Got the same result:
/VBORw0KGgoAAAANSUhEUgAAA/0AAAP9CAYAAABN/f39AAAABmJLR0QA/QD9AP39/f3

Here’s the changed cloud function:

Moralis.Cloud.define("fetchImage", async (request) => {
    const logger = Moralis.Cloud.getLogger();
    return Moralis.Cloud.httpRequest({
        method: "GET",
        url: request.params.theUrl,
    }).then(function(httpResponse) {
        logger.info(httpResponse);
        return Buffer.from(httpResponse.text, 'binary').toString('base64');;
    }, function(httpResponse) {
        logger.error('Request failed with response code ' + httpResponse.status);
    });
});
#9

if you use here logger.info(JSON.stringify(httpResponse)), what do you see in logs?

#10

{“status”:200,“headers”:{“content-type”:“image/jpeg”,“content-length”:“125966”,“connection”:“close”,“date”:“Wed, 20 Jul 2022 00:19:56 GMT”,“last-modified”:“Tue, 14 Nov 2017 18:45:22 GMT”,“etag”:"“13f01d01b5f356709254f36925145a5d”",“x-amz-version-id”:"_C042mLKEmLupO2ENS4kINrcphlKKDdy",“accept-ranges”:“bytes”,“server”:“AmazonS3”,“x-cache”:“Hit from cloudfront”,“via”:“1.1 96e54f255e90f297c13fec7ba0745ecc.cloudfront.net (CloudFront)”,“x-amz-cf-pop”:“MRS52-P4”,“x-amz-cf-id”:“dEaOj2y9yhBK3qF6u5bzvI9cfG6Gjxq0jBESGvk58efD2Ewfzmgwwg==”,“age”:“459604”},“buffer”:{“type”:“Buffer”,“data”:[255,216,255,224,0,16,74,70,73,70,0,1,1,1,1,44,1,44,0,0,255,226,12,88,73,67,67,95,80,82,79,70,73,76,69,0,1,1,0,0,12,72,76,105,110,111,2,16,0,0,109,110,116,114,82,71,66,32,88,89,90,32,7,206,0,2,0,9,0,6,0,49,0,0,97,99,115,112,77,83,70,84,0,0,0,0,73,69,67,32… (Truncated)]}, “text”:"����\u0000\u0010JFIF\u0000\u0001\u0001\u0001\u0001,\u0001,\u0000\u0000��\fXICC_PROFILE\u0000\u0001\u0001\u0000\u0000\fHLino\u0002\u0010\u0000\u0000mntrRGB XYZ \u0007�… (Truncated)

#11

try to process this buffer data directly

#12

Alrighty, I’ll let you know how it goes!

Thanks!

1 Like
#13

Hello I think you have figure it out. thank you!

what i ended up with is this:

Moralis.Cloud.define("fetchImage", async (request) => {
    const logger = Moralis.Cloud.getLogger();
    return Moralis.Cloud.httpRequest({
        method: "GET",
        url: request.params.theUrl,
    }).then(function(httpResponse) {
        logger.info(httpResponse);
        return Buffer.from(httpResponse.buffer).toString('base64');
    }, function(httpResponse) {
        logger.error('Request failed with response code ' + httpResponse.status);
    });
});

Basically I have to grab the buffer in the cloud function and return that specifically (I can’t do that on the client side because in the response the buffer is empty). for some strange reason the buffer is emptied out when it returns the httpResponse back to the client.

Anyways, it’s working great now and thank you for all your support!

1 Like