Parse.cloud.httpRequest buffer

[Amex22]

Issue Description

Hi everyone,

I'm using a third party REST API from my cloudcode.

For some requests, everything works great just like on parse.com.

example : api.server.com/1/createuser return a data array with everything. I can get httpResponse.data.id to get the new user id.

But for other ones, Parse.cloud.httpRequest returns a buffer instead of the data array.
example : api.server.com/1/createdoc return an empty buffer so I can't get httpResponse.data.id to get the new created document id. (request is working, new doc is created)
Of course, this request is working on parse.com

Someone knows what is the problem ? How to solve this ?

Thanks

Steps to reproduce

Example of my query

return Parse.Cloud.httpRequest({
            method: 'POST',
            url: this._api.endPoint + '/users/documents/',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Basic ' + this._api.auth
            },
            body: {
                Type: type
            }
        });

Expected Results

httpResponse.data.id

Actual Outcome

empty httpResponse.buffer

Environment Setup

• Server
  • parse-server version: last
  • Operating System: last debian
  • Hardware: 2 x86 64bit Cores/2GB Memory
  • Localhost or remote server? remote server
• Database
  • MongoDB version: last

Logs/Trace

You can turn on additional logging by configuring VERBOSE=1 in your environment.

parse-server-1 | 2016-06-11T15:30:19.041283837Z verbose: POST /parse/functions/createDoc { host: 'api.myparseserver.com:1337',
parse-server-1 | 2016-06-11T15:30:19.041375779Z connection: 'keep-alive',
parse-server-1 | 2016-06-11T15:30:19.041396992Z 'content-length': '282',
parse-server-1 | 2016-06-11T15:30:19.041415549Z 'x-devtools-emulate-network-conditions-client-id': '’,
parse-server-1 | 2016-06-11T15:30:19.041433814Z origin: 'file://',
parse-server-1 | 2016-06-11T15:30:19.041452010Z 'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; ONE A2003 Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/51.0.2704.81 Mobile Safari/537.36',
parse-server-1 | 2016-06-11T15:30:19.041471028Z 'content-type': 'text/plain',
parse-server-1 | 2016-06-11T15:30:19.041488502Z accept: '/',
parse-server-1 | 2016-06-11T15:30:19.041527490Z 'accept-encoding': 'gzip, deflate',
parse-server-1 | 2016-06-11T15:30:19.041546639Z 'accept-language': 'fr-FR,en-US;q=0.8',
parse-server-1 | 2016-06-11T15:30:19.041564072Z 'x-requested-with': 'com.myapp.app2' } {
parse-server-1 | 2016-06-11T15:30:19.041581647Z "type": "MYTYPE"
parse-server-1 | 2016-06-11T15:30:19.041599392Z }
parse-server-1 | 2016-06-11T15:30:19.292721015Z REQ OK
parse-server-1 | 2016-06-11T15:30:19.301017362Z verbose: {
parse-server-1 | 2016-06-11T15:30:19.301143740Z "response": {
parse-server-1 | 2016-06-11T15:30:19.301174849Z "result": {
parse-server-1 | 2016-06-11T15:30:19.301194959Z "status": 200,
parse-server-1 | 2016-06-11T15:30:19.301213155Z "headers": {
parse-server-1 | 2016-06-11T15:30:19.301231470Z "cache-control": "no-cache",
parse-server-1 | 2016-06-11T15:30:19.301249657Z "pragma": "no-cache",
parse-server-1 | 2016-06-11T15:30:19.301290570Z "content-type": "application/json; charset=utf-8",
parse-server-1 | 2016-06-11T15:30:19.301309057Z "expires": "-1",
parse-server-1 | 2016-06-11T15:30:19.301326620Z "server": "thirdpartyAPIServer »,
parse-server-1 | 2016-06-11T15:30:19.301344125Z "date": "Sat, 11 Jun 2016 15:30:19 GMT",
parse-server-1 | 2016-06-11T15:30:19.301361578Z "connection": "close",
parse-server-1 | 2016-06-11T15:30:19.301378842Z "content-length": "170"
parse-server-1 | 2016-06-11T15:30:19.301396056Z },
parse-server-1 | 2016-06-11T15:30:19.301412717Z "buffer": {
parse-server-1 | 2016-06-11T15:30:19.301430141Z "type": "Buffer",
parse-server-1 | 2016-06-11T15:30:19.301447586Z "data": []
parse-server-1 | 2016-06-11T15:30:19.301464748Z }
parse-server-1 | 2016-06-11T15:30:19.301481681Z }
parse-server-1 | 2016-06-11T15:30:19.301538854Z }
parse-server-1 | 2016-06-11T15:30:19.301557071Z }

[bohemima]

I'd move to use the 'request' package instead of Parse's "proxy" for it.

[Amex22]

Well, that's an idea, thxs @bohemima ! But it would be great to have an official explaination/tip/fix in order to use the Parse httpRequest for our migrations instead of refactoring a lot of code.

[Amex22]

Still running this problem. I can't find a way to make it work.

[Amex22]

10 days and still no official answer/tip/fix.

Please, any idea ?

[cleever]

Exactly same issue here.

HttpRequest/httpResponse seems to be broken.

[cleever]

Seems related:

#892

[flovilmart]

What version of node are you running?

[Amex22]

Hi @flovilmart !
I am actually running parse server with docker (yongjhih/docker-parse-server). Seems to be node v5.11.1 when i exec node -v on the container.

[flovilmart]

I may have found something on that, stay tuned, I'll try to push a release if the fix is working.

[flovilmart]

Let me know if the PR work for your use case.

It seems that response.body is not set (for some reason) but all tests that we run were green.

[cleever]

Thanks @flovilmart.

I've tested in Node v4.4.3 and v4.4.6 (Windows) and Node v5.11.1 (Ubuntu - Server). No one works.

I didn't test your PR yet.

[cleever]

I've just finished testing the PR and I don't have good news.

Now I have the "body" field, however it doesn't have the information that I need.

"body":{"type":"Buffer","data":[]},"buffer":{"type":"Buffer","data":[]}}"

The following code works very well in the hosted parser server (parse.com):

Parse.Cloud.define("test", function (request, response) {

  Parse.Cloud.httpRequest({
    method: 'POST',
    headers: {
      'Authorization': Key
    },
    url: server_out,
    params: {
    my_param1: "test"
    }
  }).then(function (httpResponse) {
    response.success(httpResponse.data);
  }, function (httpResponse) {
    var msg = 'Request failed with response code ' + httpResponse.status;
    response.error(msg);
  });
});

[flovilmart]

the response.body is exactly what the request module is providing in the callback. Without more information I can't move forward. As you can see in the unit tests, the body is properly set, JSON object properly set when needed etc... If you can provide a test case, that would greatly help.

[cleever]

I've found the origin of this issue:
The question is what was done in these alterations:

#892
https://github.com/ParsePlatform/parse-server/pull/892/files

This line was removed:
httpResponse.data = JSON.parse(response.body);

To be replaced for a lazy load "data" field.

get data() {
    if (!this._data) {
      try {
      this._data = JSON.parse(this.text);
      } catch (e) {}
    }
    return this._data;
 }

It's works well when we need to use this field inside cloud code, doing something like that:

}).then(function (httpResponse) {
    var data = httpResponse.data;
    response.success(data);
  }, function (httpResponse) {
    var msg = 'Request failed with response code ' + httpResponse.status;
    response.error(msg );
  });

However, when we pass entire httpResponse in success, lazy load doesn't serialize the object, as expected.

}).then(function (httpResponse) {
    response.success(httpResponse);

Resulting in a object without data field to a client.

Fortunately, it's easy to workaround.

Finally, I'd suggest to document this type of breaking changes.

[flovilmart]

I'm not sure what you wanted to show as your code snippets both do the same thing.

Would that be related to babel?

That wasn't expected to be a breaking change, but a performance optimization for everything non JSON.

[Amex22]

Same for me when I test the PR. Body is there, but still empty and buffer type.
"body": { "type": "Buffer", "data": [] }, "buffer": { "type": "Buffer", "data": [] } } } }

I don't understand either what @cleever means

[Amex22]

Ok @flovilmart I understand what @cleever says.

When I console.log(httpResponse.data.Id) on Cloud code, it works. But when i response.success(httpResponse) on Cloud Code, then, there is an empty body on the client side.

[flovilmart]

Alright, get it now. Need a proper serializer on that object!

[flovilmart]

I Just updated the PR, I believe that should cover your use case. Sorry for the breaking change, that wasn't intended as is.

[flovilmart]

@Amex22 the problem from @cleever should be fixed by that PR, for your case of the empty buffer, I'm not sure if it's related to the @cleever problem.

[cleever]

@flovilmart Sorry about the snippets. They were wrong.
I updated/corrected them, and everything makes sense now.

Now they match with what @Amex22 said.

By the way, thank you for the great job. I'm going test the PR right now.

[Amex22]

I now have a strange response. What should be in data is in _data.

_data : Object CreationDate : 14668 Id : "14288" RefusedReasonMessage : null RefusedReasonType : null Status : "CREATED" Tag : null Type : "MYTYPE" UserId : "12857" __proto__ : Object _text : "{"Type":"MYTYPE","UserId":"12857","Id":"14288","Tag":null,"CreationDate":1466867297,"Status":"CREATED","RefusedReasonType":null,"RefusedReasonMessage":null}" body : Object data : Array[0] length : 0 __proto__ : Array[0] type : "Buffer" __proto__ : Object buffer : Object data : Array[0] length : 0 __proto__ : Array[0] type : "Buffer" __proto__ : Object headers : Object status : 200

[flovilmart]

Alright, I know what's going on... Upon CloudFunction response, we use Parse._encode to serialize the object so the toJSON function is never called.

[flovilmart]

But I have good news, :) Check the latest commit!

[cleever]

I had the same issue that @Amex22 said.

But now with the latest commit the code finally works \o/

Thank you @flovilmart.

[cleever]

By the way, with a correct "data" object I think that "body" field is not needed anymore.

[Amex22]

Thank you so much @flovilmart ! It works !

[flovilmart]

I'll remove the body field as it's not in the docs http://parse.com/docs/js/api/classes/Parse.Cloud.HTTPResponse.html

[flovilmart]

I've removed the body from serialization, but it's still available on the server (for backwards compatibility). Is that ok?

[Amex22]

I think that this is ok, yes.

[flovilmart]

2.2.14 is released, it should contain the fix.