Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix .blend files with quotation marks in filename fail to import #94004

Merged

Conversation

Hilderin
Copy link
Contributor

@Hilderin Hilderin commented Jul 6, 2024

The main problem was using c_escape for paths which add \ to ' resulting to an invalid path since \ is not an escape character in XML.

Also, I added a parse and print for faultString if an error occur in Blender/Python.
I also fixed the problem where every call returned 'cannot marshal None unless allow_none is enabled' because the Python method export_gltf did not return a value.

Tested:

  • Filename with '.
  • Filename with accents (non standard characters).
  • Export without RPC for filename containing ' (do_import_direct).

Example of the blender error message from RPC response:
image

@Hilderin Hilderin requested a review from a team as a code owner July 6, 2024 17:20
@Hilderin Hilderin force-pushed the fix-blender-export-quotation-mark branch 2 times, most recently from 63b5b00 to 36b6fe1 Compare July 6, 2024 17:27
@AThousandShips AThousandShips added this to the 4.3 milestone Jul 6, 2024
@AThousandShips AThousandShips added the cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release label Jul 6, 2024
@fire fire requested a review from aaronfranke July 6, 2024 17:28
@RedMser

This comment was marked as resolved.

@akien-mga
Copy link
Member

Note that the String class has xml_escape and xml_unescape if we need to ensure proper escaping.

@Hilderin
Copy link
Contributor Author

Hilderin commented Jul 7, 2024

The values are already escaped in dict_to_xmlrpc and this function is used to created the XML payload.
image

It is important to not escape strings in parameters_map and request_options because they will be double escaped. And more, the same dictionary is used for rpc call (xml) in do_import_rpc and for direct call in do_import_direct where the values are converted in python and escaped in dict_to_python. Another c_escape is made in dict_to_python that was causing a double escape for the ' when not in RPC.

Just to be sure, I tested with a &:
image

Copy link
Contributor

@RedMser RedMser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes make sense to me, and hopefully that covers all edge cases.

Thanks for the error handling code as a bonus, that's very nice :)

@Hilderin
Copy link
Contributor Author

Hilderin commented Jul 7, 2024

I just add a little problem with this PR on my computer. I had 2 versions on Godot running, one with this modification, and one without. If the version without did a blender import first, blender was started with the old script that returns 'cannot marshal None unless allow_none is enabled'. When I import a file in Godot with this PR included, the error is trapped and the importation fails. I had to close the old version before it works again. Both version use the same blender process with the old script.

I think I should discard the error 'cannot marshal None unless allow_none is enabled' just to prevent issues for users using different versions of Godot? What do you think?

@RedMser
Copy link
Contributor

RedMser commented Jul 7, 2024

I think I should discard the error 'cannot marshal None unless allow_none is enabled' just to prevent issues for users using different version of Godot?

Not sure if it's worth adding this check, for the rare case of having multiple Godot instances running and using Blender imports on both...

I suppose if the check is simple enough to implement, documented with a comment as to why it's needed, it won't do any harm to add it. At least the error message is hard-coded on Python's side ^^

@Hilderin Hilderin force-pushed the fix-blender-export-quotation-mark branch from 36b6fe1 to 8b5e65d Compare July 7, 2024 12:01
@Hilderin
Copy link
Contributor Author

Hilderin commented Jul 7, 2024

There you go, I just added a check to discard the error 'cannot marshal None unless allow_none is enabled'.

Copy link
Contributor

@demolke demolke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was investigating #93997 and was digging through this code yesterday.

I wrote a similar thing but didn't finish it and saw yours today - demolke@7533dcd

@@ -268,7 +276,22 @@ Error EditorImportBlendRunner::do_import_rpc(const Dictionary &p_options) {
}
case HTTPClient::STATUS_BODY: {
client->poll();
// Parse response here if needed. For now we can just ignore it.
PackedByteArray response_chunk = client->read_response_body_chunk();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works if the stream can be read in one go.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following your comments, I used an easier method to determine if the exportation is successful or not. I just look if the string 'BLENDER_GODOT_EXPORT_SUCCESSFUL' is present in the output. That way, I don't have to parse the XML and the success message should always be in the first chunk. Worst case scenario, the error message is incomplete which, I think it's not a big deal (we did not have any error message before).

Copy link
Contributor

@demolke demolke Jul 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're introducing a string to look for, I think it would be safer to read the whole buffer. It's not too much extra work.

Something like

	// Wait for response.
	bool done = false;
	PackedByteArray response;
	while (!done) {
		status = client->get_status();
		switch (status) {
			case HTTPClient::STATUS_REQUESTING: {
				client->poll();
				break;
			}
			case HTTPClient::STATUS_BODY: {
				client->poll();
				response.append_array(client->read_response_body_chunk());
				break;
			}
			case HTTPClient::STATUS_CONNECTED: {
				done = true;
				break;
			}
			default: {
				ERR_FAIL_V_MSG(ERR_CONNECTION_ERROR, vformat("Unexpected status during RPC response: %d", status));
			}
		}
	}

	String response_text = "No response from Blender.";
	if (response.size() > 0) {
		response_text = String::utf8((const char *)response_chunk.ptr(), response_chunk.size());
	}

	if (client->get_response_code() != HTTPClient::RESPONSE_OK) {
		ERR_FAIL_V_MSG(ERR_QUERY_FAILED, vformat("Error received from Blender - status code: %s, error: %s", client->get_response_code(), response_text));
	} else if (response_text.find("BLENDER_GODOT_EXPORT_SUCCESSFUL") < 0) {
		// Previous versions of Godot used a Python script where the RPC function did not return
		// a value, causing the error 'cannot marshal None unless allow_none is enabled'.
		// If an older version of Godot is running and has started Blender with this script,
		// we will receive the error, but there's a good chance that the import was successful.
		// We are discarding this error to maintain backward compatibility and prevent situations
		// where the user needs to close the older version of Godot or kill Blender.
		if (response_text.find("cannot marshal None unless allow_none is enabled") < 0) {
			ERR_FAIL_V_MSG(ERR_QUERY_FAILED, vformat("Blender exportation failed: %s", response_text));
		}
	}

	return OK;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually that's wrong, let me check what's the right way :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the comment above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this code. I never worked with Godot HTTPClient before. It seems to work perfectly.

@@ -281,6 +304,75 @@ Error EditorImportBlendRunner::do_import_rpc(const Dictionary &p_options) {
return OK;
}

bool EditorImportBlendRunner::_parse_blender_http_response(const Vector<uint8_t> &p_response_data, String &r_error_message) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return Error and then you can propagate the result out of do_import_rpc()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! But this method does not exists anymore.

if (parser->get_node_name() == "string") {
parser->read();
r_error_message = parser->get_node_data().trim_suffix("\n");
// Previous versions of Godot used a Python script where the RPC function did not return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I was looking at it, I thought I would just grab all the text elements and avoid the nesting/replicating the xml structure in code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, that made the code really simple and less prone to errors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I finally created a _extract_error_message_xml that does exactly what you suggest and take only the text nodes.
Example of an error message:
image

@Hilderin Hilderin force-pushed the fix-blender-export-quotation-mark branch from 8b5e65d to b16fe04 Compare July 7, 2024 19:48
@Hilderin Hilderin force-pushed the fix-blender-export-quotation-mark branch from b16fe04 to d244d6f Compare July 7, 2024 22:13
Copy link
Contributor

@demolke demolke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@akien-mga akien-mga merged commit 3b891f5 into godotengine:master Jul 8, 2024
18 checks passed
@akien-mga
Copy link
Member

Thanks @Hilderin for the fix and everyone else for the in-depth review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cherrypick:4.2 Considered for cherry-picking into a future 4.2.x release regression topic:import topic:3d
Projects
None yet
Development

Successfully merging this pull request may close these issues.

.blend files with quotation marks in filename fail to import
6 participants