Add script API / make it scriptable

[bjorn]

Most people are not into C++ / Qt and compiling Tiled themselves, and as such small additions that would benefit their productivity are often not even started on. There should be an easy way to write scripts to perform actions in Tiled. Examples of things that should be possible:

• Running some script explicitly that performs some action on the map
• Adding a custom tool
• Hooking up to events to perform certain actions automatically
• Adding a custom map reader / writer

At the moment I think JavaScript would be the best choice of scripting language, because a JavaScript engine is provided as part of Qt and because having a JavaScript Tiled API should help with the development of a QtQuick based version of Tiled.

Examples of where scripting would have been useful (also see issues referencing this issue below):

• http://discourse.mapeditor.org/t/how-do-i-batch-automapping/3033
• http://discourse.mapeditor.org/t/selective-automapping/3093
• https://www.patreon.com/posts/22268925

[kkworden]

Did you have any idea as to how the feature would be implemented? Were you thinking a JS console? Macros and custom key binds? I'd be interested in helping add this feature.

[bjorn]

@kkworden Great to hear you'd like to help out with this! All your suggestions would be nice things to have eventually. My approach would be the following:

• First of all we need to instantiate the scripting engine. Since QtScript is deprecated, we should look at how this is expected to be done with QtQml. I think it's alright to depend on Qt 5 at this point. Maintaining compatibility with Qt 4 would be somewhat cumbersome here.
• Then, we need to look into how we can effectively expose Tiled's data types to the script. In Qt, the bindings to the scripting engine are commonly done via QObject derived classes or alternatively with classes marked with Q_GADGET. On the wip/qtquick branch, I've derived the Tiled::Object from QObject, which allows exposing properties and methods of the basic data types to the scripting engine.
• I think we should have a standard directory that Tiled scans for JS files, executing each script on startup. An API call should be available through which the script can add actions to the menu.
• Eventually we need to add more high-level API to access Tiled itself, allowing opening and saving maps, changing the current map, adding a custom tool, etc.

Since I'd like the scripting support to extend to a QtQuick based version of Tiled later, I think it may be good to do this in the wip/qtquick branch.

[tpetry]

Sounds interesting and something i would need at the moment. So i have to stick to exporting the map, modifying it with scripting and then reopening it.
But i personally would not prefer a solution allowing tiled to scan some folders for javascript-files (too much magic). Instead (like adding tilesets) the scripts should be added manually, because maybe you want to add a script to one map but not to another and don't want to seperate them to different folders.

I would suggest the PostgreSQL approach of developing such a feature: Start with an easy implementation and make it more powerful over time. The first iteration could simply be the automappig feature implemented for scripting. So whenever the map is modified the script will be called with the changed element and the script can modify the map. This would enable:

• a more powerfull automapping feature
• running scripts (by placing "special" tiles on the map which the script would detect and could remove after running)
  UI-Integrations, Export-Scripts etc. could be added later, having something up and running and waiting for user feedback is much more powerfull than adding an overly complex solution at first. Custom Export-Scripts e.g. can be trivially implemented by writing a small script in any language by reading the XML output and converting it.

[genericptr]

I wanted to suggest the using Lua is the easiest way to implement this after just replacing a huge mess of AI code in a game to Lua. It's was really easy and had a very small learning curve.

Here's a possible implementation idea:

1. Select a group of tiles in the editor which represent the "palette"
2. On the brush tool assign a script to use
3. When the user clicks to add a tile call the function "brush" on the Lua instance you created for the brush and pass the x,y coords of the cursor and the palette which is an array of tile ID's
4. In the Lua script below implement "brush" and 2 functions get_tile() and set_tile():

function brush(x, y, palette)
    if get_tile(x, y) == 1 then -- if the tile under the brush is id 1
        set_tile(x, y, palette[1])  -- set the tile under the brush to tile #1 from the palette
    end
end

5. get_tile() will return the tile ID at the x,y coords passed in by calling back to a C function in Tiled which will return that number.
6. set_tile() will add a tile to the layer at the x,y coords and tile ID. Likewise this will call a C function which basically performs the same function as the brush normally would.

Here's the basic outline of how the C bindings work but my pseudo code is Pascal:

// create a new lua instance
// you'll want to keep track of this unless the user changes scripts on the brush
l := luaL_newstate();
luaL_openlibs(l);

// bind 2 c functions (LuaGetTile, LuaSetTile) to Lua
lua_pushcfunction(l, @LuaGetTile);
lua_setglobal(l, 'get_tile');
lua_pushcfunction(l, @LuaSetTile);
lua_setglobal(l, 'set_tile');

// run the lua script from a string you loaded early
luaL_dostring(l, scriptText);

// when the user uses the brush tool retrieve the lua instance you created early and call:
lua_getglobal(l, 'brush');
lua_pushnumber(L, 1);
// ??? we need an array of tile IDs passed in! I'll provide that code also since passing arrays to lua is confusing imo.
lua_pcall(L, 1, 0, 0);

// when get_tile() is called from lua it will run this C function (mine being Pascal, sorry)
function LuaGetTile (l: Plua_State): integer; cdecl;
var
    x, y: integer;
    tileID: string;
begin
        // get the x and y coords that were passed from lua
    x := luaL_checkinteger(L, 1);
        y := luaL_checkinteger(L, 2);
        // find the tile ID at x,y and return the value back to lua
       tileID := FindTileID(x, y)
    lua_pushstring(L, tileID);
    result := 1;
end;

This is trivially easy to implement and would mean the user could have precise control over what tile the brush will insert and what tiles are around the cursor. I'm happy to help answering any questions.

[bjorn]

@genericptr Thanks for the suggestion. Actually I know Lua inside out and have used it in several C++ hobby projects as well as at work, where I essentially code Lua full time. Yet, I'm not convinced Lua is the best choice here because Qt ships with a JavaScript engine and provides an easy way of exposing existing C++ objects to it. Also, when I finally get time to work on a "Tiled 2", then probably all the high-level stuff will be written with JavaScript (as part of Qt Quick).

I think as languages go, Lua and JavaScript are very similar, though granted Lua is a lot less cryptic in its syntax. On the other hand, JavaScript is much more widely used. In the end though, it's more about what Qt supports and where I want to go with Tiled in the future.

The danger with starting with two simple API functions like in your example, is that there will be no end to requests for more functionality to be exposed. Maybe the tool wants access to tile properties, maybe it wants to base its decisions on data from other layers, etc. So in the end, we need a solution that can provide a full API and does not require extra maintenance.

[genericptr]

If it's just as easy with JS than go that route for sure. I'd assume add a larger API eventually but you could implement something really powerful even with those 2 functions and build from there. Thanks.

[starwing]

and there is a Lua module for QtScript: http://www.nongnu.org/libqtlua/

you can use this directly to get same ability as JavaScript in QtScript.

[bjorn]

@starwing I'm aware of QtLua. It would indeed help creating Lua APIs, but I'd still rather go with JavaScript since it's shipping with Qt and would anyway be the basis of a possible Qt Quick based Tiled.

[tomcashman]

Combined with #1665 I think the following would be useful:

• Create a project for a game and use the world map feature
• Use a script to hook to the new map created event to auto-setup tilesets + layers
• Use a script to paint extra tiles on different layers when a tile is painted (similar to automap but maybe easier to maintain in JS?)
• Use a script to validate tile properties against some other game data (e.g. a file somewhere else on the filesystem) when saved
• Use a script when save function is called to add some additional metadata to the map's XML

[bjorn]

Alright, I've finally pushed the initial change that adds some script evaluation capabilities to Tiled! It is already possible to automate some basic things, like applying automapping to all open maps.

(That the console mentions the Python script path is a little confusing, but those are messages from the Python plugin whereas here you can (at least currently) only execute JavaScript)

It is also possible to connect to signals, for example you can execute the following snippet to have Tiled write the file name of the current document every time you switch maps to the terminal:

tiled.documentManager.currentDocumentChanged.connect(function() {
    print(tiled.documentManager.currentDocument.fileName);
});

What is of course missing is most of the API that would actually be useful, along with ways to add custom actions or tools or some way to execute a script from a file. I'll be working on all of that in the coming weeks! Also, I need to adjust the autobuilds to ship the additional Qt module required for running JavaScript code, before I can push this in a snapshot.

@tomcashman Thanks for listing some additional use-cases that could be covered by scripting! It's very helpful to have all these examples to see where the script API is still incomplete.

[EJlol]

Is it possible to see the whole API somewhere, so that I can see what is currently already possible?

I tried to type a few commands to see whether it is possible to inspect the objects in the console:
if I typetiled it returns Tiled::Internal::ScriptModule(0x12db03b99c0). I could of course type JSON.stringify(tiled), but the stringify function is designed to only show the properties and not the functions. For example it misses the function tiled.activeAsset.layerAt(2).

Another thing I noticed was that every object has a empty string 'objectName', and that sizes always have two properties. For example: "width":100,"height":100,"size":{"width":100,"height":100}.

EDIT: Never mind, I can find out by using Object.keys(tiled).

[bjorn]

@EJlol Yep, you can use Object.keys() as you found out! Of course, the full API will be documented soon, and there remains a large amount of functionality to be implemented with still a lot of challenges (for example, for changing tile layer data or implementing custom tools). Thanks for trying it out already, any feedback is welcome!

Note that you can use tiled.trigger() for triggering most menu actions, but here as well the full list of available actions remains to be documented (and could be exposed through the API as well, for example by adding tiled.actions that returns the whole list). These registered actions will also form the basis for customizable keyboard shortcuts (another feature scheduled for Tiled 1.3) and it will of course be possible to register additional actions from the script.

[bjorn]

@Ktar5 I've doing some other things like the 1.2.4 patch release, attending the GitHub Satellite event and recently been replacing the "auto-updater" with a simpler and cross-platform new version check and re-activated the snap builds.

I did also upgrade the builds to Qt 5.12, but apart from that my previous comment is still relevant regarding the current state. As for the API, I'm making an effort to keep the reference updated continuously, it's available here.

[bjorn]

I forgot to associate some commits with this issue. Here's a bit of progress:

• 6de218a Added access to several more MapObject properties
• 74b6c94 Added access to the tilesets of a map
• e52e396 Added persistent history of commands to Console window (last 10 commands)
• 4a5dfd9 Added access to more tile properties and terrains

I did not expect this task to be so immense. It seems like each property that I want to make accessible through the script API comes with its own challenges and open questions. There's a lot of boxes in my comment above still unchecked, but progress is being made!

[bjorn]

I've missed to refer to this issue, but today I made it possible to script custom tileset formats (in addition to the already supported custom map formats). However I did not have time to test this feature yet.

A few days ago I announced the Tiled 1.3 Beta and today I have also announced the string freeze. The aim is to release Tiled 1.3 in one week. I realize there's still some scripting API gaps, like support for templates and Wang tiles, but I may choose to fill these only if there is demand because there are many more interesting improvements to be made.

[justburner]

I've been trying to port my generic C export python script to the new javascript export but sadly there's a few missing features that makes it impossible to fully port it.

It is possible to implement generic file I/O? Maybe similar to node.js fs?
The script requires to read json profiles files, write a header file and know where the script file is located or at least where is the extensions path so it can figure out where the profiles files are located, suggesting tiled.extensionsPath or tiled.scriptFilename?

It's possible to silently abort export, either by return false or return null ?

Also want to let you know that throwing exceptions using throw will act as if you're returning with the value of the exception... e.g. throw "Error occured" will write Error occured into the exported filename.

[bjorn]

Also want to let you know that throwing exceptions using throw will act as if you're returning with the value of the exception... e.g. throw "Error occured" will write Error occured into the exported filename.

Right, if you just throw a string then this is what will be returned on the C++ side apparently. For the C++ side to realize you've actually thrown an error you need to throw new Error("message").

But throwing an error will actually get a report with full stack trace, which may not be what you want. I'll look into changing the API so that you can control the error shown in the dialog without throwing it.

It is possible to implement generic file I/O? Maybe similar to node.js fs?

I didn't immediately add it due to the security concerns with running potentially other people's scripts with full file system access. That said, there should definitely be a way to read other files. Maybe it's better to just put a security warning in the documentation for now.

The script requires to read json profiles files, write a header file and know where the script file is located or at least where is the extensions path so it can figure out where the profiles files are located, suggesting tiled.extensionsPath or tiled.scriptFilename?

Once there is support for reading other files, it should be possible to load files from within the extensions path using the ext: prefix. That said, I see no reason not to make the extensions path available to the script (may return an array since I prepared a little for projects that ship with extensions, but maybe their path could be accessed separately).

Thanks for trying it out and the feedback!

[bjorn]

@justburn I've added API for reading and writing arbitrary files in either binary of text mode. Also, for reading text files you could alternatively use XMLHttpRequest. :-)

Unfortunately I'll need to delay the release, probably by another week. I'll consider pushing out another release candidate soon. I'll still try to find a solution to the error case as well.

[justburner]

Thank you for the file API! Can't wait for the next release :)

[bjorn]

Alright, I've now changed the writing function signature, such that if it returns a string it is considered the error string, and for writing to the file it is expected to use the TextFile or BinaryFile API.

That concludes the support for scripting in Tiled 1.3! I will open new issues for the remaining gaps in the scripting API.

[bjorn]

I've opened #2662, #2663 and #2664 for the features that remained unchecked in my comment listing the things still to do. I've also opened #2665 for adding an API to work with XML files.