Playpen

[gogins]

This is the old computer music playpen idea springing right back into life after my favorable experience with Python GTK used by python-abx.

The computer music playpen is an integrated development environment (IDE) for computer music that maximizes functionality and minimizes development time. The idea is to have the shortest, easiest "edit, render in real time and listen., edit..." cycle that is possible. Objectives:

• Form builder for UI design.
• HTML5 support for visual music, 3-D scores, etc.
• Csound support via ctcsound.
• CsoundAC support via Python.
• Arbitrary core generator support via <CsScore bin="generator_binary">, can be used with Common Lisp, etc.
• Ideally, the IDE should be able to edit itself, i.e. each piece is its own IDE and can edit itself. Or, the IDE supports all of the above features which create a complete program that is the piece, and one launches the piece from the IDE.
• Ideally, completely cross-platform. Haven't tested but should work on all GTK/WebKit/Gstreamer platforms.
• Maintainable going forward! Very important!

Current approaches that implement some of this:

1. Code editor for HTML5 piece that runs in NW.js. Cons: no form builder, incomplete CsoundAC support, need for script to run pieces. Pros: Full-powered, easily customizable, lightweight.
2. Code editor for CsoundAC written in Python. Cons: No form builder (but possible with addon to editor), no HTML5 support (but possible with PyWebKit). Pros: easily customizable, lightweight. Doing this one.
3. QtCreator. Cons: poor HTML5 support in QtWebEngine, more compex coding. Pros: Good debugging.

[gogins]

Investigate:

• Google Web Designer. Don't think it will do, designed to create ads, can't import existing HTML files into the form builder.
• Glade integrated in Scite to create UI for PyGTK.
• Wrap CsoundAC for Node.js using SWIG. Not doing as Python seems likely to work out.
• WebKit or CEF integrated in Python pieces: pywebview is alive, cefpython is alive but may be behind, pywebkit is dead. How to combine this with regular Python GUI? WebKit2 is in gi.repository.

[gogins]

I think the playpen is quite doable using Python GTK from a decent code editor. The editor should have custom commands for:

• Running glade.
• Running the app.
• Running the app with realtime rendering already started.

This is all doable with a naming pattern (piece.py goes with piece.glade) and with standardized command line options for the piece, as I already have for Python pieces, but internalized in the piece itself.

Then there can be a template for the piece that loads the Glade file, interprets the command options, and has standard menus and windows in the piece.

It seems better to just write a playpen application using Python GTK.

[gogins]

Changed the design to be a Python GTK3 application, as that looks like it might well be cross-platform and everything I have tried has worked pretty well so far. I'm impressed by the cleanness of GTK. Outstanding tasks:

• GUI builder for controls, using a Glade tree re-parented to the controls page.
• Handle different method names and types for the values of all widgets likely to be used for controlling pieces.
• Actually use the file naming convention.
• Run Glade with a new Glade file for a new piece, and the existing XML for an existing piece, and rebuild the GUI with the XML saved by Glade when it exits.
• User-specific settings go in a dictionary in the user settings directory.
• Customize the theme just a bit, mainly darker background with green letters for the Csound console, and a better color for the text in the code editor.
• Add the post-processing stuff.
• The GTK tutorial text editor is not really suitable. I am using GtkSource.View. Seems to work just fine.
• Create a Csound syntax file for the GtkSource.View? Python and C++ and HTML all work fine. I've looked into it, there are a lot of details and I am not going to do this unless it turns out I start relying on the playpen.
• Add a better csd example piece with a widget demo.
• Make the Csound file the basic container for Glade, Python, HTML, etc?
• Add Python example piece into the application program, so that user code is integrated into the playpen runtime. This does happen if the playpen simply calls exec(piece_python, globals(), locals()). In order to prevent the GUI from freezing, I have subclassed ctcsound.Csound as GtkCsound and overridden the perform method to use the Gtk event loop to run Csound with co-operative multiprocessing.
• Add HTML example piece, showing how to communicate between the playpen and the HTML and its JavaScript context. It looks like this can be done with WebKit Process Extensions. Here is a more extended example.
• Improve documentation, write installation stuff from Ubuntu standpoint with notes about other platforms.
• Create a Python package for this, see https://github.com/pypa/sampleproject. This militates against the JavaScript wrapper unless I build it with setuptools.

[gogins]

User-specific settings should include:

• Author.
• Publisher.
• Year.
• Notes.
• License.
• Audio output device.
• Audio input device.
• MIDI output device.
• MIDI input device.
• Soundfile editor.
• Gnome theme name.
• GtkSourceView style scheme name.

In reality, the use of a naming convention to derive the output soundfile name means that none of the Csound settings are actually required, although I will keep the audio output device as a convenience. The audio input and MIDI settings I will not implement.

[gogins]

Post-processing tasks, which if any can be done with GStreamer? A glance shows GStreamer harder to implement than adapting my existing script.

• Normalize.
• Tag.
• Convert to MP3.
• Convert to FLAC.
• Convert to MP4 with spectrogram image.

[gogins]

For triggers, it may suffice to send 1 for the pressed signal and 0 for the released signal.

It may be cleaner to have different handlers for different signals, or for different widget classes.

[gogins]

GtkSourceView looks usable but also tricky. See https://stackoverflow.com/questions/10524196/load-gui-from-a-glade-with-gtksourceview-in-pygobject. I have to make Glade aware of SourceView like this https://cjenkins.wordpress.com/2012/05/08/use-gtksourceview-widget-in-glade/, but that didn't work. But adding the old directory for the catalog did work.

API is here: https://developer.gnome.org/gtksourceview/stable/.

Looks like I have to create the editor programmatically and swap it in for the current GtkTextView. No, I got Glade to use the GtkSource.View by adding its catalog directory.

Search and replace is now working but you always have to press enter in the search entry field to get stated, even if you are then going to replace that text. This is not quite standard.

[gogins]

Just found out about GJS. But it's not relevant to my use case, nor is Seed.

Wrong! GJS is very relevant -- it contains the API references that I absolutely need. But there is no user guide!!

[gogins]

My question is, considering that ctcsound.py comes with Csound, is there any way to get it into the WebKit JavaScript context without the C compiler?

[gogins]

My problem right now is that I don't really want to re-implement csound.node for WebKit2. My choices seem to be:

1. Drop HTML support from the playpen.
2. Re-implement csound.node for WebKit2. But I don't see evidence that a lot of people are doing this kind of thing. I will look harder. But there seems to be SWIG support. I am going to try this.
3. Implement my own rudimentary RPC protocol just in Python and JavaScript for a Csound object. This ends up re-doing exactly what the QtWebEngine does, and I hate that.

[gogins]

Cleanup:

• Remove unneeded diagnostic messages.
• List all dependencies in README.md.
• If the user clicks the Edit button, create and open a new Glade file if none exists for the piece.
• See if values of widgets can be saved and restored after performing. Can't see it. Maybe I can do it myself as the Builder remains live. I only need to serialize channel values. I am going to try to use an XML DOM parser library. Didn't work. Actually just saving a dictionary of channel names and values, that seems to work fine, though in a separate file.
• Clean up settings, add metadata as well as a style for the code editor.
• If after editing/running a piece another piece is opened, things get messed up, and it is possible to wrongly overwrite a piece. This was caused by inconsistent use of a global variable for the piece filepath and the piece code, now fixed.
• After removing all widgets from the user controls layout, the controls panel is not empty as it should be.
• Better examples (see next comment).
• Add rudimentary User Guide to README.md.
• Replace build scripts with a CMakeLists.txt.
• Packaging -- keep it simple. Right now there is just a make install target, I may add a Debian package or other installers later.

[gogins]

Examples:

• xanadu.csd, plain Csound piece with no extras, just to prove that things are working.
• sierpinski-csound.py, Python and musx piece with a Glade-defined user interface; use a pluck that uses the controls! Using wgpluck2.
• message.html, HTML piece with a JQuery-based user interface.

[gogins]

The XML parser will read the .glade file into memory and listen for value changes on control widgets, updating the XML in response. When saving, the .glade file will be written from the DOM and not from the .glade file previous read. This will work only if:

• All control widget values can be saved in the .glade file. The Adjustment instances are indeed loaded from values in the .glade file. I assume other numeric control widgets would be as well.
• The XML written from the DOM will read back into the Builder and restore the control widget values. Yes it does.
• Save the current values of all control widgets to the UI file. _Not using the UI file, using a separate piece.ui.channels file.

[gogins]

The hard part is looking up elements from widgets. Some kind of extra metadata might simplify this. I can't arbitrarily add properties to the ui xml. But I could create a dictionary when I connect the signals.

[gogins]

The widget classes that I will save and restore, how their values can be accessed, and the xpath to the ui element that stores the value. This xpath begins at the element with the same id as the control channel name. "gk_channel" here stands for all channel names. For every get there is a corresponding set. Some of these are subclasses of others, but I need the specific class names in order to handle setting the values properly.

• GtkButton, event data parameter: not saved, but sent to channel.
• GtkCheckButton, get_active method, event data parameter.
• GtkEditable, get_text method.
• GtkRadioButton, get_active method, event data parameter.
• GtkScale, get_value method.
• GtkScaleButton, get_value method.
• GtkSpinButton, get_value method.
• GtkSwitch, event data parameter, get_state method.
• GtkToggleButton, get_active method.
• GtkVolumeButton, get_value method.

There are three categories: the get_value method, the get_active method, and the get_text method.

[gogins]

There is a fairly serious problem with the UI XML. Default widget values are not saved from ElementTree. I may be able to work around this, or I may use another method of storing the control channel values, e.g. with JSON for the map of channel names and values.

There are no Glade settings to control this, so ElementTree and GTK Builder are almost certainly not the problem.

I could work around this by constructing the elements for widget values in the XML, but I think I will go with a separate file, that should be less fragile.

[gogins]

I have a simple Python dict holding elementary type values for string channel names. This is saved and restored using JSON from piece.ui.channels. It is a separate file and I hate that, but the solution is simple and should be robust.

[gogins]

I'm closing this now. I'm going to start using it. No doubt there will be bugs. They will become separate issues.