Third party developer support/DevX for applets and other cosmic integration

[SUPERCILEX]

Gist: what's the long-term support story for third-party developers? Are there plans for docs eventually? Or should it be community lead?

---

I started seriously looking into what it would take to build the clipboard manager an am hoping/would love to get support from the team. My questions fall into three categories: applet/cosmic integration, system76 support for the final polish/push to 100%, and "how do the internals work."

Integration:

• How do I get an applet to run on startup?
• How can I add settings to my applet? Is there a central registry or is this ad-hoc?
• How do I register shortcuts to run code in the applet?
• Is it possible to make the applet appear at the user's cursor position? Ideally you could hit a shortcut and the applet would appear on the bottom right of the cursor, like in Windows.
• What does the installation/packaging story look like? Right now my default will be to build binaries on CI and tell people to use cargo to install locally if they have that. But is there some way to get packaged with you guys? On that note, how do I get the applet to show up in the status bar? Right now it's just a floating icon.

System76 resources:

• Would system76 be willing to provide translation support for popular applets? As in translate the final strings?
• Would system76 be willing to provide UI/UX support from designers to ensure the applet feels cohesive with the rest of the OS? Or maybe a set of design guidelines? Think Google's Material Design.
• What else should I be doing to make the integration seamless? Would system76 be willing to provide code reviews?

How do it work d'oh:

• How does iced actually work? I have a background in Android development (pre compose) and the fact that I'll be dumping the entirety of the clipboard into a single scrollable terrifies me. Unless there's something clever going on under the hood, I'm eventually going to need a recycler-view style thing (i.e. only render the visible views and reuse their resources).
• What's the difference between cosmic iced and upstream? In general, what are cosmic APIs devs should know about? Is the best path forward just browsing https://pop-os.github.io/libcosmic/cosmic/index.html?

---

Some of this discussion (especially the iced internals) might work better over a call (possibly recorded so other people can get up to speed) if that makes sense to do? I'm also thinking it'd be cool to write about the experience and have the clipboard manager be a showcase for third-party cosmic integration.

[SUPERCILEX]

Reverse index: https://www.reddit.com/r/pop_os/comments/194bugz/comment/khmat5e/?utm_source=share&utm_medium=web2x&context=3

cc @mmstick

[wiiznokes]

I was also thinking of making a clipboard history applet too. I would be happy to help with development.
You might want to take a look at https://github.com/mohamadzoh/clipop, it's made with Iced. I've just tested it, and it works on Linux. It will require serious change tho.

Ps: Great work for the Gnome extension, I thought your name remind me something 😅

[SUPERCILEX]

@wiiznokes thanks! ❤️ I see you have iced experience which is awesome if you want to collab. Not sure what you enjoy implementing—I personally don't like UI so I'd love any help I can get there. I more of a distributed systems guy with a focus on low-level performance engineering. I've started out the design work for the clipboard here if you're interested in the low level stuff: SUPERCILEX/clipboard-history#3.

[Drakulix]

First off: Thanks for your interest in this.

Let me preface answering your more specific questions with the round about answer of:

• We support a bunch of standardized wayland protocols.
• The panel is just another wayland compositor, each applet is just an app rendering a window and popups, that get placed by the panel process.
• None of this is documented, but the goal is to be compatible with the wider ecosystem and existing solutions rather than coming up with our own api where possible.
• Some of this is still in flux and more complex plugins will require cooperation of our desktop components, which may or may not be there already.
• As such you are also not required to use libcosmic in any capacity. It just implements most of these procotols already and is our primary target, but other toolkits do too.

That said I believe everything necessary to build a clipboard manager is in place.

Gist: what's the long-term support story for third-party developers? Are there plans for docs eventually? Or should it be community lead?

Yes there will be docs, but this isn't the first priority right now. The priority right now is to finalize the experience of COSMIC. Many components are built with customizability in mind, but most of this is only "documented" through code (unfortunately).

Community-lead docs are a possible way to get this of the ground faster and while I can't speak for all of System76 would certainly be welcome from my perspective. It would be good to coordinate such efforts for example in our "COSMIC Epoch" mattermost channel, should somebody want to get this started.

• How do I get an applet to run on startup?

Install it in the system, add a desktop-file for it (which will need some special keys to be picked up by the panel, see existing desktop files in the cosmic-applets repo). Then it will appear in cosmic-settings and can be added to the panel. (All of this works today and is how our applets operate. Cosmic-panel just ships with a standard configuration of applets.)

• How can I add settings to my applet? Is there a central registry or is this ad-hoc?

All our applets use "cosmic-config" for configuration (which can be found in the libcosmic repo), but that is no requirement by any means.

• How do I register shortcuts to run code in the applet?

Custom shortcuts currently can be added through cosmic-comp's config file. This will be replaced by cosmic-config at some point and then accessible manually through cosmic-settings.
Mid to long term applications should be able to register shortcuts themselves through the corresponding xdg-desktop-portal interface (not supported yet).

• Is it possible to make the applet appear at the user's cursor position? Ideally you could hit a shortcut and the applet would appear on the bottom right of the cursor, like in Windows.

Getting the exact position of the user's cursor is currently not possible. Placement is done within the constraints of either:

• an xdg-popup of the applet icon, placed relative to the icon
• or within the constraints of the wlr-layer-shell protocol (which is probably more interesting to you, when it comes to shortcuts)

I suggest checking if either of those are good enough for your use-case and otherwise open an issue to discuss your use-case (though I can't promise any results here).

• What does the installation/packaging story look like? Right now my default will be to build binaries on CI and tell people to use cargo to install locally if they have that. But is there some way to get packaged with you guys? On that note, how do I get the applet to show up in the status bar? Right now it's just a floating icon.

We do want to make custom applets an integral part of cosmic's customizablity story. As part of that we have brainstormed about something like an "Applet Store". However those plans are not set in stone in any way, shape or form. So any official solution like that is likely going to take a lot more time.

In the meantime shipping binaries from Github Releases or crates.io is certainly an easy option. If you specifically want to cater to Pop!_OS users, setting up a PPA may be another option. I am not sure, if there is a process of getting third-party packages accepted into our main repositories, maybe somebody else is able to comment on that.

• Would system76 be willing to provide translation support for popular applets? As in translate the final strings?

Translations are mostly community-organized at this point. I lack knowledge to comment on future plans.

• Would system76 be willing to provide UI/UX support from designers to ensure the applet feels cohesive with the rest of the OS? Or maybe a set of design guidelines? Think Google's Material Design.

Yes, but this is again more of a mid-long term effort. Documents for this already exist and will be polished up and shared at some point as far as I am aware.

• What else should I be doing to make the integration seamless? Would system76 be willing to provide code reviews?

Again lacking knowledge, but I would assume we don't have the resources right now to facilitate an offer like this. However you are certainly free to reach out and see, if somebody is willing to invest time once there is something to review.

For the iced related question I am not the best to answer them. Maybe @mmstick or @wash2 want to comment on those.

[wiiznokes]

@wiiznokes thanks! ❤️ I see you have iced experience which is awesome if you want to collab. Not sure what you enjoy implementing—I personally don't like UI so I'd love any help I can get there. I more of a distributed systems guy with a focus on low-level performance engineering. I've started out the design work for the clipboard here if you're interested in the low level stuff: SUPERCILEX/clipboard-history#3.

This seems over complicated for a clipboard manager 😆 . Rust is fast enough to just use a Vec imo. And a fuzzy search would be nice.

Otherwise, i prefer to wait a little before libcosmic is more stable and cosmic is release, because it will be difficult to test the app on Gnome (at least the last time i tested, this havn't worked).
And imo, using libcosmic is important to have a consistent style across the DE, and since it's a common applet, it make sens to use libcosmic.

[mmstick]

We forked iced to implement a custom sctk runtime for the Wayland integrations that we need for applets, as well as various modifications to implement features that we need in libcosmic.

You would not develop applications directly with iced though because it's only doing the bare minimum to provide a runtime with some limited widget primitives without any theming capabilities. It does not automatically handle window manager or keyboard shortcut events, so a decent amount of boilerplate is required to set up subscriptions to get started. By itself, it is more like GDK than GTK.

libcosmic uses the runtime provided by iced, but replaces its Application trait, and replaces its widgets with an alternative COSMIC themed widget library. There are various supporting types, functions, and traits to automate application setup, integration with window managers, common application behaviors, cosmic-config integration, and methods for utilizing COSMIC interface concepts such as the context drawer and navigation bar.

For applets, libcosmic provides an applet trait which automates applet setup. So by using libcosmic for applets, you get access to the full libcosmic widget library, COSMIC theming, and cosmic-config subscriptions; without needing to do most the applet integration setup.

[mmstick]

Would system76 be willing to provide translation support for popular applets? As in translate the final strings?

We do not have translators. All translations are contributed by volunteers. You would have to rely on the same.

Would system76 be willing to provide UI/UX support from designers to ensure the applet feels cohesive with the rest of the OS? Or maybe a set of design guidelines? Think Google's Material Design.

Unless we are promoting and distributing the applet in COSMIC, direct support would be on a voluntary basis. You could ask for some help in a development room in Mattermost. Though as long as you are using libcosmic and compare to existing applets, there shouldn't be a problem with consistency and theming. An interface guideline will be released after COSMIC is finished.

What else should I be doing to make the integration seamless? Would system76 be willing to provide code reviews?

Applets developed with libcosmic and its widgets will integrate seamlessly. We wouldn't provide code reviews unless it is an applet that we are promoting in our curated applet shop. Applets that become officially maintained by our team would of course have code and QA reviews for all pull requests.

[mmstick]

How does iced actually work?

It works like a direct equivalent to Elm in Rust. So if you are familiar with Elm, you will model your applications in precisely the same way here. You begin with an application struct that contains all application state. You will implement the cosmic::Application trait on that struct, which consists of at least a view method to generate interface updates, and an update method to handle messages emitted by controls generated from the view.

You have only immutable access to the application state in the view, and mutable access during an update. So when pattern matching on your application's message type in the update method, you can modify state and cache what's necessary to be used by the next view. The update method then returns an optional Command; which is an async job, or batch of async jobs; that will be executed on an async runtime (tokio) running in the background.

For optimal application performance, you should return from the update method ASAP by pushing most application logic onto background threads with commands or tokio. Commands are asynchronously awaited by the iced runtime on the main thread. Each command must also return a message type that will be passed into the update method once received. So if you want to execute some code in the background without returning a message, you can opt to spawning a future on tokio directly.

If you want to develop an application with background services that pass their messages into your application, you would spawn their event loop with either std::thread or tokio::spawn, and send their responses back to your application through a channel. Using the receiving end of the channel, you can attach it to a Subscription that you register in your application's subscription method.

[SUPERCILEX]

@Drakulix

Let me preface answering your more specific questions with the round about answer of:

This is great! Long term it would nice to have this documented in some "Create your own applet" page, but it's very nice to hear you guys are trying to work with the rest of the ecosystem.

Yes there will be docs, but this isn't the first priority right now. The priority right now is to finalize the experience of COSMIC.

That works and sounds like you have the right priorities. 😁

mattermost channel

Totally forgot you guys had a mattermost, thanks for the reminder. I'm too used to thinking anything rust => zulip.

Install it in the system, add a desktop-file

👍

Custom shortcuts currently can be added through cosmic-comp's config file. This will be replaced by cosmic-config at some point and then accessible manually through cosmic-settings.
Mid to long term applications should be able to register shortcuts themselves through the corresponding xdg-desktop-portal interface (not supported yet).

Sounds good, I'll just wait for that to be implemented if I somehow manage to beat you guys' development pace. 😅

or within the constraints of the wlr-layer-shell protocol

Ok, sounds like it should be possible long-term, so I'll cross that bridge when I get there. It's a feature request I've had for a long time but isn't really feasible with gnome AFAIK.

We do want to make custom applets an integral part of cosmic's customizability story.

Awesome! If this all pans out, I'm sure we can work something out in the meantime (the ppa idea sounds like good ux).

Translations are mostly community-organized at this point.

TIL. That works!

Yes, but this is again more of a mid-long term effort. Documents for this already exist and will be polished up and shared at some point as far as I am aware.

Perfect!

I would assume we don't have the resources right now to facilitate an offer like this. However you are certainly free to reach out and see, if somebody is willing to invest time once there is something to review.

That works, thank you!

---

@wiiznokes

This seems over complicated for a clipboard manager 😆 . Rust is fast enough to just use a Vec imo.

Lol for sure, but then I'd be bored. 😉

Otherwise, i prefer to wait a little before libcosmic is more stable and cosmic is release

Yeah, my new plan is write the app with a core backend crate and frontend crates for TUI, maybe normal iced, and a libcosmic applet. That'll cover all the bases and provide solid foundations that are almost certain to never have to be re-written.

---

@mmstick

You would not develop applications directly with iced though because it's only doing the bare minimum to provide a runtime with some limited widget primitives without any theming capabilities. It does not automatically handle window manager or keyboard shortcut events, so a decent amount of boilerplate is required to set up subscriptions to get started. By itself, it is more like GDK than GTK.

Ah, this is very interesting, thanks! It makes more sense why there wouldn't be as many docs if iced is a low-level framework.

So by using libcosmic for applets, you get access to the full libcosmic widget library, COSMIC theming, and cosmic-config subscriptions; without needing to do most the applet integration setup.

Yup, that's what I saw when running stuff from https://github.com/pop-os/cosmic-applets.

---

@mmstick

Sounds consistent with what Victoria said, thanks!

---

@mmstick

If you want to develop an application with background services that pass their messages into your application, you would spawn their event loop with either std::thread or tokio::spawn, and send their responses back to your application through a channel. Using the receiving end of the channel, you can attach it to a Subscription that you register in your application's subscription method.

This is extremely helpful, thank you! I was trying to figure out how I'd decouple background threads that are computing things for the UI vs that sit around waiting for something external to happen before telling the UI. It sounds like subscriptions precisely model that latter case of UI being changed externally.

Regarding performance, I'd specifically like to know how scrollable is implemented. A dumb implementation will run measure and layout on the entire list (which won't work when it's huge) while a smart implementation IIRC will track how far you've scrolling and only measure+layout what's going to be rendered on-screen +- a delta to give the illusion of a fully rendered list when scrolling.

[wiiznokes]

I checked the wl-clipboard-rs crate, and posted this question. I didn't looked the actual code a lot, but apparently, it will require some change (probably implementing a stream with the wayland dispatch function (You should know i'm new to wayland and async rust)).

How ever, i tried to the command wl-paste -w ls from wl-clipboard (it should emulate the actual behavior needed for a clipboard-manager), and this is the ouput on COSMIC:

$ wl-paste -w ls
Watch mode requires a compositor that supports wlroots data-control protocol

@Drakulix Maybe the compositor need this ?
cc @SUPERCILEX

[Drakulix]

We have the wlr-data-control protocol, it is just disabled by default currently, because it is a security risk. You can enable it in the cosmic-comps configuration file.

EDIT: Also applets have more privileged access to protocols, this could include wlr-data-control.

[wiiznokes]

Thanks, i will try to make an applet then

[SUPERCILEX]

For anyone wondering what happened to this on my end, I'm pretty close to a production ready clipboard: https://github.com/SUPERCILEX/clipboard-history. I still need to implement a wayland watcher and paste support, but other than that it's basically done. There's no COSMIC applet yet though, just a DE agnostic GUI—if anyone wants to work on the applet I'd be happy to accept PRs.

I don't think there's anything actionable in this issue (besides great info that a search engine will find), so closing.