Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(developer): manage shared fonts #11236

Open
mcdurdin opened this issue Apr 17, 2024 · 4 comments · May be fixed by #12336
Open

feat(developer): manage shared fonts #11236

mcdurdin opened this issue Apr 17, 2024 · 4 comments · May be fixed by #12336
Assignees
@mcdurdin
Labels
developer/ feat
Milestone
A18S14

Comments

@mcdurdin
Copy link
Member

mcdurdin commented Apr 17, 2024 •
edited
Loading

Fonts are currently difficult to manage in Keyman Developer projects, particularly because we want to share fonts between keyboards. The developer must either clone the entire keyboards repository, or else jump through convoluted hoops in order to properly use shared fonts.

Proposal, in short: allow packages to reference files from GitHub repos, not just local files. Then the compiler will download (and maybe cache) files. The keyboard author could reference, e.g. https://github.com/keymanapp/keyboards/blob/master/release/shared/fonts/aegyptus/Aegyptus.otf. (Query, should we use sha instead of branch names?)

Ideally, we can leverage an existing WSTech repository for this. Perhaps LFF. In theory, we could allow any URL in the .kps file. But we would probably want to have a whitelist for perf, stability, and security reasons for the keyboard repository.

This also resolves the single-keyboard-repo conundrum of how to share fonts.

It would be important to have a UI that helps keyboard devs to find the fonts as well.
@mcdurdin mcdurdin added this to the 18.0 milestone Apr 17, 2024
@mcdurdin
Copy link
Member Author

@DavidLRowe @LornaSIL @srl295 your thoughts appreciated. I think that this would go a long way towards addressing remaining font sharing issues and allow us to eventually deprecate having fonts in the Keyman keyboards repository itself.

Obviously this could be applied to more than just fonts but would like to limit the conversation here to the font puzzle.

@LornaSIL
Copy link

Proposal, in short: allow packages to reference files from GitHub repos, not just local files.

Yes, if that is possible it would be fine.

  • What about referencing the Noto fonts online?
    • Use the embed code:

image

  • Or referencing LFF (that would mean we would need to include all the desired fonts on LFF).

  • What would we do about online help files needing fonts? Would we still need to add them to s.keyman.com? Or could we again use the github repo or LFF for display of the help files? Or use the embed code from Noto.

  • There are fonts we don't really want in LFF (PUA encoded, hacked fonts, etc.) so the referencing the Keyman Github repo might be best.

@mcdurdin
Copy link
Member Author

What about referencing the Noto fonts online?

  • Use the embed code:

We couldn't use the embed code as that is a HTML/CSS construct, as we need to be able to download a .ttf directly for embedding into the package. That may be possible also, but it needs to be a stable URL, so I would lean towards requiring a GitHub repo that we control, or that has good stability guarantees.

  • What would we do about online help files needing fonts? Would we still need to add them to s.keyman.com? Or could we again use the github repo or LFF for display of the help files? Or use the embed code from Noto.

My thinking is we would automatically deploy all fonts found during the build to s.keyman.com. The primary problem would be if more than one package used the same font in different versions.

  • Or referencing LFF (that would mean we would need to include all the desired fonts on LFF).
  • There are fonts we don't really want in LFF (PUA encoded, hacked fonts, etc.) so the referencing the Keyman Github repo might be best.

We can certainly support more than one source, and it makes sense to make the mechanism generic for future extension, even if we limit the sources we will support in the keyboards repository. We can have LFF and other fonts in the keymanapp/keyboards repo or even in a separate keymanapp/keyboard-fonts repo to keep it clean.

Technical note

This would slow down the build, as it would now need to source online content. It also introduces some fragility relating to network availability. Caching of fonts could mitigate both of those concerns, but caching also adds complexity.

@mcdurdin mcdurdin modified the milestones: 18.0, A18S5 Apr 29, 2024
@mcdurdin mcdurdin self-assigned this Apr 29, 2024
@mcdurdin mcdurdin modified the milestones: A18S5, A18S7 Jul 5, 2024
@darcywong00 darcywong00 modified the milestones: A18S7, A18S8 Aug 2, 2024
@srl295 srl295 mentioned this issue Aug 6, 2024
Open
@darcywong00 darcywong00 modified the milestones: A18S8, A18S9 Aug 17, 2024
@mcdurdin
Copy link
Member Author

Suggest we add a Source element to kps Package.Files.File. This would be:

Question on source of truth for filename of font. Preferable to use Source filename where possible I think. But then refs in OSKFont and DisplayFont elements are harder to determine (tooling makes this easier but raw edit of XML leads to inconsistency).

At the same time CopyLocation, Description, and FileType elements should be deprecated and totally ignored by the compiler. They are busydata.

This would bump Package.System.FileVersion to e.g. 18.0.

There would be no change to the corresponding .kmp file format.

@darcywong00 darcywong00 modified the milestones: A18S9, A18S10 Aug 31, 2024
mcdurdin added a commit that referenced this issue Sep 2, 2024
@mcdurdin
feat(developer): source files from remote URLs
1a0161c 
TEMP: this is very much not ready for production; opening as a draft for
public comment.

I will split the 'file.description' removal into a separate PR later.

The concept here is that the 'Name' property for a file can now be a
remote reference, rather than a local file. There are two supported
formats in this commit:

* GitHub: This is a cutdown version of a plain github.com URL, and must
  match this exact format:

  ```
  github:<owner>/<repo>/raw/<hash>/<filepath/filename>
  ```

  This format is mandated in order to ensure that we always have a
  hashed version of a file from the origin. This gives us reproducible
  builds, which avoids churn issues when font files change.

  Example: `github:silnrsi/fonts/raw/b88c7af5d16681bd137156929ff8baec82526560/fonts/sil/alkalami/Alkalami-Regular.ttf`
  gets https://github.com/silnrsi/fonts/raw/b88c7af5d16681bd137156929ff8baec82526560/fonts/sil/alkalami/Alkalami-Regular.ttf

  An alternative could be to just have `https://github.com/silnrsi/fonts/raw/b88c7af5d16681bd137156929ff8baec82526560/fonts/sil/alkalami/Alkalami-Regular.ttf`
  which could be matched with a regex in the same way as the `github`
  prefix, and would avoid the need to munge the input URL. **Discuss!**

* fonts.languagetechnology.org: references just a font identifier. This
  is somewhat broken, because if the source file changes, we don't know
  about it and won't publish an updated version of the package. So this
  needs some more discussion (we could e.g. embed the version number in
  the request, e.g. `flo:andika@6.200`). **Discuss!**

  ```
  flo:<family>
  ```

  e.g. `flo:andika` gets https://fonts.languagetechnology.org/fonts/sil/andika/Andika-Bold.ttf

Future sources could be considered, e.g. noto. We don't want to allow
arbitrary URLs, both for stability and for security reasons.

This change is entirely compiler-side, so we don't need to make any
changes to apps, and so packages will be backwardly compatible. A lot of
work will need to be done with the Package Editor in TIKE to support
this feature.

Fixes: #11236
@mcdurdin mcdurdin linked a pull request Sep 2, 2024 that will close this issue
Draft
@darcywong00 darcywong00 modified the milestones: A18S10, A18S11 Sep 14, 2024
@darcywong00 darcywong00 modified the milestones: A18S11, A18S12 Sep 28, 2024
@darcywong00 darcywong00 modified the milestones: A18S12, A18S13 Oct 11, 2024
@darcywong00 darcywong00 modified the milestones: A18S13, A18S14 Oct 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mcdurdin mcdurdin

Labels
developer/ feat
Projects
Keyman
Status: No status
Milestone
A18S14
Development

Successfully merging a pull request may close this issue.

feat(developer): source files from remote URLs keymanapp/keyman
3 participants
@mcdurdin @LornaSIL @darcywong00