Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add glyphs to unicode reserved plane #211

Open
driusan opened this issue Jul 14, 2016 · 57 comments
Open

Add glyphs to unicode reserved plane #211

driusan opened this issue Jul 14, 2016 · 57 comments
Labels

Comments

@driusan
Copy link

driusan commented Jul 14, 2016

I'm trying to use FiraCode w/ ligatures in a text editor I'm writing (where I'm currently using Deja Vu Sans Mono), but there doesn't seem to be any way to access the ligatures from Go, where the standard font drawing methods are based on converting unicode runes to glyphs and have no knowledge of ligatures.

The Unicode private use area planes is designed for this purpose, according to wikipedia, but doesn't seem to be used by Firacode (at least as far as I can tell by printing runes starting at 0x100000 or 0x0f0000 and visually scanning). Would it be possible to map the ligatures to the supplementary plan? If they were accessible as runes in a private use area with a documented mapping, I could manually do the context-sensitive mapping when rendering.

It's possible that I'm missing something, because all the "how to use the font" documentation seems to be user documentation, and I can't find any developer documentation..

@siegebell
Copy link

siegebell commented Aug 9, 2016

Providing the fancy glyphs via private use area (PUA) codes would resolve #42 via prettify-symbols-mode.

@siegebell
Copy link

@mordocai I believe this is what you're looking for.

@siegebell
Copy link

siegebell commented Aug 11, 2016

@mordocai Here's a workaround: I've extracted the glyphs that Fira Code adds into a separate font, and assigned each a Unicode ID in the private character area. Download the symbols font: FiraCode-Regular-Symbol.zip

This symbols font should be assigned as a fallback font for either Fira Code or Fira Mono because it does not define the regular characters.

Character map:

 www  \ue100     **   \ue101    ***  \ue102    **/  \ue103
  *>  \ue104     */   \ue105     \\  \ue106    \\\  \ue107
  {-  \ue108     []   \ue109     ::  \ue10a    :::  \ue10b
  :=  \ue10c     !!   \ue10d     !=  \ue10e    !==  \ue10f
  -}  \ue110     --   \ue111    ---  \ue112    -->  \ue113
  ->  \ue114    ->>   \ue115     -<  \ue116    -<<  \ue117
  -~  \ue118     #{   \ue119     #[  \ue11a     ##  \ue11b
 ###  \ue11c   ####   \ue11d     #(  \ue11e     #?  \ue11f
  #_  \ue120    #_(   \ue121     .-  \ue122     .=  \ue123
  ..  \ue124    ..<   \ue125    ...  \ue126     ?=  \ue127
  ??  \ue128     ;;   \ue129     /*  \ue12a    /**  \ue12b
  /=  \ue12c    /==   \ue12d     />  \ue12e     //  \ue12f
 ///  \ue130     &&   \ue131     ||  \ue132    ||=  \ue133
  |=  \ue134     |>   \ue135     ^=  \ue136     $>  \ue137
  ++  \ue138    +++   \ue139     +>  \ue13a     +>  \ue13a
 =:=  \ue13b     ==   \ue13c    ===  \ue13d    ==>  \ue13e
  =>  \ue13f    =>>   \ue140     <=  \ue141    =<<  \ue142
 =/=  \ue143     >-   \ue144     >=  \ue145    >=>  \ue146
  >>  \ue147    >>-   \ue148    >>=  \ue149    >>>  \ue14a
  <*  \ue14b    <*>   \ue14c     <|  \ue14d    <|>  \ue14e
  <$  \ue14f    <$>   \ue150   <!--  \ue151     <-  \ue152
 <--  \ue153    <->   \ue154     <+  \ue155    <+>  \ue156
  <=  \ue157    <==   \ue158    <=>  \ue159    <=<  \ue15a
  <>  \ue15b     <<   \ue15c    <<-  \ue15d    <<=  \ue15e
 <<<  \ue15f     <~   \ue160    <~~  \ue161     </  \ue162
 </>  \ue163     ~@   \ue164     ~-  \ue165     ~=  \ue166
  ~>  \ue167     ~~   \ue168    ~~>  \ue169     %%  \ue16a
   x  \ue16b      :   \ue16c      +  \ue16d      *  \ue16f

@siegebell
Copy link

siegebell commented Aug 11, 2016

And if you're using the prettify-symbols-mode extension for vscode, you can use the following substitution settings. (prettify-symbols-mode for Emacs should look similar-ish, but you'll combine the pre, ugly, and post into the same regex and \ue00a0 maybe can be replaced with a normal space.)

"prettifySymbolsMode.substitutions": [
  {
    "language": "*",
    "substitutions": [
      { "ugly": "\\{2}", "pretty": "\u03bb", "post": "\\s*(?:\\w|_).*?\\s*->" },
      { "ugly": "\\.=", "pretty": " \ue123", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "\\.-", "pretty": " \ue122", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": ":=", "pretty": " \ue10c", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "=:=", "pretty": "\u00a0  \ue13b", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "==", "pretty": " \ue13c", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "!=", "pretty": " \ue10e", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "===", "pretty": "\u00a0  \ue13d", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "!==", "pretty": "\u00a0  \ue10f", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "=/=", "pretty": "\u00a0  \ue143", "pre": "[^.=:!/<>]|^", "post": "[^.=:!/<>]|$" },
      { "ugly": "<-", "pretty": " \ue152", "pre": "[^<\\->=]|^", "post": "[^<\\-=>]|$" },
      { "ugly": "->", "pretty": " \ue114", "pre": "[^<\\->=]|^", "post": "[^<\\-=>]|$" },
      { "ugly": "<--", "pretty": "\u00a0  \ue153", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "-->", "pretty": "\u00a0  \ue113", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<->", "pretty": "\u00a0  \ue154", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<<-", "pretty": "\u00a0  \ue15d", "pre": "[^<\\->=]|^", "post": "[^<\\-=>]|$" },
      { "ugly": "->>", "pretty": "\u00a0  \ue115", "pre": "[^<\\->=]|^", "post": "[^<\\-=>]|$" },
      { "ugly": "=>", "pretty": " \ue13f", "pre": "[^<\\->=]|^", "post": "[^<\\-=>]|$" },
      { "ugly": "<==", "pretty": "\u00a0  \ue158", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "==>", "pretty": "\u00a0  \ue13e", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<=>", "pretty": "\u00a0  \ue159", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<=<", "pretty": "\u00a0  \ue15a", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<<=", "pretty": "\u00a0  \ue15e", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "=>>", "pretty": "\u00a0  \ue140", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">=>", "pretty": "\u00a0  \ue146", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">>=", "pretty": "\u00a0  \ue149", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">>-", "pretty": "\u00a0  \ue148", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">-", "pretty": " \ue144", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "-<", "pretty": " \ue116", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "-<<", "pretty": "\u00a0  \ue117", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "=<<", "pretty": "\u00a0  \ue142", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<~~", "pretty": "\u00a0  \ue161", "pre": "[^<\\->=~]|^", "post": "[^<\\->=~]|$" },
      { "ugly": "<~", "pretty": " \ue160", "pre": "[^<\\->=~]|^", "post": "[^<\\->=~]|$" },
      { "ugly": "~~", "pretty": " \ue168", "pre": "[^<\\->=~]|^", "post": "[^<\\->=~]|$" },
      { "ugly": "~>", "pretty": " \ue167", "pre": "[^<\\->=~]|^", "post": "[^<\\->=~]|$" },
      { "ugly": "~~>", "pretty": "\u00a0  \ue169", "pre": "[^<\\->=~]|^", "post": "[^<\\->=~]|$" },
      { "ugly": "<<<", "pretty": "\u00a0  \ue15f", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<<", "pretty": " \ue15c", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<=", "pretty": " \ue141", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<>", "pretty": " \ue15b", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">=", "pretty": " \ue145", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">>", "pretty": " \ue147", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": ">>>", "pretty": "\u00a0  \ue14a", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<\\|", "pretty": " \ue14d", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "<\\|>", "pretty": "\u00a0  \ue14e", "pre": "[^<\\->=|]|^", "post": "[^<\\->=|]|$" },
      { "ugly": "\\|>", "pretty": " \ue135", "pre": "[^<\\->=|]|^", "post": "[^<\\->=|]|$" },
      { "ugly": "<\\$", "pretty": " \ue14f", "pre": "[^<\\->=$]|^", "post": "[^<\\->=$]|$" },
      { "ugly": "<\\$>", "pretty": "\u00a0  \ue150", "pre": "[^<\\->=$]|^", "post": "[^<\\->=$]|$" },
      { "ugly": "\\$>", "pretty": " \ue137", "pre": "[^<\\->=$]|^", "post": "[^<\\->=$]|$" },
      { "ugly": "<\\+", "pretty": " \ue155", "pre": "[^<\\->=+]|^", "post": "[^<\\->=+]|$" },
      { "ugly": "<\\+>", "pretty": "\u00a0  \ue156", "pre": "[^<\\->=+]|^", "post": "[^<\\->=+]|$" },
      { "ugly": "\\+>", "pretty": " \ue13a", "pre": "[^<\\->=+]|^", "post": "[^<\\->=+]|$" },
      { "ugly": "<\\*", "pretty": " \ue14b", "pre": "[^<\\->=*]|^", "post": "[^<\\->=*]|$" },
      { "ugly": "<\\*>", "pretty": "\u00a0  \ue14c", "pre": "[^<\\->=*]|^", "post": "[^<\\->=*]|$" },
      { "ugly": "\\*>", "pretty": " \ue104", "pre": "[^<\\->=*]|^", "post": "[^<\\->=*]|$" },
      { "ugly": "\\\\\\\\", "pretty": " \ue106", "pre": "[^<\\\\\\->=]|^", "post": "[^<\\\\\\->=]|$" },
      { "ugly": "\\\\\\\\\\\\", "pretty": "\u00a0  \ue107", "pre": "[^<\\\\\\->=]|^", "post": "[^<\\\\\\->=]|$" },
      { "ugly": "\\{-", "pretty": " \ue108", "pre": "[^<\\\\\\->={}]|^", "post": "[^<\\\\\\->={}]|$" },
      { "ugly": "-}", "pretty": " \ue110", "pre": "[^<\\->={}]|^", "post": "[^<\\->={}]|$" },
      { "ugly": "//", "pretty": " \ue12f", "pre": "[^<\\->=/]|^", "post": "[^<\\->=/]|$" },
      { "ugly": "///", "pretty": "\u00a0  \ue130", "pre": "[^<\\->=/]|^", "post": "[^<\\->=/]|$" },
      { "ugly": "/\\*", "pretty": " \ue12a", "pre": "[^<\\->=/*]|^", "post": "[^<\\->=/*]|$" },
      { "ugly": "/\\*\\*", "pretty": "\u00a0  \ue12b", "pre": "[^<\\->=/*]|^", "post": "[^<\\->=/*]|$" },
      { "ugly": "\\*\\*/", "pretty": "\u00a0  \ue103", "pre": "[^<\\->=/*]|^", "post": "[^<\\->=/*]|$" },
      { "ugly": "\\*/", "pretty": " \ue105", "pre": "[^<\\->=/*]|^", "post": "[^<\\->=/*]|$" },
      { "ugly": "</", "pretty": " \ue162", "pre": "[^<\\->=/]|^", "post": "[^<\\->=/]|$" },
      { "ugly": "<\\!--", "pretty": "\u00a0 \u00a0\ue151", "pre": "[^<\\->=!]|^", "post": "[^<\\->=!]|$" },
      { "ugly": "</>", "pretty": "\u00a0  \ue163", "pre": "[^<\\->=]|^", "post": "[^<\\->=]|$" },
      { "ugly": "/>", "pretty": " \ue12e", "pre": "[^<\\->=/]|^", "post": "[^<\\->=/]|$" },
      { "ugly": "x", "pretty": "\ue16b", "pre": "[0-9a-fA-F]", "post": "[0-9a-fA-F]" },
      { "ugly": ":", "pretty": "\ue16c", "pre": "[0-9]{1,2}", "post": "[0-9]{1,2}" },
      { "ugly": "\\+", "pretty": "\ue16d", "pre": "[0-9a-zA-Z]", "post": "[0-9a-zA-Z]" },
      { "ugly": "-", "pretty": "-", "pre": "[0-9a-zA-Z]", "post": "[0-9a-zA-Z]" },
      { "ugly": "\\*", "pretty": "*", "pre": "", "post": "" },
      { "ugly": "www", "pretty": "\u00a0  \ue100", "pre": "\\b", "post": "\\b" },
      { "ugly": ";;", "pretty": " \ue129", "pre": "[^;]|^", "post": "[^;]|$" },
      { "ugly": "::", "pretty": " \ue10a", "pre": "[^:]|^", "post": "[^:]|$" },
      { "ugly": ":::", "pretty": "\u00a0  \ue10b", "pre": "[^:]|^", "post": "[^:]|$" },
      { "ugly": "!!", "pretty": " \ue10d", "pre": "[^!]|^", "post": "[^!]|$" },
      { "ugly": "\\?{2}", "pretty": " \ue128", "pre": "[^?]|^", "post": "[^?]|$" },
      { "ugly": "%%", "pretty": " \ue16a", "pre": "[^%]|^", "post": "[^%]|$" },
      { "ugly": "&&", "pretty": " \ue131", "pre": "[^&]|^", "post": "[^&]|$" },
      { "ugly": "\\|{2}", "pretty": " \ue132", "pre": "[^|=]|^", "post": "[^|=]|$" },
      { "ugly": "\\.{2}", "pretty": " \ue124", "pre": "[^.<]|^", "post": "[^.<]|$" },
      { "ugly": "\\.{3}", "pretty": "\u00a0  \ue126", "pre": "[^.<]|^", "post": "[^.<]|$" },
      { "ugly": "\\.\\.<", "pretty": "\u00a0  \ue125", "pre": "[^.<]|^", "post": "[^.<]|$" },
      { "ugly": "\\[\\]", "pretty": " \ue109", "pre": "[^[\\]]|^", "post": "[^[\\]]|$" },
      { "ugly": "--", "pretty": " \ue111", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "---", "pretty": "\u00a0 \ue112", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "\\+{2}", "pretty": " \ue138", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "\\+{3}", "pretty": "\u00a0  \ue139", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "\\*{2}", "pretty": " \ue101", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "\\*{3}", "pretty": "\u00a0  \ue102", "pre": "[^\\-+*=<>~]|^", "post": "[^\\-+*=<>~]|$" },
      { "ugly": "~=", "pretty": " \ue166", "pre": "[^~=\\-]|^", "post": "[^~=\\-]|$" },
      { "ugly": "~-", "pretty": " \ue165", "pre": "[^~=\\-<>]|^", "post": "[^~=\\-<>]|$" },
      { "ugly": "-~", "pretty": " \ue118", "pre": "[^~=\\-<>]|^", "post": "[^~=\\-<>]|$" },
      { "ugly": "~@", "pretty": " \ue164", "pre": "[^~=\\-@~]|^", "post": "[^~=\\-@~]|$" },
      { "ugly": "\\^=", "pretty": " \ue136", "pre": "[^~=\\-^]|^", "post": "[^~=\\-^]|$" },
      { "ugly": "\\?=", "pretty": " \ue127", "pre": "[^~=\\-/?]|^", "post": "[^~=\\-/?]|$" },
      { "ugly": "/=", "pretty": " \ue12c", "pre": "[^~=\\-/]|^", "post": "[^~=\\-/]|$" },
      { "ugly": "/==", "pretty": "\u00a0  \ue12d", "pre": "[^~=\\-/]|^", "post": "[^~=\\-/]|$" },
      { "ugly": "\\|=", "pretty": " \ue134", "pre": "[^~=\\-/|]|^", "post": "[^~=\\-/|]|$" },
      { "ugly": "\\|{2}=", "pretty": "\u00a0  \ue133", "pre": "[^~=\\-/|]|^", "post": "[^~=\\-/|]|$" },
      { "ugly": "##", "pretty": " \ue11b", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "###", "pretty": "\u00a0  \ue11c", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "####", "pretty": "\u00a0\u00a0 \ue11d", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#{", "pretty": " \ue119", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#\\[", "pretty": " \ue11a", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#\\(", "pretty": " \ue11e", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#\\?", "pretty": " \ue11f", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#_", "pretty": " \ue120", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" },
      { "ugly": "#_\\(", "pretty": "\u00a0  \ue121", "pre": "[^#_?({[\\]]|^", "post": "[^#_?({[\\]]|$" }
    ]
  }
]

@ghost
Copy link

ghost commented Aug 11, 2016

@siegebell There's a bug and i'm not sure whether it is fira code or emacs.

So with font set to Fira Code and setting the fontset to use fira code symbol for \ue100 to \ue16f I'm getting some weird behavior.

The symbols appear to the left of the original text a significant amount. So, for instance, if I do <-- as the first text on the line i'll get most of it rendering off the left side of the screen. Including emacs config and screenshot here, not yet sure what the problem is.

Edit: If there is other text to the left this bug will actually cause the symbol to overlap with the text!

Edit2: Loading it with no other emacs config has the same effect so shouldn't be some package interfering. In addition, it looks like it is about two spaces/characters to the left of where it should be.

Edit3: I see now that you are dealing with this via \u00a0 in the vscode config? Not sure if I can do that with emacs or not.

Emacs config:

;;; Prettify symbols
(setq prettify-symbols-unprettify-at-point t)

(set-fontset-font t '(#Xe100 . #Xe16f) "Fira Code Symbol")

(defconst fira-code-font-lock-symbols-alist
  (mapcar (lambda (s)
            (cons (car s) (decode-char 'ucs (car (cdr s)))))
          (list '("www" #Xe100)
                '("<--" #Xe153))))

(add-hook 'prog-mode-hook
          (lambda ()
            (dolist (alias fira-code-font-lock-symbols-alist)
              (push alias prettify-symbols-alist))
            (prettify-symbols-mode)))

fira-code-emacs-bug

@ghost
Copy link

ghost commented Aug 11, 2016

@siegebell So this definitely looks like it is because emacs expects to be replacing multiple characters with a character with a width of one where as the fira code symbols are designed to take up multiple character widths. Haven't yet figured out a solution.

@ghost
Copy link

ghost commented Aug 11, 2016

Found an uglier way that works.

(add-hook 'prog-mode-hook
          (lambda ()
            (font-lock-add-keywords nil
                                    `(("\\(###\\)"
                                       (0 (prog1 ()
                                            (compose-region (match-beginning 1)
                                                            (match-end 1)
                                                            ;; The first argument to concat is a string containing a literal tab character inserted via C-q <tab>. It is important!
                                                            ,(concat "  " (list (decode-char 'ucs #Xe11c)))))))))))

I'm going to turn it into a helper function then I just need to figure out emacs's awful regex syntax for each thing and we should be good.

@tonsky
Copy link
Owner

tonsky commented Aug 11, 2016

@mordocai FYI in Fira symbols visually take multiple character places but marked as taking just one (rightmost) single place:

screen shot 2016-08-11 at 13 14 42

So in fact when I do ligature substitution in font code, I replace e.g. hyphen hyphen greater with space space hyphen_hyphen_greater.liga (CR stands for space):

    sub CR CR greater' by hyphen_hyphen_greater.liga;
    sub CR hyphen' greater  by CR;
    sub hyphen' hyphen  greater  by CR;

The reason for that is that editors will still see result of substitution as three single-width characters, and will allow to “step inside” the ligature glyph. Without that, ligature will act as it’s a single character, which changes editing behaviour which we want to avoid. We want to keep feeling that substitution is purely visual and underlying code is there unchanged.

The downside of that is that glyphs span outside their container. This is usually perfectly fine with all known editors, but might surprise anyone who tries to use them directly without knowing about that gotcha.

Hope that helps.

@ghost
Copy link

ghost commented Aug 11, 2016

@tonsky Yeah, I ended up figuring out what was going on! Luckily the new way I'm doing it with emacs knows how to handle things like that by using a special string beginning with a tab.

Everyone: I got it working! Here is the emacs code:

https://gist.github.com/mordocai/50783defab3c3d1650e068b4d1c91495

Linking to gist so I can update it later if necessary.

Edit: Screenshot. Also, sorry for spamming this issue so bad. I kept thinking I was going to give up for the night, but then didn't...
fira-code-emacs-working

@emmanueltouzery
Copy link

emmanueltouzery commented Aug 13, 2016

@mordocai thanks for the work! I tried the gist but I get mostly garbage. See the picture. Maybe I'm using the wrong version of the font or something? I'm pretty sure I'm using 0.200 although I did install older versions in the past.

EDIT OK SORRY IT WORKS. Need to install the Fira code symbol font additionnally. The "x" does look weird, in "Text" for instance. It's replaced by a cross. Also the ligature for =<< doesn't work while fira code appears to support it.

@emmanueltouzery
Copy link

emmanueltouzery commented Aug 13, 2016

to fix the =<< in the gist =>

          ("\\(=<<\\)"                   #Xe142)    (changed from ==<)
          ("[^-=]\\(<<\\)"               #Xe15c)    (excluded the = before the << as well)

this is for the overzealous cross:

          ;; ("\\(x\\)"                     #Xe16b)

would need to require non-word characters before & after but my emacs regex-fu is weak for that, so I just commented that locally.

@siegebell
Copy link

siegebell commented Aug 13, 2016

For example, this should only match 'x' if the preceding character is a number and the subsequent character is either a number or letter:

          ("[0-9]\\(x\\)[0-9a-fA-F]"                     #Xe16b)

edit: changed a-F to a-f

@emmanueltouzery
Copy link

@siegebell i think the ideal would be a "word boundary" regex before & after the x. I was a bit lazy earlier, but this seems to work great =>

          ("\\b\\(x\\)\\b"               #Xe16b)

emmanueltouzery added a commit to emmanueltouzery/dotspacemacs that referenced this issue Aug 13, 2016
@emmanueltouzery
Copy link

hmm actually the 'x' just bothers me :-)
in haskell we often use the 'x' variable (x:xs) and that triggers it. I'll keep that line commented for my use.

@ghost
Copy link

ghost commented Aug 13, 2016

Updated my gist with @emmanueltouzery's changes. I also had ended up commenting out the the X so I included that too.

@siegebell
Copy link

@emmanueltouzery that's why I didn't use \b :)

Anyhow, the advantage of prettify-symbols-mode is that every user can easily pick & choose their favorite substitutions. And fancy 0xFF notation is useful in only a few kinds of projects.

@emmanueltouzery
Copy link

@siegebell ohhh... that 'x' was for hex numbers!!! I totally didn't get it. I thought some languages use it for multiplication or something like that, so I didn't undersand your regex. I thought you were trying to be helpful by showing some generic example of regex =D
Then maybe your regex should be used instead.

@emmanueltouzery
Copy link

enabled @siegebell 's 'x' regex for me locally. There is a little typo, @siegebell put a-FA-F instead of a-fA-F btw (so it only worked for uppercase hex digits).

otherwise I agree this mode is great. You can enable per-mode and per-glyph. It's just great. And also you can combine the symbols with other fonts. I'm actually using these new symbols in combination with the usual fira mono, NOT with the base fira code!

Probably we should update the FiraCode wiki with these instructions? The question is whether this way works also on OSX? Do we have two sets of instructions, OSX & linux or we just put these new instructions for all emacs versions? This way requires to install a separate font which @tonsky may not be willing to support in the future. On the other hand, the OSX-specific way seems to cause all kinds of hangs in all kinds of situations...

@siegebell
Copy link

@emmanueltouzery there's no particular reason why this needs to be a separate symbols font. I created it this way for expediency/convenience because the free font editor I have access to, Font Forge, is a buggy UI-nightmare to work with. Creating a separate [symbols] font is best suited for when you want to add glyphs to a non-free font that you cannot redistribute, but this approach is limited to text editors that support font fallback.

I think it would be best for @tonsky to merge these changes back into Fira Code to avoid needless forking. (I would submit a PR if only I could run the trial of Glyphs App on Windows...)

@emmanueltouzery
Copy link

some feedback after using @siegebell & @mordocai 's solution daily for a while: it works great, two little things.

  1. The regex for substitution should probably blacklist symbols which can repeat more times than the symbol covers. For instance ";;;" doesn't look good, we get ";; ;". Ideally the regex for substitution should be something like ";;[^;]" or something like that. Same for some others, like "##", "//", "!!!" and probably more. I am currently too lazy to deal with it :-p
  2. that's a tougher one: company completion renders completion combos as ascii art more or less. Ligatures/glyph substitution breaks expectations that company makes and the completion combos end up looking ugly (see screenshot). I guess there's no easy fix for that.

screenshot from 2016-09-04 11-42-52

I'm still keeping it

@davidar
Copy link

davidar commented Sep 5, 2016

@emmanueltouzery It looks like there might be a problem with the widths of the ligatures in your screenshot (they should be an integer multiple of the normal character width)

@siegebell
Copy link

siegebell commented Sep 5, 2016

@emmanueltouzery most of the ugliness in the regex I posted was essentially "blacklisting"; applying a substitution only when it is clearly not part of a larger symbol [which may have no ligature]. Converting it to emacs regex should look something like ("${pre}\\(${ugly}\\)${post}" ${unicodeSymbol}). Most of it has been left out of @mordocai's translation to emacs, probably because it was very tedious and [I suppose] he chose to only keep "blacklisting" for the more likely collisions in found in Haskell programming.

@ghost
Copy link

ghost commented Sep 6, 2016

@siegebell Actually ruby/javascript/lisp programming but otherwise correct :). Since emacs regexes are different it was a pain to copy the originals and modify them so instead I just made new ones from scratch and I didn't check for all such "blacklisting". It hasn't bothered me too much, so far, so I haven't fixed it.

As far as the widths, i've noticed using the "tab as first character to compose-region string argument" method in emacs, described in docs as

If it is a string, the elements are alternate characters. In
this case, TAB element has a special meaning. If the first
character is TAB, the glyphs are displayed with left padding space
so that no pixel overlaps with the previous column. If the last
character is TAB, the glyphs are displayed with right padding
space so that no pixel overlaps with the following column.

Doesn't seem to line up the columns correctly, which may also be the issue with company. Currently its doing the tab as the first character(so left padding space), i haven't tried tab as last character (so right padding space). If right padding space is just as bad/doesn't work(my guess is it won't work), may be best to try and use the unicode spaces like @siegebell did in the other examples. The function (and configuration) will need to take an extra argument for number of spaces to add though, if that is the case.

@dominikh
Copy link

@mordocai using TAB doesn't seem to be a perfect solution, either. TAB makes sure that the replacement doesn't overlap with the previous column, but it still gets spacing wrong for the || ligature. Doing x||y will show it noticeably closer to x than to y, as opposed to the centered look it should have.

@clembu
Copy link

clembu commented Mar 19, 2017

Okay folks. This is all great. I thank you very much for your work.

But I'm now wondering, would you go about updating @siegebell 's font with the new ligatures? It'd be awesome if this font could be updated with Fira Code.

@TheAntimist
Copy link

Hey, out of curiosity, what's stopping the ligature glyphs from being moved to the private area? Is there any blocking issue?

@tonsky Has there been any update to this request? It'd be great if we could get this for the latest release of FiraCode.

@tonsky
Copy link
Owner

tonsky commented Nov 4, 2019

I’ll plan it for the next update

@tonsky tonsky added this to the 3 milestone Nov 4, 2019
@TheAntimist
Copy link

@tonsky Thanks, glad to hear that. I found out only recently, I believe that emacs27 is adding support for HarfBuzz, and also possibly proper ligature support too. [1] Not sure if this change should be made, after finding that out.

[1] https://www.reddit.com/r/emacs/comments/byddvm/emacsdevel_harfbuzz_is_now_available_on_master/eqi90r8/?context=1

@clembu
Copy link

clembu commented Nov 4, 2019

emacs27 is still a long way though it feels

@wedens
Copy link

wedens commented Nov 4, 2019

And ligatures support is unlikely to be implemented until emacs28 https://www.reddit.com/r/emacs/comments/dcryg1/tab_support_landed_in_emacs_master/f2kevco/

@DogLooksGood
Copy link

Just given a try, almost everything looks great, but the some symbol like .- will only work when it is the whole word, it won't work in something like .-a.

@clembu
Copy link

clembu commented Nov 26, 2019

thankfully!

@xieve
Copy link

xieve commented Dec 1, 2019

@rohit507

How can we generate the fira-symbol font from the current version of fira-code?

I updated the Gist by @reiver-dev to work with the latest version. All you'll have to do is patch the font installed in the system and enable the fira-code-mode provided by the elisp sniippet. Using Arch, it's

# ./fira_code_patch.py -o /usr/share/fonts/OTF /usr/share/fonts/OTF/Fira-*.otf

@AllanZyne
Copy link

AllanZyne commented Dec 12, 2019

@rohit507

How can we generate the fira-symbol font from the current version of fira-code?

I updated the Gist by @reiver-dev to work with the latest version. All you'll have to do is patch the font installed in the system and enable the fira-code-mode provided by the elisp sniippet. Using Arch, it's

# ./fira_code_patch.py -o /usr/share/fonts/OTF /usr/share/fonts/OTF/Fira-*.otf

I have tested this command and it doesn't work.
After I read script code, I guess you can't override write font while it's opening.
I use this command and it works:

./fira_code_patch.py -o ~/.local/share/fonts/ /usr/share/fonts/OTF/FiraCode*.otf

@xieve
Copy link

xieve commented Dec 12, 2019 via email

@AllanZyne
Copy link

AllanZyne commented Dec 12, 2019

The invocation of the script itself may be different across different installations. I only provided mine as an example of usage. Your suggestion will only work for making a local font to override the system one, but not all programs even look in the user-specific fonts folder. Also, note how your wildcard differs from mine: Yours will match FiraCode-Regular.otf and FiraCode-Bold.otf, while mine has an extra hyphen in it, so it'll match Fira-Code-Regular.otf and the likes. As to the "error" that you found: that is simply not the case, at least on my system. I overrode my fonts without a problem, fontforge doesn't seem to lock a file opened for reading. It might be interesting to submit the error message you got (if you got one) as a comment of my Gist, so that we don't spam this issue. lazy notifications@github.com schrieb am Do., 12. Dez. 2019, 05:38:

@rohit507 https://github.com/rohit507 How can we generate the fira-symbol font from the current version of fira-code? I updated the Gist https://gist.github.com/xieve/d5a01cc59896c3973cb16df9ba8d30d4 by @reiver-dev https://github.com/reiver-dev to work with the latest version. All you'll have to do is patch the font installed in the system and enable the fira-code-mode provided by the elisp sniippet. Using Arch, it's # ./fira_code_patch.py -o /usr/share/fonts/OTF /usr/share/fonts/OTF/Fira-.otf I have tested this command and it doesn't work. After I read script code, I guess you can't override write font while it's opening. I suggest ./fira_code_patch.py -o ~/.local/share/fonts/ /usr/share/fonts/OTF/FiraCode.otf — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#211?email_source=notifications&email_token=AE2JI5NBLVWWIARQJQKCZZTQYG53VA5CNFSM4CJTMHP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGVN4NQ#issuecomment-564846134>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE2JI5ME275UOL2SKFR7WWLQYG53VANCNFSM4CJTMHPQ .

Oh, now maybe this is my system font cache issue.

I changed wildcard because I didn't know "Fira-Code-*.otf" this naming style. In fact I can't match anything by "Fira-*" because my font files are "FiraCode-*.otf".

Anyway, thanks for your script and it really works!

@tonsky tonsky added the Unicode label Mar 9, 2020
Losangelosgenetics pushed a commit to Losangelosgenetics/FiraCode that referenced this issue Mar 12, 2020
200: Update rake requirement from ~> 10.0 to ~> 12.3 r=bronzdoc a=dependabot[bot]

Updates the requirements on [rake](https://github.com/ruby/rake) to permit the latest version.
<details>
<summary>Changelog</summary>

*Sourced from [rake's changelog](https://github.com/ruby/rake/blob/master/History.rdoc).*

> === 12.3.1
> 
> ==== Bug fixes
> 
> * Support did_you_mean >= v1.2.0 which has a breaking change on formatters.
>   Pull request [tonsky#262](https://github-redirect.dependabot.com/ruby/rake/issues/262) by FUJI Goro.
> 
> ==== Enhancements:
> 
> * Don't run task if it depends on already invoked but failed task.
>   Pull request [tonsky#252](https://github-redirect.dependabot.com/ruby/rake/issues/252) by Gonzalo Rodriguez.
> * Make space trimming consistent for all task arguments.
>   Pull request [tonsky#259](https://github-redirect.dependabot.com/ruby/rake/issues/259) by Gonzalo Rodriguez.
> * Removes duplicated inclusion of Rake::DSL in tests.
>   Pull request [tonsky#254](https://github-redirect.dependabot.com/ruby/rake/issues/254) by Gonzalo Rodriguez.
> * Re-raise a LoadError that didn't come from require in the test loader.
>   Pull request [tonsky#250](https://github-redirect.dependabot.com/ruby/rake/issues/250) by Dylan Thacker-Smith.
> 
> === 12.3.0
> 
> ==== Compatibility Changes
> 
> * Bump `required_ruby_version` to Ruby 2.0.0. Rake has already
>   removed support for Ruby 1.9.x.
> 
> ==== Enhancements:
> 
> * Support `test-bundled-gems` task on ruby core.
> 
> === 12.2.1
> 
> ==== Bug fixes
> 
> * Fixed to break Capistrano::Application on capistrano3.
> 
> === 12.2.0
> 
> ==== Enhancements:
> 
> * Make rake easier to use as a library
>   Pull request [tonsky#211](https://github-redirect.dependabot.com/ruby/rake/issues/211) by [**drbrain**](https://github.com/drbrain)
> * Fix quadratic performance in FileTask#out_of_date?
>   Pull request [tonsky#224](https://github-redirect.dependabot.com/ruby/rake/issues/224) by [**doudou**](https://github.com/doudou)
> * Clarify output when printing nested exception traces
>   Pull request [tonsky#232](https://github-redirect.dependabot.com/ruby/rake/issues/232) by [**urbanautomaton**](https://github.com/urbanautomaton)
> 
> ==== Bug fixes
> 
> * Account for a file that match 2 or more patterns.
>   Pull request [tonsky#231](https://github-redirect.dependabot.com/ruby/rake/issues/231) by [**styd**](https://github.com/styd)
></table> ... (truncated)
</details>
<details>
<summary>Commits</summary>

- See full diff in [compare view](https://github.com/ruby/rake/commits/v12.3.1)
</details>
<br />

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

---

**Note:** This repo was added to Dependabot recently, so you'll receive a maximum of 5 PRs for your first few update runs. Once an update run creates fewer than 5 PRs we'll remove that limit.

You can always request more updates by clicking `Bump now` in your [Dependabot dashboard](https://app.dependabot.com).

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)

Finally, you can contact us by mentioning @dependabot.

</details>

Co-authored-by: dependabot[bot] <support@dependabot.com>
Losangelosgenetics pushed a commit to Losangelosgenetics/FiraCode that referenced this issue Mar 12, 2020
211: RSpec: config.disable_monkey_patching! r=bronzdoc a=olleolleolle

This PR configures RSpec to avoid some now-unnecessary metaprogramming.

See http://rspec.info/documentation/3.8/rspec-core/RSpec/Core/Configuration.html#disable_monkey_patching!-instance_method

> Enables zero monkey patching mode for RSpec. It removes monkey patching of the top-level DSL methods (describe, shared_examples_for, etc) onto main and Module, instead requiring you to prefix these methods with RSpec.. It enables expect-only syntax for rspec-mocks and rspec-expectations. It simply disables monkey patching on whatever pieces of RSpec the user is using.





Co-authored-by: Olle Jonsson <olle.jonsson@gmail.com>
@tonsky tonsky modified the milestones: 3, 4 Apr 9, 2020
@tonsky tonsky modified the milestones: 4, 5 May 16, 2020
@tonsky tonsky modified the milestones: 5, 6 Jun 9, 2020
@MonsieurPi
Copy link

@xieve I tried your solution but I didn't manage to make it work on my configuration. Here are all the infos I can provide in case you have an answer to my problem:

  • Ubuntu 20.04 LTS
  • emacs 26.3

The fonts are properly installed (I hope)

❯ fc-list | grep -i "fira"
/usr/local/share/fonts/f/FiraCode_Regular_Symbol.otf: Fira Code Symbol:style=Symbol-Regular
/usr/share/fonts/opentype/firacode/FiraCode-Retina.otf: Fira Code,Fira Code Retina:style=Retina,Regular
/usr/share/fonts/opentype/firacode/FiraCode-Bold.otf: Fira Code:style=Bold
/usr/share/fonts/opentype/firacode/FiraCode-Regular.otf: Fira Code:style=Regular
/usr/share/fonts/opentype/firacode/FiraCode-Medium.otf: Fira Code,Fira Code Medium:style=Medium,Regular
/usr/share/fonts/opentype/firacode/FiraCode-Light.otf: Fira Code,Fira Code Light:style=Light,Regular

ligatures.el is in .emacs.d/custom/

In my .emacs I have the following lines:

(custom-set-faces
 '(default ((t (:family "DejaVu Sans Mono" :foundry "unknown" :slant normal :weight normal :height 113 :width normal :inverse-video nil :box nil :strike-through nil :overline nil :underline nil)))))

(use-package fira-code-mode
  :load-path "~/.emacs.d/custom/"
  :hook prog-mode)

I tried applying your script only to Fira Code Symbol:

❯ sudo ./fira_code_patch.py -o /usr/local/share/fonts/f/ /usr/local/share/fonts/f/Fira*.otf

And to the rest of the FiraCode fonts:

❯ sudo ./fira_code_patch.py -o /usr/share/fonts/opentype/firacode/ /usr/share/fonts/opentype/firacode/Fira*.otf
The glyph named triangleright is mapped to U+25BA.
But its name indicates it should be mapped to U+22B2.
The glyph named triangleright is mapped to U+25BA.
But its name indicates it should be mapped to U+22B2.
The glyph named triangleright is mapped to U+25BA.
But its name indicates it should be mapped to U+22B2.
The glyph named triangleright is mapped to U+25BA.
But its name indicates it should be mapped to U+22B2.
The glyph named triangleright is mapped to U+25BA.
But its name indicates it should be mapped to U+22B2.

But it's still all messed up (example below showing ==)

image

Thing is, I don't even know if it's using the fonts since when I delete them and reload the font cache (fc-cache -vs) I still have the same display so it looks like I'm missing something.

As a sidenote, would it be easier if I just installed emacs 27 since it looks like it should be able to use ligatures? And if so, how am I supposed to use it? Set FiraCode as the default font?

@xieve
Copy link

xieve commented Jul 16, 2020

@MonsieurPi Have you restarted Emacs since reinstalling them? If so, please confirm that your fonts are actually patched by looking at them using FontForge and going to character U+E100 (Ctrl+Shift+>, U+E100). If it contains the www ligature, your font is patched. Next, please confirm that you are actually using Fira Code as your main font, or at least for the range U+E100 through U+E187 (although I doubt this would look visually pleasing). The two digits visible on your screenshot don't seem to be in Fira Code, if I'm not mistaken. There also was a function that would display all characters (and, if available, corresponding glyphs) in a certain range for easier debugging, but I can't find it anymore. Maybe someone else can help out.

In other news, I updated my snippet to include the changes that have been made since I wrote the ligature list, it should now be working again, even with newly patched and updated versions.

Edit: About Emacs 27, I genuinely don't know. If you can get it working, throw me a hint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests