Ignores candidates that do not match prefix

[oscarfv]

I'm using flx-rs as the sorting backend.

For some reason, only candidates that have a prefix matching the entered string are shown. This is with the etags backend.

This means that if I have foo-bar in TAGS and write fba, foo-bar is suggested, but not when I write bar.

My config so far is:

(require 'company)
(require 'company-fuzzy)
(setq company-fuzzy-sorting-backend 'flx-rs)
(setq company-fuzzy-prefix-on-top nil)

(defun my-mode-hook ()
  (setq-local completion-styles '(substring)) ;; Try to not discard candidates.
  (setq-local company-backends '(company-etags))
  (company-mode 1)
  (company-fuzzy-mode 1)
)

Any suggestion?

[jcs090218]

That's because of the search engine. fbr and bar score 0 because there is no f in there. The way how the fuzzy matching works (at least in flx) is to match all characters if possible. The one you require doesn't match any fuzzy matching algorithms I know (sublime-fuzzy, flx, flex, orderless, flxy, fuz, etc.).

If something pops out like the word bar, then it can be powered by some semantic engine in the backend (like language server protocol).

[oscarfv]

I'm a flx user for many years now, mostly with ido, so I'm quite familiarized with how flx works.

foo-bar should be a match under flx for both fbr and bar. Specifically, fbr is a match because [f]oo-[b]a[r] (matched chars marked with []) and bar because foo-[b][a][r].

On a Elisp file, except with company-capf instead of company-etags, find-file-at-point is a match for at-point, so there is no requirement for a prefix in that case. Dunno what's special about company-etags (company-ctags suffers from the same.)

[jcs090218]

What does it print in (message "%s" (flx-score "bar" "fbr"))? 🤔 I got nil.

[oscarfv]

Never mind. company-etags--candidates was filtering out those candidates that don't start with the given prefix parameter (prefix contains the first char of the string to be completed). Here, the correct thing when using flx is to return all candidates, unfiltered. This simple change makes the trick:

+++ #<buffer company-etags.el>
@@ -82,7 +82,7 @@
          (fboundp 'tags-completion-table)
          (save-excursion
            (visit-tags-table-buffer)
-           (all-completions prefix (tags-completion-table))))))
+           (tags-completion-table)))))
 
 ;;;###autoload
 (defun company-etags (command &optional arg &rest ignored)

Of course, this change is not correct for all company-fuzzy-sorting-backends. Maybe company-fuzzy shouldn't pass a prefix to the company-backend when it is using a sorting backend that does its own filtering?

BTW, the candidate I was using as an example is foo-bar, not bar as you put on your previous question. With that change:

(message "%s" (flx-score "foo-bar" "fbr"))
-> (157 0 4 6)
(message "%s" (flx-score "foo-bar" "bar"))
-> (203 4 5 6)

Thank you for company-fuzzy, just discovered this package and seems that I will be using it a lot from now on 😃

[jcs090218]

Here, the correct thing when using flx is to return all candidates unfiltered.

By default, company-fuzzy will try to get all candidates (as many as possible), so the scoring engine can filter.

Of course, this change is not correct for all company-fuzzy-sorting-backends. Maybe company-fuzzy shouldn't pass a prefix to the company-backend when it is using a sorting backend that does its own filtering?

I don't remember the details, but if I recall correctly, “most” backends need a prefix to get the list of candidates.

BTW, the candidate I was using as an example is foo-bar, not bar, as you put on your previous question.

Oops, did I misread it? Sorry about that... 😓

Thank you for company-fuzzy, just discovered this package and seems that I will be using it a lot from now on 😃

I'm glad you like it! ❤️

[oscarfv]

Here, the correct thing when using flx is to return all candidates unfiltered.

By default, company-fuzzy will try to get all candidates (as many as possible), so the scoring engine can filter.

Of course, this change is not correct for all company-fuzzy-sorting-backends. Maybe company-fuzzy shouldn't pass a prefix to the company-backend when it is using a sorting backend that does its own filtering?

I don't remember the details, but if I recall correctly, “most” backends need a prefix to get the list of candidates.

On this case, if the prefix is just the empty string, all candidates are returned, otherwise the prefix parameter is used as a required prefix for the returned candidates.

If I understand correctly, that prefix which is passed to company-etags--candidates comes from company-fuzzy, right? When using flx and related backends, this is incorrect as otherwise valid candidates are filtered out by the company backend, so could company-fuzzy just pass an empty string as prefix when using flx?

I must admit that I know nothing about the requirements of company backends, so it is possible that using the empty string as prefix may break some company backend...

[jcs090218]

I must admit that I know nothing about the requirements of company backends, so it is possible that using the empty string as prefix may break some company backend...

No worries. All backends are different, so there is no right or wrong answer. Here is how company-fuzzy deal with specific backends:

company-fuzzy/company-fuzzy.el

Lines 500 to 576 in cb0628a

	(defun company-fuzzy--backend-prefix-filter (backend)
	"Return prefix for each BACKEND while doing the first basic filerting.
	This is some what the opposite to function `company-fuzzy--backend-prefix-get'
	since it's try get as much candidates as possible, but this function returns
	a prefix that can filter out some obvious impossible candidates."
	(cl-case backend
	(`company-files (company-fuzzy--valid-prefix backend))
	(`company-paths (company-fuzzy--backend-prefix 'company-files 'match))
	(t (company-fuzzy--backend-prefix backend 'match))))
	(defun company-fuzzy--backend-prefix-match (backend)
	"Return prefix for each BACKEND while matching candidates.
	This function is use for scoring and matching algorithm. It returns a prefix
	that best describe the current possible candidate.
	For instance, if there is a candidate function `buffer-file-name' and with
	current prefix `bfn'. It will just return `bfn' because the current prefix
	does best describe the for this candidate."
	(cl-case backend
	((company-capf) (company-fuzzy--valid-prefix backend))
	(`company-c-headers
	(when-let ((prefix (ht-get company-fuzzy--prefixes backend)))
	;; Skip the first < or " symbol
	(substring prefix 1 (length prefix))))
	(`company-files
	;; NOTE: For `company-files', we will return the last section of the path
	;; for the best match.
	;;
	;; Example, if I have path `/path/to/dir'; then it shall return `dir'.
	(when-let* ((prefix (ht-get company-fuzzy--prefixes backend))
	(splitted (split-string prefix "/" t))
	(len-splitted (length splitted))
	(last (nth (1- len-splitted) splitted)))
	last))
	(`company-paths
	(when-let ((prefix (ht-get company-fuzzy--prefixes backend)))
	(if (string-suffix-p "/" prefix) 'anything
	(nth 0 (last (split-string prefix "/" t))))))
	(t company-fuzzy--prefix)))
	(defun company-fuzzy--backend-prefix-get (backend)
	"Return prefix for each BACKEND while getting candidates.
	This function is use for simplify prefix, in order to get as much candidates
	as possible for fuzzy work.
	For instance, if I have prefix `bfn'; then most BACKEND will not return
	function `buffer-file-name' as candidate. But with this function will use a
	letter `b' instead of full prefix `bfn'. So the BACKEND will return something
	that may be relavent to the first character `b'.
	P.S. Not all backend work this way."
	(cl-case backend
	(`company-c-headers
	;; Skip the < or " symbol for the first character
	(ignore-errors (substring (ht-get company-fuzzy--prefixes backend) 1 2)))
	(`company-files
	(when-let ((prefix (ht-get company-fuzzy--prefixes backend)))
	(let* ((splitted (split-string prefix "/" t))
	(len-splitted (length splitted))
	(last (nth (1- len-splitted) splitted))
	(new-prefix prefix))
	(when (< 1 len-splitted)
	(setq new-prefix
	(substring prefix 0 (- (length prefix) (length last)))))
	new-prefix)))
	(`company-paths
	(when-let ((prefix (ht-get company-fuzzy--prefixes backend)))
	(if (string-suffix-p "/" prefix) prefix
	(file-name-directory prefix))))
	(t
	;; Return an empty string or first character is likely going to return a
	;; full list of candaidates. And this is what we want.
	(when (ht-get company-fuzzy--prefixes backend)
	company-fuzzy--prefix-first))))

company allows every backends have it's "own" rule, so there is no "absolute" way to get all the candidates! There are many backends that has similar rule (company doesn't have good documentation about creating backend, so many user just copy & paste from other existing packages). That's why I have created an issue in #12.

Hope the information helps! ;)

[oscarfv]

Thanks.

This is a case of conflicting responsibilities: company-fuzzy supports methods that expect to do the filtering (i.e. flx) while company backends are supposed to do the filtering.

We can convince some company backends to effectively refrain from filtering. With company-ctags we can set company-ctags-fuzzy-match-p, with company-capf we can set completion-styles to substring. Then both will not discard relevant candidates with current implementation of company-fuzzy, which passes the first letter. However, other backends (company-etags) uses that letter as a prefix, which discards relevant candidates.

company-fuzzy could pass an empty string to those backends that are known to not break on that case, this way the user does not need to figure out how to configure Emacs to see the candidates he expects and benefit from the backends that do not support configuration.