Configure group based sorting in agenda

[gerazov]

Does this feature exist in Emacs orgmode core?

N/A

Orgmode link

No response

Feature value

The sorting of tasks in the agenda view seems a bit arbitrary - I get a mix of NEXT and TODO items, and the files they come from are also mixed even if all the tasks are for the same date. I couldn't find how sorting is done in the help file...

For e.g. I get:

понеделник 01 ноември 2021
  work:       Scheduled:  NEXT task name
  nehm:       Scheduled:  NEXT task name
  nehm:       Scheduled:  NEXT task name
  pros:       Scheduled:  NEXT task name
  work:       Scheduled:  NEXT task name
  nehm:       Scheduled:  NEXT task name
  nehm:       Scheduled:  TODO task name
  evoc:       Scheduled:  TODO task name
  nehm:       Scheduled:  NEXT task name

It would be very helpful if there is a way to group items based on keywords and files, and possibly set different priorities for sorting them in the agenda, e.g. DONE > DOING > NEXT > TODO 👍

Additional context

No response

[kristijanhusak]

Currently this logic is used: https://orgmode.org/manual/Sorting-of-agenda-items.html

But it seems this should be configurable with this: https://orgmode.org/worg/doc.html#org-agenda-sorting-strategy. It seems that there is no a specific setting to sort by the todo keyword. Both TODO and NEXT keywords are basically a TODO state, and orgmode allows sorting by TODO -> DONE or vice versa.

[gerazov]

Ok so it says:

For the daily/weekly agenda, the items for each day are sorted. The default order is to first collect all items containing an explicit time-of-day specification. These entries are shown at the beginning of the list, as a schedule for the day. After that, items remain grouped in categories, in the sequence given by org-agenda-files. Within each category, items are sorted by priority (see Priorities), which is composed of the base priority (2000 for priority ‘A’, 1000 for ‘B’, and 0 for ‘C’), plus additional increments for overdue scheduled or deadline items.

In orgmode.nvim the specific time todos are listed after the todos without a time of day, this is less useful, i.e. the emacs way sounds better.

And the items are not grouped in categories in my case, even if I pass an explicit list of files rather than `org/*' they don't get sorted - I get the same mix as before.

👍 for sorting them by todo state too even if not emacs style 😉

[levouh]

Something like this (code here and from an article here) would be something that users would have to add manually, right? Based on the linked code, it appears that jethrokuan has items sorted based on their TODO state in separate files, then just has a title and list if items that come together to make up a given view.

This looks like it would be something like this, which would accommodate for not just this use-case but realistically any "custom agenda view" use case.

Not 100% sure of the state right now, but would something like this require changes to the core code (this repo.) @kristijanhusak?

[kristijanhusak]

@levouh If we would add some similar configuration, then yeah, it would require some changes, and probably not so small ones. I'm not even sure what's possible with the custom agenda command, I would have to investigate. There is like an AgendaItem that basically wraps all logic around determining what and where to show, so that can be reused, but rendering itself would need changes.

[levouh]

Makes sense, I'll see if I can't come up with a good summary before taking a stab at what the code might actually look like. Can open up a separate issue to track it as well, whichever you'd prefer.

The main start on the issue would be:

1. Summary of how the functionality works in Emacs
2. What is applicable change-wise here, and what changes should be a part of the core

[kristijanhusak]

Makes sense, I'll see if I can't come up with a good summary before taking a stab at what the code might actually look like. Can open up a separate issue to track it as well, whichever you'd prefer.

The main start on the issue would be:

1. Summary of how the functionality works in Emacs
2. What is applicable change-wise here, and what changes should be a part of the core

Awesome, thanks! That will help tremendously. You can open up a separate enhancement issue, since this is related only to sorting.

[gerazov]

Ok, after some playing around with the sorting in agenda/init.lua I've finally fixed this 😅

The problem was that the all the tasks were sorted based on their timestamp. This meant that date_only tasks were randomly sorted and all of them went before the tasks scheduled with times of day (I guess os.time() was defaulting to 00:00 AM).

Now tasks with time of day are only sorted and they come first in the agenda view, followed by the tasks set without.

Before (notice how the two tasks from ea without time of day get separated):

After:

One thing that I still want to improve is for the tasks to follow the order of the files as given to orgmode in the config. This would allow users to e.g. put their personal task org last so work tasks comes before or vice versa. To this end I've commented out the file sorting in parser/files.lua in Files.all(), but still the order is not the same as in the config - I guess it's the async reading of the files (every reload gives a different order in the agenda):

    vim.schedule_wrap(function(err, content)
      if err then
        return callback(nil)
      end
      return callback(File.from_content(content, category, path, ext == 'org_archive'))
    end)

[gerazov]

Ok the ordering is now fixed 🤟 and the files are still read asyncly.

Here's what I get for the same date (assuming work > ea > valence)

[kristijanhusak]

@gerazov can you double check how Emacs orgmode is doing the sorting? I think I adjusted it to work the same way as there, but maybe I missed something.
If it's the same as there, lets go with some solution to allow configuring the sorting as a user, and leave this one as default. As a first (temporary) step we could have a sorting function exposed in the configuration, which you can override to your needs. Once #135 is done, this function would be removed or changed to adapt the new functionality.

[gerazov]

Yeah it's like that in the specs as far as I can tell (and it feels right too 😅 )

For the daily/weekly agenda, the items for each day are sorted. The default order is to first collect all items containing an explicit time-of-day specification. These entries are shown at the beginning of the list, as a schedule for the day. After that, items remain grouped in categories, in the sequence given by org-agenda-files. Within each category, items are sorted by priority (see Priorities), which is composed of the base priority (2000 for priority ‘A’, 1000 for ‘B’, and 0 for ‘C’), plus additional increments for overdue scheduled or deadline items.

Plus the randomization is minimized to within category listings, which is a bit weird since they are read from the same file 🤔

I haven't added the priorities though for the no time-of-day entries, will do that and update. I'm not sure about the

additional increments for overdue of scheduled or deadline items.

It reads to me that first comes the overdue deadline, then the overdue scheduled item, then the priority items, then the regular items, no?

I can also implement this for the todo-s list.

[gerazov]

Ok, it's now taking deadlines into account, but deadlines are sorted before overdue scheduled items. I'm not sure how it is in emacs as the spec doesn't read very exact. Here's what I get for 3 deadlines (1 overdue, 1 today, 1 tomorrow), 1 overdue scheduled item and 1 today item:

I'm happy with this, how does it look to you guys?

[kristijanhusak]

@levouh @gerazov I did some refactoring on agenda to allow easier introduction of custom commands. If you find any issues with current functionality let me know.

@gerazov you will have to rebase your sort PR because I split different agenda view in different files.