Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pins): apply pins to messages being loaded by fetching more #16897

Merged
merged 1 commit into from
Jan 7, 2025

Conversation

jrainville
Copy link
Member

@jrainville jrainville commented Dec 5, 2024

What does the PR do

Fixes #16896

status-go PR status-im/status-go#6226

The problem was that we relied on the SIGNAL_PINNED_MESSAGES_LOADED event to apply the pinned status to messages, but that only happens at the start and a lot of messages are not loaded at start if they are not in the first 30 messages.

To fix this, I just added pinnedBy to the Message object in status-go. This way, we can easily tell straight from the MessageDto if a message is pinned and by whom.

Affected areas

Messages (list)

Screenshot of functionality (including design for comparison)

old-pins.webm

Impact on end user

Fixed a bug where old pins showed as normal messages

How to test

  • Have a chat with a lot of messages
  • Pin an old one
  • Restart
    Scroll back up

Risk

Tick one:

  • Low risk: 2 devs MUST perform testing as specified above and attach their results as comments to this PR before merging.
  • High risk: QA team MUST perform additional testing in the specified affected areas before merging.

Worst case, the bug still happens, or the table of pins in the code doesn't get up to date and we get a weird state

@jrainville jrainville requested a review from a team as a code owner December 5, 2024 19:43
@jrainville jrainville requested review from noeliaSD, a team, iurimatias, igor-sirotin and osmaczko and removed request for a team and noeliaSD December 5, 2024 19:43
@status-im-auto
Copy link
Member

status-im-auto commented Dec 5, 2024

Jenkins Builds

Click to see older builds (28)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 308337a #1 2024-12-05 19:52:38 ~9 min tests/nim 📄log
✔️ 308337a #1 2024-12-05 19:53:46 ~10 min macos/aarch64 🍎dmg
✔️ 308337a #1 2024-12-05 19:57:07 ~13 min tests/ui 📄log
✔️ 308337a #1 2024-12-05 19:59:27 ~15 min macos/x86_64 🍎dmg
✔️ 308337a #1 2024-12-05 20:00:47 ~17 min linux-nix/x86_64 📦tgz
✔️ 308337a #1 2024-12-05 20:08:24 ~25 min linux/x86_64 📦tgz
✔️ 308337a #1 2024-12-05 20:12:13 ~28 min windows/x86_64 💿exe
✔️ 27ef71f #2 2024-12-06 15:06:37 ~6 min macos/aarch64 🍎dmg
✔️ 27ef71f #2 2024-12-06 15:09:05 ~8 min tests/nim 📄log
✔️ 27ef71f #2 2024-12-06 15:11:44 ~11 min macos/x86_64 🍎dmg
✔️ 27ef71f #2 2024-12-06 15:13:19 ~12 min tests/ui 📄log
✔️ 27ef71f #2 2024-12-06 15:13:54 ~13 min linux-nix/x86_64 📦tgz
✔️ 27ef71f #2 2024-12-06 15:22:40 ~22 min linux/x86_64 📦tgz
✔️ 27ef71f #2 2024-12-06 15:26:56 ~26 min windows/x86_64 💿exe
✔️ 7df8fa8 #3 2024-12-17 16:47:10 ~6 min macos/aarch64 🍎dmg
✔️ 7df8fa8 #3 2024-12-17 16:48:34 ~7 min tests/nim 📄log
✔️ 7df8fa8 #3 2024-12-17 16:53:38 ~12 min tests/ui 📄log
✔️ 7df8fa8 #3 2024-12-17 16:54:43 ~14 min macos/x86_64 🍎dmg
✔️ 7df8fa8 #3 2024-12-17 16:55:27 ~14 min linux-nix/x86_64 📦tgz
✔️ 7df8fa8 #3 2024-12-17 16:58:29 ~17 min linux/x86_64 📦tgz
✔️ 7df8fa8 #3 2024-12-17 17:09:54 ~29 min windows/x86_64 💿exe
✔️ 666b15b #4 2024-12-18 19:15:48 ~4 min macos/aarch64 🍎dmg
✔️ 666b15b #4 2024-12-18 19:18:20 ~7 min tests/nim 📄log
✔️ 666b15b #4 2024-12-18 19:23:06 ~12 min tests/ui 📄log
✔️ 666b15b #4 2024-12-18 19:23:35 ~12 min macos/x86_64 🍎dmg
✔️ 666b15b #4 2024-12-18 19:25:27 ~14 min linux-nix/x86_64 📦tgz
✔️ 666b15b #4 2024-12-18 19:27:58 ~17 min linux/x86_64 📦tgz
✔️ 666b15b #4 2024-12-18 19:35:32 ~24 min windows/x86_64 💿exe
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ c71a9ef #5 2025-01-06 15:23:25 ~6 min macos/aarch64 🍎dmg
c71a9ef #5 2025-01-06 15:24:14 ~7 min windows/x86_64 📄log
✔️ c71a9ef #5 2025-01-06 15:25:43 ~8 min tests/nim 📄log
✔️ c71a9ef #5 2025-01-06 15:29:12 ~12 min tests/ui 📄log
✔️ c71a9ef #5 2025-01-06 15:30:48 ~13 min macos/x86_64 🍎dmg
✔️ c71a9ef #5 2025-01-06 15:33:12 ~16 min linux-nix/x86_64 📦tgz
✔️ c71a9ef #5 2025-01-06 15:38:36 ~21 min linux/x86_64 📦tgz
✔️ cdeaf56 #6 2025-01-06 17:00:46 ~4 min macos/aarch64 🍎dmg
✔️ cdeaf56 #6 2025-01-06 17:03:48 ~7 min tests/nim 📄log
✔️ cdeaf56 #6 2025-01-06 17:08:30 ~12 min macos/x86_64 🍎dmg
cdeaf56 #6 2025-01-06 17:12:15 ~16 min tests/ui 📄log
✔️ cdeaf56 #6 2025-01-06 17:15:21 ~19 min linux-nix/x86_64 📦tgz
✔️ cdeaf56 #6 2025-01-06 17:16:53 ~20 min linux/x86_64 📦tgz
✔️ cdeaf56 #6 2025-01-06 17:24:11 ~28 min windows/x86_64 💿exe
✔️ cdeaf56 #7 2025-01-06 18:26:27 ~11 min tests/ui 📄log

@jrainville jrainville force-pushed the fix/old-pins-looking-normal branch from 308337a to 27ef71f Compare December 6, 2024 15:00
Copy link
Member

@caybro caybro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@igor-sirotin
Copy link
Contributor

hmmm, doesn't status-go provide pinnedBy already for each message? 🤔

@jrainville
Copy link
Member Author

hmmm, doesn't status-go provide pinnedBy already for each message? 🤔

I would feel quite silly if that was the case. Let's see

@jrainville
Copy link
Member Author

hmmm, doesn't status-go provide pinnedBy already for each message? 🤔

Ok, I don't feel silly in the end @igor-sirotin . It's indeed not available from the fetchMessages API. We could add it I guess. It would slow down the query a little though

{
      "id": "abe40742d35c03ed75c32c2def0a137b5b93a12bac6dd2daca7472f3623f1a24",
      "whisperTimestamp": 1733419934000,
      "from": "0x04ea5307274c69ee2075c82e9cd22de9bcc370d2d05620c06b1255a99cae6d697075525d747f3698ed1358c50f022ffd25d7d200e0aa51d7f6ce6c504457392e2f",
      "alias": "zQ3...FqioYB",
      "identicon": "",
      "seen": true,
      "quotedMessage": {
        "id": "0xe386b48e38591d3dd79532cc83b6bb55780c877a10644f7c53658468a21702b1",
        "contentType": 1,
        "from": "0x04ea5307274c69ee2075c82e9cd22de9bcc370d2d05620c06b1255a99cae6d697075525d747f3698ed1358c50f022ffd25d7d200e0aa51d7f6ce6c504457392e2f",
        "text": "allo 2",
        "parsedText": [
          {
            "type": "paragraph",
            "children": [
              {
                "literal": "allo 2"
              }
            ]
          }
        ],
        "albumImagesCount": 0
      },
      "rtl": false,
      "lineCount": 0,
      "text": "",
      "chatId": "0x04ea5307274c69ee2075c82e9cd22de9bcc370d2d05620c06b1255a99cae6d697075525d747f3698ed1358c50f022ffd25d7d200e0aa51d7f6ce6c504457392e2f",
      "localChatId": "0x04ea5307274c69ee2075c82e9cd22de9bcc370d2d05620c06b1255a99cae6d697075525d747f3698ed1358c50f022ffd25d7d200e0aa51d7f6ce6c504457392e2f",
      "clock": 1733419933645,
      "replace": "",
      "responseTo": "0xe386b48e38591d3dd79532cc83b6bb55780c877a10644f7c53658468a21702b1",
      "ensName": "",
      "displayName": "",
      "gapParameters": {},
      "timestamp": 1733419934000,
      "contentType": 14,
      "messageType": 1,
      "compressedKey": "zQ3shvQoxa3rU3HkuGFgjgxi8ZCwQkiiHgTUNQy86pNFqioYB",
      "emojiHash": [
        "skipped"
      ]
    },

@jrainville jrainville force-pushed the fix/old-pins-looking-normal branch from 27ef71f to 7df8fa8 Compare December 17, 2024 16:40
Copy link
Contributor

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jrainville you're right, it doesn't exist indeed. But do you think we should just add it there and not mess on the Nim side?

common.Message already provides data from not this specific message, like quotedMessage. So I think we could hydrate it with pinnedBy somewhere in prepareMessage.

What do you think?

@jrainville
Copy link
Member Author

common.Message already provides data from not this specific message, like quotedMessage. So I think we could hydrate it with pinnedBy somewhere in prepareMessage.

@igor-sirotin I think it would need to be done in the persistence level since the message itself doesn't have any idea it has been pinned. Therefore, prepareMessage would have to do a query to know if the message is pinned, which is not optimal.

IMO, the real solution would have been to just have the pin_messages columns in the messages table, since they are one to one.
It would have simplified everything.

Now, we can still just JOIN the two tables. In theory it would be very simple and it shouldn't slow down. We can always add an index if needed.

I don't know if we need to ask other people before making that decision in case there is something else we haven't thought of in terms of performance?

@igor-sirotin
Copy link
Contributor

IMO, the real solution would have been to just have the pin_messages columns in the messages table, since they are one to one. It would have simplified everything.

I prefer not to mess up and keep tables as separate as possible, especially for the user_messages, which already contains too much stuff.

I think it would need to be done in the persistence level

Yes I think you're right, better LEFT JOIN pin_messages.

I don't know if we need to ask other people before making that decision in case there is something else we haven't thought of in terms of performance?

I don't know, not a big pro in database world, I guess we can probably do some benchmarks or ask @osmaczko what he thinks.
But, comparing to the Nim-level loop over all pinned messages, it should be faster anyway 😄

@caybro
Copy link
Member

caybro commented Dec 18, 2024

IMO, the real solution would have been to just have the pin_messages columns in the messages table, since they are one to one. It would have simplified everything.

I prefer not to mess up and keep tables as separate as possible, especially for the user_messages, which already contains too much stuff.

I think it would need to be done in the persistence level

Yes I think you're right, better LEFT JOIN pin_messages.

I don't know if we need to ask other people before making that decision in case there is something else we haven't thought of in terms of performance?

I don't know, not a big pro in database world, I guess we can probably do some benchmarks or ask @osmaczko what he thinks. But, comparing to the Nim-level loop over all pinned messages, it should be faster anyway 😄

Our custom LeftJoinModel C++ model enters the stage... 💪 😆

On a serious note, I'd keep them separate, I'd even simplify the NIM part and do the join in QML using the above ^^

@jrainville jrainville force-pushed the fix/old-pins-looking-normal branch from 7df8fa8 to 666b15b Compare December 18, 2024 19:10
@jrainville
Copy link
Member Author

@igor-sirotin I went with the LEFT JOIN. Seeing the amount of things that are joined already for messages, I doubt joining the pin_messages will add any performance issue, especially since it's a pretty small table.

You can find the status-go PR here: status-im/status-go#6226

I changed the code here completely too. It's way simpler now

@@ -107,7 +107,7 @@ proc createMessageItemFromDto(self: Module, message: MessageDto, communityId: st
imagesAlbum.add(msg.image)
albumMessageIds.add(msg.id)

return msg_item_qobj.newMessageItem(msg_item.initItem(
return msg_item_qobj.newMessageItem(msg_item.initMessageItem(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I renamed this function because it was so annoying trying to see which function to modify when every Item has the same init function. Now it's gonna be way clearer.

Copy link
Contributor

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! I like the simplicity 👍

Copy link
Member

@caybro caybro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@jrainville
Copy link
Member Author

@iurimatias you have been randomly chosen to test this PR

There were 4 items in your list. Here they are in random order:

iurimatias
igor-sirotin
caybro
osmaczko

Fixes #16896

The problem was that we relied on the `SIGNAL_PINNED_MESSAGES_LOADED` event to apply the pinned status to messages, but that only happens at the start and a lot of messages are not loaded at start if they are not in the first 30 messages.

To fix this, I just added `pinnedBy` to the Message object in status-go. This way, we can easily tell straight from the MessageDto if a message is pinned and by whom.
@jrainville
Copy link
Member Author

The allure report failed to generate, but the tests passed. I'll force the merge because I've been stuck on that for 2 days

@jrainville jrainville merged commit 723dc71 into master Jan 7, 2025
8 of 9 checks passed
@jrainville jrainville deleted the fix/old-pins-looking-normal branch January 7, 2025 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Old pinned messages do not have the right background
6 participants