Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[filesystem] Fix missing "out of sync error" #7139

Merged
merged 2 commits into from
Feb 18, 2020

Conversation

AlexTugarev
Copy link
Contributor

@AlexTugarev AlexTugarev commented Feb 12, 2020

which should be shown if one try to save pending editor changes to a file which was modified in background.

Fixes #7102

Thanks @akosyakov to provide this fix!

How to test

  • do some changes to a file
  • modify the opened file on terminal
  • try to save the dirty editor
    • it should warn you:
      Screen Shot 2020-02-12 at 11 20 34
  • test closing with dirty and not against the file modified in the background and not
  • test typing fast and slow with auto save
  • test reloading a page with a file deleted in the meantime
  • test file move and delete with dirty and not editors

Review checklist

Reminder for reviewers

@akosyakov akosyakov added the filesystem issues related to the filesystem label Feb 12, 2020
@akosyakov akosyakov self-requested a review February 12, 2020 10:47
@akosyakov
Copy link
Member

Just had to check now why we broke it, probably we tried to fix something else :)

@akosyakov
Copy link
Member

ok, it seems i overlooked it by while introducing incremental updates: bace685#diff-bc50ceab9ceca36de64ebebbe8a2987bL64 How did we live with that for such long time 😬

@akosyakov
Copy link
Member

It does not work very well :( It does ask, but if i say don't save then it asks again and then it does not allow me to save anymore.

@akosyakov
Copy link
Member

It would be good to create integration tests for it as well.

@akosyakov
Copy link
Member

@AlexTugarev Can i push to this PR?

@AlexTugarev
Copy link
Contributor Author

Please, yes!

@akosyakov akosyakov changed the title [filesystem] Fix missing "out of sync error" WIP [filesystem] Fix missing "out of sync error" Feb 12, 2020
@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch 3 times, most recently from f076519 to 574bf69 Compare February 12, 2020 15:58
@akosyakov akosyakov self-assigned this Feb 12, 2020
@akosyakov akosyakov removed their request for review February 12, 2020 15:58
@akosyakov
Copy link
Member

@AlexTugarev i'm going to close it and open a new PR, fine? that we can revert author and reviewer roles?

@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch from 574bf69 to 5a80e67 Compare February 12, 2020 21:38
@akosyakov
Copy link
Member

akosyakov commented Feb 12, 2020

@AlexTugarev It's turned out to be really complicated. The main issue is that monaco model does not know about resource stat while dirty but it still can fetch a resource and reset it to latest stat because of it. In such case, monaco model will overwrite fs. I wonder should we expose something like stat on resource.

btw I've added integration tests for saving to make it reproducible: https://github.com/eclipse-theia/theia/pull/7139/files#diff-74f4047f6db63505c37031b5e5228464R66-R124

@akosyakov
Copy link
Member

Discussed with @svenefftinge that we will add optional version attribute to Resource which clients can pass to save operation.

@akosyakov
Copy link
Member

akosyakov commented Feb 13, 2020

Also noticed that on reload deleted files are not marked as deleted and overwrite content on the disk if saved after file is added again to the disk in background.

Also add tests for:

  • removing file
  • moving file
  • deleted files cannot be opened (or deleted files should be closed on reload)

@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch 2 times, most recently from e3a44d1 to 8505ac1 Compare February 13, 2020 14:18
@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch 5 times, most recently from bc383db to f3dd0a9 Compare February 14, 2020 12:06
@akosyakov
Copy link
Member

@AlexTugarev @marcdumais-work @vince-fugnitto @lmcbout Could you test please? I've added integration tests for all kind of cases of save cycle, but it would be good to test manually as well that saving works well generally and against changed state on the disk.

@akosyakov akosyakov changed the title WIP [filesystem] Fix missing "out of sync error" [filesystem] Fix missing "out of sync error" Feb 14, 2020
@akosyakov
Copy link
Member

@marcdumais-work could you try again please?

@akosyakov
Copy link
Member

akosyakov commented Feb 14, 2020

There is still something fishy with move of dirty editors when I test with Java, now editor is properly moved, but there is an error in the editor. It looks like Java LS is missing some updates for the target editor.

@marcdumais-work
Copy link
Contributor

@marcdumais-work could you try again please?

Works better now - the problem I had earlier is fixed. Will test some more.

@marcdumais-work
Copy link
Contributor

marcdumais-work commented Feb 14, 2020

Also add tests for:

  • removing file

Seems to work well (same as master):

  • remove a file that's open in editor (dirty) -> editor kept, marked as deleted on disk
  • remove a file that's open in editor (not dirty) -> editor immediately closed. (note: VS Code keeps the editor in this case - I have no strong opinion which is better)
  • moving file

Seems to work well (same as master):

  • move file opened in editor (dirty) -> editor kept and marked as deleted from disk
  • move file opened in editor (not dirty) -> editor closed (note: VS Code keeps the editor in this case - I have no strong opinion which is better)
  • deleted files cannot be opened (or deleted files should be closed on reload)
  • remove file opened in editor (dirty) -> editor not re-opened after frontend reload
  • remove file opened in editor (not dirty) -> editor closed immediately and not re-opened after reload

With this PR the editor for the deleted file is not restored after reload 👍

test typing fast and slow with auto save

On its own, autosave works fine as far as I can tell. However if we add concurrent modifications done from outside Theia, it does not work great compared to VS Code. (but no worse than theia master as far as I can tell). So maybe this could be tackled separately.

The result seems to depend on the timing of auto-save vs timing of outside modifications. It's very easy for the changes from outside to be silently overwritten without the user noticing it's the case if the changes are concurrent.

  • create/open a file in Theia: e.g. toto
  • in a terminal run the following one-liner script that appends to the file periodically: for i in {1..1000}; do sleep 1; echo "OOO" >> toto; done
  • in a second terminal run tail -f toto to see the updates to the file as they happen

At times, depending on timing, I can see :

  • the outside change is seen being added to the file by tail but not shown at all in Theia editor, and overwritten by autosave
  • the outside change is seen in the editor. However it's selected and so overwritten as soon as a key is pressed in the editor.

Peek 2020-02-14 15-33

Peek 2020-02-14 15-39

@akosyakov
Copy link
Member

akosyakov commented Feb 14, 2020

The result seems to depend on the timing of auto-save vs timing of outside modifications. It's very easy for the changes from outside to be silently overwritten without the user noticing it's the case if the changes are concurrent.

Do you test with auto on save in both in Theia and VS Code?

@marcdumais-work
Copy link
Contributor

The result seems to depend on the timing of auto-save vs timing of outside modifications. It's very easy for the changes from outside to be silently overwritten without the user noticing it's the case if the changes are concurrent.

Do you test with auto save on both in Theia and VS Code?

yes, both.

@akosyakov
Copy link
Member

akosyakov commented Feb 17, 2020

@marcdumais-work I cannot reproduce, i get consistently a dialog asking me to overwrite in Theia.

I also don't think behaviour in VS Code is good, on overwrite it appends some content to the editor from the disk instead of overwriting the disk with the editor content. Do you think it is expected? Ok, so it happens after overwrite, i got the same in Theia.

@akosyakov
Copy link
Member

akosyakov commented Feb 17, 2020

move file opened in editor (dirty) -> editor kept and marked as deleted from disk

that sounds bogus, on move, old editor should be closed regardless whether dirty or not, target editor should be opened instead with the same state. It is bogus as it behaves on master, e.g. see #7054

@marcdumais-work Could you check that the build was successful for this PR? Maybe you were testing without changes applied?

@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch 2 times, most recently from a838e0c to adf9023 Compare February 17, 2020 15:00
@akosyakov
Copy link
Member

@AlexTugarev Could you have a look whether you can reproduce @marcdumais-work issues? See #7139 (comment)

@AlexTugarev
Copy link
Contributor Author

AlexTugarev commented Feb 18, 2020

@marcdumais-work and @AlexTugarev, in the test from above where you're trying to catch periodic changes from outside, that's actually what I would even assume what should happen, given:

  1. the frequency of the change cycles is 1 Hz
  2. nsfw debounces at debounceMS = 500 by default
    So, chances are good that you just miss changes quite often, or am I wrong?

edit

that's actually not what the ☝️ comment was about. Anton just explained, that it's about the dirty state, etc. 🤦‍♂ nvm!

@AlexTugarev
Copy link
Contributor Author

2020-02-18 10 02 09

@marcdumais-work I re-tested the "outside-changes" scenario with auto-save enabled from above, and it looks good. are you 100% sure, that it's not the master state you're looking at? I tried several times to make sure not posting bs again ;-)

@AlexTugarev
Copy link
Contributor Author

Screen Shot 2020-02-18 at 10 03 59

@akosyakov that "deleted from disk" still exists when using mv from terminal. (auto-save is disabled, and editor was dirty in the screen shot ☝️ )

@akosyakov
Copy link
Member

akosyakov commented Feb 18, 2020

that "deleted from disk" still exists when using mv from terminal. (auto-save is disabled, and editor was dirty in the screen shot ☝️ )

@AlexTugarev I know, we don't participate in move from outside, only when they triggered by Theia, there is no reliable way to detect it. It is the same on master. I'm more concerned with cases when we programmatically rename or move something, like in #7054 Could you file a new issue? I'm not sure that it is solvable since we have to take snapshot of the state before move and events come only after.

@AlexTugarev
Copy link
Contributor Author

@akosyakov thanks for clarification!

@AlexTugarev
Copy link
Contributor Author

@marcdumais-work, it would be great if you can retest 🙏 This PR solves a lot of issues already.

which should be shown if one try to save pending editor changes to a file which was modified in background.

Signed-off-by: Anton Kosyakov <anton.kosyakov@typefox.io>
@akosyakov akosyakov force-pushed the alextugarev/dirty-editor-ignores-7102 branch from adf9023 to e4d8b54 Compare February 18, 2020 10:01
@marcdumais-work
Copy link
Contributor

Hi @AlexTugarev , @akosyakov ,

Back to work this morning. I pulled the latest version of this PR and it's building. Let me retest a bit and comment.

This PR solves a lot of issues already.

👍

@marcdumais-work
Copy link
Contributor

Update: I can't reproduce the "concurrent file update" issue. It's possible that there is a race-condition in the scenario where we may wrongly overwrite outside updates, but if so it's not so easy to trigger anymore.

that sounds bogus, on move, old editor should be closed regardless whether dirty or not, target editor should be opened instead with the same state.

I confirm that's what happens when renaming a dirty file from inside Theia 👍

As I think @AlexTugarev figured, I was originally renaming the file from the terminal using mv - sorry for the confusion.

Conclusion: looks good to merge, behavior-wise.

@akosyakov
Copy link
Member

akosyakov commented Feb 18, 2020

@marcdumais-work @AlexTugarev thank you for the review could someone approve please?

I've also pushed one more commit to insulate tests against nsfw failures: bd6f947 It should make sure that file resource operates even if nsfw failed given that file operations are triggered via Theia UI (FileSystem API). If not then we are out of luck if nsfw failed.

It seems to work. There are nsfw failures on travis (https://travis-ci.com/eclipse-theia/theia/jobs/288440321#L6249), but tests don't fail this time, i.e. file resources keep operating.

@akosyakov
Copy link
Member

@AlexTugarev said (offline) that he will review the code once

Integration tests revealed that nsfw fails from time to time as well as introduce delays. It affects the file resource especially for the delete operation. Instead of relying only on nsfw, file resource now listens to file delete triggered by a user from Theia. It allows to react faster as well as tolerate failures of file watching.

Signed-off-by: Anton Kosyakov <anton.kosyakov@typefox.io>
@AlexTugarev
Copy link
Contributor Author

✔️ I went through the changes and tested once again. It looks very good! Thanks for investing so much time into this issue, Anton!

Copy link
Member

@akosyakov akosyakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved on behalf of @AlexTugarev, since we changed roles: #7139 (comment)

@marcdumais-work is it good to merge with you?

@akosyakov akosyakov merged commit 00c1747 into master Feb 18, 2020
@akosyakov akosyakov deleted the alextugarev/dirty-editor-ignores-7102 branch February 18, 2020 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
filesystem issues related to the filesystem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dirty editor ignores when file was changed in background
3 participants