Startup process executes twice causing conflicts when syncing updates #133
Comments
|
Hi @sky76, really sorry to hear you're having these issues, and many thanks for providing such a detailed issue report. From the log you posted everything looks fine until... The log shows that the post-startup update check executes twice, which it shouldn't do. That leads to the event handlers being re-enabled after the first update sync ( I'm currently trying to finish v1.5.1 which is mainly an Android update, but I'll do some investigation and try to find the reason why the post-startup update check is sometimes executing twice. |
|
Hi @nero120 - thank you for your reply. Here I have added the sync logs from all 3 machines: xBrowserSyncLog_20191114210444370_Machine1.txt Machine3 is my MacBookPro, the other two machines are iMac 27" Retina models. All Chrome's are identical with the exception of the MacBookPro, where I have a local extension installed called "Google PageSpeed Insights" which is now deprecated (the MacBookPro is from late 2012). Other than that these should be nearly identical. The Chrome profile I am using is the main (first) profile (I have approx. 5 to 25 profiles on each Chrome browser for client stuff) and the native Chrome bookmark sync is explicitly disabled on these profiles. Anyhow, I don't think the sync problem has to do with any of the extensions. The bookmark count extension I have installed recently is also very silent - it only updates the number of bookmarks if you click on it. I hope it won't be too much of a pain to find what's causing the problem. For now I'll try not to touch anything regarding bookmarks in case I need to provide additional information. On the MacBookPro I have disabled the sync of the xBrowserSync extension - there I have my 2000+ bookmarks (and of course I have a bookmark html-export at hand, just in case). All my bookmarks are regular http(s)-only bookmarks, no bookmarklets or similar stuff. |
|
Thanks @sky76. From best I can tell, I think the problem was in the startup process, the other logs don't reveal much else. I'll track that as an issue for now, if you get more solid info on how to reproduce the issue in future please post it here. |
|
Ok. I'll redo everything using a new sync ID and when the problem occurs again, I'll post detailed info again. |
|
I think I am having a similar issue. I sometimes lose big chunks of bookmarks for seemingly no reason. It happens every couple of weeks without any warning (no errors, no sign of issues). It actually just happened today where every other bookmark disappeared in one section of my bookmark list. I can't figure it out but I have a feeling that firefox is to blame somehow. I say this because I use chrome 99% of the time and it doesn't seem like I ever have this problem. I only open firefox if I need to use a different browser for whatever reason. Maybe its a coincidence but I feel like whenever I have this problem I have used firefox that day. I think the extension has trouble with bookmark folders or nested folders in firefox? I noticed after the first 200 bookmars in the JSON file the "id" field for each has the number "id": "84911111111111111111111111" with a varying amount of "1"s that can get very long. Is there a way to clean or rebuild the bookmark database without losing any of the tags, descriptions or titles I added? My setup has the bookmark server self-hosted on my ubuntu-server 19.04, I only access it from my internal network with my 2 computers (a desktop and laptop running Windows 10), I use chrome 99% of the time and only use firefox once in a while. No sync profiles on any browser. For now, what I do is backup my whole bookmark database via the chrome extension which outputs a JSON file. I do it at least every day and sometimes more frequently if I have added a lot of bookmarks. I then compare the file size of this backup to a prior saved backup that I know is good. Please don't feel like I'm complaining because I absolutely love the xBrowserSync! I just felt like mentioning it on this thread because it sounded familiar. Thank you! |
|
Hi @rob4226 , I use the Chrome Extension "Über Simple Bookmark Count". |
|
Just a quick note - the size of the JSON file might vary if your favicon data is saved there as well. When I export my bookmarks manually, the favicon data (BASE64 encoded or something like that) makes up for more than half of the file size. Don't know whether this gets exportet by your extension. Anyhow - when I clear my bookmarks entirely(!) and import the backup HTML file, no "Bookmarks backup folder" is created. So I have instant restor of bookmarks. After 1-2 minutes also the favicons are "back" (this takes some time) - at least the ones that are still in the cache. And no, my extensions does not solve the problem, but I have an eye on bookmark changes because I remember the last "highest" number of bookmarks. |
|
Thanks @sky76 for the "Über Simple Bookmark Count" Chrome extension idea! This will definitely make it much easier for me to track the number of bookmarks so maybe I will be able to pinpoint the conditions that causes a loss of bookmarks for me. The JSON backup doesn't have the favicon images, only the bookmark data so that's why I can use the file size as a good indicator of lost bookmarks. As a test, I restored the backup from yesterday (when I noticed this issue occurred again because the JSON backup file size was smaller) and used the bookmark counter extension that you recommended...it showed 503 bookmarks. I then restored the last known good backup and it showed 807 bookmarks. So 300 went missing at some point yesterday lol. It's nice to know the exact number that disappeared now using the counter extension. That's why I constantly backup my bookmarks using the extension's JSON file backup feature so I don't lose any. Since I self-host the server, I decided what I am going to do is create a cron script to automatically backup the database on the server like every hour and keep a few days worth of the backups in case. I may also include a mechanism to check the size of the current to the last backup (or actual number of bookmarks if possible) and if it's smaller by a certain amount, save the good backup permanently with a timestamp so I can go back and fix it manually. |
Unfortunately I've not managed to find a reproducable instance of the issue which makes it very hard to debug. It's definitely there as users are experiencing it, I can't say whether it's Firefox only where the issues occurs. Can you tell me if you had made any bookmark changes when the issues have occurred or when you've noticed the missing bookmarks? I'm wondering if it happens when checking for updates on browser startup or whether it's triggered by a bookmark change on the part of the user.
Ok that's definitely a bug so I'll look into that. The ids should all be unique and incremental, note that as xBrowserSync does not store added dates it uses the id to determine the order the bookmarks were added. There is no automatic way to reset the bookmark ids at the moment, so unfortunately it would be a manual clean up job, sorry. I'll look into adding an id clean up process when backing up for a future release. Could you test something for me please? Could you disable your current sync, then back up your bookmarks and see if the new back up file still contains the broken ids?
Not at all, I'm just very sorry when users lose data on account of a bug and I really appreciate your input and help in getting to the bottom of this! |
Very useful suggestion, thanks!!
Just to confirm, no the favicons are not exported/stored with the xBrowserSync back up file. |
As bookmark data is encrypted client side, you'd need to write the decryption process into your script in order to get the raw JSON, then parse the JSON and count the number of bookmarks from there. If you'd like to attempt this then this issue might assist you. |
@nero120 Thanks for the replies! I did the above test and yes it fixed the broken id's perfectly!! I wish I could just use that as my main bookmark sync but it doesn't keep the descriptions or tags. I looked at my regular backup closer and found where the ids stop counting correctly, it stops at 213 (out of a total of 807)...And now I see what is happening, the next id is "8491" including the parentheses so its being used as a string instead of a number. That is why I see tons of 1's because the string "8491" is being incremented by concatenating a "1" as it goes down the list of the remaining 600 bookmarks lol. I have no idea why that's happening. I attached a screenshot of the JSON, showing when it changes over to "8491". I can use the regular expression "8491.*" to find all the bad id numbers but is there something that I can replace it with that will fix my database? If I at least replace all 398 bad id numbers with an integer (instead of a string being there), even if its the same integer, will the program be able to fix the ids? Thanks!
|
When creating a new sync it gets the bookmark data from the Chrome API, since Chrome does not support descriptions or tags it cannot keep them, so yes not the best solution to your issue.
If you can manually replace the broken string ids with unique integers that increment from the highest existing integer id then it will fix the issue once you restore the edited backup file. |
|
Yeah, I figured a new sync gets its data directly from chrome which only has a url and title. To fix my bookmark sync I am going to write a python script that recursively traverses the new syncs JSON that I know has correct structure and id numbers, and then add the missing data like description and tags from the messed up sync. The script will match on the url field and output a new JSON file that correctly includes everything and lastly restore that JSON into the chrome extension. This way I have a sync where I know the id's and structure are correct but still have my descriptions and tags data. |
|
@nero120 Awesome, thank you very much! |
|
I've got the exactly same issue. Hopefully i restored everything from a backup but this is the third time i've got this. What about to add some sort of rollback function? So if anything goes wrong you can restore back to some previous version... |
|
@sky76 @rob4226 @graphixillusion v1.5.1 has now been released, give it a try and see if it resolves your issues. |
We can finally use XBrowserSync on Brave! Thanks for this important update!I spoke too early. Correction .. It does not synchronize :( |
|
I may have encountered the same issue with v1.5.1. It happened on my firefox in ubuntu when I bookmark one website slowly. By slowly, I mean I clicked on the star in the URL bar and spent 1 minute to find the right folder for it. The log is also attached. |
|
@powpowshen thank for this, it looks like a slightly different issue in your case. Do you see the error every time you add a bookmark using the native Add Bookmark button? |
|
@nero120 |
|
@powpowshen no I think this is the same issue, leave it with me for now and I'll get a fix sorted as soon as I can. |
|
@nero120 |
|
Just chiming in that I'm seeing the same issue when adding bookmarks in FF. 3/4 of my bookmarks are no longer there. |
|
@truncsphere ok I need a log file from the browser where the bookmarks dissappeared, and can you tell me exactly what happened when you noticed they had gone (e.g. you turned on your machine, opened your browser, xbs synced and they were gone?), and let me know the time this happened please. |
|
@nero120 Sure I can repeat it easily (just did it again now). In FF add a new bookmark put it in a folder and I can see the sync icon light up. If I open my bookmarks folder I can see them actively all disappear and some of the folders are added back. It syncs for several |
|
Thanks @truncsphere that helps a lot, I think I see what the issue is. Leave it with me for now. |
|
@truncsphere I think I have a fix sorted, would you be willing to test it for me? 🙏 I have released a new beta which you can download here. To install, remove your current xBrowserSync version from Firefox then drag the .xpi file onto the |
|
@nero120 Add several new bookmarks and everything is working as expected. Thanks for addressing this quickly! |
|
Great news, many thanks @truncsphere! @sky76, as the issue opener could you also please give 1.5.2-beta1 a try and see if it resolves the original issue? |
|
I guess this solution is only for the problem of this post. Not even the Brave browser .. |
|
@nero120 Do we have an ETA on when the update will be pushed out? |
|
Just tested it on FF and it seems to work. @truncsphere Which action did you exactly try that originally caused that behaviour? Adding a bookmark, then moving it to another folder? |
|
@sky76 I detailed my steps in an earlier post. I didn't move them, it was just from adding them when it occurred. |
@truncsphere I've got one more bug to fix so v1.5.2 should be out within the next two weeks. You'll just need to remove the beta and re-install xBrowserSync once it's out. Watch out for your live data being cleared out if you're not accessing it before then... |
nero120
changed the title
|
This lost my bookmarks too. No where to get them back???? |
Backup, if you have one... |
Yes exactly, I cannot stress to take backups enough!! xBrowserSync is an open source project and thus will inevitably be subject to bugs that may in the very worst case result in data loss/corruption. Obviously I work hard to try to limit and resolve these issues but users need to take responsibility for their own data, and make sure you take backups. Even if you do it once a month, that's better than never. The app even tells you to backup multiple times!! |
|
Any update on release? Not being able to sync bookmarks is beginning to be painful. Thanks! |
|
@truncsphere it's pretty much done, just needs thorough testing. However I'm moving house this week so have no spare time, will hopefully pick up again at some point next week. |
What is the problem?
Sometimes syncing 1 new bookmark kills 80% of my bookmarks on 3 perfectly synced machines!
How can the issue be reproduced?
I use xBrowserSync on 3 machines (see specs below) with Chrome (3x Version 78.0.3904.97) and Firefox, but 99% with Chrome.
I use the native Chrome Sync Profiles, but on each of the 3 machines I have disabled just the native bookmark sync part of Google Chrome for my primary profile.
I do not use Chrome Sync on mobile phones, I use Brave there.
I set up the bookmarks, create a new sync ID, and populate each of these machines (let's call them A, B and C) with approx. 2272 bookmarks.
I'm using "simple bookmark count" to keep track of the number/irregularities of my bookmarks.
Everything is going well for some weeks, my bookmarks have been expanded only very cautiously up to 2279 bookmarks on each of my machines - A, B and C - for several days.
I am using Chrome on these 3 devices on a daily basis and everything is in sync. :)
But then I add just 1 bookmark to Chrome on machine A - now I have 2280 bookmarks there.
Then I start Chrome on machine C and probably later on B. Suddenly my CPU usage raises up to 99% for 3 minutes on machine B while syncing. After that the count for B is still 2280.
But on machine C my number of bookmarks suddenly drops to 185.
And soon after also on B.
The structure of folders and the sorting order of bookmarks is intact, but some folders are empty, some bookmarks are missing, completely random!
Then I clear everything, de-install the extension, populate the browser with new bookmarks, re-install the extension, create a new sync ID and start from scratch. This whole process I do repeat on each of my 3 machines.
Now I am in "phase VI" - I have already done this "complete cleanup" 5 times...
Sometimes I even don't dare to touch or alter my bookmarks just in fear of screwing everything up.
But then I could also replicate a bookmarks.html file from time to time and it would be safer.
The idea of a sync tool should be to sync.
I love xBrowserSync and backed it from the early days on Patreon after looking for an alternative to (fo)Xmarks.
I am also willing to help providing more insights getting this resolved.
Nevertheless: Keep up the good work, guys!
Desktop browser
Please complete the following information for all affected desktop browsers:
Application log
Please include the logs from the affected clients:
Device B:
Log entries
The text was updated successfully, but these errors were encountered: