Data loss #4148
Comments
|
@MohitMaliFtechiz Please treat in priority. |
|
@ABHISHEKCHHIKARA Thanks for opening the issue, and providing the detailed steps. I will get back to you ASAP after testing your provided scenario.
For this, we have already removed the system's DownloadManager from our application in #4143. |
|
@ABHISHEKCHHIKARA, I have tested all the scenarios you described, and the ZIM files remain in the storage (They are not deleted from the storage until I uninstall the application). So here i have some questions for you. You had version Both applications download the ZIM files in
As you described, you saw an
The blur bar shows the progress of scanning the storage, if there are lots of big files in the storage it takes time to scan. |
Yes kiwix was installed as 2 different apps so I moved the files from kiwix standalone to kiwix mobile folder using z archiver , upon disconnecting the hdd , i delete the right app by checking its data usage ( I used 5g internet connection not wifi ) (as I had downloaded stuff through it ) in app info . Then I reconnected the hdd , opened the new app and started loading them in library , i waited but it didn't load for a long time . There was atleast 50 gbs of data in there , I don't know how much time it should take . |
|
@MohitMaliFtechiz Here's a link to screen recording the library isn't loading up even when there are just 5 small files |
As you describe it is taking time to load the ZIM files, so it means the ZIM files exist in HDD and are not deleted. The application was trying to scan the HDD for ZIM files.
Clearing the cache does not delete this media directory, it only deletes the cache directory of the application.
I have tested this scenario as well, and the ZIM files remain in the HDD. If I disconnected the HDD, uninstalled the app, and reinstalled it, then connected the HDD, and selected the HDD in settings. The ZIM files remain in the storage. There are possibly two reasons behind it:
However, if you are thinking it is OS related issue, please do this to confirm:
|
@ABHISHEKCHHIKARA, For this, can you please share the device logs? After showing this blue progress bar at least for 10-15 seconds, please go to the "Help" screen, and send a diagnostic report. It will generate the file please attach that here. Also, can you please me is this blue bar always shows or if it hides after some time? |
It shows always once I reload , only hides sometimes if I change apps or there are too many downloads for a long time and when the screen is turned off after 2 minutes or manually , also the most it reaches sometimes is a little less than midway , it never went far from that. |
Okay I will be testing this , first i will download some files in internal , then I will switch to external , download them again and see the changes in library . Last time this time this happened it got crashed i guess .
I tested this by first deleting the app when hdd is disconnected , then if the hdd is connected and the app is installed through playstore again , the files get destroyed in hdd ( I will try it when manually installing the app through file manager too later on.) |
@ABHISHEKCHHIKARA Please share the device logs for this. It helps us to diagnose the problem.
I have tested this scenario as well. Disconnected the HDD, and removed the app, connected the HDD, installed the app from Play Store. Opened the application, and set up the permission and the ZIM files were showing in the application when I refreshed the library screen.
Yes, it will be great to have the device logs and videos, to find the exact scenario where you have lost your data to identify whether this is your OS problem or not. |
I sent the logs via email to kiwix , please check them , also sending them here --> logs1735479945810.txt , MountInfo(device=/dev/block/dm-52, mountPoint=/apex/com.android.mediaprovider@351113060, fileSystem=ext4) /storage/9E56BD0056BCDA6B/Android/data/org.kiwix.kiwixmobile/files I think the above directory storage/9E56BD0056BCDA6B/Android/data/org.kiwix.kiwixmobile/files is wrong as in hdd it is /kiwix , i don't know why is it showing it as /files I just changed the directory , but the same error again.. also I am sending the screen recording now of files being deleted due to play store install |
@ABHISHEKCHHIKARA It is internal files app-specific directory, and we are downloading the ZIM files in our public app-directory. This is normal.
Okay, by seeing your shared video it seems it is your OS problem. When you are isntalling the Kiwix app from playStore it is deleting the all previous data about this application from the device as well as from all the contented devices, since the app is freshly installed.
As you described in your previous comment, that time this happened and you lost your data. So it is your OS-related problem. To prevent this type of data loss, when you gonna uninstall the application, before that please move all the data from this directory to another directory to prevent data loss. Since uninstalling and reinstalling the application causes data loss.
Interesting, thanks i will fix this ASAP. |
|
@ABHISHEKCHHIKARA We have also open #4150 in an attempt to better warn the user at the app uninstalling time. |
Nice! Although in my case it happened on reinstall (I did remember to disconnect it when uninstalling) , the hdd was still connected, I will remember not to reinstall or delete apps with the hdd connected next time 👍👍 Here is how it probably happened : https://www.dropbox.com/scl/fi/m7xsyh9jfbewpkm8wxsaw/Record_2024-12-29-20-06-19.mp4?rlkey=lfcvo335btn2wqlb2dptq5ilb&st=bvckjd5k&dl=0 |
Describing the bug
So this bug happened as follows :
I first had 3.9 kiwix version , used it for some time , it had the library bugs as stated many times before about zim files not loading and getting detected in the sd card location of app storage .
Then I installed the latest 3.13 version from GitHub , the "standalone" combined version of the app , I downloaded various files for a few days , the process was like : I have an external 2tb hdd so I will charge my Realme Pad X ( Android 12 ) to 100 % , connect the hdd and download the files until it dropped to 20 - 30 % , sometimes I did this , other times I downloaded files in the local storage and then moved them to the same storage location in hdd as it is in local storage and specified by the app.
Now , the app wasn't able to determine the downloaded files in hdd upon disconnection and reconnection even if I saved all things properly and ejected perfectly with interrupting any process . This too wasn't the problem for me for a while as all the files existed and I could see which one I downloaded and which ones I need to download later on ( although the app displayed all the files available for download as it wasn't able to determine the downloaded ones ) (the same functionality was given in previous versions of kiwix like 3.9 or even far back as 3.3.3 , i didn't notice any change in the app's function other than the individual zim file picker being improved however it could only pick 1 file at a time ) , the problem was the fact upon installing the google play store version .
I disconnected the hdd , installed the app to make sure it doesn't destroy the files in hdd , then opened the app set up its storage permissions , plugged in the hdd , saw that the directory was slightly different as it was org.kiwix.mobile so I moved the folder kiwix in this directory and then went to settings to select hdd as storage location , I left it to download khan academy and gutenberg project as there are 2 simultaneous downloads supported . I went outside home and came back after 5 hours , apparently there was a network error and it stopped downloading but also fried all the remaining files .
I believe this issue was not due to internet or file size but rather at the moment when I selected the external storage as download directory in settings , any file during that moment got vanished ( I tried this again )
One more thing to notice is that the app uses the system's download manager which may be good for various number of device able to have the download functionality in sd card working properly but it itself is also a source of error , I noticed many time the realme download manager is laggy and it downloads the files in .zim".oplusdownload" extension and upon full download it removes this .oplusextension extension but often times if there is a small error in internet connection or a moment of disconnection from hdd for whatever reasons , it kills the ongoing download and makes everything buggy as hell . If I try to cancel it and redownload the app is unable to do so because the downloader renamed the new file with -01 at end and doesn't remove the previously failed task , then I would forcefully kill the app or it kills itself often times when I try to stop the download as it will repeat that download process but unable to do anything . I would then deleted both of the files and freshly start the download for it to work .
Let me know if you want more details of any specific event or any other curious point of what happens if I do something else .
Expected behavior
Steps to reproduce the behavior:
1.) Open the app , connected the hdd , change storage location to sd card (Now it shouldn't clear all the files in this directory ) ( I would like an option to force make external as the default directory even if it is not connected and keep it that way , it shouldn't start the download to internal storage immediately as it causes so many conflictions and errors in meanwhile something goes wrong in the connection ) ( the app should ask if I want to start the download in internal storage and a check mark to keep it like that for all the files from now on )
2.) Load up all the .zim files in Kiwix folder in the library of the external directory in Android/media/org.kiwix.mobile and sync them up to the ones available online (I had them downloaded from kiwix but it wasn't able to identify its own downloaded files , something like the size is same if the name changes slightly like -01 in the end shouldn't make it crash and otherwise )
3.) Even if it isn't able to pick them up the + option should allow more than 1 files to be saved all at once in the library .
4.) An option to close all the ongoing downloads all at once in the downloads and a multi select UI to select various files to be downloaded all at once .
5.) Upon downloading all the files to the sd card , stop it from changing the directory to internal and refreshing the library as I need to remove the hdd from the device and the app shouldn't forget all the work it did and the locations .
Screenshots
I could add screenshots of the loading blue bar stuck in the library but which else screenshots should I provide ?
Environment
The text was updated successfully, but these errors were encountered: