Extended expiry date rolls back to Original expiry && expiry date disappears after restart

[xd003]

Describe the bug
As implied by the title ,the expiry duration when extended rolls back to the original expiry after restart

To Reproduce

1. Invite a new user with expiry enabled ( eg 1 Month )
2. After the user signs up ,the expiry date correctly shows the date after 1 month
3. Now select that user , click on extend expiry and extend the expiry for 1 Month . At this point , the expiry date correctly increases 1 Month and its now expiry is 2 months from today
4. Now go to settings and click on restart . When i go back to Accounts and check the expiry of this user its again back to what it was set originally ie 1 Month from now ( like step 2 )

Platform/Version
jfa-do docker:unstable

[xd003]

Sometimes it might even take 2 restarts for the expiry date to roll back to original but its definitely not persistent for me

[hrfee]

Could you post the contents of <folder you mounted to /data in your container>/users.json before and after you extend the expiry, and then after you restart?

[hrfee]

nvm, just happened to me as well after like 3 restarts.

[xd003]

Sorry i missed to reply quickly but good to know that you were able to reproduce it

[hrfee]

Weird, I can't repeat the issue now. Could you check the logs (docker logs <jfa-go container name>) for any errors starting with "Failed to store user duration:"?

[xd003]

Weird, I can't repeat the issue now. Could you check the logs (docker logs <jfa-go container name>) for any errors starting with "Failed to store user duration:"?

I can't seem to find any error starting with such keywords but here's the log file nevertheless
jfa-go.txt

[hrfee]

Yeah, your logs look normal. I think i might have just read the date the wrong way round when I thought it happened to me. I've added some extra log lines and I'll keep mine running for a while to see if it happens again.

[xd003]

Also I don't know if it's worth mentioning but sometimes after restart , the expiry duration in Accounts option just go blank for some reason ( i am 100% sure expiry was enabled for the users and it was visible sometime ago ) . Could be somehow related to this expiry issue

[hrfee]

Also I don't know if it's worth mentioning but sometimes after restart , the expiry duration in Accounts option just go blank for some reason ( i am 100% sure expiry was enabled for the users and it was visible sometime ago ) . Could be somehow related to this expiry issue

Could you check the file I mentioned above and see how many entries there are?

[hrfee]

Could you post the contents of <folder you mounted to /data in your container>/users.json before and after you extend the expiry, and then after you restart?

Actually can you do this as well?

[xd003]

@hrfee firstly the expiry which was showing blank yesterday started showing up fine today when i wake up but lets just ignore it for now . So now when initially everything was okey , i took a backup of users.json and after around 2 restarts , i was able to make the expiry date again so blank so i took the users.json file backup again. And expectedly both of the file had difference in size . These both backups were literally taken after a 30 seconds gap ( 2 restarts )
ps / i have renamed .json to .txt as github wasn't allowing to uplaod json

users_before.txt
users_after_restart.txt

Although the docker logs doesn't seem to be showing anything unusual but here it is nevertheless ( could check from bottom as the issue was reproduced just a min ago )
jfa-go_dockerlog.txt

[hrfee]

One of the expiry dates was deleted from the file, despite it's expiry being later than all of the others. This is super confusing, as the program will should log in all the places where any modifications occur to the file. I've added some more logging and some other stuff that should help us figure out the problem in 76b8222, give that a try once its built.

[xd003]

I tested the latest docker: unstable build . The User expiry for a test user which was showing normally immediately upon creation vanished after a single restart

Here's the docker logs . You can see line 165 where user test got created
jfa-go_dockerlogs.txt

The users.json file was same before and after the restart. I know it doesn't help much but i managed to record a small screen recording of 50 seconds showing the issue. You can watch it here - https://drive.google.com/file/d/1C_DeLMJsUR_exura1jx_9-SUk9DO_4Ki/view?usp=drivesdk ( keep on on Test user having the email xd003@mail.com )

[hrfee]

Thanks for the video. After you've restarted and the expiry time has disappeared, if you select that use, does the "Extend Expiry" button still appear?

[xd003]

Thanks for the video. After you've restarted and the expiry time has disappeared, if you select that use, does the "Extend Expiry" button still appear?

No the Extend expiry button doesn't appear at top after selecting that user

[xd003]

I did a interesting test if its helpful .In jfa-go , i have set any user to get deleted on expiry . So i created a test user with 2 min Expiry and got its expiry date vanished after 1 restart and waited for 2 more mins . The test user did got deleted even though expiry date was absent for it at that very moment

I will do similar test for a longer Expiry duration for 12 hours and see how it does

[hrfee]

ooh, thats good news then i guess. I've just put a special build for you on docker hub (hrfee/jfa-go:debug) that includes extra log lines so we can see what's going on. Can you try it out before and after the expiry disappears, then send the logs?

[xd003]

ooh, thats good news then i guess. I've just put a special build for you on docker hub (hrfee/jfa-go:debug) that includes extra log lines so we can see what's going on. Can you try it out before and after the expiry disappears, then send the logs?

Yeah sure

[xd003]

Analysis on jfa:debug ( User Test is the one being evaluated )

I am not sure if you just enabled more verbose logging or did some other changes too as the behaviour was little different this time. So firstly i created the user and set expiry for 5 mins and as always upon creation the expiry was showing fine . Now I restarted 3,4 times in a row ,but the expiry date was still persistent , looked good till now. Then i extended expiry for 1 month , all good till here , restarted again and it waa still displaying expiry well (5 Mins + 1 Month ) . Now i extended expiry for 1 day to test and upon restart the account itself got deleted somehow. . To summarise , this time there was no Expiry disappearance nor expiry rollback issue but user account got deleted as per the original expiry , it ignored the extensions . I have logged it all properly . I think we might be pretty close to get it fixed

users-before.txt
users-afterRestart.txt
jfa-go_dockerlogs.txt

[hrfee]

I think i've figured it out, your logs were super helpful.

I noticed the user daemon (Daemon: Checking for user expiry) was being run wayy too much in your logs. Its meant to run every minute, but your logs show it was being run twice 3 seconds apart.

jfa-go doesn't actually restart when you press restart, it basically ends the main function and then runs it again. The user daemon isn't being stopped when a "restart" happens, so gradually more and more instances are being run. I think this is where the weird behaviour when you restarted like 3-4 times came from.

I'll work on a fix now.

[xd003]

That's good to here that my logs were helpful , i will test the upcoming version tonight if possible but here its almost 11 pm so i might send the next report tomorrow morning

[hrfee]

Its up on the same docker tag now, np if you need to do it tomorrow.

[xd003]

Analysis on jfa-go:debug ( User Test is the one being evaluated )

I did this test with a preplanned 3 stage process to verify eveything properly in 1 go

1. Check for expiry disappearance
2. Check for expiry rollback
3. Check if jfa-go considers expiry extension

So after running the debug build , i invited the test user with a duration of 15 mins . Within 2,3 mins itself , i did like 2,3 restarts and didin't noticed any expiry disappearance . So it did fine here
After that i extended the expiry for 24 hours and did 3,4 further restarts within the next few mins and yes ,it never rolled back to orignal expiry , did fine here as well
At last , i was waiting for 15 mins to get over ( 15 mins since the user got invited ) to check whether jfa-go considers expiry extension and to my relief it did

So as of now ,it has passed all 3 tests that i did . Although i think its a good idea to upload logs even when result is positive

Logs
users.json ( before any test , just after user creation )
{"0d2e20b972db485ca2d1f27dd19aac2a":"2021-04-06T17:58:29.486970303Z","e0cafd54be7c4a6a80bdf08e7a5aacb5":"2021-05-05T05:07:56.253012695Z"}

users.json ( after all the tests + extension )
{"0d2e20b972db485ca2d1f27dd19aac2a":"2021-04-07T18:58:29.486970303Z","e0cafd54be7c4a6a80bdf08e7a5aacb5":"2021-05-05T05:07:56.253012695Z"}

Jfa-go:debug docker logs here - https://del.dog/jfa-go-debug.txt

[xd003]

i think it would now fine to merge all the debug commits in unstable, lemme know when you do so . i will switch back to unstable. This issue was seemingly bit tricky to get hold of but now seems to have been resolved finally : )

[hrfee]

Already did merge, even if it wasn't the fix it still should've been there. You can switch back to unstable. Is it safe to mark this as closed for now?

[xd003]

yes , this could be closed now . I just had one one last question , its kinda related to expiry so i thought better to ask it here instead of opening a new issue altogether as its a very small thing . So the thing is the time displayed in expiry never matches my time zone . Sure when i set it to 1 min , it deletes invited user after 1 min only but the timing is something else together . So this time-zone mismatch is a jfa-go thing ? or i will have do something on my vps to fix it ? ( Assuming jfa-go fetches the time from my vps . This is the docker command i use to run jfa-go if it helps - docker run -d --name=jfa-go --restart=unless-stopped -e PGID=1001 -e PUID=1001 -v /opt/jfa-go:/data -v /etc/localtime:/etc/localtime:ro -p 8056:8056 hrfee/jfa-go:unstable . i am from india so i need ISD ( Indian Standard time ) to be used for expiry date

[hrfee]

Does the date command output the correct time on your vps?

[hrfee]

I've just realised how weird my date handling is, right now a date is converted to DD/MM/YY on the server instead of the client, so if your server is in a different timezone it'll appear weird. I'll start changing it now.

[xd003]

Does the date command output the correct time on your vps?

it displays Coordinated Universal Time (UTC) by default . i just changed my timezone to display the correct one - https://linuxize.com/post/how-to-set-or-change-timezone-in-linux/ . Now date shows my exact time and accordingly fixed in jfa-go expiry as well . You can add this link in readme for changing timezone for users who don't know . xd finally i can sleep . Thanks for the cooperation in fixing this issue

its just 2 commands if you want to add it directly

timedatectl list-timezones
sudo timedatectl set-timezone <your_time_zone>

[xd003]

@hrfee i just updated to unstable. Saw some time related commits . Earlier i was having just timezone mismatch issue which had got fixed by changing my timezone although the format was fine still then ( it was just UTC timezone )

Now coming back to topic , the time format currently has become quite weird now . It's looking like some random numbers . Here's a pic

[hrfee]

Could you try updating? This should be fixed.

[xd003]

I was already on latest jfa-go:unstable .That pic was from last night . I checked just now its not displaying those random numbers , maybe jfa-go was adjusting timezone or something when i checked it yesterday. Although i noticed one thing , currently the expiry date format is MM/DD/YY , i think earlier DD/MM/YY was better

Or better yet if you can provide these as options options in jfa-go ( like there's one for 12/24 hr ) , that would be most ideal

[hrfee]

I mistakenly thought the function I used would respect the browser language/locale, but it didn't. 7196361 should fix that.

Currently the 12/24 selection is only for emails, but I think i'm going to expand the language menu (in the top left) to integrate with time formatting and also have a 12/24h selector.

[hrfee]

Added a 12/24h selector to the language menu in 66b7df7.

[xd003]

That's nice , do you want me to test it now or i can do it all together when expiry date format also gets added

[hrfee]

don't mind, the expiry one should be done in 5 mins though so you may as well wait

[hrfee]

I was already on latest jfa-go:unstable .That pic was from last night . I checked just now its not displaying those random numbers , maybe jfa-go was adjusting timezone or something when i checked it yesterday. Although i noticed one thing , currently the expiry date format is MM/DD/YY , i think earlier DD/MM/YY was better

Or better yet if you can provide these as options options in jfa-go ( like there's one for 12/24 hr ) , that would be most ideal

You can select the "English (GB)" to get DD/MM/YY back.

[xd003]

That's great , will give it a try now

[xd003]

I just tested both 12/24 and the language option

If i select either of 12/24 h time format and reload the page , the earlier saved option is retained , that's pretty good
But the same doesn't seem to happen with language . If i select any language it changes but when i reload my jfa page, it reverts back to default lang. I am guessing this happens because while choosing any particular language , jfa-go adds ( ?lang=$lang ) to the url. So when i open my subdomain.domain.xyx link , it simply opens it with the default language

I can propose 2 ideas to fix this issue

1. Maybe some way to change language without altering the url so it can be done like 12/24 hr format which retains itself on reloading
2. Or maybe you can make the https://IP:PORT // https://jfa_domain_url to automatically open with the saved language by adding the necessary part in url
   Eg
   If my domain is https://signup.jfa.com
   And i have selected en-gb

Then upon loading https://signup.jfa.xyz , it could automatically load https://signup.jfa.xyz/?lang=en-gb ( invite page )
https://signup.jfa.xyz/accounts?lang=en-gb ( accounts page )
https://signup.jfa.xyz/settings?lang=en-gb ( settings page )

Obviously you might know some even better way around this , i am not a developer .. These are my inputs just as a user

[hrfee]

For now you can just set the default language in settings, but remembering the clients choice is a good idea too.

[hrfee]

language preference now stored in a cookie as of a8f71c8.