Realtime Database stays offline and doesn't reconnect after sleeping #9682
Comments
|
We have a similar issue since iOS15. It seems to work 99% of the time, but once in a while, the reconnection does not happen. |
|
This keeps happening to us. It's been like 3 times I hit this myself since I reported the issue. Our users are hitting it a lot too. |
|
Hey all, apologies for the slow response. Do you have any reliable repro steps or are you able to enable logging and share the logs when this bug occurs? |
|
This is happening to me as well, running locally at the moment, haven't tested this issue in prod, but every time my macbook sleeps, the realtime connection is ended and I have to refresh the browser to reconnect. I am using for user presence as well so it becomes an issue when any mac users in my app encounter this bug expecting to see others online to communicate. |
|
Hey @morganchen12, sorry about the delayed response on my side this time. I couldn't reproduce it myself but we have reports that it is still happening to users. Unfortunately we don't have any reliable repro steps. When you say enable logging is just doing |
|
Yep, or by calling this method. |
|
Ok, I could reproduce it and collect logs and other information! In my experience, it is more likely to hit this issue on Monday mornings, that is, after having my Mac sleeping during the weekend with the app running. So last Friday at the end of the day I set up debug mode for Realtime Database, started recording logs left the app running attached to Xcode and put my mac to sleep. Yesterday morning, while the mac was sleeping the app tried reconnecting to the Realtime Database. This behavior is normal as far as I know: even if the computer is "sleeping" the app would wake app every now and then for some seconds and try to reconnect to servers because it doesn't know the mac "is actually sleeping". Some seconds later, it actually goes to sleep so the app stop running. The problem is that, during these spontaneous wake ups, there are high chances that we are not connected to the network, so connection issues might happen: As there were network problems, opening this connection timed out. And the Realtime Database client hanged here. Today I started my workday, opened my mac and this got logged. I checked server side state and confirmed this issue was happening: I was offline for the rest of the world but my app was running and had a good Internet connection. Using an internal debugging feature I tried to interact with the Realtime Database, and this is what I got: This confirmed previous observations on this issue. The Realtime Database client was "stuck". Using Xcode, I checked two Realtime Database client internal bits of state:
So it looks like the app hanged in this In the past, I had seen the app recovering itself after several hours so I'm still waiting to collect logs in case that could be helpful. I still have Xcode attached so if you want to debug it with me now, let me know. :) I have more logs if that might be useful to you but they have some (slightly) sensitive data so I am fine with sending them to you directly instead of attaching them here publicly if you need them. Thank you! |
|
Ok, after around 2 hours, the app reconnected. These are the logs: Then it properly connected again. Note I had a breakpoint so that might explain the delay in 13:08:36. Let me know if you need more information. Thanks! |
|
Thanks, it sounds like the right way to fix this is to just retry on socket connection failures, respecting the |
|
Thanks! You might also consider the timeout case. I think the socket error in this case was 2 hours late so it's not safe to wait for it unless the SDK can force it to happen earlier somehow. Alternatively, could the SDK discard the timed out web socket (and future notifications) and try a new one? |
|
Maybe it'd be better to deliberately terminate the socket connection on sleep and restore it on wakeup then? @IanWyszynski what do you think? |
|
Thanks! In my opinion, terminating the connection on sleep and restore it on wake up might reduce the chances of this from happening, but it doesn't fix the root problem. The main problem is that the SDK doesn't retry after the timeout (it just logs an error), it waits for the socket error and that might take a lot of time (if it actually happens). |
|
Hi, this issue is more widespread than just macos, it definitely happens on iOS as well in ionic cordova apps (environment info below). A couple observations and research items of note that may help:
It's quite a bit of effort to setup a new 'minimal' repo, ionic, angularfire and a subscription to a listval with detection events on resume/background. If someone else has that handy then go for it, I hope this post helps someone else so they don't spend days searching and experimenting! Note: using modular angularfire. ENVIRONMENT |
|
How is this still an issue? Seems to be a pretty critical bug imo... |
|
Hello, this is still an issue on both Firebase and Firestore. Not only that, but, it's found in the web & Android versions, not just iOS. Do we have to implement a watchdog or an app lifecycle event ourselves for something this critical? There have been countless times where I get a push notification while the app is on the background / sleep, I open it and the data from Firestore never arrives without restarting the app. |
|
this cost me a lot of negative reviews on my app, I guess we need to remove rtdb. I tried @markterrill solution listening the lifecycle of the app but it doesn't work, rtdb never recovers and it doesn't show an error or something in debug window. |
|
In my case, when the app goes offline then online the listener to ".info/connected" is never called again. This means when the app reconnects to WiFi again, my app is not able to update its online status again in the realtime database. |
|
Hello, import { Injectable } from '@angular/core';
import { getApp } from 'firebase/app';
import { getDatabase, goOffline, goOnline, onValue, ref } from 'firebase/database';
import { Network } from '@capacitor/network';
@Injectable({
providedIn: 'root'
})
export class DatabaseService {
public connectedToDatabase: boolean;
private app = getApp();
private database = getDatabase(this.app, "...firebasedatabase.app/");
constructor() {
// Get network status changes
// Network status changes when switching between wifi, cellular and offline
Network.addListener('networkStatusChange', status => {
console.log('Network status changed', status);
if (status.connected == true) {
// We need to go offline first, because realtime database can get stuck when switching between wifi and cellular
goOffline(this.database);
goOnline(this.database);
}
if (status.connected == false) {
goOffline(this.database);
}
});
// Now the callback on connectedRef is triggered at each network change
// And write operations like (set()) don't get stuck anymore
const connectedRef = ref(this.database, ".info/connected");
onValue(connectedRef, (snap) => {
if (snap.val() === true) {
this.connectedToDatabase = true;
} else {
this.connectedToDatabase = false;
}
});
}
} |
|
@UZ8VLqifK36Z Thanks for sharing. I'm going to give this a try at some point with my React app. I just discovered this window event, which hopefully works the same. |
|
Hey there, having the same issue here, react native firebase though. I believe it's related. |
|
Does Firebase 11.x make a difference? A new socket implementation was introduced in 10.27.0 This issue may be related to #13877 |
|
I'll try downgrade my version before 10.27.0 and see if that improves things :) Issue does seem related! |
|
Hey @fbarbat. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically. If you have more information that will help us get to the bottom of this, just add a comment! |
[REQUIRED] Step 1: Describe your environment
[REQUIRED] Step 2: Describe the problem
Steps to reproduce: Have the mac going to sleep. After waking up, sometimes (with really low frequency), the Realtime Database client would stay offline indefinitely, even if network conditions are good.
We have an app that uses Realtime Database for implementing Presence in Firestore. https://firebase.google.com/docs/firestore/solutions/presence. If we have the app running and the mac goes to sleep, sometimes, after waking up, the Realtime Database client doesn't reconnect.
We have strong evidence that
database.reference(withPath: ".info/connected").observereported afalsevalue while the mac was sleeping but it never reportedtrueagain, even after waking up and having good network conditions.If we use any operation while Realtime database is offline, like statusReference.setValue to write to the database, the operation never runs the completion callback.
Firestore is able to reconnect properly without problems but Realtime Database client is not working properly.
We saw one case that recovered by itself after a some hours (12 hours after disconnected, 2 hours after the mac woke up). But as we need the Realtime Database to implement Presence, that delay is not acceptable.
Restarting the app makes it work again.
The text was updated successfully, but these errors were encountered: