[DataImporter] Two fidi docker container instances for two different users cross-loading data

[diamant-x]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem
• ... it's definitely me, not you

Description

I have one Firefly3 setup with two different users, i want each of them to have an automatic fidi to import their independent transactions.

I have setup already Fidi for one of the users, with its own Personal Access Token.
I have setup Fidi with a unique set of Nordigen app keys.
Nordigen import works fine and when creating the config, the dropdown of firefly accounts matches those available for user1.

I have deployed a new FiDi container, with a separate config folder and instance.
I have setup FiDi with a new Personal Access Token under user2. The tokens are clearly different between those of user1 and user2.
I have setup FiDi2 with new Nordigen app keys from the same organization, so different app keys from user1.
When creating a new nordigen config flow in the new container for user2, the lsit of available accounts are the same as those for user1, and those of user2 are not available at all.
user2 uses the same bank and bankaccount
user2 is a FF3 admin account.

Thanks!

Expected behaviour

FiDi from user1 sees its own firefly accounts only
FiDi from user2 sees its own firefly accounts only.

Debug information

Can't find any specific mention about the connection to firefly3 from fidi, all the logs are related to the (succesful) connection to nordigen.

Steps to reproduce

1. setup FF3 with 2 users
2. seutp different asset accounts for each user
3. setup one fidi for user1 with personal acces token
4. confirm fidi for user1 shows accounts for user1 only
5. seutp a second fidi for user 2 with its personal access token
6. check the available account to be mapped
   error occurs, accounts from user 1 appear.

Additional info

No response

[JC5]

The (encrypted) access token is cached in a cookie, and could also be present in the FIREFLY_III_ACCESS_TOKEN environment variable. If the two data importers share the same (sub)domain, the information from one FIDI instance can leak to the other one.

[JC5]

Can't find any specific mention about the connection to firefly3 from fidi, all the logs are related to the (succesful) connection to nordigen.

Look for stuff like this:

[2022-04-13 14:42:17] local.DEBUG: Now at App\Http\Controllers\TokenController::index  
[2022-04-13 14:42:17] local.DEBUG: No access token in hasAccessToken(), will return config variable.  
[2022-04-13 14:42:17] local.DEBUG: No client id in hasClientId(), will return config variable.  
[2022-04-13 14:42:17] local.DEBUG: No base url in getBaseUrl(), will return config variable.  
[2022-04-13 14:42:17] local.DEBUG: No vanity url in getVanityUrl(), will return config variable.  
[2022-04-13 14:42:17] local.INFO: The following configuration information was found:  
[2022-04-13 14:42:17] local.INFO: Personal Access Token: "" (limited to 25 chars if present)  
[2022-04-13 14:42:17] local.INFO: Client ID            : "0"  
[2022-04-13 14:42:17] local.INFO: Base URL             : "http://firefly.jc.home"  
[2022-04-13 14:42:17] local.INFO: Vanity URL           : "http://firefly.jc.home"  
[2022-04-13 14:42:33] local.DEBUG: Now at App\Http\Controllers\TokenController::submitClientId  
[2022-04-13 14:42:33] local.DEBUG: Submitted data:  {"client_id":"4","base_url":"https://firefly.jc.home/"} 
[2022-04-13 14:42:33] local.DEBUG: Base URL is "http://firefly.jc.home"  
[2022-04-13 14:42:33] local.DEBUG: Vanity URL is now "http://firefly.jc.home"  
[2022-04-13 14:42:33] local.DEBUG: Base URL is now "https://firefly.jc.home/"  
[2022-04-13 14:42:33] local.DEBUG: Now in App\Http\Controllers\TokenController::redirectForPermission(request, "https://firefly.jc.home", "http://firefly.jc.home", 4)  
[2022-04-13 14:42:33] local.DEBUG: Query parameters are {"client_id":4,"redirect_uri":"https://firefly-data.jc.home/callback","response_type":"code","scope":"","state":"UPxnrSwGEp4lAOspwLpRcw1E2oJY7FyAP2QNITky","code_challenge":"d9kcEJhV6k_vwSOYorDmfrvlH5kexzmaB_-nuVtOAR4","code_challenge_method":"S256"} 
[2022-04-13 14:42:33] local.DEBUG: Now redirecting to "http://firefly.jc.home/oauth/authorize?" (params omitted)  
[2022-04-13 14:42:34] local.DEBUG: Now at App\Http\Controllers\TokenController::callback  
[2022-04-13 14:42:34] local.DEBUG: State is valid!  
[2022-04-13 14:42:34] local.DEBUG: Params for access token {"form_params":{"grant_type":"authorization_code","client_id":4,"redirect_uri":"https://firefly-data.jc.home/callback","code_verifier":"1m...e"}} 
[2022-04-13 14:42:34] local.DEBUG: Will contact "https://firefly.jc.home/oauth/token" for a token.  
[2022-04-13 14:42:36] local.DEBUG: Response {"token_type":"Bearer","expires_in":1209598,"access_token":"eyJ0eXAi...b902169c65e79078c0869"} 
[2022-04-13 14:42:36] local.DEBUG: Return redirect with cookies to "https://firefly-data.jc.home"  

[diamant-x]

The (encrypted) access token is cached in a cookie, and could also be present in the FIREFLY_III_ACCESS_TOKEN environment variable. If the two data importers share the same (sub)domain, the information from one FIDI instance can leak to the other one.

Both FiDis are deployed in the same docker host, so they share the same subdomain, one is working on port 3474 and the other in port 3475. I'm not sure to understand how that would be an issue?

This is the log i'm getting. I do see that the beginning of the personal access token is identical for both users, not sure if that might clash?

[2022-04-13 14:58:01] production.DEBUG: Now at App\Http\Controllers\IndexController::reset  
[2022-04-13 14:58:01] production.INFO: The following configuration information was found:  
[2022-04-13 14:58:01] production.INFO: Personal Access Token: "<redacted-token>" (limited to 25 chars if present)  
[2022-04-13 14:58:01] production.INFO: Client ID            : "0"  
[2022-04-13 14:58:01] production.INFO: Base URL             : "http://mydomain:3473"  
[2022-04-13 14:58:01] production.INFO: Vanity URL           : "http://mydomain:3473"  
[2022-04-13 14:58:01] production.DEBUG: Found personal access token + URL "http://mydomain:3473" in config, set cookie and return to index.  
192.168.10.26 - - [13/Apr/2022:14:58:01 +0200] "GET /token HTTP/1.1" 302 4651 "http://mydomain:3475/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36 Edg/100.0.1185.39"
[2022-04-13 14:58:01] production.DEBUG: Now in App\Http\Controllers\IndexController::index  
[2022-04-13 14:58:01] production.DEBUG: App\Services\Shared\Authentication\SecretManager::hasValidSecrets  

I see that my logs don't include the 'Response' from the FF3 like yours. any way to force that to appear?

[diamant-x]

Thanks for the followup. I have since been unable to get the auto-import to work anymore with just one FiDi anyways. Manually doing the steps works fine. When being triggered on its own through cron (within docker), it fails under Authentication failure. Tried with the vanity set to my subdomain and different IPs attemps, but they all fail the same way. I think when running through cron in docker, Communications go through the internal docker network and then the callback url in FF3 fails to match the previously authorized one.

[JC5]

I was able to get a single instance to work as if it were two, by visiting one in a private browser window. For this experiment, I used a reverse proxy to give them different names but I'm convinced I can do the same with the same host name.

[diamant-x]

Thanks for the test. So just to understand (correct me if i'm wrong).

1 FF3 instance.
1 FiDi instance.
2 reverse proxy hostname (different one to another) pointing to the fidi instance and port.

in the private browser, you had to a) login to FF3 instance with second account and b) authorize the FiDi instance through the second proxy hostname as callback url?

What authorization method did you use?

Thanks!

[JC5]

in the private browser, you had to a) login to FF3 instance with second account and b) authorize the FiDi instance through the second proxy hostname as callback url?

Yes. Both needed authentication separately. I set no authentication variables at all. Each user had to input their client ID and the Firefly III url.

[diamant-x]

I'm just coming back to this, as I have now a proper proxy setup I can use to try subdomains and such.
However, I'm realizing that by removing the authentication variables, then FiDi wouldn't be able to auto-import to a 'default' account.

• I guess ideally the auto_import config files could/should store authentication variables for each user so that a single FiDi instance could work and autoimport different configs for different user profiles.

Or at least a way for FiDi to just, 'ignore' any authentication variables so that it can run as an 'incognito mode' and be able to use it for sporadical loads to a secondary account.

Maybe I misunderstood something but something like that would be great to properly enable Firefly3 for multi-users.

EDIT: Or maybe to be able for the client ID to be defined by the http headers on the reverse proxy, so that I could set the subdomains to add the client ID which they should use and use those.... Although I guess that wouldn't help for auto-import as it would run without any http query.

EDIT2: Just to add to the community, I was able to do it just as mentioned in the last answer. No authentication variables. 2 different subdomains pointing to the same container. Opening one FF3 session in incognito (or different browser) and one of the subdomains as well. Setting the client id as not-confidential and with the chosen subdomain as callback.

[diamant-x]

Another approach I saw in the documentation was
You can generate your own client ID on your Profile page (under OAuth). This is the ID you need when you want to share FIDI with multiple people, or when you want to allow others to use the same instance of FIDI.

• How would the architecture work for this scenario to happen?
• Is there a way to automate the cron jobs for different users sharing the same FiDi? Or does this documentation description only apply to manual web-browser-based imports?

Thanks.

[JC5]

If it's on the command line, you can take the two tokens and run the jobs.