Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3.16.3 FIX/CTCI login on ibgateway 10.22 #207

Closed
yujinkai1978 opened this issue Apr 12, 2023 · 20 comments
Closed

3.16.3 FIX/CTCI login on ibgateway 10.22 #207

yujinkai1978 opened this issue Apr 12, 2023 · 20 comments

Comments

@yujinkai1978
Copy link

I tried with FIX/CTCI login and passed the 2FA check, but the IBC and ibgateway still exit after 1 minute, as below log:

2023-04-11 21:14:38:587 IBC: Detected frame entitled: Loading...; event=Opened
2023-04-11 21:14:38:588 IBC: Found Gateway main window
2023-04-11 21:14:41:882 IBC: Detected dialog entitled: Second Factor Authentication; event=Opened
2023-04-11 21:14:41:883 IBC: Second Factor Authentication initiated
2023-04-11 21:14:41:890 IBC: Detected frame entitled: Loading...; event=Lost focus
2023-04-11 21:14:41:890 IBC: Detected frame entitled: Loading...; event=Deactivated
2023-04-11 21:14:41:891 IBC: Detected dialog entitled: Second Factor Authentication; event=Activated
2023-04-11 21:14:41:892 IBC: Detected dialog entitled: Second Factor Authentication; event=Focused
2023-04-11 21:14:50:307 IBC: Detected dialog entitled: Second Factor Authentication; event=Lost focus
2023-04-11 21:14:50:307 IBC: Detected dialog entitled: Second Factor Authentication; event=Deactivated
2023-04-11 21:14:50:312 IBC: Detected dialog entitled: Second Factor Authentication; event=Closed
2023-04-11 21:14:50:313 IBC: Duration since login: 14 seconds
2023-04-11 21:14:50:314 IBC: If login has not completed, IBC will exit in 40 seconds
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-11 21:15:30:315 IBC: IBC closing because login has not completed after Second Factor Authentication
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-11 21:15:30:315 IBC: Exiting with exit code=1111
IBC returned exit status 87
@yujinkai1978
Copy link
Author

I found there's same issue in early version 3.12 + ibgateway 10.21, but it can be ignored by setting ExitAfterSecondFactorAuthenticationTimeout=no.

But in version 3.16.3 this setting is deprecated, below please find the logs from IBC 3.12

2023-04-09 02:32:29:599 IBC: Detected dialog entitled: Second Factor Authentication; event=Deactivated
2023-04-09 02:32:29:599 IBC: Second Factor Authentication dialog event: Deactivated
2023-04-09 02:32:29:599 IBC: Detected dialog entitled: Second Factor Authentication; event=Closed
2023-04-09 02:32:29:599 IBC: Second Factor Authentication dialog event: Closed
2023-04-09 02:32:29:599 IBC: Second Factor Authentication dialog closed
2023-04-09 02:32:29:599 IBC: ExitAfterSecondFactorAuthenticationTimeout is set to 'no'

@rlktradewright
Copy link
Member

Quoting small extracts from the logfiles isn't very helpful. Please just attach the whole logfile. It contains a mass of information that can be very helpful in diagnosing problems.

@yujinkai1978
Copy link
Author

yujinkai1978 commented Apr 14, 2023 via email

@rlktradewright
Copy link
Member

I asked you to ATTACH the logfile, and you've QUOTED three logfiles. I can't work with that.

Please do as I ask and ATTACH the files. When you reply, there is a link below where you're typing which says 'Attach files by dragging & dropping, selecting or pasting them'. Click on that and select the files.

@yujinkai1978
Copy link
Author

Sorry I replied to the email with attachment files and don't know it convert to text in this post. please check the attached file again. sorry for inconvenience.

Attached please find three log files, the IB API is success, but FIX only
or FIX+IB failed after 2FA check. all the settings and versions are same
except the "FIX=yes" and "FIXLoginId/FIXPassword" parts.

Please take a look, thank you.

FIX and IB - failed - ibc-3.16.3_GATEWAY-1022_Friday.txt
IB API only - success - ibc-3.16.3_GATEWAY-1022_Friday.txt
FIX only - failed - ibc-3.16.3_GATEWAY-1022_Friday.txt

@rlktradewright
Copy link
Member

Ok, thanks for that. The logs are very helpful.

I'll look into this but I may have to ask you to do something to increase the level of detail in the logs. First I'll have to run some experiments myself, which I'll try to do this evening.

@yujinkai1978
Copy link
Author

It seems to check window's title to determine whether the login success, but FIX/CTCI's window is much simpler than IB-API, for example it no setting dialog and the window's title always showing "Loading...", as attached screenshot.

1

@rlktradewright
Copy link
Member

This is all a bit tricky because I don't have a FIX-capable account, so I can't test it fully.

However, from the experiments I've done this evening, it's possible that it it just needs a bit more time to get the FIX gateway stuff loaded and running (hence the 'LOADING' in the window title).

So could you please change the SecondFactorAuthenticationExitInterval setting in config.ini to a higher value: the default is 40 seconds, so perhaps set it to 90 seconds. This gives Gateway more time to get everything loaded before it decides that it's not going to happen and trying again.

If it still doesn't work with 90 seconds, it probably means that Gateway just behaves differently when FIX is involved and I'll have to try to work out a way to determine exactly what I need to detect. In which case attach the logfile again so I can see if anything else is going on in the extra time.

@yujinkai1978
Copy link
Author

Hi, according to my observation, the "LOADING..." title will keep during whole session if login successfully with FIX/CTCI even after one week, it may be a minor bug from ibgateway.

I just tried some scenario as below:

  1. SecondFactorAuthenticationExitInterval=180, it doesn't work as same problem with previously one, attached please find the log file, and last part of logs as below:

2023-04-15 23:03:59:494 IBC: Detected dialog entitled: Second Factor Authentication; event=Activated
2023-04-15 23:03:59:496 IBC: Detected dialog entitled: Second Factor Authentication; event=Focused
2023-04-15 23:04:06:836 IBC: Detected dialog entitled: Second Factor Authentication; event=Lost focus
2023-04-15 23:04:06:836 IBC: Detected dialog entitled: Second Factor Authentication; event=Deactivated
2023-04-15 23:04:06:837 IBC: Detected dialog entitled: Second Factor Authentication; event=Closed
2023-04-15 23:04:06:837 IBC: Duration since login: 11 seconds
2023-04-15 23:04:06:837 IBC: If login has not completed, IBC will exit in 180 seconds
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-15 23:07:06:840 IBC: IBC closing because login has not completed after Second Factor Authentication
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2023-04-15 23:07:06:840 IBC: Exiting with exit code=1111
IBC returned exit status 87

  1. I test not respond for the 2FA of FIX/CTCI for a long time, the IBC and ibgateway keep waiting for 2FA dialog, and the logs stuck as below:
    2023-04-15 23:13:42:801 IBC: Detected dialog entitled: Second Factor Authentication; event=Activated
    2023-04-15 23:13:42:878 IBC: Detected dialog entitled: Second Factor Authentication; event=Focused
    2023-04-15 23:13:42:879 IBC: Detected dialog entitled: Second Factor Authentication; event=Opened
    2023-04-15 23:13:42:880 IBC: Second Factor Authentication initiated
    (Keep waiting for a long time and no more logs...)

But if I tried no respond for the 2FA for IB-API login, it will closing 2FA dialog after 180 seconds and return to login dialog, the log as below:

2023-04-15 23:25:59:675 IBC: Detected dialog entitled: Second Factor Authentication; event=Activated
2023-04-15 23:25:59:676 IBC: Detected dialog entitled: Second Factor Authentication; event=Focused
2023-04-15 23:29:02:175 IBC: Detected dialog entitled: Second Factor Authentication; event=Lost focus
2023-04-15 23:29:02:175 IBC: Detected dialog entitled: Second Factor Authentication; event=Deactivated
2023-04-15 23:29:02:175 IBC: Detected frame entitled: IB Gateway; event=Activated
2023-04-15 23:29:02:179 IBC: Detected frame entitled: IB Gateway; event=Focused
2023-04-15 23:29:02:180 IBC: Detected dialog entitled: Second Factor Authentication; event=Closed
2023-04-15 23:29:02:180 IBC: Duration since login: 187 seconds
2023-04-15 23:29:02:180 IBC: Re-login after second factor authentication timeout not required

  1. If I cancel the 2FA dialog of FIX/CTCI, it will exit immediately. If I cancel the 2FA dialog of IB-API, it will return to login dialog.

as a conclusion, for the FIX/CTCI login, may I suggest using the SecondFactorAuthenticationExitInterval timer after 2FA event=Activated, and believe it login success if got 2FA event=Closed before timeout, or exit/restart if no more event after timeout.

ibc-3.16.3_GATEWAY-1022_Saturday.txt

@rlktradewright
Copy link
Member

Can you please tell me what happens if you set SecondFactorAuthenticationExitInterval to something large, say 3600 seconds.

Is Gateway then actually usable, including for FIX orders, after the 2FA completes, for the next hour?

In other words, is everything working apart from it shutting down after SecondFactorAuthenticationExitInterval seconds?

@yujinkai1978
Copy link
Author

yujinkai1978 commented Apr 17, 2023 via email

@rlktradewright
Copy link
Member

Ok, thanks.

What I'll do is give you a beta version that behaves slightly differently when Gateway is using FIX.

In 'normal' Gateway without FIX, the close event for the 2FA dialog doesn't mean that Gateway is ready to go: after this event, Gateway displays a splash screen for some time, and only when that splash screen closes are the Gateway menus operable. So IBC normally takes the close event for the splash screen to mean that it can now do things that involve the menus (things like setting autorestarttime or autologofftime, but quite a number of other things too).

But when FIX is involved, there is now nothing that tells me that the menus are operable. There seem to be no other events after 2FA close, and even the window title is not helpful, since you say it just displays "LOADING" for as long as Gateway is running.

So I'll give you a version that, when running FIX, assumes that the menus are operable once 2FA close event has occurred. And I'll have to get you to test all the things that might require IBC to access the Gateway menus to see whether they work or not at that point. If they don't then I'm going to have to write some 'probing' code that periodically tries to use the menus and defers any menu-using stuff until that has succeeded.

I might be able to get this new version to you later today, but tomorrow is more likely.

@yujinkai1978
Copy link
Author

yujinkai1978 commented Apr 17, 2023 via email

@rlktradewright
Copy link
Member

Thanks for that info. It explains a lot.

Have IB given you details of the jts.ini settings? Some of them, like TrustedIPs, are obvious, but if you have complete info that would be helpful. I can make IBC set these jts.ini settings automatically.

This update is going to take longer than I hoped at first. So maybe not today (I also have other things to do...).

@yujinkai1978
Copy link
Author

yujinkai1978 commented Apr 18, 2023 via email

@rlktradewright
Copy link
Member

That's what I thought, but it was worth checking whether there are any other settings that could be changed.

So the new version will have a new setting in config.ini called TrustedTwsApiClientIPs, which is a list of IP addresses that will be used to set the TrustedIPs in jts.ini. And the existing OverrideTwsApiPort setting will be used to set LocalServerPort in jts.ini.

Of course you can set these manually in jts.ini rather than config.ini, but there are situations where that won't be a viable option. The config.ini settings take precedence over existing values in jts.ini.

@rlktradewright
Copy link
Member

Here is a new version 3.16.4-beta.1 that I hope will fix this issue.

The only files that have changed are IBC.jar, version and config.ini (new setting TrustedTwsApiClientIPs).

So just extract the IBC.jar and version files to overwrite your existing ones. And edit your config.ini to include the new section.

Once you tell me this is working properly, I'll merge the changes I've made into the master branch and publish a new release.

IBCLinux-3.16.4-beta.1.zip

@yujinkai1978
Copy link
Author

Thank you, I tried the 3.16.4 just now and it seems works fine, and the FIX protocol works fine too. attached please find the log.

But I didn't test TrustedTwsApiClientIPs param because I don't have another computer to access it.

ibc-3.16.4-beta.1_GATEWAY-1022_Thursday.txt

@rlktradewright
Copy link
Member

Good, that's great. No problem about the TrustedTwsApiClientIPs setting: I know that IBC sets this correctly in jts.ini from my testing.

I'll include this in the next release, which will probably be this weekend.

rlktradewright added a commit that referenced this issue May 10, 2023
@rlktradewright
Make FIX handling work properly (issue #207)
909c868
@skister
Copy link

skister commented May 16, 2023

FYI, I could not log in to the IB gateway 10.22 (non-FIX) with IBC 3.16.2
I got an error "IBC closing after TWS/Gateway failed to display login dialog"
Even though the login dialog screen was open.
I tried 3.16.4-beta.1 and it works. I can upload the log file if that is of interest, but it is fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@skister @rlktradewright @yujinkai1978