Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FT8 demod for monitoring #1561

Closed
f4exb opened this issue Jan 7, 2023 · 7 comments
Closed

FT8 demod for monitoring #1561

f4exb opened this issue Jan 7, 2023 · 7 comments
Assignees
Labels
Milestone

Comments

@f4exb
Copy link
Owner

f4exb commented Jan 7, 2023

I've been playing with FT8 reception recently connecting the audio output of the SSB plugin to the WSJT-X program which is an easy set up. Then I could analyze the CALLxxx.TXT file(s) to get statistics and a visualization of the received locator squares using a Python Jupyter notebook.

However there are some shortcomings when you would like to go a bit further...

  • It is not easy to do multi-band monitoring while some set ups like RedPitaya with Metis plugin and Pavel's firmware permits to have 8 receivers in parallel with a single device and antenna. SDRangel also allows multi reception of heterogeneous receivers or multiple instances of KiwiSDR plugin for example
  • There is no connection between the base frequency recorded and the actual received frequency which does not make it easy to switch bands. I have resorted to mute audio, change band in SDRangel, change band in WSJT-X and put back audio so that the recorded frequency is consistent.
  • Although some people have designed smart scripts it is not easy to work headless for example let a Raspberry-Pi run for hours or days recording traffic. It would be much easier if FT8 support was built in SDRangel so SDRangel server flavor could be used for that purpose.

Having a FT8 demod plugin would also make it possible to collect data from multiple receivers in a specific FT8 feature plugin (not covered in this issue) so as to be able to collate results in a single file with the correct frequencies. This would be a bit similar to what the AIS demod / AIS feature couple does. In addition this opens the possibility to display locator squares live on a map.

At least for now the plugin would not be intended for real traffic. There is a significant delay induced with data buffering that makes it difficult to handle Rx and Tx. Moreover you have a delay in each way. It is easy to overcome the Rx delay if you do Rx only because you can start the 15s recording arbitrarily but for Tx-ing with only a reasonable delay you would need to anticipate. This leaves very little time for decision even if since the first CQ all the information is available. You still need to know if the answer to the CQ was received or not.

There is a very interesting port of the original Fortran WSJT-X code in c++ here: https://github.com/rtmrtmrtmrtm/ft8mon Moreover it seems that the core process resides in only one FT8 class which would facilitate integration. However it uses standard threads so for portability it may be necessary to rewrite parts of the code using QThreads. Also this seems to work as well or better than the original WSJT-X code.

@f4exb f4exb added the feature label Jan 7, 2023
@f4exb f4exb self-assigned this Jan 8, 2023
@srcejon
Copy link
Collaborator

srcejon commented Jan 23, 2023

Might not be ready yet, but couldn't resist giving it a try :)

On Ubuntu 22.04 with 2m preset, I get:

ft8/unpack0.cpp:53: uint64_t FT8::un64(int*, int, int): Assertion `len < (int)sizeof(x) * 8 && start >= 0 && start + len <= 63' failed.

#7 0x00007fff6ec299b4 in FT8::un64(int*, int, int) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#8 0x00007fff6ec2625c in FT8::Packing::unpack(int*, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#9 0x00007fff6ec8bc02 in FT8DemodWorker::FT8Callback::hcb(int*, float, float, char const*, float, int, int) () at /opt/install/sdrangel/lib/sdrangel/plugins/libdemodft8.so
#10 0x00007fff6ec0bee4 in FT8::FT8::try_decode(std::vector<float, std::allocator > const&, float*, float, int, float, float, int, char const*, std::vector<std::vector<std::complex, std::allocator<std::complex > >, std::allocator<std::vector<std::complex, std::allocator<std::complex > > > > const&) ()
at /opt/install/sdrangel/lib/sdrangel/libft8.so
#11 0x00007fff6ec0a90b in FT8::FT8::one_iter1(std::vector<float, std::allocator > const&, int, float, float, float) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#12 0x00007fff6ec094ef in FT8::FT8::one_iter(std::vector<float, std::allocator > const&, int, float) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#13 0x00007fff6ec09336 in FT8::FT8::one(std::vector<std::complex, std::allocator<std::complex > > const&, int, float, int) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#14 0x00007fff6ec02589 in FT8::FT8::go(int) () at /opt/install/sdrangel/lib/sdrangel/libft8.so
#15 0x00007fff6ec00052 in FT8::FT8::start_work() () at /opt/install/sdrangel/lib/sdrangel/libft8.so

If I disable the assertion, I can receive packets though. Nice!

@srcejon
Copy link
Collaborator

srcejon commented Jan 23, 2023

Just saw this assertion:

/opt/build/srcejon/sdrangel/ft8/fft.cpp:97: FT8::FFTEngine::Plan* FT8::FFTEngine::get_plan(int, const char*): Assertion `p->fwd_' failed.

Although has only happened once (whereas the above is pretty frequent)

@f4exb
Copy link
Owner Author

f4exb commented Jan 23, 2023

I see quite a few issues with "their" implementation of FFT. I might try to use "my" FFT factory instead (with some adaptations). Moreover it does not seem to use a wisdom file which can result in a very slow startup on Raspberry Pi (not tried yet).

And yes... assertions were OK for a standalone program but not suitable here. I might have to do some cleanup in that sense.

@N5BRG
Copy link

N5BRG commented Jan 23, 2023 via email

@f4exb
Copy link
Owner Author

f4exb commented Jan 23, 2023

The turn around time is critical for FT8 so keep this in mind when moving forward.

Well... hence the "for monitoring" in the title. For now it is not intended for traffic. In order to do this you need a modulator and a feature to do the QSO logic to control the Rx/Tx sequence. This may be done in a later step. For now with a decoding time budget of 0.5s results are comparable if not better than the with the original WSJT-X code thanks to Robert Morris, AB1HL code .

0.5s is not bad but the time budget in the protocol is very tight even without considering the buffering that will occur both in the Rx and the Tx path:

  • 174 message bits including FEC => 58 symbols
  • 21 sync symbols
  • Total: 79 symbols of 0.16s length = 12.64s
  • There is a 0.5s delay from the dot clock before actual transmission.

This leaves 2.36s of turnaround time but you have to accommodate some slack for late messages. Not much time even for human decision so if one day traffic mode is implemented it will be only automatic. For now let's observe how it goes in receiving mode with 15s buffers. For real traffic the decoding should start ahead of time with a shorter portion of real samples padding the end of the buffer with zeros.

I tried to get the FT8 maintainers to allow a bit more turnaround time and they said no and to go find a different rig

As demonstrated above you cannot really move the walls with the FT8 protocol so this probably explains.

@f4exb
Copy link
Owner Author

f4exb commented Jan 31, 2023

Actually on the transmission side (TBD) it is possible to benefit from the first sync period which is a fixed sequence (Costas sequence) thus this adds an extra slack of 7 symbols that is 1.12 seconds.

@f4exb f4exb added this to the v7.9.0 milestone Feb 1, 2023
@f4exb
Copy link
Owner Author

f4exb commented Feb 1, 2023

Implemented in v7.9.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants