Looking for some help with a user c module for the unix port

[kdschlosser]

I keep on getting a segfault and for the life of me I don't know why it is happening.

It is occurring on line 565 of the attached file below
sdl_bus.c.txt

if (event->window.windowID != window_id) return 0;
                if (self->window_callback == mp_const_none) return 0;

                mp_obj_t res6[5];
                res6[0] = MP_OBJ_FROM_PTR(self);
                res6[1] = mp_obj_new_int_from_uint(event->type);
                res6[2] = mp_obj_new_int_from_uint(event->window.event);
                res6[3] = mp_obj_new_int(event->window.data1);
                res6[4] = mp_obj_new_int(event->window.data2);

                mp_printf(&mp_plat_print, "SDL_WINDOWEVENT %d\n", event->window.event);

                mp_call_function_n_kw(self->window_callback, 5, 0, res6);        <<========== line 565

                mp_printf(&mp_plat_print, "SDL_WINDOWEVENT\n");

maybe someone can shed some light as to why this is not working. Would a method being passed as the callback instead of a function cause that kind of an issue?

The same kind of code is used to pass arguments to a callback function in other places in Micropython code. I added passing self just to see if that would fix the problem thinking that the problem might be because of using a method as a callback. That is not the case.

Any assistance on this would be really helpful. I am starting to go bald from pulling my hair out.

[stinos]

Would a method being passed as the callback instead of a function cause that kind of an issue?

You're passing self as first argument, is that method effectively a member of self?

[kdschlosser]

I actually figured out what the problem was last night. I am not able to enter Python code from a thread that was not started in Python code. I am sure there is some way it can be done, I couldn't figure out how to do it. The user c module I wrote has things that need to move between Python and C code.

I was using SDL to create 2 threads, one was to handle writing frame buffer to SDL. This is a double buffering design and there is a flag that gets set to inform the graphics framework when a frame buffer has been written. It does this to let the frame work know it can write to the buffer. The second thread was to poll for events. because those threads were started in C code and they are running in C code I am not able to make the jump from C code to Python code by calling a user supplied callback. IDK I have tried scheduling the callback to be called from the main thread and that doesn't seem to work either even after specifically telling MicroPython to run any scheduled tasks. It doesn't work.

I am am sure it can be done, I have not been able to locate any code I could use as an example for doing this.

[stinos]

I actually have applications which run multiple threads, one of them running MicroPython, and IIRC the key thing to get this working was setting MICROPY_STACK_CHECK to 0. For the rest I don't think MicroPython really cares on what thread it runs, as long as you don't simultaneously call MicroPython code (allocator/interpreter) from multiple threads. But if that is guarded by a mutex then it works as well.

[kdschlosser]

MICROPY_STACK_CHECK is not defined for the unix port so that has no bearing because it not being defined is the same as it being a zero.

I get the segfault when mp_call_function_n_kw is called from inside of a thread that was started inside of C code. I am not able to call a python function from a thread that was created in C code. If I create the thread in python code then call a C function that ends up calling the python callback then it works.

I have also tried using mp_sched_schedule to have the function called via the main thread and that still results in a segfault.

I have to mess about with it some more to see if I can get it to work the way that I want. I have changed so many things to get it to work so I may but the thread code back the way I had originally had it and see if it will run. If it does then IDK what the issue would have been. LOL

[stinos]

MICROPY_STACK_CHECK is not defined for the unix port so that has no bearing because it not being defined is the same as it being a zero.

You sure about that? The standard variant has #define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES) and mpconfig.h has #define MICROPY_STACK_CHECK (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES).

Again, threading itself is not really an issue, but there are some things you should take care of.

[kdschlosser]

You have this defined in py/mpconfig.h

#ifndef MICROPY_STACK_CHECK
#define MICROPY_STACK_CHECK (MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES)
#endif

you have this define in py/mpconfig.h

#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES (MICROPY_CONFIG_ROM_LEVEL >= MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES)

there is this in py/mpconfig.h

#define MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES (30)

you have this defined in ports/unix/mpconfigport.h

#ifndef MICROPY_CONFIG_ROM_LEVEL
#define MICROPY_CONFIG_ROM_LEVEL (MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES)
#endif

you have this defined in py/mpconfig.h

#define MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES (10)

This is some nested stuff right here.

This is what you end up with

#define MICROPY_CONFIG_ROM_LEVEL_CORE_FEATURES (10)
#define MICROPY_CONFIG_ROM_LEVEL_EXTRA_FEATURES (30)

#ifndef MICROPY_CONFIG_ROM_LEVEL
#define MICROPY_CONFIG_ROM_LEVEL (10)
#endif

#define MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_EXTRA_FEATURES (10 >= 30)

#ifndef MICROPY_STACK_CHECK
#define MICROPY_STACK_CHECK (10 >= 30)
#endif

I was correct in that MICROPY_STACK_CHECK is not defined in any of the unix port header files. It is defined in py/mpconfig.h. This macros is defined in other ports directly which is why I took it as not being defined for the unix port.

If I have traced through the macros correctly you wopuld be incorrect about the stack check being enabled. From what I am able to tell it is disabled by default.

Now the thing is you mentioned not accessing the interpreter by more than one thread at a time. How would someone go about controlling that? The main thread is going to be continually. The whole purpose to what I am trying to do is to collect events for things like mouse movements and window resizing at the moment they actually take place. I need to be able to push those events into the main the GUI is able to be updated. I know there is mp_sched_schedule and I believe that this will schedule a task to run on the main thread where the supplied function will get called.

I believe where I am running into an issue with the segfault is because of the call to mp_sched_schedule and it only allowing a single parameter to be passed to the supplied function. Because only a single argument is allowed to be passed I have to create a list, dict or a tuple to hold the data that I want to relay to the main thread. So that is what I do. The issue is that list, dict or tuple gets created inside of a c function so it is allocated on the stack and as soon as the function exits the pointer that was just passed to the schedule function is no longer valid. The scheduler doesn't make a copy of the data that has been passed to it. I have tried using mp_sched_run_pending directly after mp_sched_schedule and that doesn't keep the pointer valid because the task doesn't get run even tho it is pending.

maybe I need to have a while loop to check the number of pending tasks to ensure that it is a zero after I schedule the task. I would need to use mp_sched_num_pending to do this.

I still haven't figured out why the stack comes into play with this. I need to get the data pushed into the main thread and I am unsure if calling mp_sched_schedule and mp_sched_run_pending does this. I also would have imagined that the GIL comes into play here but I have not seen any code surrounding locking the GIL when running a scheduled task.

[kdschlosser]

I changed the code and did as you said and the segfault still happens. It seems to be happening when I create the list to store the data that is to be passed to the scheduler. I am going to attempt to just have the scheduler call the callback passing None as the argument. lets see if that will work.