[rcore] Porting raylib to iOS and implement rcore_ios.c

[blueloveTH]

I would like to take the initial step to add iOS support for raylib.
I am going to add rcore_ios.c and implement an iOS's platform layer.
I will update this pr on my progress.

Current Demo (iPhone 8)

RPReplay_Final1711259172.MP4

Steps

• Create an empty iOS game project in XCode
• Compile ANGLE and get libEGL.xcframework and libGLESv2.xcframework
• Fill rcore_ios.c and compile raylib
• Link all of the above with an example source file

Functions

• void PollInputEvents(void)
• int InitPlatform(void)
• void ClosePlatform(void)

Prebuilt ANGLE libraries for iOS

Users need to add the following ANGLE libraries into Xcode.
libEGL.xcframework.zip
libGLESv2.xcframework.zip

References

• will it support iOS? #2681
• Linking raylib against ANGLE on MacOS #2998
• https://medium.com/@grplyler/building-and-linking-googles-angle-with-raylib-on-macos-67b07cd380a3
• [build] Try to support iOS platform #330
• https://github.com/levinli303/ANGLESwiftUI/tree/main

[blueloveTH]

Hi @raysan5 . I need your help. iOS always use a UIApplicationMain function to start the game. UIApplicationMain is an extern function which is hidden for me. It runs forever.

int main(int argc, char * argv[]) {
    return UIApplicationMain(argc, argv, nil, NSStringFromClass([AppDelegate class]));
}

It seems does not expose detailed control to allow us to achieve the following:

int main() {
  library_init();
  // game init code here
  while(we_have_not_quit_the_game) {
    library_message_loop();
    library_init_render();
    // render stuff
    library_end_render();
    // update game state
  }
  library_shutdown();
}

See https://stackoverflow.com/questions/2187684/how-can-i-remove-uiapplicationmain-from-an-iphone-application.

What we can do is registering render callbacks via CADisplayLink. I cannot find a way to give users full control of the game loop.

I can try a callback-based api like this. However, this requires a minor change of existing raylib projects if they want to run on iOS.

extern void ios_ready();
extern void ios_update();
extern void ios_destroy();

Old project should adapt their main function into this in order to be cross-platform between iOS and other platforms:

#ifndef PLATFORM_IOS
int main(int argc, char** argv){
    // store as global variables if you need
    ios_ready();
    while(!WindowShouldClose()) ios_update();
    ios_destroy();
    return 0;
}
#endif

[blueloveTH]

Question: I see there is a feature: InitWindow(0, 0, "title") will set as full screen. How is it achieved?

Should I override CORE.Window.screen.width and CORE.Window.screen.height in InitPlatform with device width and height?

[blueloveTH]

Now rcore_ios.c has basic functionalities including graphics, audio and touch input. I've marked this PR as ready for review. I will continue to work for more details and wait for raysan5's comment.

[neondeex]

Interesting.. will colaborate soon

[zeroxer]

Any new progress？

[jonpittock]

also interested :D?

[blueloveTH]

I am a bit busy recently :/ It works on my branch. I may sync it with the latest raylib if I have time.

[celioreyes]

@blueloveTH do you have local changes on your branch?

I have this running on my mac/iphone now and planning on allocating some free time on this as I also need raylib ios for a project I'm working on.

Anything you need for collaboration?

Do you have your Makefile/CMake changes to compile Raylib to ios local?

[lzralbu]

To use raylib with Emscripten it's already necessary to give up the main loop control(unless one's willing to add ASYNCIFY which has a nonnegligible overhead).

So I like the idea of a callback based design. Maybe something along the lines of SDL3 main callback functions.

Here's how they handle the quirks of each platform, including iOS.

[leftbones]

Definitely interested in this. This is the dream, being able to make cross platform mobile apps and games with raylib would be amazing.

[raysan5]

@leftbones Unfortunately the code structure required for this iOS implementation is hardly compatible with raylib structure, not sure if it's possible to use another structure, more similar to Android one.

[blueloveTH]

There may be internal functions that Apple does not expose allowing us to take full control of the main loop.

[jackwlee01]

Perhaps the newly merged SDL_GPU as a backend would be a better approach for targeting iOS, and consoles!!!

[calvin2021y]

@leftbones Unfortunately the code structure required for this iOS implementation is hardly compatible with raylib structure, not sure if it's possible to use another structure, more similar to Android one.

sokol seems support IOS event loop, can we learn some idea from them....

[martinfinke]

It seems does not expose detailed control to allow us to achieve the following:

int main() {
  library_init();
  // game init code here
  while(we_have_not_quit_the_game) {
  ...
}

sokol seems support IOS event loop, can we learn some idea from them....

I'm totally new to this discussion (and raylib), but familiar with iOS and Apple APIs in general. So maybe the following totally misses the problem, but I thought I'd share it 🙂

Sokol subclasses MTKView when using Metal, or GLKView when using OpenGL ES. Then creates this view and sets the FPS. This is done inside application:didFinishLaunchingWithOptions:, and I would recommend doing it there on iOS. So in main() we still call UIApplicationMain as is. Then, iOS will call the view's drawRect: method at that FPS. This function is basically one iteration of the "game loop". So in there, we would do our updates, and call raylib's BeginDrawing() and EndDrawing() once (not writing our own blocking loop).

iOS touch events arrive in touchesBegan/Moved/Ended/Cancelled. From there, we would need to "feed them" into raylib. Maybe a queue that raylib polls every frame, when we call EndDrawing().

[martinfinke]

I've now tested and looked over the rcore_ios.c from @blueloveTH 👍
UIApplicationMain does all kinds of setup (partial disassembly at the end of that article). Anything hacky (setjmp, 0.1 millisecond timer), while certainly clever, I don't see it viable in practice. If I use raylib for an iOS project, I'd much rather accept some entry point dictated by iOS, than something that might break with an iOS update, or get rejected later on. Every new app version has to go through App Store review again. Even if something passes once, a future app version may get rejected.

I think the only way to support iOS is to do it via UIApplicationMain and then getting called back for rendering, touch events, etc.

[raysan5]

@martinfinke thank you very much for looking into this new iOS platform. All raylib platform backends, including Android, allow building the examples with no change for target platform.

Is it possible same behaviour for iOS?

[martinfinke]

Hi @raysan5, thank you for creating raylib!
I'm afraid I don't see a way to do the while (!WindowShouldClose()) style API on iOS. Maybe someone with more internal knowledge will come up with a way, but anything hacky will just cause Apple to reject the app.
I agree that the while (!WindowShouldClose()) is a nice style, but it's not how iOS expects it.

Just for the sake of sharing information: I came across a StackOverflow discussion, which is about GLFW on macOS (not iOS). It creates the app manually, then calls run, which would normally block until exit. However, they override applicationDidFinishLaunching:, and in there they "stop" the app, which exits the app's own blocking event loop.
So now they have the app booted up, but without blocking, which is what we're after on iOS. But the app isn't receiving events anymore, so they poll manually.

Problems with this approach on iOS: To my knowledge,

• There's no way to stop the app's event loop, like the stop: method GLFW uses on macOS. Other than calling exit(), which terminates the entire process and looks like a crash to the user.
• There's no API to poll events manually, like the one GLFW uses on macOS.

I also checked GLFM, but it uses the "official" UIApplicationMain approach and you implement glfmMain, but not your own main with a while loop like the raylib examples.

[tommylau-exe]

SDL has prior art in this domain which, in addition to callback APIs (as previously discussed), can use a macro to transform main into SDL_main in the preprocessor phase. Broader explanation here. This allows SDL to define it's own true main which gives control to iOS via UIApplicationMain to perform the platform-specific initialization, and then call SDL_main once iOS delegates control back over to the application.

It involves a bit of macro trickery, but perhaps a similar solution could be implemented in Raylib so that existing examples are able to build for iOS without platform-specific changes as @raysan5 mentioned.

[gilzoide]

Hey folks, I'd love seeing raylib apps on iOS as well.

About the main problem, here's an idea: we could build the user's app into a dynamic framework (a .framework with a dynamically linked library inside). We can then have our iOS main function calling UIApplicationMain as usual, making all iOS-specific setups as usual, and at the right moment we dlopen the framework and run its main function, which is the user's app entrypoint. Each executable can have a single main function, but since the main executable is a separate file from the dynamic library, both can have a main function!

I made a quick test here in XCode with a dummy iOS app and it does work! (yes, my framework is called "actualframework" and I mispelled "framwork" in the code =P)

About the event loop, we could do what raylib Android does: run the user's main function in a separate thread, handle events and rendering the same way current implementation does, and synchronize both using mutexes/semaphores. WindowShouldClose() would simply check for a variable that the main thread sets on -[GameViewController viewDidDisappear:] or -[AppDelegate applicationWillTerminate:] or something like that.

[coffeebe4code]

it feels we are close!

[martinfinke]

we could build the user's app into a dynamic framework

Creating and loading a dynamic library just so I can name the entry point function main seems inadequate to me. In my own project, I would much rather just do the init/update/destroy functions as originally suggested. I don't need Raylib examples to run unchanged on iOS 🙂

To me, the best compromise seems to not even attempt to run the Raylib examples unmodified on iOS. Instead, add the basic support from this PR, and then just provide a guide: "How to bring your Raylib project to iOS: Implement ios_init, ios_update, ios_destroy like this: …". There could be a basic iOS example Xcode project.
It's better, and IMO in the spirit of Raylib, to do 3 simple and explicit functions, than doing all kinds of intransparent hackery in the background.

About the event loop, we could do what raylib Android does: run the user's main function in a separate thread, handle events and rendering the same way current implementation does, and synchronize both using mutexes/semaphores.

Apart from a lot of opportunities for deadlock, an immediate major problem with this: Cocoa code must be called from the main thread only. The dynamic framework's main() runs on a non-main thread, so we can't create a window, poll input, or make any draw calls in there.
We would have to push all of that into e.g. a lock-free queue, which would then be processed by the main thread…everything becomes asynchronous, we lose the call stacks (hard to debug)…no thanks 😅

[gilzoide]

I don't need Raylib examples to run unchanged on iOS 🙂

You may not need that, but I guess it's important for the project, or else raysan wouldn't be asking for it.
Anyway, people that know what they are doing can call the functions by themselves in the native iOS code without the hassle of loading a dynamic framework.

Apart from a lot of opportunities for deadlock

There would be little opportunities for deadlock actually. The main points of sync between the view controller and the game thread would be input handling, window flags change and making EndDrawing wait for the frame to be presented by the underlying UIView. Deadlock only occurs if you leave room for it, this case is straightforward enough that we would likely have no issues after a first successful implementation.

an immediate major problem with this: Cocoa code must be called from the main thread only. The dynamic framework's main() runs on a non-main thread, so we can't create a window, poll input, or make any draw calls in there.
We would have to push all of that into e.g. a lock-free queue, which would then be processed by the main thread…everything becomes asynchronous, we lose the call stacks (hard to debug)…no thanks 😅

You can actually make all graphics / draw calls in the game thread, even swap render buffers.
As for input, the main thread gets events from UIKit and just push the data to shared memory (again, just like raylib Android does), game thread only needs to read that data that was already sent to it (protected with a mutex, of course). The only case where game thread sends stuff to be processed by main thread would be window flags change, and even this can be as simple as writing to shared data that will only by processed later on by main thread.

There are few call stacks to lose in this case, I don't think people will bother debugging platform code after we implement it correctly. Every thread has its own call stack, the game thread call stack is still there, in full, if you need to debug your game, it will be there for you to debug like in all other platforms.