When will there be support for Android and iOS? #537

Ltty · 2015-12-15T12:05:21Z

Ltty
Dec 15, 2015

Hi,

do you have a concrete roadmap when Android and iOS will be supported?

Best,
Florian

Danielku15 · 2015-12-28T12:27:29Z

Danielku15
Dec 28, 2015
Maintainer

Hi Florian.
No, currently I don't have a concrete roadmap but Android will be definitly the first platform I will support. I started to investigate in possible compilers to translate the C# codebase to other languages like Java and C++ but I cannot tell how long it will take to create such a port.

For now there are only 3 solutions to bring alphaTab to those platforms:

Use Xamarin
Use a Browser component and use the JavaScript version
Use a JavaScript engine to use alphaTab in the background (render SVG and access model details).

I will do my best to support those platforms as a native library but currently I am struggling with the open features for 1.0.0 release.

0 replies

jason0539 · 2016-08-13T07:41:27Z

jason0539
Aug 13, 2016

Hi,Danielku
How about the progress on Android and iOS，and i saw that there are plans to support MusicXML,do you have a concrete roadmap？

0 replies

Danielku15 · 2016-08-16T18:32:20Z

Danielku15
Aug 16, 2016
Maintainer

No, there is currently no concrete roadmap when this will be available.

0 replies

gburlet · 2017-01-20T18:18:23Z

gburlet
Jan 20, 2017

For those who want to use AlphaTab on iOS (a similar solution exists for Android) and run everything locally with no connection, use a UIWebView and import the needed javascript files, font files, and any css you want. Your project directory should look like this:

Also make sure all of your files in the alphatab folder are copied over to your app under Build Phases > Copy Bundle Resources:

Then in your html file, make sure to put relative paths to all of the files so they can be loaded from the bundle. Here's a barebones example HTML:

<!DOCTYPE html>
<html>
<head>
  <meta http-equiv="content-type" content="text/html; charset=UTF-8">

  <script type="text/javascript" src="js/jquery.min.js"></script>
  <script type="text/javascript" src="js/AlphaTab.min.js"></script>
  <script type="text/javascript" src="js/jquery.alphaTab.js"></script>

  <script type='text/javascript'>
    $(window).on('load', function() {
        $('#alphaTab').alphaTab(); 
    }); 
  </script>
</head>
<body>
  <h2>AlphaTab Test</h2>
  <div id="alphaTab" data-tex="true">
    \title "The Greatest Song In The World"
    \artist "Greg Burlet"
    \tempo 90
    .
    :4 2.3 4.4 2.2 3.4
  </div>
</body>
</html>

Then in your ViewController:

@IBOutlet weak var webView: UIWebView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let alphaTabURL = Bundle.main.url(forResource: "alphatab", withExtension: "html", subdirectory: "alphatab")
        let request = URLRequest(url: alphaTabURL!)
        webView.loadRequest(request)
    }

The result:

Note that this doesn't open up the page in an installed web browser on the phone (eg., Safari, Chrome) but renders right in your app.

Maybe @Danielku15 has a different or better approach, but this seems to be working well for me. Hopefully that helps someone out there!

0 replies

Danielku15 · 2017-01-20T18:58:25Z

Danielku15
Jan 20, 2017
Maintainer

Thanks for the more concrete example @gburlet . Using a WebView directly is one way how to use alphaTab. I mentioned it before but another solution would be to use directly a JavaScript engine (like JavaScriptCore ) to let alphaTab generate the SVG. The SVG could then be rendered separately in a native way using libraries like SVGKit. Same goes for the MidiGeneration, you can let alphaTab generate the midi binary and use then a native library for playback.

Native library support is likely not achievable in mid-term since there is no cross-compiler available like for JavaScript. I plan to create a simple wrapper over a JavaScript engine and use the JavaScript version internally. It will therefore has less access to internals but might be sufficient to add rendering to iOS and Android applications.

Currently I am finalizing some rendering related features on alphaTab and also MusicXML is a big topic which keeps me busy for the next stable release. Providing a native lib that is usable in Android and iOS is definitly still on my list (iOS might be a bit tricky to test since I do not own a Mac).

0 replies

gburlet · 2017-01-20T19:02:47Z

gburlet
Jan 20, 2017

Sounds like a good plan. I agree that rendering related tasks and MusicXML parsing is higher priority than native iOS & Android libraries, especially since there are good workarounds to use in the meantime on these devices. You can always run OS X in a virtual machine to test. It's sort of a pain to set up but works great.

0 replies

Danielku15 · 2017-01-21T07:14:00Z

Danielku15
Jan 21, 2017
Maintainer

I just found a very interesting library for iOS and Android which does basically exactly what I explained many times: It wraps native JavaScript engines (JavaScript Core / V8) and extends it by a special canvas API that does native drawing on views.
https://github.com/phoboslab/Ejecta
https://github.com/godmodelabs/ejecta-v8/

It might require direct usage of the ScoreRenderer since the normal JavaScript API of alphaTab relies on a DOM.

0 replies

WNervous · 2018-07-30T03:57:31Z

WNervous
Jul 30, 2018

[AlphaTab][undefined] An unexpected error occurred when I load using webview in android

0 replies

Danielku15 · 2018-11-18T00:09:28Z

Danielku15
Nov 18, 2018
Maintainer

Its been a while since there was an update on the native Android and iOS version. Beside some minor open issues I am now able to compile alphaTab to Kotlin which will hopefully allow native Android and iOS apps soon.

As you can see there is a simple Android App with an "alphaTab Control" embedded using native Kotlin code. (Open the Gif in a new window for better quality)

In parallel I'm working on a unified API that will hopefully ensure that all new features will go evenly to the different platforms.

0 replies

matt15wish · 2019-02-11T01:15:53Z

matt15wish
Feb 11, 2019

Danielku15.. Using android studio, I got it set up in an application of mine just using a Webview. But playback is really choppy.. Is there anything I'm missing?? Or is the android studio Webview not powerful enough to handle it. Thanks

0 replies

matt15wish · 2019-04-23T03:00:41Z

matt15wish
Apr 23, 2019

Good day.
I just tried to use alphatab and load the demo showcased at alphatab.net main site and playback is sluggish. Is alphasynth compatible with mobile? I would really like to get it going on mobile as a web app or with Android studio. Any help would be appreciated, thanks

0 replies

Danielku15 · 2021-01-17T17:30:20Z

Danielku15
Jan 17, 2021
Maintainer

It was long promised but never really tackled: Native Android and iOS support.
Now it's finally time. alphaTab 1.3 will address the tasks to provide Native Android and iOS Flavors of alphaTab.

I prepared the roadmap here which will allow everyone to follow the progress:
https://github.com/orgs/CoderLine/projects/4

Unfortunately I cannot promise still any ETA but it will be the full focus of the next release (beside potential bugfixing for 1.2.x). From a first guess I would say we can expect at least an android version somewhen within Q2-2021.

1 reply

mkorobtsev Sep 10, 2021

Nadiel, do you have Xamarin example?

Danielku15 · 2021-12-17T21:37:59Z

Danielku15
Dec 17, 2021
Maintainer

Hey everyone, I wanted to give you some update on the status of native Android support.

Unfortunately I have to admit that I am heavily surprised how terrible the performance on Android native development is. It is giving me a really hard time to ship something that is really production worthy. I am considering to fully drop the Kotlin/JVM approach and only ship a WebView based wrapper around the JS Version.

Compiliation
To ship alphaTab for Android I translate now the codebase to Kotlin to compile it with Kotlin/JVM for Android (and Kotlin/Native for iOS). While it needed some efforts this was not such a big deal at the end. But the problems start afterwards: The Kotlin Compiler is extremely slow and resource hungry. TypeScript compiles in ~2m to JavaScript, the C# code compiles in 3m and Kotlin needs >12m. On GitHub Actions most of the time the compilation simply crashes.

I was in contact with many people around this Kotlin compilation performance. The only recommendations were to use more caching for incremental compilations. The compiler is known to be slow, and the build system with Gradle on top requires additional time. But even with all the improvements in place that the Kotlin community and compiler experts recommended (caching, incremental compilation, special gradle settings), the Kotlin compilation is still a lot slower than any clean compile on other platforms.

In my company we have a >400k LoC C++ codebase, and a clean compilation there is faster than compiling the Kotlin version of alphaTab. At this point I am lost on whether Kotlin is simply so slow and there is nothing to do about it, or whether there is something in this codebase that makes Kotlin go crazy.

Due to this almost everything I do takes 10x longer than usual. Even simple code changes sometimes lead to 5m compilations.

Development/Tooling

I would say alphaTab has quite a complex codebase but nothing a modern development environment shouldn't be able to handle. Nevertheless Android Studio struggles to handle the codebase. Until some latest updates the IDE simply got fully stuck at 100% CPU and froze. I reported it to JetBrains but the issue is still open at this point and there was not much more feedback than: they were able to reproduce it.

In the latest Android Studio Updates at least the IDE doesn't get 100% stuck which allowed me to continue at least the development. As you can imagine not even being able to write some code without having the IDE get stuck leads to quite some frustration. This is another reason why the topic got delayed already so much.

But the improvements in the latest updates of Kotlin and Android Studio still don't fix all the issues. The CPU load is still constantly high, the memory load of the IDE easily reaches 4-5GB and the Gradle Daemons also float around at 1-2GB easily.

This is also another indication that there is something fishy with Kotlin and the alphaTab codebase.

Java Runtime

Another issue I am tackling constantly is the runtime performance of Java. On my tests with my Pixel 2 XL I faced following issues:

The key topic here is the boxing of primitives and poor Generics supprot. As most of you might know, Java does not support generic types with primitives. Any List<> or Map<> requires to box/unbox primitives into objects. With alphaTab having quite many Lists and Maps this caused a huge performance issue during runtime. There was huge pressure to the GC and the runtime due to this boxing.

This forced me to do quite some adaptions on the translation to Kotlin and the related runtime implementations but I could solve it mostly. In Kotlin we generate now some strong typed lists and maps which do not rely on generics but are built on primitives. For this I borrowed the Dictionary<,> and List<> implementation aspects of the .net framework. Through inline methods with reified generics I could eliminate most of the code duplication.

For Android we need to be able to persist/restore states quite a lot due to the way Activities work. But as the JSON Serialization of the Data Model is based on Map<string, any> we run again into boxing related performance issues.

The second issue is the number to string conversion in Android/Java. alphaTab needs to create quite some numbers to strings for the tabs, bar numbers and various effects like bends. I tried to use the DecimalFormat class, but again it is just terribly slow. I added some micro-optimizations for now but I'm not really happy with this solution. Even such basics like converting numbers to string already cause issues, this again causes quite some frustration.

The third topic seems that also the rendering pipeline of Android seems to be rather small when using the built in Canvas APIs for rendering. Unfortunately Skija does not have an Android flavor and it would also blow up the image size quite significant due to the native libs for all cpus and platforms.

It is crazy that even with all optimizations and having Kotlin code, rendering a full score using Kotlin/JVM is a lot slower than rendering the same file in Chrome of the same Smartphone.

	Kotlin	Web
File 1	~6300ms	~520ms
File 2	~7800ms	~550ms

Android Integration

I have to admit that my experience in Android development is quite limited. This is another cause for quite some delays. Developing a component which is then really usable and stable for integrators is a challenge when you never have done it before. Especially as Android again seems to lack quite some fundamentals a developer from other domains would expect to be available out of the box.

The first thing is the optimized layout with scrolling. After some rounds on different approaches I have settled for the RecyclerView with the FlexboxLayoutManager from Google to host the rendered images. This already works out quite well, but there are still issues related to horizontal and vertical layouts. But when it comes to horizontal layouts again the issues start. Having horizontal and vertical scrolling at the same time seems to be not supported out of the box and again the workarounds start.

I need to manually change configurations in the layout managers and even then scrolling is not comparable to how it is in a WebView where you can scroll in any direction. A challenge still to be solved.

Beside that there are still many problems to solve on Android:

Handling the Playback Cursors as Overlays with correct positioning
Dynamic Scrolling
Player Output for playing the synthesized samples
UI Interactivity to select playback ranges and seek.

Conclusion

As you can see, targeting Android is quite a pain. The whole tooling to even compile a codebase which is there is a pain and on top come all the runtime performance issues and only then the actual Android integration topics start.

I can only imagine what pain it will become to natively target iOS with not even havin a Mac to actually test things on iOS during development.

If things will not become better I will likely have to fully drop the pure native support of alphaTab on these platforms. The alternative will be to provide a WebView wrapper with some more convenient to use APIs.

If we have any Android experts around here I am happy for any input in this regards.

0 replies

Danielku15 · 2021-12-31T15:51:59Z

Danielku15
Dec 31, 2021
Maintainer

Promising news. With all the optimizations and reworks I did lately, I was able to get a decent performance (CPU load, memory consumption and speed) which I consider as acceptable for alphaTab to be released.

Compilation: I was able to make some slight improvements but it is still quite terrible to work with. All development take quite some time but it's not a full blocker. GitHub actions will have a tough time compiling all versions in future unless I can improve this later.

https://github.com/CoderLine/alphaTab/actions/runs/1641096328

Development/Tooling The runtime development is now complete and it seems when developing only within the android area, the IDE does not go crazy anymore. I hope I don't need to touch these runtime areas often.

Java Runtime I now use a custom implementation of most data structures to work around the Java/Android runtime issues. For the string formatting I have some basic workaround which works fine. Since #689 alphaTab now even supports lazy rendering. This means, we only do the layout of the score but don't render all partials already. Only when they become visible we dynamically request render. As rendering of a single chunk is not taking much time, the UI feedback is still fast.

Android Integration Here I went down to the core on rendering UI things. I treat the UI as basic canvas and draw the partials myself. I added some UI virtualization mechanism on top, all at low-level without relying on any Android built-ins. The CPU and memory load is stable and acceptable. Also on some older phones I tested with, the rendering was fast.

device-2021-12-31-163732.mp4

Now I can advance in adding all functionalities beside the plain rendering:

Audio Playback
Cursor display
UI interactivity

0 replies

Danielku15 · 2022-01-07T13:56:33Z

Danielku15
Jan 7, 2022
Maintainer

Making progress: Audio Playback is working, cursor display too.

It's all still not perfect but functional wise working. Especially the audio latency topics are a bit painful to have the cursor properly synchronized with the playback.

Untitled.mp4

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When will there be support for Android and iOS? #537

{{title}}

Replies: 15 comments 1 reply

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

When will there be support for Android and iOS? #537

Replies: 15 comments · 1 reply

Danielku15 Dec 28, 2015 Maintainer

Danielku15 Aug 16, 2016 Maintainer

Danielku15 Jan 20, 2017 Maintainer

Danielku15 Jan 21, 2017 Maintainer

Danielku15 Nov 18, 2018 Maintainer

Danielku15 Jan 17, 2021 Maintainer

Danielku15 Dec 17, 2021 Maintainer

Danielku15 Dec 31, 2021 Maintainer

Danielku15 Jan 7, 2022 Maintainer

Replies: 15 comments 1 reply

Danielku15
Dec 28, 2015
Maintainer

Danielku15
Aug 16, 2016
Maintainer

Danielku15
Jan 20, 2017
Maintainer

Danielku15
Jan 21, 2017
Maintainer

Danielku15
Nov 18, 2018
Maintainer

Danielku15
Jan 17, 2021
Maintainer

Danielku15
Dec 17, 2021
Maintainer

Danielku15
Dec 31, 2021
Maintainer

Danielku15
Jan 7, 2022
Maintainer