iOS Support #10
Comments
To potential iOS contributorsThis post describes the work to be done on the iOS side and what might be the best way to break it down. Android and iOS differencesThe architecture of this plugin was really dictated by the nature of Android's component system, so first let's try to understand why things must be the way they are if the plugin is to work on both Android and iOS. In Android, an app is made up of a set of individual components which may be dynamically spawned and destroyed over the lifetime of the app. An audio playing app needs an "Activity" component which is only allowed to run foreground code (i.e. the UI), and a "Service" component which is only allowed to run background code (i.e. playing audio). The idea is that the Activity could be swiped away by the user, i.e. destroyed and freed from memory, while the Service component is allowed to live on and play audio in the background. Flutter automatically sets up an isolate inside the activity to run your Dart code, but since this activity could be swiped away at any time, we can't put the audio playing code here. We therefore use a relatively new Flutter API to create our own isolate attached to the Service for running Dart code, and this is where the audio playing code needs to go. Flutter's API for background execution of Dart code is described in this Medium article. iOS developers in the thread discussion below have pointed out that things fortunately may be a lot simpler on iOS since there is no such separation of foreground and background code. This may mean that the iOS implementation of the plugin would not need to use Flutter's background execution API. However, to main compatibility with Android's behaviour, I would suggest that the iOS implementation should still spawn a plain old isolate for running the background audio code. This would ensure call-by-value semantics is maintained on both platforms. Architecture overviewThis plugin sets up a Dart isolate for running audio playback code in a way that continues to run when the app is not in the foreground. The dart code that runs in this isolate is called the "background task". The architecture allows one or more "clients" to connect to the background task to control audio playback and be notified of state changes to update the UI. One of those clients is the Flutter user interface, but other clients may include a Bluetooth headset or a smart watch. There are distinct APIs for the Flutter client side and for the background task:
For example, Stage 0: FoundationA good starting point would be to implement the method call to start the background isolate, and to implement all other method calls either to directly pass through from one isolate to the other, or even implement them as no-ops. The only methods that absolutely need to work in this stage are:
With this much, the demo example app should start playing audio and will stop once the audio has run out by itself, but will not allow clients to control playback and will not notify the clients of state changes. Stage 1: Basic functionalityThe basic functionality provides the ability to control playback with the basic play/pause operations (which can also be controlled with a headset button click), and also the ability to stop the background task on request. The relevant method calls to implement are: There are also a set of method calls that allow jumping to different positions and tracks that don't require any special set up on the native side so all they do is forward the messages on to the background task. They are: The plugin should also have a default case for custom actions (whose method names begin with " Stage 2: Queue functionalityThis functionality adds the ability to manipulate the playlist/queue and jump to an arbitrary media items. The relevant method calls are: Stage 3: Browsing functionalityThis adds the ability for clients to browse media items offered by the background task. The relevant methods are: Addressing API differences between Android and iOSWhen there are equivalent features on iOS and Android, the preference is to implement both sides with the same API. If things would be easier on the iOS side if the API were changed, please make the suggestion below! I would definitely prefer to change the API if it means that the Android and iOS APIs can be harmonised. In cases where a purely iOS-specific feature is desirable, it can be added as long as it is named with an |
Read the Android docs to see what these methods should actually do: https://developer.android.com/guide/topics/media-apps/audio-app/building-a-mediabrowserservice |
|
I started working on the iOS implementation, but running into several issues. The flutter background isolate example does not run on iOS. If anyone knows how the above can be fixed, please let me know. In the meantime, I will working on a standalone audio_player_service plugin that does not use a background isolate, as it doesn't seem to be required for iOS. Also, the |
|
Hi @alexelisenko and first thank you so much for trying to get the iOS side working. I unfortunately don't own a Mac to test this on, however I did do a Google search for the missing type name "FlutterPluginRegistrantCallback" and found that apparently this type was only committed to git very recently: flutter/engine@1ba3295#diff-dc093c279508589dcc672b1f75d69a11 So your two options would be:
It is probably better to do the latter since there's a chance you'll want to use the latest APIs. First up, of course: If the plugin still doesn't work, then you need to switch to a channel that is updated more regularly. First, you can try the beta channel which is what I personally use: The beta channel is updated once a month, so I'd expect it should have this missing datatype since it was added last month. If it doesn't have the missing data type yet, then you can try: You could then switch back to beta within a month after the missing datatype has had time to make its way into beta. Regarding the project template, I agree and would love to update it, although would you like me to generate a Swift or Objective C template? |
|
@ryanheise switching to the I think you should go with Objective-C since it reduces the complexity of setup for the background execution. The swift template actually uses both swift and obj-c by adding bridge headers, which complicates the setup and examples that are available for background execution. I am currently flushing out all the iOS requirements in a standalone xcode project before integrating into the plugin. |
|
Waiting for this implementation 👍 |
|
@alexelisenko you've become a very popular person :-) Regarding the iOS template, I'll try to update it tonight. I watched an introductory video about Objective C recently, and while it's still a bit foreign to me, I might try my hand at filling in the basic plugin structure in a separate branch (it's bound to have syntax errors though since I don't have a Mac / Xcode to test it on). Swift looks a bit easier to me, but I'll trust you that the bridge headers add complexity. |
|
After playing around with the Flutter background Dart execution, I abandoned that approach for iOS to keep my project deadlines. The good news is I do have a working audio service for iOS (For Audio and Video files), which I will share here soon (have a few tweaks to make before making it public). This does mean that I had to resort to using things like @ryanheise We could potentially create a wrapper interface for my iOS plugin, without adding the background dart execution. One of the main reasons I wrote my own plugin for iOS, was that the I will post the link to the plugin here, hopefully in the next few days. |
|
Here is the iOS plugin: I would consider this in BETA. While it does work, it is lacking in documentation and thorough testing. The example project does give you everything you need to use it, but it does not include examples of all the methods available, so some code reading will be required for some use cases. Please keep in mind that I will be making updates to this repo, which may or may not be breaking changes. EDIT I still intend to integrate this into |
|
Excellent! I fully expect iOS to have a different, though partially overlapping, set of features, so it will be interesting to get an iOS developer's input to help shape the Does your plugin handle button clicks on the headset to play/pause media playback, and if so, which part of the code is responsible for it? I'd be interested to know if this is just handled automatically, or whether iOS gives you the flexibility to handle these button clicks with your own callbacks. After a stack overflow search, I found this (https://stackoverflow.com/questions/9797562/iphone-headset-play-button-click) which gives me hope that it's possible. |
|
iOS uses whats called the CommandCenter to display media metadata and provide controls. The plugin supports the following togglePlayPauseCommand The unimplemented commands available are: The above can be added very easily. I have currently tested this with various bluetooth headphones and over bluetooth audio in several cars As far as what code is responsible, look at this file: https://github.com/alexelisenko/audio_player_service/blob/master/ios/Classes/AudioPlayer.m The init method sets up the CommandCenter with the callbacks. NOTE: The iOS simulator does not show the command center, so it has to be tested on a device, unfortunately. |
|
So then when you click the headset button, does iOS itself maintain a play/pause toggle state and flip it on a click? That would be unfortunate, since what I'm really hoping for is to be able to just listen to button click events and allow the application to process them and manage its own state. |
|
Each of the command I listed requires a callback, which are implemented in the plugin. The state is managed by the plugin entirely, meaning it's the plugins job to decide what to do with the command events. |
|
Sorry, I’m on my phone and haven’t had a chance to inspect the plugin yet.
Though I’m curious, it looks like skip controls are in but you mention no
background excecution, can you create. Playlist with your plugin?
On Wed, Apr 3, 2019 at 8:19 AM alexelisenko ***@***.***> wrote:
Each of the command I listed require a callback, which are implemented.
The state in managed by the plugin entirely, meaning its the plugins job to
decide what to do with the command events.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#10 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AlhueJ6mGJY9NzTk-YETUtVu2-z3jVEAks5vdLh5gaJpZM4ZmGjT>
.
--
Michelle Murray - Manager
|
|
@adriancmurray Yes, the plugin actually requires that you initialize a Queue of items (which can be just one item). The plugin does not automatically play the next item in the queue, although this could be added easily. Im currently playing the next item in the queue in the actual flutter project using the plugin by watching that playback state and running |
|
This is excellent news! Thank you!
On Wed, Apr 3, 2019 at 8:25 AM alexelisenko ***@***.***> wrote:
@adriancmurray <https://github.com/adriancmurray> Yes, the plugin
actually requires that you initialize a Queue of items (which can be just
one item). The plugin does not automatically play the next item in the
queue, although this could be added easily. Im currently playing the next
item in the queue in the actual flutter project using the plugin by
watching that playback state and running next as needed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#10 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AlhueEax8PVjoRBrrLz0QmKtLT8F_6svks5vdLnRgaJpZM4ZmGjT>
.
--
Michelle Murray - Manager
|
|
@adriancmurray There is one known bug with this though: When supplying multiple items in the queue, any bluetooth devices that display the queue position, i.e. 2/10 etc, do not display the correct index, Im still working on a fix This is related to not being able to easily go to the previous item, and having to reinit the queue, which screws up the metadata for external devices that rely on this data. This does not affect devices that do not look for this data, or general playlist playback via command center |
|
Ah, nice! So for example, on receiving a Well... It sounds like you have all of the features I would need for my own use case! 👍 |
|
Thanks for the heads up. Once I implement the plugin I’ll be sure to check
out the bug and see what solutions I can come up with myself.
On Wed, Apr 3, 2019 at 8:29 AM alexelisenko ***@***.***> wrote:
@adriancmurray <https://github.com/adriancmurray> There is one known bug
with this though:
When supplying multiple items in the queue, any bluetooth devices that
display the queue position, i.e. 2/10 etc, do not display the correct
index, Im still working on a fix
This is related to not being able to easily go to the previous item, and
having to reinit the queue, which screws up the metadata for external
devices that rely on this data.
This does not affect devices that do not look for this data, or general
playlist playback via command center
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#10 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AlhueDoVFEGIGcmSLWR2ob7tiYmux4p2ks5vdLqrgaJpZM4ZmGjT>
.
--
Michelle Murray - Manager
|
|
@ryanheise Yes, if the callback does not actually play or pause the Player, the command center will not flip the play/pause button state. The state is provided to the OS via the |
|
I see, that is more or less the way it also works in Android, so it should fit nicely into the current plugin API. I've just copied the latest iOS template across to |
|
@alexelisenko Interesting. That way you're one step closer to the way this plugin works. |
|
@hacker1024 What do you mean by queuing URLs? |
|
@alexelisenko The plugin maintains a list of audio URLs to stream and will play through them. It preloads upcoming queue items, which is what I couldn't get any other plugin to do. EDIT: The Android implementation uses |
|
@hacker1024 So if you pass it a list of URLs, it will download the, then load them from local filesystem? |
|
@alexelisenko Yes, but to RAM or a cache only. The downloads aren't made to be accessed. |
|
Great Job Guys. Waiting... |
|
For those waiting for background audio plugins... might I suggest just writing it yourself. I was hoping for a plugin to be made and briefly used one mentioned on here earlier in the thread (as in I ran my own code on it for about 5 hours until I realized it wasn't going to offer what I needed). My needs are rather specific so I decided to code one myself in swift. It's not that difficult and I had never coded in swift up until writing my own plug in. This allows you a whole lot more freedom in how you implement the native APIs (and there are many for audio). I tried to alter some of the existing plugins for iOS but they all use Objective C, which kind of feels like reading a book written in the 1800's when you're used to languages like Dart/Javascript, so I used Swift. This also allows you to have a tighter coupling between your dart code and the native APIs. Hope this helps. Thanks to everyone who has been working on audio plugins. Cheers! |
|
Hi, would be pretty good, if via your plugin we can configure hide/show NextTrack, PreviousTrack, SeekToPlaybackPosition for iOS notification. |
|
@ryanheise Hi, thanks for all awesome work you're putting on this I wanted to know how are things going with remaining ios stuff?
|
|
Thank you, @bardiarastin There are some other items on my priority list right, such as Android v2 support, and also web support for |
|
Using the Example on a physical iPhone iOS 13. I don't see the command center, is the example old and not updated to use command center? |
|
I don't have a physical iPhone, but my understanding was that the command center is always accessible and there is nothing my plugin can do to make it inaccessible. Please correct me if this is not the case, as I don't have a physical device to check. Alternatively, do you mean you can see the command center, but you just don't see any media displayed on it when you call |
|
The command center is there but it doesn’t show the current playing media. It is just blank and interactions with the buttons don’t do anything. I am using the Example app from this code base. I have not altered it in any way. Just installing it on my phone. I do see from previous comments that the command center does indeed work. |
|
ios not show music control bar, how to fix please, |
|
@ryanheise after some checking of old commits the iOS Command Center broke at commit 7578d7745ee7546302f5f7317575be9740279518 The change that broke it is changing I tested on a Physical iPhone 11 Pro Max iOS 13.3.1 with Flutter: |
|
@YaarPatandarAA , that's certainly interesting, thanks for isolating the specific plugin and version. I don't see anything obviously problematic with Note: This particular issue is now being tracked in #202 . I am closing this "iOS Support" issue finally, now that it has for the most part served its original purpose. From now on I would encourage people to open new issues for specific iOS features or bugs. It almost feels sad to close it after such a long journey, and with all of the suggestions and contributions from you all. It was a true community effort. Thank you! |
|
Not sure if this counts as Apple CarPlay support but here is this package working on Apple CarPlay. The shown buttons also work, this screen is under Now Playing app on Apple CarPlay. I know there is way more that can be done with Apple CarPlay than just this, but it's a start. 👍🏻
|
|
@YaarPatandarAA can you tell me what this package is? It's just bold not hyperlinked 😄 |
This package refers to this package on which we are commenting on, in this packages issue. |
|
@YaarPatandarAA Thanks for clearing that! I was reading it in a completely different way 👍 |
|
Ah, that makes perfect sense :-) I also had the same misunderstanding as @snaeji This plugin might now actually have better support for Apple CarPlay than for Android Auto, which is an interesting twist. |
|
Hey there. I am working on a project which requires bookmarking functionality for both iOS and Android. I am wondering, has anyone managed to add "bookmarkCommand" for iOS? |
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs, or use StackOverflow if you need help with audio_service. |
I'm creating this issue to keep track of an iOS implementation. I myself am an Android developer, and I'd love to see an iOS developer contribute.
The text was updated successfully, but these errors were encountered: