cocoa-cb: initial implementation via opengl-cb API

this is meant to replace the old and not properly working vo_gpu/opengl
cocoa backend in the future. the problems are various shortcomings of
Apple's opengl implementation and buggy behaviour in certain
circumstances that couldn't be properly worked around. there are also
certain regressions on newer macOS versions from 10.11 onwards.

- awful opengl performance with a none layer backed context
- huge amount of dropped frames with an early context flush
- flickering of system elements like the dock or volume indicator
- double buffering not properly working with a none layer backed context
- bad performance in fullscreen because of system optimisations

all the problems were caused by using a normal opengl context, that
seems somewhat abandoned by apple, and are fixed by using a layer backed
opengl context instead. problems that couldn't be fixed could be
properly worked around.

this has all features our old backend has sans the wid embedding,
the possibility to disable the automatic GPU switching and taking
screenshots of the window content. the first was deemed unnecessary by
me for now, since i just use the libmpv API that others can use anyway.
second is technically not possible atm because we have to pre-allocate
our opengl context at a time the config isn't read yet, so we can't get
the needed property. third one is a bit tricky because of deadlocking
and it needed to be in sync, hopefully i can work around that in the
future.

this also has at least one additional feature or eye-candy. a properly
working fullscreen animation with the native fs. also since this is a
direct port of the old backend of the parts that could be used, though
with adaptions and improvements, this looks a lot cleaner and easier to
understand.

some credit goes to @pigoz for the initial swift build support which
i could improve upon.

Fixes: mpv-player#5478, mpv-player#5393, mpv-player#5152, mpv-player#5151, mpv-player#4615, mpv-player#4476, mpv-player#3978, mpv-player#3746, mpv-player#3739,
mpv-player#2392, mpv-player#2217

DOCS/interface-changes.rst

@@ -70,6 +70,8 @@ Interface changes
       pad must be connected either to another filter, or to a video/audio track
       or video/audio output). If they are disconnected at runtime, the stream
       will probably stall.
+    - deprecate the OpenGL cocoa backend, option choice --gpu-context=cocoa
+      when used with --gpu-api=opengl (use --vo=opengl-cb)
  --- mpv 0.28.0 ---
     - rename --hwdec=mediacodec option to mediacodec-copy, to reflect
       conventions followed by other hardware video decoding APIs

DOCS/man/options.rst

@@ -4862,7 +4862,7 @@ The following video options are currently all specific to ``--vo=gpu`` and
     auto
         auto-select (default)
     cocoa
-        Cocoa/OS X
+        Cocoa/OS X (deprecated, use --vo=opengl-cb instead)
     win
         Win32/WGL
     winvk

DOCS/man/vo.rst

@@ -429,7 +429,9 @@ Available video output drivers are:
         Specify the directory to save the image files to (default: ``./``).
 
 ``opengl-cb``
-    For use with libmpv direct OpenGL embedding; useless in any other contexts.
+    For use with libmpv direct OpenGL embedding. As a special case, on OS X it
+    is used like a normal VO within mpv (cocoa-cb). Otherwise useless in any
+    other contexts.
     (See ``<mpv/opengl_cb.h>``.)
 
     This also supports many of the options the ``gpu`` VO has.

osdep/macOS_mpv_helper.swift

@@ -0,0 +1,255 @@
+/*
+ * This file is part of mpv.
+ *
+ * mpv is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * mpv is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with mpv.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+import Cocoa
+import OpenGL.GL
+import OpenGL.GL3
+
+class MPVHelper: NSObject {
+
+    var mpvHandle: OpaquePointer?
+    var mpvGLCBContext: OpaquePointer?
+    var mpvLog: OpaquePointer?
+    var inputContext: OpaquePointer?
+    var mpctx: UnsafeMutablePointer<MPContext>?
+
+    init(_ mpv: OpaquePointer) {
+        super.init()
+        mpvHandle = mpv
+        mpvLog = mp_log_new(UnsafeMutablePointer<MPContext>(mpvHandle),
+                            mp_client_get_log(mpvHandle), "cocoacb")
+        mpctx = UnsafeMutablePointer<MPContext>(mp_client_get_core(mpvHandle))
+        inputContext = mpctx!.pointee.input
+
+        mpv_observe_property(mpvHandle, 0, "ontop", MPV_FORMAT_FLAG)
+        mpv_observe_property(mpvHandle, 0, "border", MPV_FORMAT_FLAG)
+        mpv_observe_property(mpvHandle, 0, "keepaspect-window", MPV_FORMAT_FLAG)
+    }
+
+    func setGLCB() {
+        if mpvHandle == nil {
+            sendError("No mpv handle available.")
+            exit(1)
+        }
+        mpvGLCBContext = OpaquePointer(mp_get_sub_api2(mpvHandle, MPV_SUB_API_OPENGL_CB, false))
+        if mpvGLCBContext == nil {
+            sendError("libmpv does not have the opengl-cb sub-API.")
+            exit(1)
+        }
+    }
+
+    func initGLCB() {
+        if mpvGLCBContext == nil {
+            setGLCB()
+        }
+        if mpv_opengl_cb_init_gl(mpvGLCBContext, nil, getProcAddress, nil) < 0 {
+            sendError("GL init has failed.")
+            exit(1)
+        }
+    }
+
+    let getProcAddress: mpv_opengl_cb_get_proc_address_fn = {
+            (ctx: UnsafeMutableRawPointer?, name: UnsafePointer<Int8>?) -> UnsafeMutableRawPointer? in
+        let symbol: CFString = CFStringCreateWithCString(
+                                kCFAllocatorDefault, name, kCFStringEncodingASCII)
+        let indentifier = CFBundleGetBundleWithIdentifier("com.apple.opengl" as CFString)
+        let addr = CFBundleGetFunctionPointerForName(indentifier, symbol)
+
+        if symbol as String == "glFlush" {
+            return glDummyPtr()
+        }
+
+        return addr
+    }
+
+    func setGLCBUpdateCallback(_ callback: @escaping mpv_opengl_cb_update_fn, context object: AnyObject) {
+        if mpvGLCBContext == nil {
+            sendWarning("Init mpv opengl-cb first.")
+        } else {
+            mpv_opengl_cb_set_update_callback(mpvGLCBContext, callback, MPVHelper.bridge(obj: object))
+        }
+    }
+
+    func setGLCBControlCallback(_ callback: @escaping mpv_opengl_cb_control_fn, context object: AnyObject) {
+        if mpvGLCBContext == nil {
+            sendWarning("Init mpv opengl-cb first.")
+        } else {
+            mp_client_set_control_callback(mpvGLCBContext, callback, MPVHelper.bridge(obj: object))
+        }
+    }
+
+    func reportGLCBFlip() {
+        if mpvGLCBContext == nil { return }
+            mpv_opengl_cb_report_flip(mpvGLCBContext, 0)
+    }
+
+    func drawGLCB(_ surface: NSSize) {
+        if mpvGLCBContext != nil {
+            var i: GLint = 0
+            glGetIntegerv(GLenum(GL_DRAW_FRAMEBUFFER_BINDING), &i)
+
+            mpv_opengl_cb_draw(mpvGLCBContext, i, Int32(surface.width), Int32(-surface.height))
+        } else {
+            glClearColor(0, 0, 0, 1)
+            glClear(GLbitfield(GL_COLOR_BUFFER_BIT))
+        }
+    }
+
+    func setGLCBICCProfile(_ profile: NSColorSpace) {
+        if mpvGLCBContext == nil { return }
+        var iccData = profile.iccProfileData
+        iccData!.withUnsafeMutableBytes { (u8Ptr: UnsafeMutablePointer<UInt8>) in
+            let icc = bstrdup(nil, bstr(start: u8Ptr, len: iccData!.count))
+            mp_client_set_icc_profile(mpvGLCBContext, icc)
+        }
+    }
+
+    func setGLCBLux(_ lux: Int) {
+        if mpvGLCBContext == nil { return }
+        mp_client_set_ambient_lux(mpvGLCBContext, Int32(lux))
+    }
+
+    func command(_ cmd: String) {
+        if mpvHandle == nil { return }
+        mpv_command_string(mpvHandle, cmd)
+    }
+
+    func commandAsync(_ cmd: [String?], id: UInt64 = 1) {
+        if mpvHandle == nil { return }
+        var mCmd = cmd
+        mCmd.append(nil)
+        var cargs = mCmd.map { $0.flatMap { UnsafePointer<Int8>(strdup($0)) } }
+        mpv_command_async(mpvHandle, id, &cargs)
+        for ptr in cargs { free(UnsafeMutablePointer(mutating: ptr)) }
+    }
+
+    func getBoolProperty(_ name: String) -> Bool {
+        if mpvHandle == nil { return false }
+        var value = Int32()
+        mpv_get_property(mpvHandle, name, MPV_FORMAT_FLAG, &value)
+        return value > 0
+    }
+
+    func getIntProperty(_ name: String) -> Int {
+        if mpvHandle == nil { return 0 }
+        var value = Int64()
+        mpv_get_property(mpvHandle, name, MPV_FORMAT_INT64, &value)
+        return Int(value)
+    }
+
+    func getStringProperty(_ name: String) -> String? {
+        if mpvHandle == nil { return nil }
+        let value = mpv_get_property_string(mpvHandle, name)
+        let str = value == nil ? nil : String(cString: value!)
+        mpv_free(value)
+        return str
+    }
+
+    func canBeDraggedAt(_ pos: NSPoint) -> Bool {
+        if inputContext == nil { return false }
+        let canDrag = !mp_input_test_dragging(inputContext!, Int32(pos.x), Int32(pos.y))
+        return canDrag
+    }
+
+    func setMousePosition(_ pos: NSPoint) {
+        if inputContext == nil { return }
+        mp_input_set_mouse_pos(inputContext!, Int32(pos.x), Int32(pos.y))
+    }
+
+    func putAxis(_ mpkey: Int32, delta: Double) {
+        if inputContext == nil { return }
+        mp_input_put_wheel(inputContext!, mpkey, delta)
+    }
+
+    func sendVerbose(_ msg: String) {
+        send(message: msg, type: MSGL_V)
+    }
+
+    func sendInfo(_ msg: String) {
+        send(message: msg, type: MSGL_INFO)
+    }
+
+    func sendWarning(_ msg: String) {
+        send(message: msg, type: MSGL_WARN)
+    }
+
+    func sendError(_ msg: String) {
+        send(message: msg, type: MSGL_ERR)
+    }
+
+    func send(message msg: String, type t: Int) {
+        if mpvLog == nil {
+            sendFallback(message: msg, type: t)
+        } else {
+            let args: [CVarArg] = [ (msg as NSString).utf8String! ]
+            mp_msg_va(mpvLog, Int32(t), "%s\n", getVaList(args))
+        }
+    }
+
+    func sendFallback(message msg: String, type t: Int) {
+        var level = "\u{001B}"
+        switch t {
+        case MSGL_V:
+            level += "[0;30m[VERBOSE]"
+        case MSGL_INFO:
+            level += "[0;30m[INFO]"
+        case MSGL_WARN:
+            level += "[0;33m"
+        case MSGL_ERR:
+            level += "[0;31m"
+        default:
+            level += "[0;30m"
+        }
+
+        print("\(level)[osx/cocoacb] \(msg)\u{001B}[0;30m")
+    }
+
+    func deinitGLCB() {
+        mpv_opengl_cb_set_update_callback(mpvGLCBContext, nil, nil)
+        mp_client_set_control_callback(mpvGLCBContext, nil, nil)
+        mpv_opengl_cb_uninit_gl(mpvGLCBContext)
+        mpvGLCBContext = nil
+    }
+
+    func deinitMPV() {
+        mpvHandle = nil
+        mpvLog = nil
+        inputContext = nil
+        mpctx = nil
+    }
+
+    // (__bridge void*)
+    class func bridge<T: AnyObject>(obj: T) -> UnsafeMutableRawPointer {
+        return UnsafeMutableRawPointer(Unmanaged.passUnretained(obj).toOpaque())
+    }
+
+    // (__bridge T*)
+    class func bridge<T: AnyObject>(ptr: UnsafeRawPointer) -> T {
+        return Unmanaged<T>.fromOpaque(ptr).takeUnretainedValue()
+    }
+
+    // *(char **) MPV_FORMAT_STRING on mpv_event_property
+    class func mpvStringArrayToString(_ obj: UnsafeMutableRawPointer) -> String? {
+        let cstr = UnsafeMutablePointer<UnsafeMutablePointer<Int8>>(OpaquePointer(obj))
+        return String(cString: cstr[0])
+    }
+
+    // MPV_FORMAT_FLAG
+    class func mpvFlagToBool(_ obj: UnsafeMutableRawPointer) -> Bool? {
+        return UnsafePointer<Bool>(OpaquePointer(obj))?.pointee
+    }
+}

osdep/macOS_swift_bridge.h

@@ -0,0 +1,54 @@
+/*
+ * This file is part of mpv.
+ *
+ * mpv is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * mpv is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with mpv.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+// including IOKit here again doesn't make sense, but otherwise the swift
+// compiler doesn't include the needed header in our generated header file
+#import <IOKit/pwr_mgt/IOPMLib.h>
+
+#include "player/client.h"
+#include "libmpv/opengl_cb.h"
+
+#include "player/core.h"
+#include "input/input.h"
+#include "video/out/win_state.h"
+
+#include "osdep/macosx_application_objc.h"
+#include "osdep/macosx_events_objc.h"
+
+
+// complex macros won't get imported to Swift so we have to reassign them
+static int SWIFT_MBTN_LEFT       = MP_MBTN_LEFT;
+static int SWIFT_MBTN_MID        = MP_MBTN_MID;
+static int SWIFT_MBTN_RIGHT      = MP_MBTN_RIGHT;
+static int SWIFT_WHEEL_UP        = MP_WHEEL_UP;
+static int SWIFT_WHEEL_DOWN      = MP_WHEEL_DOWN;
+static int SWIFT_WHEEL_LEFT      = MP_WHEEL_LEFT;
+static int SWIFT_WHEEL_RIGHT     = MP_WHEEL_RIGHT;
+static int SWIFT_MBTN_BACK       = MP_MBTN_BACK;
+static int SWIFT_MBTN_FORWARD    = MP_MBTN_FORWARD;
+static int SWIFT_MBTN9           = MP_MBTN9;
+
+static int SWIFT_KEY_CLOSE_WIN   = MP_KEY_CLOSE_WIN;
+static int SWIFT_KEY_MOUSE_LEAVE = MP_KEY_MOUSE_LEAVE;
+static int SWIFT_KEY_MOUSE_ENTER = MP_KEY_MOUSE_ENTER;
+static int SWIFT_KEY_STATE_DOWN  = MP_KEY_STATE_DOWN;
+static int SWIFT_KEY_STATE_UP    = MP_KEY_STATE_UP;
+
+// dummy function to override glFlush()
+static void glDummy() {}
+static void *glDummyPtr(void) __attribute__((unused));
+static void *glDummyPtr() { return &glDummy; }