Is header-translator viable for SDK usage code linting?

[complexspaces]

At 1Password I've been trying to passively plan out a solution for resolving a lot of clunky code and tech debt in our "foundation" layer. Without making this sound like a product shill/pitch, I'll briefly explain the architecture and constraints. We have three layers of code in the iOS (macOS shares the lower two) app:

• Frontend, which is written entirely in Swift
• "Core" which is written entirely in Rust
• "Foundation" which is a mixture of Rust and Swift code, compiled and linked in by a custom solution I designed and a larger xcodebuild produced .a file.

The FFI layer between the Core and foundation layer isn't great, primarily because the C ABI exposed by Swift is pretty limiting and cumbersome. For example passing state between them opaquely is fragile, returning errors with context out of deeply nested functions doesn't work well, etc.

To resolve this I'm hoping to do two things:

• Use bridging classes (like mentioned in Use extern_class macro with classes declared in swift via @objc #642) for fairly complicated logic or Swift-only functionality where the files are still needed.
• Convert a large amount of the existing Swift code to use the objc2_* family of framework crates and "inline" the FFI directly into the calling Rust code.

That last point is what I'm wanting to inquire on. A blocking concern I've found to adopting the objc2 crate family is deprecation and the liveness usually provided by compiling with swiftc from XCode and the most recent SDK in our build systems. For example, if we are using a function in Swift today that Apple decides to get rid of, we get an XCode-provided deprecation warning since its compiling against the latest SDK. If the same was used via objc2, we might not know until an arbitrary crate update in the future.

My workaround for this idea was developing a custom linting tool that scans the known callsites of Apple frameworks in Rust, collects the referenced symbols (functions, statics, and constants), and then parses their latest metadata from the current system's SDK. If anything deprecated (and not explicitly allowed) or missing is found, it raises an error to the developer or CI pipeline. For a while I wasn't sure how to do this but I found the header-translator project while looking at the other objc2 crates. I saw that it collects the deprecation information specifically and translates that nicely to Rust's built-in attributes but beyond that its semi-opaque to me.

The Rust collection side would need to be syn based, but do you think building a "diffing" tool like this would be feasible with the functionality offered by header-translator today? Could there potentially be a better way that doesn't damage performance which the framework crates themselves handled at build-time?

[madsmtm]

Thanks for the question! Let me answer this one first:

A blocking concern I've found to adopting the objc2 crate family is deprecation and the liveness usually provided by compiling with swiftc from XCode and the most recent SDK in our build systems. For example, if we are using a function in Swift today that Apple decides to get rid of, we get an XCode-provided deprecation warning since its compiling against the latest SDK. If the same was used via objc2, we might not know until an arbitrary crate update in the future.

That's definitely true! To add a bit of nuance, for you to receive deprecation warnings from objc2-*, the following has to happen:

1. Apple adds a deprecation warning to their SDK headers.
2. Apple releases a stable version of Xcode with this SDK.
3. I update the framework crates to this Xcode version.
4. I do a new release of the framework crates.
5. You update your crate to the new (possibly breaking) version.

For point 2, I've intentionally restricted us to a stable Xcode versions, since it happens fairly often that Apple adds an API in an Xcode beta, and then later removes it in the next beta. A possible mitigation here could be for me to release unstable beta versions of the framework crates, matching the Xcode betas? Though then again, I have also promised to be mindful of the crates.io team and not do too frequent releases. I think if you want this, I'd suggest that you patch.crates-io with a fork of objc2 + your own generated objc2-generated while the betas are out.

For point 3, the biggest blocker for me has usually been GitHub Actions, but that could honestly just be ignored if speed is a concern. There's also of course the question of whether I'll stay as active as I am currently (bus factor etc.), I probably won't, but I am pretty committed to keeping the project alive (and it's usually a quite small amount of work for me to update to a new Xcode version).

For point 4, my focus has been (and is currently) on other improvements, so my release schedule hasn't been prioritized for getting the latest and greatest SDKs out (when we didn't even support CoreFoundation until recently). But, as the project grows more stable, this is definitely something I'd change. I'm not ready to give any kind of commitment or timelines or "framework crates will be released at least X days after the Xcode release" guarantees here though, sorry ;).

For point 5, this should be easy for you if the version bump is free from SemVer breaking changes (and those should happen less as the project matures). Looking far ahead, I'm somewhat unlikely to ever release a v1.0, since Rust has a more strict type system that allows fewer API changes (e.g. it's a breaking change in Rust to make a parameter nullable, while it's not (or at least less so) in Swift/Objective-C), and thus to keep up with SDK changes it's likely that we will need a breaking version every third Xcode release or so. But I'll have to get more experience over the years to make a final statement here.

[madsmtm]

My workaround for this idea was developing a custom linting tool that scans the known callsites of Apple frameworks, collects the referenced symbols (functions, statics, and constants), and then parses their latest metadata from the current system's SDK.

Out of curiosity, do you have an idea of the amount of code/how many calls that would fall into this category? Given Apple's fairly strong backwards compatibility guarantees, the amount of APIs that aren't deprecated, and the long timeline Apple usually has for deprecations, I'd argue that anything lower than like at least 1.000, probably higher, might just be wasted effort on your part?

Honestly, if I were to name a fear that I would think 1Password to have for moving to objc2 (apart from generally less upstream support from Apple), it would rather be lack of availability annotations (i.e. you don't get warnings when using new APIs and not guarding these behind a dynamic check, so it's harder to support older OS versions). But the tool you propose could also help with that!

The Rust collection side would need to be syn based

Probably. Note that that won't catch weird macro invocations, but you're probably already aware of that.

For a while I wasn't sure how to do this but I found your header-translator project while looking at the other objc2 crates. I saw that it collects the deprecation information specifically and translates that nicely to Rust's built-in attributes but beyond that its semi-opaque to me.
do you think building a "diffing" tool like this would be feasible with the functionality offered by header-translator today?

Yes and no. Yes, in that you can definitely do this, but no in that you should not be using header-translator! It is meant to be an internal tool, and is honestly one of my more crappy codebases 🙄. Using the clang crate and extracting deprecation out of Entity::get_platform_availability directly is a much better idea.

For your use-case, I think you can get away with basically a single .visit_children returning EntityVisitResult::Recurse, and collect deprecation information from all methods, properties and functions across the entirety of Xcode's SDKs. If you don't use -fmodules, and just create a file with a bunch of #include <.../...>s for each framework you care about, then I think Clang will visit all the entities.

Feel free to ask further if you need help with setting that up!

Would there potentially be a better way that doesn't damage performance that could be done at build-time in the framework crates themselves?

I don't see one unfortunately; (one of) the selling points of objc2-* is that they are pre-generated, and thus don't need to ever touch the Xcode SDKs. And opting in to it without adding a build.rs in the first place would probably be technically challenging in Cargo (?).