Vision framework compatibility

[flaushi]

I am using the Vision framework but the program crashes as soon as I access the pixelBuffer property of the result observation. This is my code:

    VNCoreMLModel * model = [VNCoreMLModel modelForMLModel:fns.model  error:&error];
    VNCoreMLRequest * request = [[VNCoreMLRequest alloc] initWithModel:model completionHandler:^(VNRequest * _Nonnull request, NSError * _Nullable error) {
            if(error) {
                NSLog(@"completionHandler Error %@", error);
                return;
            }
            for (VNObservation * observation in request.results){
                if(observation.class == [VNPixelBufferObservation class]){
                    VNPixelBufferObservation * pxbufobs = (VNPixelBufferObservation*)observation;
                    
                      
                    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pxbufobs.pixelBuffer]; // <<---EXC_BAD_ACCESS here!!!
                    
                    CIContext *temporaryContext = [CIContext contextWithOptions:nil];
                    CGImageRef videoImage = [temporaryContext
                                             createCGImage:ciImage
                                             fromRect:CGRectMake(0, 0,
                                                                 CVPixelBufferGetWidth(pxbufobs.pixelBuffer),
                                                                 CVPixelBufferGetHeight(pxbufobs.pixelBuffer))];
                    
                    // resulting UIImage
                    UIImage * image = [UIImage imageWithCGImage:videoImage];
                    CGImageRelease(videoImage);
                }
                    
            }
        }];

The fns is a .mlmodel converted with this project and dragged into my xcode project.

Interestingly, the models offered here do not work in xcode 9.1. It complains about the missing header file (although it is there). So, I was not able to test with mlmodels from alternative sources.

[opedge]

I also have bad_access in VNPixelBufferObservation. I think it is because errors on Vision framework side as direct CoreML model usage is ok. I can suggest a simple workaround:

• Don't use image output in CoreML model. You need to convert models with these lines https://github.com/prisma-ai/torch2coreml/blob/master/example/fast-neural-style/convert-fast-neural-style.py#L155-L161 commented or deleted. You model will have multi-array output instead of pixelBuffer.
• Convert multi-array to image manually. Handy library for array->image function as well as multi-array manipulation functions: https://github.com/hollance/CoreMLHelpers

I create a quick and straightforward example using CoreMLHelpers (only for demo purpose, you need to handle errors more cleverly): https://gist.github.com/opedge/1e3a80528e2d30d2238bc7b18e0a2020
Please note that you need to add bias to output image and convert it from BGR to RGB manually.