Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make fails on a Mac - sg.h not found #38

Open
mccodesurfer opened this issue Feb 4, 2019 · 46 comments
Open

make fails on a Mac - sg.h not found #38

mccodesurfer opened this issue Feb 4, 2019 · 46 comments

Comments

@mccodesurfer
Copy link

Hi, There seems to be a missing file in the repository. make fails with "scsi/sg.h not found" after cloning the repo and running make on a Mac.

@asalamon74
Copy link
Owner

There is no SCSI Generic driver for Mac, that's why you get this error message.

And this is the reason why Mac is not supported.

@asalamon74 asalamon74 changed the title make fails - sg.h not found make fails on a Mac - sg.h not found Feb 5, 2019
@mccodesurfer
Copy link
Author

mccodesurfer commented Feb 5, 2019 via email

@asalamon74
Copy link
Owner

Maybe they are using some different kind of protocol. Can you share a link to the USB driver you are referring to.

@mccodesurfer
Copy link
Author

mccodesurfer commented Feb 5, 2019 via email

@rlancaste
Copy link

Yes, the INDI GPhoto driver is built on libgphoto2. This library builds on Linux, Mac, and even Windows. So we are able to provide INDI support on Macs and Linux for Canon, Nikon, and any other cameras that libgphoto2 supports. My understanding is that the library talks directly to the camera to instruct it to take exposures and that the issue with Pentax is that it doesn't include the ability to take exposures this way. Is that correct?

@rlancaste
Copy link

Does PKTrigger Cord completely control via USB?

@rlancaste
Copy link

So I am guessing that PKTriggerCord uses a different type of connection to the camera than libgphoto2?

@asalamon74
Copy link
Owner

It seems that modifying the bsd port of the program is more promising. With a simple hack I was able to compile it on a Mac. It still has lots of warnings and need testing. I'll investigate it soon.

@mccodesurfer
Copy link
Author

If you think you can gain access to the Mac USB then I'll hold off making a dylib. Let me know as soon as you can. We may incorporate pktriggercord directly into Kstars and bypass gphoto2 for Pentax.

@Blueshawk
Copy link

Yes, the INDI GPhoto driver is built on libgphoto2. This library builds on Linux, Mac, and even Windows. So we are able to provide INDI support on Macs and Linux for Canon, Nikon, and any other cameras that libgphoto2 supports. My understanding is that the library talks directly to the camera to instruct it to take exposures and that the issue with Pentax is that it doesn't include the ability to take exposures this way. Is that correct?

I don't have a Mac but maybe I can help explain the state of things. The way the indi_pentax driver works is ekos>>indi>>libgphoto>>pktriggercord. The indi driver wraps to libgphoto which is in turn wrapping to PkTriggercord for calls to the camera using "MSC" (usb:scsi) through the usb port to the camera instead of the PTP used by the other brands.
Cheers,
Ray

@rlancaste
Copy link

Ok, does libgphoto2 run pktriggercord as a separate program where it executes commands or does it use it as a library with functions that libgphoto2 calls?

@asalamon74
Copy link
Owner

They import the source code of pktriggercord, you can find it here: https://github.com/gphoto/libgphoto2/tree/master/camlibs/pentax

They sync the source code sometimes. Occasionally there ask for changes in pktriggercord to make it easiere to import it, but it's quite rate. For instance: #22

@rlancaste
Copy link

Ok I got it, so the pktriggercord code in libgphoto2 just goes into one of the camlibs in the camlibs folder. You don't have to launch a separate program that gets called. Thank you for the clarification.

@mccodesurfer
Copy link
Author

Thanks @asalamon74 . I looked at the camlibs/pentax folder and it doesn't include USB/SCSI driver so gphoto2 guys must provide one, which makes me wonder why its not working on the Mac, unless they also use a generic scsi driver. I'll see if I can find it but so far its looking like I'll have to write one.

@asalamon74
Copy link
Owner

I created a hack to compile for Mac OS X. The macosx branch contains it: https://github.com/asalamon74/pktriggercord/tree/macosx

Basically I tried to use the BSD port of the program. Since scsiio.h was missing I copied the file from BSD. It is possible to compile the program. GTK is a bit tricky so it's better to compile it using make cli.

When I connect my camera (K-x) the system recognized it, at least I can get info using ioreg -p IOUSB -l. The BSD port was assuming /dev/rsd* device which is mising from Mac OS X. The new devices after I connect the camera: /dev/disk2, /dev/disk2s1, /dev/rdisk2, /dev/rdisk2s1. I've tried all the devices but only got Device open while querying: and IOCTL failed in query error messages. The first one can be eliminated by umounting.

I don't know too much about Mac OS X ioctl, so any help would be appreciated.

@mccodesurfer
Copy link
Author

That’s excellent! I have written the ioctl part. I was just trying to test it. Seems to be working but your Mac build will help a lot. Now I can try sending actual camera commands. I copied the scsiio.h also but I think in the end I won’t need it. Question though. My stuff is written in objective C. Can I add it to pk and compile? How to compile with C ?

@mccodesurfer
Copy link
Author

Cause I’m using the Xcode environment. Will gcc recognize objective C?

@rlancaste
Copy link

You could do this in terminal on OS X, it worked fine for me in an experiment:

mkdir ~/Desktop/test
cd ~/Desktop/test
git clone https://github.com/asalamon74/pktriggercord.git
cd pktriggercord
git checkout macosx
make cli

@asalamon74
Copy link
Owner

@mccodesurfer Please only use C, not objective C. Gcc might recognize objective C, but we need to compile pktriggercord on different platforms, so the best is to use only ANSI C.

@mccodesurfer
Copy link
Author

May have to provide a library then because the code is not written from scratch. Uses macOS frameworks and libraries. The way I understand it. I just have to put the dylib file in the /usr/local/lib but not sure how to link it in the make file

@asalamon74
Copy link
Owner

@mccodesurfer If you provide a library then the language of the library is not that important, only the few calls should be written in C.

Just hack together something with using the -I -L -l switches (-I for the directory containing the headers, -L for the directory containing the lib, -l for the library linking) and I try to clean it later.

@mccodesurfer
Copy link
Author

Ok. Will do. It may take a week or two. I’m not on it as much as I’d like.

@mccodesurfer
Copy link
Author

@asalamon74 I am close with this macOS driver - getting status from camera failing on commands- I haven't been able to relate your USB description "pentax_scsi_protocol.md" to this doc: https://www.beyondlogic.org/usbnutshell/usb6.shtml#SetupPacket

In particular, the pslr_init sets up ioctl for BSD. I have a similar ioctl that uses the setupPacket structure. Your cmd structure is similar (I guess) but I haven't figured out how to form it into the packet structure the macOS uses.

@mccodesurfer
Copy link
Author

Any advice would be great. Thanks

@asalamon74
Copy link
Owner

@mccodesurfer

pentax_scsi_protocol.md describes a higher level protocol. It describes the byte arrays we send to the camera, the bytes array read from the camera. Probably you don't really need this.

The low level interfaces define scsi_read and scsi_write methods to send/receive data.

The pslr_init tries to find the device and initialize it. Probably you are referring to the get_drive_info method called by pslr_init. Is that correct? On OpenBSD it uses ioctl, but it's possible that on Mac there are other ways to get the information. For instance the linux version reads the info from the /sys filesystem without using ioctl. Unfortunately it seems to me that there is no such filesystem on Mac.

@mccodesurfer
Copy link
Author

Yes, I'm trying to emulate the low level SCSI_read, SCSI_write that call ioctl (in bsd version). In particular, I replace ioctl - this seems to be where the rubber meets the road - with some macOS interfaces. One called "DeviceRequest" use is very similar to ioctl; build a USB byte stream structure with USB byte codes (per the link I gave you) and call DeviceRequest on the structure.

From what you're saying, I guess I need to encapsulate the Pentax protocol into the DeviceRequest protocol. I was thinking they were the same level. They seamed to be the same in many respects, like the F0 and TT codes look like bmRequestType and bRequest. Since F0 code defines vendor specific codes to follow, it looks like this byte stream protocol is USB, not Pentax specific. However, F0 in particular does not exactly match the USB protocol in the link - if F0 where bmRequestType, it would be a reserved code. Just thinking out loud here.

I guess, I have to find out more about using DeviceRequest.

@mccodesurfer
Copy link
Author

I decided to shift the interface with Wireshark on a pc and see that indeed there is 30 bytes or so preamble to the pentax command codes... still working on it!

@mccodesurfer
Copy link
Author

Got Wireshark on Mac now. Looks like I have to make "interface" transactions, which I had working a while ago but its not working right now. Anyway, just letting you know the status.

@rlancaste
Copy link

Good to see you are making progress! I’m wondering if you have given any thought yet as to whether we want to get the code for controlling this into libgphoto or whether we want to make a new INDI driver for pentax entirely ? Obviously the library you are writing could work either way, but I’m just trying to plan ahead for when you finish. Personally I think the best route due to encapsulation and simplicity would be if we can get it into gphoto

@mccodesurfer
Copy link
Author

Thanks Rob! I like the idea of a Pentax driver. Its been almost a year and I don't believe GPhoto2 has yet been updated with some significant fixes in pktethercord. For the future, Pentax has finally come out with an SDK for their cameras but only support releases from the first k-1 and later. But future ports from their SDK could be truly full featured.

@tempelmann
Copy link

tempelmann commented Apr 1, 2019

Any news on this project? It's not clear to me whether there's so far an actual working Mac version of pktriggercord.

I tried rlancaste's instructions, but when I run it with sudo, I only keep getting a "Resource busy" message.

I also wonder if SIP has to be turned off for this to work (mine is off).

I'm testing on 10.13.6

@rlancaste
Copy link

Which command gives "resource busy"

@mccodesurfer
Copy link
Author

I'm still working on it...slowly. Ran into a roadblock on the USB access and got some help on Codementor. The guy steered me in a new direction using new high level Apple API for camera access and control, which looks really easy to use but I'm just getting going on it. It seems to be undocumented - very new I guess.

@tempelmann
Copy link

tempelmann commented Apr 1, 2019

@rlancaste I get this error when I invoke the cmd without arguments.

@mccodesurfer Typical for Apple these days. They can't keep up any more. But that API sounds like it won't use SCSI cmds, so are you sure you're not going in the wrong direction there? Or are you implementing a "driver" for that new API?

I'm still investigating the "macosx" branch. Am currently getting the project to run in Xcode to make debugging easier.

@tempelmann
Copy link

tempelmann commented Apr 1, 2019

Here's what I've found out so far:

this:
fd = open(deviceName, O_RDWR);
fails always (10.13.6) with error "Device busy".

Changing it to
fd = open(deviceName, O_RDWR | O_SHLOCK);
makes it work.

However, the next call fails again:
ioctl(fd, SCIOCCOMMAND, &screq)
It gives "Inappropriate ioctl for device" as the error msg.

@rlancaste On which macOS version did you test this? And where is _IOWR('Q' documented?

This SO question about using SCSI cmds on OSX suggests we'd need a IOKit driver for that: https://stackoverflow.com/a/7349373/43615. So, am I missing something? Do I need to install an extra kext first for this to work? @rlancaste Could it be that you already have some 3rd party SCSI-supporting driver installed that facilitates this, while I do not?

An interesting thing is that the cmd ioreg -r -c IOSCSIPeripheralDeviceNub lists my attached Pentax as a SCSI device. So, macOS know about SCSI, it just doesn't let us talk to it via SCSI-Pass-Through. According to all the info I could find so far, the only (and proper) way would be to write a IOKit driver that matches on Pentax vendor codes and then injects itself as a block device driver, with providing the added functionality for SCSI ioctl calls. I've done basic IOKit drivers before, but I have no idea how to do the SCSI part of it as I do not understand the interaction of it all.

@mccodesurfer
Copy link
Author

@tempelmann It looks like the API will give access to low level commands that will allow me to tunnel the camera commands through the USB interface. So its experimental.

@rlancaste
Copy link

So just to be clear. I just test compiled the code. I cannot actually test the code because I don't have a compatible camera. Originally the code would not compile, so I was just verifying that it could be compiled.

@tempelmann
Copy link

@rlancaste Oh, bummer then. That means my assessment is correct. Gotta hope @mccodesurfer can get something working, then. Good luck!

@mccodesurfer
Copy link
Author

Well, I'm sorry guys but I've given up on the trying to port to the Mac. It seems every turn ends up a dead end with lack of information. I'm not prepared to put any more time into it. I think the fact that the Pentax is supported on linux is something I can live with. Since Pentax has released new software dev tools, maybe there will be another solution soon.

@tempelmann
Copy link

tempelmann commented Feb 9, 2022

I had another look after learning how to use Apple's native SCSI APIs:

  1. There is really no pass-thru method in macOS via ioctl(), so the BSD access method won't work here.
  2. To use the native API (via <IOKit/scsi/SCSITaskLib.h>), the device must not be claimed already by a macOS driver, which is will if the Camera is in MSC mode: Then it'll present itself as a mass storage device, and macOS will take over, and thereby denying access via aforementioned SCSI APIs.

Which means we'd need to use PTP mode so that macOS won't claim the camera's access for itself. But AFAIK, pktriggercord does not work with the camera in PTP mode. So that's not an option.

So we need the camera to be in MSC mode - but then macOS takes over. A way out of this dilemma might be to make a simple IOKit KEXT that prevents macOS from inserting its mass storage driver. Technically, the kext probably won't need any code, but only a plist that reserves itself for the Vendor ID of Pentax cameras (0x25fb, I think). That should prevent Apple's IOKit drives from taking over.

But: Apple has announced that KEXTs won't be allowed in macOS in the future, for (in this case unnecesary and counterproductive) security reasons. Still, with older macOS versions (before macOS 12 "Monterey"), a KEXT might be possible, maybe even still thereafter; Apple is quite vague about this (they say that some types may still be allowed).

So, let's assume we can make such a kext. Then the next problem is that the SCSI driver won't get loaded either, I believe, and therefore the SCSI API won't be usable either.

But without the SCSI API, we'll have to talk to the Camera via the USB API instead.

So the big question is: How do we send SCSI commands over USB?

That's the part I don't understand yet. How does one use the lower level USB protocol to talk SCSI-over-USB? Are there special USB channels involved in that, or do the scsi commands get wrapped somehow? Does someone have insight in this? Or does pktriggercord even do this already in cases where there's no SCSI-pass-thru?

@rlancaste
Copy link

So, one thing that might help here is that we already disable the PTP Camera program because we have to do that to control Canon cameras and I think other DSLR's as well. That means that it does not get taken over by MacOS and is left open for gphoto control. That's already a part of KStars https://github.com/KDE/kstars/blob/ff1ac9cfeb1f11b4fb8725b643ae9e0c8950ab09/kstars/ekos/manager.cpp#L857

I don't know if this will help resolve the issue you mentioned here, but maybe.

@tempelmann
Copy link

tempelmann commented Feb 10, 2022

@rlancaste Thanks for keeping an eye on this topic. The code you've shown is handling a user-owner process, though, which is easy to deal with (i.e. simply force-quit it) in order to release any hold on the PTP device. With the block storage device this is more difficult because it's macOS (i.e. the kernel) that takes over control of it. And there are no APIs to force it to give it up.

I am currently exploring a different path, though. I have been able to open the endpoints to my K-5 when it's in PTP mode (but not in MSC mode - there I can only open the "usb device" services, but not the "usb interface" services). I've also found the docs on how to send SCSI cmds over USB, at least for the "BOT" mode.

Can all supported Pentax cameras be used with pktriggercard in PTP mode, or do some of them only work with MSC?

@56Tbird
Copy link

56Tbird commented Feb 10, 2022 via email

@rlancaste
Copy link

No problem, sorry I can't be more help with it, I don't have as much experience working with lower level stuff. I modified a couple of INDI drivers to make them work better on Macs, but they didn't need the kind of work you are talking about. But if you get the pktriggercord program working on Mac, I can help by getting it in the 3rdParty build and I can get it building with the rest of the MacOS build for KStars and INDI on craft. I'm currently working toward the goal of getting it to do nightly builds on the server.

@tempelmann
Copy link

Anything on my question whether pgtriggercord works with the camera being in PTP mode instead of MSC? I can't find a definitive answer in the docs or the code on this.

@mccodesurfer
Copy link
Author

This PKTriggercord doc shows that it uses MSC - I tried to work on this a long time ago and gave up. I later employed Codementor help but it seems one would have to use ImageCapture (or its successor) to talk to the Camera. But it would be great if someone found an alternative
pentax_scsi_protocol.md
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants