Sony ptpip feature request

[falk0069]

This isn't an issue but rather a future feature request. I'm hoping one day gphoto2 will be able to tell the Sony cameras when it is transferring files and when it is done. You don't need to do this but I wanted to at least bring to your attention what I've figured out so far.

Basically I started up a mini project and I'm hoping to get other Sony users to try it out to verify it works. Please check it out if you have a chance:
https://github.com/falk0069/sony-pm-alt

It is quite a hack job I did to the ptpip.c code. I basically inject 29 data packets in order to tell the Sony camera this info. Also the transactionIDs are all over the board, but the camera doesn't seem to care.

Lastly, You'll notice I created a 500 byte sonyrequest buffer. I only should have needed to defined it as 26 bytes but when I made it smaller I occasionally got a variety of segmentation fault issues. I couldn't figure out where the issue was but I believe it might be the same issue that alik55 reported (#15). I feel that it might be in the logging code since this is where the core dumps generally blew up. Of course I might be doing something completely brain dead with my packet injections. By making the buffer large enough I seemed to avoid any illegal memory boundary writes.

I'm hoping you might entertain this idea and you don't find it upsetting. Please let me know what you think.

Thanks!

[msmeissn]

i added parts of this in real ptp code.

if you could try out current git of libgphoto2 and see if that already helps as well?

(it is not fully the same as you do yet.)

[msmeissn]

(check ptp_sony_9280 and ptp_sony_9281 implementation and calls in camera_init and camera_exit )

[falk0069]

Excellent! I'll check it out this weekend.
--Andy

[falk0069]

I'm still studying what you have done. From what I can tell, this is going to be great. Let me tell you what I've done so far.

1. My A5000 doesn't seem to have the SONY Extension ID. It comes across as 0xffffffff. Here is my debug_deviceinfo:
   0.847206 print_debug_deviceinfo (2): Device info:
   0.847227 print_debug_deviceinfo (2): Manufacturer: Sony Corporation
   0.847245 print_debug_deviceinfo (2): Model: ILCE-5000
   0.847263 print_debug_deviceinfo (2): device version: 1.0
   0.847282 print_debug_deviceinfo (2): serial number: '00000000000000003282651203392641'
   0.847301 print_debug_deviceinfo (2): Vendor extension ID: 0xffffffff
   0.847320 print_debug_deviceinfo (2): Vendor extension version: 100
   0.847459 print_debug_deviceinfo (2): Vendor extension description: (null)
   0.847485 print_debug_deviceinfo (2): Functional Mode: 0x0000
   0.847503 print_debug_deviceinfo (2): PTP Standard Version: 100
   0.847676 print_debug_deviceinfo (2): Supported operations:
   0.847905 print_debug_deviceinfo (2): 0x1002 (Open session)
   0.847946 print_debug_deviceinfo (2): 0x1003 (Close session)
   0.847970 print_debug_deviceinfo (2): 0x1001 (Get device info)
   0.847993 print_debug_deviceinfo (2): 0x1004 (Get storage IDs)
   0.848015 print_debug_deviceinfo (2): 0x1005 (Get storage info)
   0.848037 print_debug_deviceinfo (2): 0x1006 (Get number of objects)
   0.848059 print_debug_deviceinfo (2): 0x1007 (Get object handles)
   0.848081 print_debug_deviceinfo (2): 0x1008 (Get object info)
   0.848103 print_debug_deviceinfo (2): 0x1009 (Get object)
   0.848124 print_debug_deviceinfo (2): 0x100a (Get thumbnail)
   0.848174 print_debug_deviceinfo (2): 0x1014 (Get device property description)
   0.848198 print_debug_deviceinfo (2): 0x1015 (Get device property value)
   0.848221 print_debug_deviceinfo (2): 0x1016 (Set device property value)
   0.848243 print_debug_deviceinfo (2): 0x101b (Get partial object)
   0.848266 print_debug_deviceinfo (2): 0x9801 (Get object properties supported)
   0.848289 print_debug_deviceinfo (2): 0x9802 (Get object property description)
   0.848311 print_debug_deviceinfo (2): 0x9803 (Get object property value)
   0.848377 print_debug_deviceinfo (2): 0x9805 (Get object property list)
   0.848411 print_debug_deviceinfo (2): 0x9280 (Unknown PTP_OC)
   0.848434 print_debug_deviceinfo (2): 0x9281 (Unknown PTP_OC)
   0.848456 print_debug_deviceinfo (2): 0x9282 (Unknown PTP_OC)
   0.848478 print_debug_deviceinfo (2): 0x9283 (Unknown PTP_OC)
   0.848500 print_debug_deviceinfo (2): 0x9284 (Unknown PTP_OC)
   0.848522 print_debug_deviceinfo (2): 0x9285 (Unknown PTP_OC)
   0.848540 print_debug_deviceinfo (2): Events Supported:
   0.848558 print_debug_deviceinfo (2): 0x4004
   0.848576 print_debug_deviceinfo (2): 0x4005
   0.848594 print_debug_deviceinfo (2): 0x4006
   0.848611 print_debug_deviceinfo (2): Device Properties Supported:
   0.848628 print_debug_deviceinfo (2): 0x5001
   0.848645 print_debug_deviceinfo (2): 0xd402
   0.848662 print_debug_deviceinfo (2): 0xd406
   0.848678 print_debug_deviceinfo (2): 0xd407
   0.848695 print_debug_deviceinfo (2): 0xd303

So, I added this extra piece of code in the fixup_cached_deviceinfo function:
/* Observed a Sony A5000 report as PTP_VENDOR_MTP - Restore Sony vendor extid. */
if (di->VendorExtensionID == PTP_VENDOR_MTP && di->Manufacturer && !strcmp(di->Manufacturer,"Sony Corporation"))
{
di->VendorExtensionID = PTP_VENDOR_SONY;
}

That got it so the new code would be executed.

2. The next thing I noticed is during the camera_exit a failed message appeared. Here is a clip of the log:
   32.290719 gp_camera_exit (2): Exiting camera ('PTP/IP Camera')...
   32.290987 ptp_ptpip_sendreq (3): ptpip/oprequest data: (hexdump of 22 bytes)
   0000 16 00 00 00 06 00 00 00-01 00 00 00 81 92 a6 01 ................
   0010 00 00 04 00 00 00 - ......

32.299992 ptp_ptpip_generic_read (3): ptpip/generic_read header: (hexdump of 8 bytes)
0000 14 00 00 00 09 00 00 00- ........

32.300065 ptp_ptpip_generic_read (3): ptpip/generic_read data: (hexdump of 12 bytes)
0000 a6 01 00 00 00 00 00 00-00 00 00 00 ............

32.343034 ptp_ptpip_generic_read (3): ptpip/generic_read header: (hexdump of 8 bytes)
0000 0c 00 00 00 0a 00 00 00- ........

32.343106 ptp_ptpip_generic_read (3): ptpip/generic_read data: (hexdump of 4 bytes)
0000 a6 01 00 00 - ....

32.343352 ptp_ptpip_getresp ptpip.c:409: response type 10 packet?
32.343450 camera_exit library.c:2293: 'ptp_sony_9281(params, 0x4)' failed: (null) (0x9281)

So, tried adding an additional case entry in the ptp_ptpip_getresp function to support respType 10 (DATA_PACKET):
case PTPIP_DATA_PACKET:
GP_LOG_D("PTPIP_DATA_PACKET");
resp->Transaction_ID = dtoh32a(&data[0]);
free (data);
data = NULL;
goto retry;

This let it get further and it received this from the camera:
31.178738 ptp_ptpip_getresp (2): PTPIP_CMD_RESPONSE
31.178801 camera_exit library.c:2293: 'ptp_sony_9281(params, 0x4)' failed: PTP Device Busy (0x2019)

I figure this might be appearing because the initial 'sending' status isn't working yet, so it can't shut things down.

3. I also tried uncommenting some of the items you had in the camera_init. The first item that says it returns an error 2006, seems fine for me. The second one that says it returns 18 bytes instead of 16, doesn't work at all for me since there are too many parameters being passed to the function. I'm not sure yet how to correctly call these new functions.

4. Lastly, I stuck in some additional debug statements so I could see when the new functions fire. I just added this to the two new functions:
   ptp_debug( params, "ptp_sony_928X");
   I can definitely see them being called. What I noticed is every time the ptp_sony_9281 is called a huge data response comes back providing some sort of DeviceInfoXML. It looks pretty much the same the three times it is called in the camera_init. So, I'm not sure why it keep returning the same thing.

Attached is a recent log if you care to look (Note: I had to name it a gif file in order to upload).

I'll continue experimenting and report back anything I think might be useful.

Thanks
--Andy

[msmeissn]

instead of adding this specific case, can you go to camera_exit
and swap the ptp_sony_9280 and ptp_sony_9281 lines?

I think the 9280 call must come always before the 9281 call.

[msmeissn]

I applied your change from (1) to git. also swapped the ptp_sony_928x lines in camera_exit in current git.

[falk0069]

This is starting to get really exciting. I swapped the two lines and not only did it not fail, but it also shut down the camera like I want!

Also, using how you call the 9280 followed by the data packet I started playing with my raw packet injections again. I was really able to bring down the number of packets needed by following this standard.

So after the GetDeviceInfo call these are the only packets required to get the 'sending' to appear:
/* Op Req: 0x9280 /
16000000060000000000000080922300000004000000
/ Start Data Packet /
1400000009000000230000001200000000000000
/ Data Packet /
1e0000000a00000023000000020000000200000000000000000000000101
/ End Data Packet /
0c0000000c00000023000000
/ 14 bytes are received */

Then to get it to shut off I send:
/* Op Req: 0x9281 /
1600000006000000000000008192c602000004000000
/ 8250 bytes are received /
/ Op Req: 0x9280 /
1600000006000000000000008092c702000004000000
/ Start Data Packet /
1400000009000000c70200001000000000000000
/ Data Packet /
1c0000000a000000c702000000000000050000000000000000000000
/ End Data Packet /
0c0000000c000000c7020000
/ 14 bytes are received */

The 9281 I send prior to it shutting off probably is pending from the 9280 send to get it into sending mode. And probably a trailing 9080 might be needed after the camera turns off to be fully completed. I'm not sure how important it is to have these calls alternate. Still more experimenting needed

So, what I have right now in the camera_init is just this:
C_PTP (ptp_sony_9280(params, 0x4,2,2,0,0));

I found this will display 'connected' on the camera (not 'sending...'). This might be good enough to prevent the camera from disconnecting. I think the extra "0101" might need to be passed in to get the 'sending' to appear instead of the 'connected'

I'll keep playing around and let you know what I find.

Thanks!

--Andy

[falk0069]

Hi,

Okay last update of the day. I realized you can merge the Data Packet and End Data Packet into one by just making sure the 0x0c byte is set instead of the 0x0a byte. This saved another packet--which is what you are already doing.

Also, it does look like 9281 function needs to follow the 9280 function in the camera_init. However, in the camera_exit, I think only 9280 is needed. Looking back on some older network traces, I never see a final 9281 when shutting down the connection,

I also played around a bit with the ptp_sony_9280 function so it can take in those extra two bytes (01 01). I know this probably isn't the best way to do it, but this is how I coded it:

uint16_t
ptp_sony_9280 (PTPParams* params, uint32_t param1,
        uint32_t data1, uint32_t data2, uint32_t data3, uint32_t data4, uint8_t data5, uint8_t data6)
{
        PTPContainer    ptp;
        unsigned char   buf[18];
        unsigned char   *buffer;
        int             less = 0;
        /* may not need to full 18 bytes*/
        if (0 == data5 && 0 == data6)
        {
                less = 2;
        }
        ptp_debug( params, "ptp_sony_9280");

        PTP_CNT_INIT(ptp, 0x9280, param1);

        htod32a(&buf[0], data1);
        htod32a(&buf[4], data2);
        htod32a(&buf[8], data3);
        htod32a(&buf[12], data4);
        htod8a(&buf[16], data5);
        htod8a(&buf[17], data6);
        buffer=buf;
        return ptp_transaction(params, &ptp, PTP_DP_SENDDATA, sizeof(buf)-less, &buffer, NULL);
}

Then this is all I have in the camera_init:

C_PTP (ptp_sony_9280(params, 0x4,2,2,0,0, 0x01,0x01));
C_PTP (ptp_sony_9281(params, 0x4));

And this is all I have in the camera_exit

C_PTP (ptp_sony_9280(params, 0x4,0,5,0,0,0,0));

Thing are working exactly how I think they should now and I'm able to transfer pictures successfully.

Let me know what you think. And thank you for your kindness and wisdom.

--Andy

[msmeissn]

i did the ptp_sony_9280 a bit different, i think the first argument is actually the number of additional bytes sent.

commited your suggestions of 9280 and 9281 usage in camera_init and camera_exit.

[falk0069]

Brilliant! I just pulled the latest and everything works perfectly. I'll try to gather up other Sony users and verify it works for them as well. I'll provide feedback if needed. I think for now we can close this request. Thanks so much for helping.
--Andy

[msmeissn]

very nice :)

[falk0069]

Marcus,
I noticed the message displayed on the camera wasn't quite right. I did a little bit more network tracing and I noticed the extra two bytes are shifted to the left one. I then noticed this also in the new ptp_sony_9280 function. Do you think you could update the function to have this:

        /* only sent in the case where additional is 2 */
        buf[16]= x; buf[17]= y;

Things work as I would expect after doing this.

Let me know if you don't agree.

Thanks!

--Andy

[falk0069]

Hi Marcus,
I'm reopening this discussion because I'm not sure if you got notified of my last comment about the extra bytes being shifted. Hopefully you agree with my analysis.
Thanks
--Andy

[msmeissn]

i saw it and already applied the fix to git

[falk0069]

yea! Thanks. Sorry didn't mean to spam you.