Skip to content

Commit

Permalink
Metadata documentation update
Browse files Browse the repository at this point in the history
Tracked On: DSO-6850

Change-Id: If07db154139d4e77a2226ec6c711f3a336747b58
Signed-off-by: Evgeni Raikhel <evgeni.raikhel@intel.com>
  • Loading branch information
ev-mp committed Aug 27, 2017
1 parent 8634989 commit f2d8d33
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 50 deletions.
2 changes: 1 addition & 1 deletion doc/distribution_linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,4 @@ realsense-sdk-dev | Header files and symbolic link for developers | realsense-sd
realsense-sdk-dbg | Debug symbols for developers | realsense-sdk

**Note** The packages include binaries only.
Use the github repository to obtain the source code.
Use the github repository to obtain the source code.
24 changes: 8 additions & 16 deletions doc/frame_metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ When a new device is recognized by `librealsense2` the internal resources for it
Note that each metadata attribute requires an explicit parser, and querying an unregistered attribute will result in an appropriate exception raised by the library.
With the metadata handlers in place, the inbound frames for the specific endpoint can be queried for the requested attributes.

The metadata registration flow is summarized in the following diagram*:
The metadata registration flow is summarized in the following diagram*:
>**Note** the diagrams present the high-level design. The actual implementation may differ.
![register for metadata](./metadata/metadata_registration_flow.bmp)
Expand All @@ -34,7 +34,6 @@ Metadata attributes propagation from the library to the user code is described i
![metadata acquisition](./metadata/metadata_acquisition_flow.bmp)

## Metadata Query API

`librealsense2` introduces two functions into its public API to query metadata attributes:
```cpp
/**
Expand Down Expand Up @@ -62,7 +61,7 @@ verifies that the hardware and software-applied preconditions for metadata parsi
```
will invoke the metadata parser to read the actual value from metadata payload. In case the attribute's origin is other than the payload, such as Auto-Exposure for Fisheye stream, then its value will be calculated internally by librealsense2.

# API Utilization
# Librealsense2 API for Metadata
The envisaged mode of querying metadata attribute is `check-then-query`
```cpp
rs2_metadata_t val = 0;
Expand Down Expand Up @@ -91,31 +90,24 @@ AUTO_EXPOSURE|1

![Demo Presenting metadata attributes](./metadata/metadata-cpp-config-ui.png)



# Metadata Support for Intel® RealSense™ devices
In order for `librealsense2` to get access for device-generated attributes the following system preconditions shall be met:
- OS shall support metadata provision - for Linux the specific kernel patches shall be applied.
- Device firmware shall declare and implement metadata attributes payload.

## OS support
Linux OS - the standard Linux UVC driver does not provide metadata support.
`librealsense2` package includes metadata patch for `uvcvideo` kernel object. The patches are intended to be deployed with Ubuntu 14 and Ubuntu 16.01 LTS with kernel version 4.4.0-xx., and are applied as part of Linux backend installation . See [Linux installation-guide](./installation.md) for more details.
***Linux OS*** - the standard Linux UVC driver does not provide metadata support.
`librealsense2` package includes metadata patch for `uvcvideo` kernel object. The patches are intended to be deployed with Ubuntu 14, 16.01/02 LTS with kernel versions 4.4, 4.8 and 4.10, and are applied as part of Linux backend installation . See [Linux installation-guide](./installation.md) for more details.
The patches were also successfully ported to Yocto-based distribution and made available for Yocto Reference Kit based on Poky 2.1.2 with kernel version 4.4.26.

Windows OS - the metadata is enabled if following requirements are met (see [installation manual](./installation_windows.md) for details):
- Windows10 RedStone1 or later.
- WinSDK 10.0586 is installed
- Dedicated registry keys are present for each camera device.

**Note** Windows10 has a known bug with providing metadata for multi-pin device - the metadata is available for the first pin only. In case of RS4XX device, the metadata for the Depth stream will be provided ,but not for the Infrared1/2 streams.
***Windows OS*** - Metadata extraction is supported by Microsoft starting with Windows10. Check [Windows installation guide](./installation_windows.md) for details.
**Note** Extracting metadata attributes is currently available for all UVC stream except for Infrared.

## Metadata attributes in RS4xx Devices
Support for metadata attributes for Depth, Infrared and Fisheye streams is provided with firmware version 5.5.0 or later.
## Metadata attributes in RS400 Devices
The device firmware implements a custom metadata payload compatible with [Microsoft Extensions for UVC ](https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/uvc-extensions-1-5) spec, and emits metadata attributes in the frame payload header.

The custom payload comprise of several data chunks ("C" structs) with multiple attributes each. The chunks are categorized as `calibration/configuration/capture/status`.
The chunks are further arranged into sets. For instance, given attributes chunks named `a,b,c,d,e,f,g`, the possible sets could be `{a,b,c,d}`, `{a,f,g,d}`,`{d,e,f,g}`. The design requires ,though, that the *essential frame attributes, such as Frame Number and Timestamp shall be included in all configured sets*, to allow a consistent track of the frames arrived.
During streaming the chunks are arranged into ordered sets. For instance, given attributes chunks named `a,b,c,d,e,f,g`, the possible sets could be `{a,b,c,d}`, `{a,f,g,d}`,`{d,e,f,g}`. The design requires ,though, that the *essential frame attributes, such as Frame Number and Timestamp shall be included in all configured sets*, to allow a consistent track of the frames arrived.

During the regular operation course the firmware internal state machine decides which attributes set to generate for the current frame. For instance, the first frames will be dispatched with `configuration/calibration` payloads, while the rest will carry the `capture` attributes.
From the user's perspective it means that a properly registered metadata attribute may still not be available for some frames, as metadata generation is governed by firmware design.
Binary file modified doc/img/windows_cmake.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
70 changes: 37 additions & 33 deletions doc/installation_windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,64 +2,68 @@

**Note:** Due to the USB 3.0 translation layer between native hardware and virtual machine, the librealsense team does not recommend or support installation in a VM.

librealsense shall be built on Windows using CMake and Visual Studio 2015:
(MSVC2013 and older are not compatible with the C++11 features set).
![Windows CMake](./img/windows_cmake.png)
librealsense shall be built on Windows using [CMake](https://cmake.org/download/) and Visual Studio 2015 / 2017:
(MSVC2013 and older are not fully compatible with the C++11 feature-set).
<a name="cmake_snapshot_win">
![Windows CMake](./img/windows_cmake.png)
</a>
![link text][Win_cmake_image]

Don't forget to check `BUILD_EXAMPLES` if you wish to use librealsense samples.

### Enabling per-frame metadata:
## Windows 8.1:
When working on Windows 8.1, make sure you have [KB3075872](https://support.microsoft.com/en-us/kb/3075872) and [KB2919355](https://support.microsoft.com/en-us/kb/2919355) installed. These patches are addressing issues with 8.1 video drivers, resolved in Windows 10.

## Enabling metadata on Windows
Metadata attributes is an advanced capability provided by librealsense.
Read more on this feature in [link](./frame_metadata.md).
Follow the instructions to enable metadata generation:

Prerequisites:
- Metadata support is available starting with Windows 10 build 1607 (RedStone1)
- Windows 10 with administrator login.
- WinSDK ver 10 (10.0.15063)
- A Registry key for metadata buffering shall be manually created.

#### Installation:
- Verifying OS version:
- Run "winver" command from desktop/terminal - "Version 1607" or later is expected.
- Verify OS version:
- Run `winver` command from desktop/terminal - "Version 1607" or later is expected.
![winver](./img/winver_Win10.png)


- Installing WinSDK ver10:
- Install WinSDK ver10:
- Navigate to "Control Panel" -> "Programs and Features"
- Double-click on "Microsoft Visual Studio" and select "Modify"
- Check that SDK version 10.0.10586 or later is present, install if needed:
- Check that SDK version 10.0.10586 or later is present, install if needed:
![winsdk](./img/WinSDK_10.0.10586.png)


- Registering Intel® RealSense™ device for metadata:
Windows OS requires a dedicated registry entry to be present for each unique video device in order to provide metadata support
- Windows OS requires a dedicated registry entry to be present for each unique video device in order to provide metadata support:
- Connect Intel® RealSense™ device to the host
- Navigate to "Control Panel" -> "Device Manager"
- Browse for Intel® RealSense™ devices
- Select the first device from the list, e.g. `Intel® RealSense™ Camera SR300 Depth` (Step 1)
- Find its path (Step 2) and additional interfaces (Step 3)
- Find device's path (Step 2) and the additional interfaces (Step 3)

![win_dev_master_interface](./img/win_device_interface.png)
![win_dev_master_interface](./img/win_device_sibling_interfaces.png)
- Modify Registry
- For each interface (Steps 2 and 3) perform
- Using Registry Editing tool such as "regedit" navigate to `HKLM\SYSTEM\CurrentControlSet\Control\DeviceClasses\{e5323777-f976-4f5b-9b55-b94699c46e44}` branch (Step 1)
- Browse into the subdirectory with the name identical to the `Device instance path` obtained from the previous step
- Expand the entry into `#GLOBAL` -> `Device Parameters` (Step 2)
- Add `DWORD 32bit` value named `MetadataBufferSizeInKB0` with value 4 (Step 3)
- Modify Windows Registry:
- For each interface (Steps 2 and 3) perform
- Using Registry Editing tool such as "regedit" navigate to `HKLM\SYSTEM\CurrentControlSet\Control\DeviceClasses\{e5323777-f976-4f5b-9b55-b94699c46e44}` branch (Step 1)
- Browse into the subdirectory with the name identical to the `Device instance path` obtained from the previous step
- Expand the entry into `#GLOBAL` -> `Device Parameters` (Step 2)
- Add `DWORD 32bit` value named `MetadataBufferSizeInKB0` with value 4 (Step 3)
![win_md_reg_key](./img/win_md_reg_key.png)

- Repeat the last two steps for
- Repeat the last two steps for
`HKLM\SYSTEM\CurrentControlSet\Control\DeviceClasses\{65E8773D-8F56-11D0-A3B9-00A0C9223196}` branch
- Repeat the procedure for all the additional RealSense devices (e.g `Intel® RealSense™ Camera SR300 RGB`)

#### Compilation:
- Repeat the procedure for all the additional RealSense devices (e.g. `Intel® RealSense™ Camera SR300 RGB`)

#### Compiling Librealsense with Metadata support:
During compilation the library will automatically detect and enable/disable metadata functionality according to the WinSDK version selected.
In case the active WinSDK doesn't provide for metadata, the compilation will complete successfully, but the user will be notified that the build does not include metadata.
The user may alter the policy and require that the metadata block will be always included regardless of the active WinSDK by enabling `ENFORCE_METADATA` Cmake option.
When set, the compiler will check for target platform version, and actively interrupt the build if the found version is below 10.0.10586.

When WinSDK10 is installed, CMake will automatically create MSVC project files targeted for the active platform (WinSDK8.1/10). In case it fails to select the required version correctly, it can be manually altered by retargetting the solution for the proper WinSDK version:
![win_retarget_platform](./img/win_retarget_platform.png)
In case the installed WinSDK doesn't expose metadata the user will be notified that the build does not include metadata generation:
`Librealsense notification: Featuring UVC Metadata requires WinSDK 10.0.10586.0 toolset.
The library will be compiled without the metadata support!`

In order to ensure compilation with metadata support check the `ENFORCE_METADATA` in the [Cmake configuration stage](#cmake_snapshot_win).
When set, the compiler will check the target WinSDK version, and abort the build if it is not compatible with metadata requirements.


## Additional Info
**Note:** When working on Windows 8.1, make sure you have [KB3075872](https://support.microsoft.com/en-us/kb/3075872) and [KB2919355](https://support.microsoft.com/en-us/kb/2919355) installed. These patches are addressing issues with 8.1 video drivers, resolved in Windows 10.
**Note** In case of multiple WinSDK version installed, CMake automatically selects the latest version of SDK. In some cases, however, the automatic selection may fail. Then you need to manually retarget the solution for the proper WinSDK version:
![win_retarget_platform](./img/win_retarget_platform.png)

0 comments on commit f2d8d33

Please sign in to comment.