Skip to content

Commit

Permalink
Merge pull request #179 from ko80/dev/add-mcm-ffmpeg-audio-plugin
Browse files Browse the repository at this point in the history 
Add FFmpeg MCM audio plugin
  • Loading branch information
@ko80
ko80 authored Aug 29, 2024
2 parents 43290bb + 2a5b1aa commit d2a7619
Show file tree
Hide file tree
Showing 9 changed files with 780 additions and 201 deletions.
48 changes: 29 additions & 19 deletions ffmpeg-plugin/6.1/0001-mcm-add-in-out-dev-support.patch
View file
Original file line number Diff line number Diff line change
@@ -1,44 +1,47 @@
From 07fac1a4532e1d10660e8e138360b0e3673aac9e Mon Sep 17 00:00:00 2001
From: Ilichev, Konstantin <konstantin.ilichev@intel.com>
Date: Tu, 14 May 2024 05:48:00 +0100
Subject: [PATCH 1] Enable FFmpeg support in MCM
From fbb0700ea7bd9b055ae9e43746a7751be88a8955 Mon Sep 17 00:00:00 2001
From: Konstantin Ilichev <konstantin.ilichev@intel.com>
Date: Wed, 21 Aug 2024 14:55:08 +0000
Subject: [PATCH] Enable FFmpeg support in MCM

---
configure | 6 ++++++
libavdevice/Makefile | 2 ++
libavdevice/alldevices.c | 2 ++
3 files changed, 10 insertions(+)
configure | 9 +++++++++
libavdevice/Makefile | 5 +++++
libavdevice/alldevices.c | 5 +++++
3 files changed, 19 insertions(+)

diff --git a/configure b/configure
index 96b181fd21..e5320f8751 100755
index 5af693c954..60caa4ebdf 100755
--- a/configure
+++ b/configure
@@ -292,6 +292,7 @@ External library support:
@@ -285,6 +285,7 @@ External library support:
--enable-libvo-amrwbenc enable AMR-WB encoding via libvo-amrwbenc [no]
--enable-libvorbis enable Vorbis en/decoding via libvorbis,
native implementation exists [no]
+ --enable-mcm enable Media Communications Mesh library [no]
--enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no]
--enable-libwebp enable WebP encoding via libwebp [no]
--enable-libx264 enable H.264 encoding via x264 [no]
@@ -1904,6 +1905,7 @@ EXTERNAL_LIBRARY_LIST="
@@ -1841,6 +1842,7 @@ EXTERNAL_LIBRARY_LIST="
jni
ladspa
lcms2
+ mcm
libaom
libaribcaption
libass
@@ -3689,6 +3691,8 @@ xwma_demuxer_select="riffdec"
@@ -3567,6 +3569,11 @@ xwma_demuxer_select="riffdec"
android_camera_indev_deps="android camera2ndk mediandk pthreads"
alsa_indev_deps="alsa"
alsa_outdev_deps="alsa"
+mcm_indev_deps="libmcm_dp"
+mcm_outdev_deps="libmcm_dp"
+mcm_audio_indev_deps="libmcm_dp"
+mcm_audio_outdev_deps="libmcm_dp"
+mcm_audio_pcm16_outdev_deps="libmcm_dp"
avfoundation_indev_deps="avfoundation corevideo coremedia pthreads"
avfoundation_indev_suggest="coregraphics applicationservices"
avfoundation_indev_extralibs="-framework Foundation"
@@ -6867,6 +6871,8 @@ enabled libaribb24 && { check_pkg_config libaribb24 "aribb24 > 1.0.3" "ar
@@ -6692,6 +6699,8 @@ enabled libaribb24 && { check_pkg_config libaribb24 "aribb24 > 1.0.3" "ar
enabled libaribcaption && require_pkg_config libaribcaption "libaribcaption >= 1.1.1" "aribcaption/aribcaption.h" aribcc_context_alloc
enabled lv2 && require_pkg_config lv2 lilv-0 "lilv/lilv.h" lilv_world_new
enabled libiec61883 && require libiec61883 libiec61883/iec61883.h iec61883_cmp_connect -lraw1394 -lavc1394 -lrom1394 -liec61883
Expand All @@ -48,30 +51,37 @@ index 96b181fd21..e5320f8751 100755
enabled libbluray && require_pkg_config libbluray libbluray libbluray/bluray.h bd_open
enabled libbs2b && require_pkg_config libbs2b libbs2b bs2b.h bs2b_open
diff --git a/libavdevice/Makefile b/libavdevice/Makefile
index c30449201d..f6a44520ca 100644
index c30449201d..62bda757c3 100644
--- a/libavdevice/Makefile
+++ b/libavdevice/Makefile
@@ -20,6 +20,8 @@ OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o
@@ -20,6 +20,11 @@ OBJS-$(CONFIG_AUDIOTOOLBOX_OUTDEV) += audiotoolbox.o
OBJS-$(CONFIG_AVFOUNDATION_INDEV) += avfoundation.o
OBJS-$(CONFIG_BKTR_INDEV) += bktr.o
OBJS-$(CONFIG_CACA_OUTDEV) += caca.o
+OBJS-$(CONFIG_MCM_INDEV) += mcm_rx.o
+OBJS-$(CONFIG_MCM_OUTDEV) += mcm_tx.o
+OBJS-$(CONFIG_MCM_INDEV) += mcm_video_rx.o mcm_common.o
+OBJS-$(CONFIG_MCM_OUTDEV) += mcm_video_tx.o mcm_common.o
+OBJS-$(CONFIG_MCM_AUDIO_INDEV) += mcm_audio_rx.o mcm_common.o
+OBJS-$(CONFIG_MCM_AUDIO_OUTDEV) += mcm_audio_tx.o mcm_common.o
+OBJS-$(CONFIG_MCM_AUDIO_PCM16_OUTDEV) += mcm_audio_tx.o mcm_common.o
OBJS-$(CONFIG_DECKLINK_OUTDEV) += decklink_enc.o decklink_enc_c.o decklink_common.o
OBJS-$(CONFIG_DECKLINK_INDEV) += decklink_dec.o decklink_dec_c.o decklink_common.o
OBJS-$(CONFIG_DSHOW_INDEV) += dshow_crossbar.o dshow.o dshow_enummediatypes.o \
diff --git a/libavdevice/alldevices.c b/libavdevice/alldevices.c
index 8a90fcb5d7..4dd6194b02 100644
index 8a90fcb5d7..fe0eabdd67 100644
--- a/libavdevice/alldevices.c
+++ b/libavdevice/alldevices.c
@@ -25,6 +25,8 @@
@@ -25,6 +25,11 @@
/* devices */
extern const AVInputFormat ff_alsa_demuxer;
extern const FFOutputFormat ff_alsa_muxer;
+extern const AVInputFormat ff_mcm_demuxer;
+extern const FFOutputFormat ff_mcm_muxer;
+extern const AVInputFormat ff_mcm_audio_demuxer;
+extern const FFOutputFormat ff_mcm_audio_muxer;
+extern const FFOutputFormat ff_mcm_audio_pcm16_muxer;
extern const AVInputFormat ff_android_camera_demuxer;
extern const FFOutputFormat ff_audiotoolbox_muxer;
extern const AVInputFormat ff_avfoundation_demuxer;
--
2.34.1

144 changes: 136 additions & 8 deletions ffmpeg-plugin/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,32 @@ Install dependencies and build MCM as described in the top level README.md, para
./build-ffmpeg.sh
```

## Arguments
TBD
## MCM connection configuration

The next arguments are supported to configure a connection to MCM

| Argument | Type | Description | Default |
| --------------- | :-----: | -------------------------------------------------------- | :--------------: |
| `ip_addr` | String | Remote IP address | `"192.168.96.1"` |
| `port` | String | Remote port (Sender), or Local port (Receiver) | `"9001"` |
| `protocol_type` | String | MCM Protocol type (`"auto"`, `"memif"`, etc.) | `"auto"` |
| `payload_type` | String | ST2110 payload type (`"st20"`, `"st22"`, `"st30"`, etc.) | `"st20"` |
| `socket_name` | String | Memif socket name | - |
| `interface_id` | Integer | Memif interface id | `0` |

## Video configuration

The next arguments are supported to configure a video transmission

| Argument | Type | Description | Default |
| -------------- | :----: | ----------------------------------------------- | :-------------: |
| `video_size` | String | Video frame size (`"640x480"`, `"hd720"`, etc.) | `"1920x1080"` |
| `pixel_format` | String | Video pixel format | `"yuv422p10le"` |
| `frame_rate` | String | Video frame rate (`25`, `50`, `60`, etc.) | `25` |

## Run
## Example – Run video transmission

This test run demonstrates sending a video file from the 1st FFmpeg instance to the 2nd FFmpeg instance via MCM and then stream it to a remote machine via UDP.
This example demonstrates sending a video file from the 1st FFmpeg instance to the 2nd FFmpeg instance via MCM and then stream it to a remote machine via UDP.

### NIC setup

Expand All @@ -46,8 +66,13 @@ TBD
2. Start FFmpeg to receive frames from MCM and stream to a remote machine via UDP
```bash
sudo MCM_MEDIA_PROXY_PORT=8002 ffmpeg -re -f mcm \
-frame_rate 24 -video_size nhd -pixel_format yuv422p10le -protocol_type auto \
-payload_type st20 -ip_addr 192.168.96.1 -port 9001 \
-frame_rate 24 \
-video_size nhd \
-pixel_format yuv422p10le \
-protocol_type auto \
-payload_type st20 \
-ip_addr 192.168.96.1 \
-port 9001 \
-i - -vcodec mpeg4 -f mpegts udp://<remote-ip>:<remote-port>
```

Expand All @@ -60,8 +85,13 @@ TBD
2. Start FFmpeg to stream a video file to the receiver via MCM
```bash
sudo MCM_MEDIA_PROXY_PORT=8001 ffmpeg -i <video-file-path> -f mcm \
-frame_rate 24 -video_size nhd -pixel_format yuv422p10le -protocol_type auto \
-payload_type st20 -ip_addr 192.168.96.2 -port 9001 -
-frame_rate 24 \
-video_size nhd \
-pixel_format yuv422p10le \
-protocol_type auto \
-payload_type st20 \
-ip_addr 192.168.96.2 \
-port 9001 -
```

### VLC player setup
Expand All @@ -71,6 +101,104 @@ On the remote machine start the VLC player and open a network stream from the ne
udp://@:1234
```

## Audio configuration

The table below shows a proper way to configure the sender and the receiver depending on the audio PCM encoding format

| Audio encoding | Sender configuration | Receiver configuration |
| --- | --- | --- |
| PCM 24-bit | Output device `mcm_audio` | Input device `mcm_audio` and argument `-pcm_fmt pcm24`
| PCM 16-bit | Output device `mcm_audio_pcm16` | Input device `mcm_audio` and argument `-pcm_fmt pcm16`

The next arguments are supported to configure an audio transmission

| Argument | Type | Description | Default |
| ------------- | :-----: | ------------------------------------------------ | :-------: |
| `channels` | Integer | Number of audio channels (`1`, `2`, etc.) | `2` |
| `sample_rate` | Integer | Audio sample rate (`44100`, `48000`, or `96000`) | `48000` |
| `ptime` | String | MTL audio packet time (`"1ms"` or `"125us"`) | `"1ms"` |
| `pcm_fmt` | String | PCM audio format (`"pcm24"` or `"pcm16"`) | `"pcm24"` |

## Example – Run audio transmission

This example demonstrates sending an audio file from the 1st FFmpeg instance to the 2nd FFmpeg instance via MCM.

There are two options of configuration:
* **Option A** for PCM 24-bit encoded audio
* **Option B** for PCM 16-bit encoded audio

### NIC setup

TBD

### Receiver side setup

1. Start media_proxy
```bash
sudo media_proxy -d 0000:32:11.1 -i 192.168.96.2 -t 8002
```
2. Start FFmpeg to receive packets from MCM and store on the disk

**Option A – PCM 24-bit audio**
```bash
sudo MCM_MEDIA_PROXY_PORT=8002 ffmpeg -re -f mcm_audio \
-channels 2 \
-sample_rate 48000 \
-ptime 1ms \
-pcm_fmt pcm24 \
-protocol_type auto \
-payload_type st30 \
-ip_addr 192.168.96.1 \
-port 9001 \
-i - output.wav
```

**Option B – PCM 16-bit audio**
```bash
sudo MCM_MEDIA_PROXY_PORT=8002 ffmpeg -re -f mcm_audio \
-channels 2 \
-sample_rate 48000 \
-ptime 1ms \
-pcm_fmt pcm16 \
-protocol_type auto \
-payload_type st30 \
-ip_addr 192.168.96.1 \
-port 9001 \
-i - output.wav
```

### Sender side setup

1. Start media_proxy
```bash
sudo media_proxy -d 0000:32:11.0 -i 192.168.96.1 -t 8001
```
2. Start FFmpeg to stream an audio file to the receiver via MCM

**Option A – PCM 24-bit audio**
```bash
sudo MCM_MEDIA_PROXY_PORT=8001 ffmpeg -i <audio-file-path> -f mcm_audio \
-channels 2 \
-sample_rate 48000 \
-ptime 1ms \
-protocol_type auto \
-payload_type st30 \
-ip_addr 192.168.96.2 \
-port 9001 -
```

**Option B – PCM 16-bit audio**
```bash
sudo MCM_MEDIA_PROXY_PORT=8001 ffmpeg -i <audio-file-path> -f mcm_audio_pcm16 \
-channels 2 \
-sample_rate 48000 \
-ptime 1ms \
-protocol_type auto \
-payload_type st30 \
-ip_addr 192.168.96.2 \
-port 9001 -
```

## Known Issues

### Shared libraries error:
Expand Down
Loading

0 comments on commit d2a7619

Please sign in to comment.