Multienc is broken

[stan-smith]

Amlogic Hardware Video Encoder Investigation Report

Khadas VIM4 - Amlogic S905X4

Executive Summary

This report documents a comprehensive investigation into the Amlogic hardware video encoder on the Khadas VIM4 (Amlogic A311D2 SoC). After extensive testing and analysis, we have determined that the hardware encoder is non-functional due to an incomplete driver implementation. While the encoder accepts input frames, it provides no mechanism to retrieve encoded output, making it unsuitable for production use.

Table of Contents

1. Hardware and Software Environment
2. Initial Discovery
3. Low-Level Device Interface Analysis
4. Memory Mapping Investigation
5. High-Level API Testing
6. Driver Source Code Analysis
7. Test Programs and Results
8. Conclusions and Recommendations

---

Hardware and Software Environment

• Device: Khadas VIM4
• SoC: Amlogic A311D2
• Kernel: Linux 5.15.137
• Encoder Device: /dev/amvenc_multi
• Libraries:
  • libamvenc.so - High-level encoder API
  • libamvenc_api.so - Multi-encoder API wrapper
  • libvpcodec.so - VP codec library

Encoder Device Information

crw-rw-rw- 1 root root 464, 0 Jul 24 11:59 /dev/amvenc_multi

Related Kernel Modules

amvenc_multi           65536  2
encoder_common         16384  2 jpegenc,amvenc_multi

---

Initial Discovery

Original Symptoms

When attempting to use the Amlogic encoder through GStreamer:

// Original GStreamer usage attempt
GstElement *encoder = gst_element_factory_make("amlvenc", NULL);
// Result: Segmentation fault when processing frames

The encoder would crash with segmentation faults when attempting to encode video frames, suggesting fundamental issues with the driver implementation.

---

Low-Level Device Interface Analysis

IOCTL Discovery

We systematically discovered the following working IOCTLs:

IOCTL	Direction	Size	Description
0x00000002	None	-	TEST/INIT
0x40045603	Write	16 bytes	CONFIG (width, height, framerate, bitrate)
0x40045604	Write	8 bytes	Unknown (returns zeros)
0x40205600	Write	32 bytes	Unknown (fails with valid params)
0x40205601	Write	32 bytes	Unknown (accepts but returns nothing)
0x40205605	Write	32 bytes	PROCESS_FRAME
0x40205606	Write	32 bytes	Returns physical addresses
0x40205608	Write	32 bytes	Returns different physical addresses

IOCTL Test Program

/*
 * test_ioctl_discovery.c - Systematically discover working IOCTLs
 */

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <errno.h>
#include <string.h>
#include <stdint.h>

int main() {
    int fd = open("/dev/amvenc_multi", O_RDWR);
    if (fd < 0) {
        printf("Failed to open device: %s\n", strerror(errno));
        return 1;
    }
    
    printf("Testing IOCTLs on /dev/amvenc_multi\n\n");
    
    /* Test data structure */
    struct {
        uint64_t field0;
        uint64_t field1;
        uint64_t field2;
        uint64_t field3;
    } test_data = {0};
    
    /* Test range of IOCTLs */
    for (uint32_t cmd = 0; cmd <= 0x10; cmd++) {
        errno = 0;
        int ret = ioctl(fd, cmd, &test_data);
        if (ret == 0 || (ret == -1 && errno != ENOTTY)) {
            printf("IOCTL 0x%02x: ret=%d, errno=%d (%s)\n", 
                   cmd, ret, errno, strerror(errno));
        }
    }
    
    /* Test Write IOCTLs with different sizes */
    uint32_t types[] = {0x56}; /* Type V */
    uint32_t sizes[] = {4, 8, 16, 32};
    
    for (int t = 0; t < sizeof(types)/sizeof(types[0]); t++) {
        for (int s = 0; s < sizeof(sizes)/sizeof(sizes[0]); s++) {
            for (uint32_t nr = 0; nr <= 0x10; nr++) {
                uint32_t cmd = (0x40 << 24) | (sizes[s] << 16) | 
                              (types[t] << 8) | nr;
                
                errno = 0;
                test_data.field0 = test_data.field1 = 0;
                test_data.field2 = test_data.field3 = 0;
                
                int ret = ioctl(fd, cmd, &test_data);
                if (ret == 0 || (ret == -1 && errno != ENOTTY && errno != EINVAL)) {
                    printf("IOCTL 0x%08x: ret=%d, errno=%d (%s)\n", 
                           cmd, ret, errno, strerror(errno));
                    if (ret == 0) {
                        printf("  Response: %016lx %016lx %016lx %016lx\n",
                               test_data.field0, test_data.field1, 
                               test_data.field2, test_data.field3);
                    }
                }
            }
        }
    }
    
    close(fd);
    return 0;
}

Key IOCTL Findings

1. IOCTL 0x40205605 (PROCESS_FRAME)

   • Accepts frame data pointer and size
   • Returns consistent response:
     • field0: 0x7f55ed9000 (always the same userspace address)
     • field1: 0xffffffc026517000 (kernel virtual address)
     • field2: 0xffffffc026517000 (same as field1)
     • field3: 0x100 (always)
   • No mechanism to retrieve encoded output

2. IOCTL 0x40205606 and 0x40205608

   • Return physical addresses:
     • 606: 0x00400000, 0x97c00000
     • 608: 0x06400000, 0x98000000
   • These addresses contain no encoded data when checked

---

Memory Mapping Investigation

mmap Analysis

The encoder supports memory mapping with specific characteristics:

/*
 * test_encoder_mmap.c - Test encoder with mmap
 */

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <errno.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>

int main()
{
    int fd;
    void *mapped;
    uint8_t *frame;
    size_t frame_size = 640 * 480 * 3 / 2;
    size_t map_size = 1441632256;  /* ~1.4GB - discovered via strace */
    
    fd = open("/dev/amvenc_multi", O_RDWR);
    if (fd < 0) {
        printf("Failed to open device: %s\n", strerror(errno));
        return 1;
    }
    
    /* Initialize encoder */
    ioctl(fd, 0x02, 0);
    
    struct {
        uint32_t width;
        uint32_t height;
        uint32_t framerate;
        uint32_t bitrate;
    } config = {640, 480, 30, 4000000};
    
    ioctl(fd, 0x40045603, &config);
    printf("Encoder configured: %dx%d @ %d fps\n", 
           config.width, config.height, config.framerate);
    
    /* Map encoder memory */
    mapped = mmap(NULL, map_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    if (mapped == MAP_FAILED) {
        printf("mmap failed: %s\n", strerror(errno));
        close(fd);
        return 1;
    }
    printf("Mapped %zu bytes at %p\n", map_size, mapped);
    
    /* Submit frame */
    frame = malloc(frame_size);
    memset(frame, 128, frame_size);
    
    struct {
        uint64_t field0;
        uint64_t field1;
        uint64_t field2;
        uint64_t field3;
    } req = {
        (uint64_t)frame,
        frame_size,
        0,
        0x100
    };
    
    printf("\nSubmitting frame...\n");
    int ret = ioctl(fd, 0x40205605, &req);
    printf("PROCESS_FRAME ret=%d\n", ret);
    printf("Response: %016lx %016lx %016lx %016lx\n",
           req.field0, req.field1, req.field2, req.field3);
    
    /* Check if field0 is within mmap region */
    if (req.field0 >= (uint64_t)mapped && req.field0 < (uint64_t)mapped + map_size) {
        size_t offset = req.field0 - (uint64_t)mapped;
        printf("Output offset in mmap: 0x%zx\n", offset);
        
        uint8_t *output = (uint8_t *)req.field0;
        printf("First 64 bytes at offset:\n");
        for (int i = 0; i < 64; i++) {
            printf("%02x ", output[i]);
            if ((i + 1) % 16 == 0) printf("\n");
        }
    }
    
    munmap(mapped, map_size);
    free(frame);
    close(fd);
    
    return 0;
}

mmap Results

• The encoder supports mmap up to 1.4GB
• field0 from PROCESS_FRAME points to offset 0x1ef02000 in the mmap region
• This location contains only zeros, no encoded data
• No H.264 NAL units found anywhere in the mapped memory

---

High-Level API Testing

amvenc API Test

/*
 * test_amvenc_correct.c - Test using amvenc high-level API
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <amvenc.h>

int main()
{
    amvenc_handle_t encoder;
    
    /* Setup encoder parameters */
    amvenc_info_t enc_info = {0};
    enc_info.width = 640;
    enc_info.height = 480;
    enc_info.frame_rate = 30;
    enc_info.bit_rate = 4000000;
    enc_info.gop = 30;
    enc_info.prepend_spspps_to_idr_frames = true;
    enc_info.img_format = AML_IMG_FMT_NV12;
    enc_info.qp_mode = 0;
    
    printf("Initializing encoder: %dx%d @ %d fps, %d bps\n",
           enc_info.width, enc_info.height, 
           enc_info.frame_rate, enc_info.bit_rate);
    
    /* Initialize encoder */
    encoder = amvenc_init(AML_CODEC_ID_H264, enc_info, NULL);
    if (encoder <= 0) {
        printf("Failed to initialize encoder: %ld\n", encoder);
        return 1;
    }
    printf("Encoder initialized successfully (handle=%ld)\n", encoder);
    
    /* Create test frame */
    size_t frame_size = enc_info.width * enc_info.height * 3 / 2;
    unsigned char *input_buffer = malloc(frame_size);
    unsigned char *output_buffer = malloc(1024 * 1024);
    
    memset(input_buffer, 128, frame_size);
    
    /* Encode frame */
    amvenc_frame_info_t frame_info = {0};
    frame_info.frame_size = frame_size;
    frame_info.pitch = enc_info.width;
    frame_info.height = enc_info.height;
    frame_info.coding_timestamp = 0;
    
    amvenc_buffer_info_t in_buffer_info = {0};
    in_buffer_info.buf_type = AML_VMALLOC_TYPE;
    in_buffer_info.buf_info.in_ptr[0] = (unsigned long)input_buffer;
    in_buffer_info.buf_info.in_ptr[1] = (unsigned long)(input_buffer + enc_info.width * enc_info.height);
    in_buffer_info.buf_info.in_ptr[2] = 0;
    in_buffer_info.buf_stride = enc_info.width;
    in_buffer_info.buf_fmt = AML_IMG_FMT_NV12;
    
    amvenc_buffer_info_t ret_buffer_info = {0};
    
    amvenc_metadata_t metadata = amvenc_encode(encoder, 
                                               frame_info,
                                               AML_FRAME_TYPE_IDR,
                                               output_buffer,
                                               &in_buffer_info,
                                               &ret_buffer_info);
    
    printf("Encode result:\n");
    printf("  is_valid: %s\n", metadata.is_valid ? "true" : "false");
    printf("  encoded_data_length: %d bytes\n", metadata.encoded_data_length_in_bytes);
    printf("  error_code: %d\n", metadata.err_cod);
    
    amvenc_destroy(encoder);
    free(input_buffer);
    free(output_buffer);
    
    return 0;
}

amvenc API Results

• Initialization succeeds
• Encoding fails with error code 127
• No encoded data produced
• The API exists but the underlying implementation is broken

---

Kernel Module Investigation

DMA Buffer Access

We created kernel modules to directly access the DMA buffers:

/*
 * kernel_mem_extract.c - Direct kernel memory extraction module
 */

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/device.h>
#include <linux/cdev.h>
#include <linux/mm.h>
#include <linux/io.h>
#include <linux/uaccess.h>
#include <linux/slab.h>
#include <asm/io.h>

#define DEVICE_NAME "kmem_extract"

static int read_physical_memory(u64 phys_addr, void __user *ubuf, u32 size)
{
    void __iomem *virt_addr;
    u8 *temp_buf;
    int ret = 0;
    
    temp_buf = kmalloc(size, GFP_KERNEL);
    if (!temp_buf)
        return -ENOMEM;
    
    virt_addr = ioremap(phys_addr, size);
    if (!virt_addr) {
        kfree(temp_buf);
        return -EIO;
    }
    
    memcpy_fromio(temp_buf, virt_addr, size);
    
    if (copy_to_user(ubuf, temp_buf, size)) {
        ret = -EFAULT;
    }
    
    iounmap(virt_addr);
    kfree(temp_buf);
    
    return ret;
}

/* ... module init/exit code ... */

Kernel Module Results

• Successfully accessed kernel virtual addresses (0xffffffc0XXXXXXXX)
• Found no H.264 encoded data at any of the addresses
• Physical addresses from IOCTLs 606/608 contained no encoded data

---

Driver Source Code Analysis

Source Repository Analysis

We analyzed the Khadas common_drivers repository:

• URL: https://github.com/khadas/common_drivers/tree/khadas-vims-5.15.y/drivers/media_modules/frame_sink/encoder

Key findings:

1. The source code shows a different IOCTL interface than what the device implements
2. Expected pattern for working encoder:
   • Pre-allocated output buffer
   • Hardware register HCODEC_VLC_TOTAL_BYTES for output size
   • IOCTL to retrieve encoded data
3. None of these mechanisms exist in the actual device driver

IOCTL Mismatch

Source code IOCTLs (VDI_IOCTL_*) do not match device IOCTLs (0x4020XXXX), indicating:

• Wrong driver loaded on device
• Incomplete driver implementation
• Missing output retrieval mechanism

---

Test Programs and Results

1. IOCTL Discovery Test

• Purpose: Systematically discover all working IOCTLs
• Result: Found 8 working IOCTLs, none provide output retrieval

2. Memory Mapping Test

• Purpose: Test if encoded output is available via mmap
• Result: mmap succeeds but contains no encoded data

3. High-Level API Test

• Purpose: Test amvenc API functionality
• Result: Initialization works, encoding fails with error 127

4. Kernel Module Test

• Purpose: Direct access to kernel/physical memory
• Result: No H.264 data found at any memory location

5. Additional IOCTL Tests

• Purpose: Test newly discovered IOCTLs for output retrieval
• Result: No IOCTL provides encoded output

---

strace Analysis

Analysis of aml_enc_test (Amlogic's test program):

open("/dev/amvenc_multi", O_RDWR) = 5
ioctl(5, _IOC(_IOC_WRITE, 0x56, 0x5, 0x20), 0x7fb74edcd8) = 0
mmap(NULL, 1441632256, PROT_READ|PROT_WRITE, MAP_SHARED, 5, 0) = 0x7f5a127000
open("/tmp/vpu_mutex.map", O_RDWR|O_CREAT, 0777) = -1 EACCES (Permission denied)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x10} ---

The official test program also crashes, confirming the encoder is non-functional.

---

Conclusions and Recommendations

Findings Summary

1. Driver Status: Incomplete/broken implementation
2. Input Processing: Accepts frames via IOCTL 0x40205605
3. Output Retrieval: No working mechanism exists
4. Memory Access: Encoded data trapped in kernel space with no retrieval method
5. API Status: High-level APIs exist but fail during encoding

Root Cause

The Amlogic hardware encoder driver on the Khadas VIM4 is missing the critical output retrieval mechanism. While it can accept input frames and appears to process them (based on the kernel addresses returned), there is no way to retrieve the encoded H.264 data from the kernel.

Recommendations

1. Do not use the Amlogic hardware encoder on Khadas VIM4 for production
2. Alternative solutions:
   • Use software encoding (x264/x265)
   • Consider different hardware with working encoder support
   • Wait for Khadas/Amlogic to provide a complete driver implementation

---

Appendix: Complete Test File List

All test programs created during this investigation are available in the amv_encoder_debug directory:

1. test_ioctl_discovery.c - IOCTL enumeration
2. test_encoder_mmap.c - Memory mapping tests
3. test_amvenc_correct.c - High-level API test
4. kernel_mem_extract.c - Kernel module for memory access
5. test_additional_ioctls.c - Extended IOCTL testing
6. create_nv12_test.c - Test pattern generator
7. test_encoder_example.c - Comprehensive encoder test

---

Report compiled: July 24, 2025
Investigation duration: Extensive multi-day analysis
Hardware: Khadas VIM4 (Amlogic S905X4)

[stan-smith]

Apologies, wrong cpu, I know its A311D2

• Stan

[numbqq]

@stan-smith what errors you get? I checked on my side H264/H265 encoding works well [1]

[1] - https://docs.khadas.com/products/sbc/vim4/applications/hardware-encoding