Release Blog: Top 3 Highlights in the CV-CUDA Releases - Summer 2024

[shiremathNV]

Discussed in #207

^{Originally posted by shiremathNV September 26, 2024}

Top 3 Highlights in the CV-CUDA Releases this Summer (2024)

CV-CUDA is an open-source library from NVIDIA that went live on GitHub in Q2 2022, and since then we have enabled thousands of developers to accelerate the pre- and post-processing stages of their vision AI pipelines using GPU-accelerated image processing and computer vision operations, like resize, normalize, color convert, remap, filter, morphology, SIFT, and many more.

Faster turnaround through monthly CV-CUDA releases
As the CV-CUDA developer community continues to grow, we are seeing more engagement on GitHub with questions, feedback, issue reporting, and enhancement requests. Earlier this summer, the CV-CUDA team decided to move to a monthly release cadence to have a faster turnaround to release new requested features and capabilities, and to address bugs/issues. This has helped us to support our developer community better. In addition to our release announcements on GitHub, moving forward, we aim to publish a release update blog for each release to help our developers get a deeper insight into some of the key features, improvements, and bug fixes. We will also use this blog to provide any insight into what to expect in the upcoming releases.

In this blog, we will cover the top 3 improvements made in the last three monthly releases of CV-CUDA that went live from June to August.

1. Controlling Cache Memory Limit

The CV-CUDA 0.10 release went live in July 2024 on GitHub, and it includes a critical cache memory consumption issue that has been fixed, along with multiple other bug fixes that were reported by developers. The release also now offers the ability to independently build the NVCV library.
The unbounded cache memory consumption is a critical issue that can occur in vision use cases dealing with web data that are typically long-running workflows involving a wide range of image and video sizes causing the cache to grow significantly; for example, if one keeps creating non-wrapped tensors of different shapes. To address this issue, CV-CUDA now provides a user-configurable’ cache limit and automatic clearance mechanism.

import cvcuda

# Get the cache limit (in bytes)
current_cache_limit = nvcv.get_cache_limit()

# Set the cache limit (in bytes)
my_new_cache_limit = 12345 # in bytes
nvcv.set_cache_limit(my_new_cache_limit)

To learn more about this cache consumption and the features added in the release to control the cache limit, please refer to the [Best Practices] (https://cvcuda.github.io/CV-CUDA/nvcvobjectcache.html) section added in CV-CUDA GitHub documentation.

Other Fixes and Improvements in v0.10
In addition to this critical bug, there were multiple other bugs, some were reported by developers, that were fixed in this release:
Addressed potential crashes caused by the Resize operator's linear and nearest neighbor interpolation from non-aligned vectorized writes.
Python CvtColor operator can now correctly handle NV12 and NV21 outputs.
Addressed Resize and RandomResizedCrop linear interpolation weight for border rows and columns.
Updated the fused ResizeCropConvertReformat operator for a missing parameter in C API

2. Performance Improvements in Resize and Operators with Cubic Interpolation

In CV-CUDA v0.9, there were major performance improvements that were made to:

1. Resize operator, boosting performance up to 4x for u8 inputs, and up to 3x for RGB8
2. Cubic interpolation support in operators like Rotate, WarpAffine and WarpPerspective, boosting performance up to 2x
3. The release also addressed a fixed-coordinate rounding bug in the Resize operator with nearest neighbor interpolation. The implementation previously was using the top-left pixel coordinates instead of the center coordinates for samples, resulting in the rounding down versus rounding to nearest coordinates. The fix now takes care of the erroneous subpixel shift in the resampled output image.

3. Potential race condition with Python garbage collection fixed with Pybind upgrade
In CV-CUDA v0.11, there was a fix to potential race conditions or crashes with Python garbage collection associated with an older version of Pybind11 (2.10). This user-reported issue was fixed with a simple upgrade to Pybind11 from 2.10 to 2.13. This Pybind upgrade now also enables CV-CUDA compatibility with NumPy 2.

Learn more about what other features and improvements went live in these releases here.

Summary
NVIDIA’s CV-CUDA team continues to strive towards our mission to make accelerated computing more accessible to data scientists and engineers. We can’t wait to see what people do with these new capabilities.

Have you encountered any of these issues when using CV-CUDA? Leave a comment if these fixes resolved your issue and/or how we could do better in a future release.

CV-CUDA Team

---

Resources

• Learn more about CV-CUDA
• Get Started with CV-CUDA on GitHub
• Documentation
• NVIDIA Computer Vision SDKs and Libraries
• Preview Vision AI-based NIMs
• GTC Session: Runway optimizes AI content creation tools using CV-CUDA
• GTC Session:Netflix builds streamed video processing pipelines for cloud-scale services

Why CV-CUDA?
One of the major challenges that Computer Vision developers face when building AI cloud-scale inference pipelines is achieving end-to-end performance throughput, whether it be for online or offline workloads. With these pipelines ingesting billions of images or million hours of videos, every percent of throughput speed up can result in significant cost savings at this scale. With online systems, this could also mean staying within the expected latency budgets when providing your end-user with real-time experiences. When deploying such pipelines, CV-CUDA and other NVIDIA acceleration libraries play a pivotal role in enabling our customers and partners to achieve significant end-to-end performance improvements.