Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[doc][dash-p4] Update metering bucket design #605

Closed
wants to merge 6 commits into from

Conversation

r12f
Copy link
Collaborator

@r12f r12f commented Jul 17, 2024

Problem

The current metering design uses SAI object to identify each metering bucket. This is not efficient from both CPU and memory perspective, because of multiple reasons:

  1. Whenever an ENI is created, we need to use a loop to create all metering buckets. Say, in future, if each ENI supports 1M metering buckets, we need to call the create meter bucket API 1M times.
  2. The SWSS need to maintain all metering bucket object ids, which takes memory. The metering class and bucket are essentially the same or having 1:1 mapping, so this could be simplified.

What we are doing in this change

To solve these issues, we are changing the metering bucket from SAI object to SAI table entry. With this, we could add assumption that all metering buckets will be created along with the ENI without additional calls, which saves all the CPU and memory resources.

@r12f
Copy link
Collaborator Author

r12f commented Jul 17, 2024

ok, looks like the SAI API doesn't have the stats API support for the table entries. I could not find any existing SAI API using this pattern. This change will need more time to change now.

image

@mukeshmv
Copy link
Collaborator

mukeshmv commented Jul 19, 2024

After fixing the SAI/meta/style.pm to allow table entry stats, running into the following error.
ERROR: SAI_OBJECT_TYPE_METER_BUCKET is not defined in OBJTOAPIMAP, missing sai_XXX_api_t declaration?

We may need to push a direct commit to OCP SAI to replace the existing experimental METER_BUCKET object with the METER_BUCKET_ENTRY and the corresponding APIs generated by this PR.

@r12f
Copy link
Collaborator Author

r12f commented Jul 20, 2024

This is because the entry stats functions are not generated in libsai. Feel free to take a look at the saiapi.cpp.j2. It should be straightforward to add (by copying the sai object one and do some small modifications)

@r12f
Copy link
Collaborator Author

r12f commented Jul 20, 2024

Better not do direct commit into SAI repo and diverge these 2 repos.

@mukeshmv
Copy link
Collaborator

e the entry stats functions are not generated

@r12f, No its not about the libsai entry stats function. This is OCP SAI meta error complaining about the missing SAI API headers for the original Meter bucket object type which is not removed.
With the Dash P4 changes in this PR we are now generating the APIs only for the new Meter bucket table entry instead of the original bucket object.

@mukeshmv
Copy link
Collaborator

mukeshmv commented Jul 21, 2024

Better not do direct commit into SAI repo and diverge these 2 repos.

I was thinking to create SAI OCP PR with the new Meter bucket table entry APIs exactly as it is generated here.
As a one-off special case here since we are replacing existing object.
So it should not be a diverge.

@r12f
Copy link
Collaborator Author

r12f commented Jul 22, 2024

Hi Mukesh, I see. The relationship between DASH SAI and OCP SAI is that - almost at any moment, the SAI API in DASH repo is always a superset of the OCP SAI. But once we updates the API directly there, everyone in DASH repo will produce the SAI without the new meter bucket API, unless they cherry pick the changes in this PR, hence the diverge.

Since what we need to do is to pass the SAI checks, could we only submit the change that will make this PR pass to SAI, then bring the SAI back to DASH to unblock us? This is what we do for solving all previous SAI related issues too, such as spellchecks, SAI style checks and etc.

mukeshmv added a commit to mukeshmv/SAI that referenced this pull request Jul 24, 2024
Bring in SAI API changes from this DASH PR
sonic-net/DASH#605

Signed-off-by: mukeshmv <mukesh@pensando.io>
mukeshmv added a commit to mukeshmv/SAI that referenced this pull request Jul 24, 2024
Bring in SAI API changes from this DASH PR
sonic-net/DASH#605

Signed-off-by: mukeshmv <mukesh@pensando.io>
@mukeshmv
Copy link
Collaborator

mukeshmv commented Jul 24, 2024

But once we updates the API directly there, everyone in DASH repo will produce the SAI without the new meter bucket API, unless they cherry pick the changes in this PR, hence the diverge.

@r12f , I still don't see the issue. Even if we directly commit to OCP SAI, DASH repo wont see the new SAI meter bucket entry APIs unless we update the SAI submodule in DASH, so there should be no diverge. We can update the SAI submodule as part of this PR so that the generated APIs are in sync with the SAI repo. I have created a OCP SAI PR, please take a look -
opencomputeproject/SAI#2056

@KrisNey-MSFT
Copy link
Collaborator

KrisNey-MSFT commented Jul 24, 2024

Not easy to fix, not a backward compatible change. Update OCP SAI first, move SAI submodule to the latest. Per DASH Community call, review and accept, and push into OCP SAI.

prsunny pushed a commit to opencomputeproject/SAI that referenced this pull request Jul 24, 2024
@mukeshmv
Copy link
Collaborator

@r12f the SAI PR has been merged. Can you update the SAI submodule to latest in this PR ?

@r12f
Copy link
Collaborator Author

r12f commented Jul 25, 2024

Hi Mukesh, ok. Do you mind to help with it? My internet access is horrible. Opening this page took me more than 10mins or so to retry... I don't feel I can get any real work done...

@r12f
Copy link
Collaborator Author

r12f commented Jul 25, 2024

I have commented the SAI PR as well. We will need get SAI merged back to DASH and get this PR merged. @mukeshmv

The ideal way is to check in the style.pm file then update SAI in DASH to get the change unblocked. This keeps the DASH SAI being superset of OCP SAI and avoid the chances of unexpected change being merged into OCP SAI, as the generated API from main branch will going to revert this work.

@mukeshmv
Copy link
Collaborator

@r12f sure we will take care of this. As I explained before just fixing the style.pm was not sufficient to get this PR in as we were running into API missing errors.

Since I don't have permission to add commit to this PR, I have created a duplicate PR with all your commits + updating to latest SAI.
#611 - all checks have passed here.
@prsunny merging #611 should resolve this issue.

@r12f
Copy link
Collaborator Author

r12f commented Jul 25, 2024

Perfect! Will try my best to open it and check it! Thanks Mukesh! (Do you still remember a mobile network called 3G? XD)

@r12f r12f closed this Jul 25, 2024
@r12f r12f deleted the user/r12f/meter-update branch July 25, 2024 08:17
@KrisNey-MSFT
Copy link
Collaborator

KrisNey-MSFT commented Jul 25, 2024 via email

@mukeshmv
Copy link
Collaborator

Perfect! Will try my best to open it and check it! Thanks Mukesh! (Do you still remember a mobile network called 3G? XD)

3G - Which corner of the world are you in :)

siqbal1986 pushed a commit to siqbal1986/SAI that referenced this pull request Sep 30, 2024
)

Bring in SAI API changes from this DASH PR
sonic-net/DASH#605

Signed-off-by: siqbal1986 <shahzad.iqbal@microsoft.com>
erohsik pushed a commit to erohsik/SAI that referenced this pull request Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants