Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Duplicated BTFs files #17

Open
mauriciovasquezbernal opened this issue Nov 18, 2021 · 2 comments
Open

Duplicated BTFs files #17

mauriciovasquezbernal opened this issue Nov 18, 2021 · 2 comments
Labels
area/bpf area/repository kind/chore kind/feature

Comments

@mauriciovasquezbernal
Copy link
Collaborator

This is rather not an issue but something that could be improved.

I found out that there are some BTF files with the exact same content for different kernel versions:

$ fdupes -r ./archive/ 
./archive/centos/8/x86_64/4.18.0-147.3.1.el8_1.x86_64.btf
./archive/centos/8/x86_64/4.18.0-147.5.1.el8_1.x86_64.btf

./archive/centos/7/x86_64/3.10.0-1062.4.2.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-1062.4.3.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-1160.2.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-1160.2.2.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-1160.24.1.el7.x86_64.failed
./archive/centos/8/x86_64/4.18.0-240.el8.x86_64.failed

./archive/centos/7/x86_64/3.10.0-1160.41.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-1160.45.1.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-327.36.2.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-327.36.3.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-327.3.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-327.4.4.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-1062.1.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-1062.1.2.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-693.1.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-693.2.2.el7.x86_64.btf

./archive/centos/7/x86_64/3.10.0-514.6.1.el7.x86_64.btf
./archive/centos/7/x86_64/3.10.0-514.6.2.el7.x86_64.btf

./archive/ubuntu/18.04/x86_64/4.15.0-106-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-108-generic.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1020-azure.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1022-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1047-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1048-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1050-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1033-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1034-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-115-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-117-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1014-azure.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1018-azure.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1045-aws.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1047-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1022-aws.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1024-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-121-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-122-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-123-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-124-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-109-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-111-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-112-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1103-azure.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1104-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-69-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-70-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-129-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-130-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-132-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-134-generic.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1034-aws.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1035-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-24-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-29-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1028-azure.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1030-azure.btf

./archive/ubuntu/18.04/x86_64/5.4.0-40-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-42-generic.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1028-aws.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1029-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-45-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-47-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1039-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1040-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-51-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-52-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-53-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-54-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1093-azure.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1095-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1077-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1079-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1086-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1087-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-156-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-158-generic.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1023-azure.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1025-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-74-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-76-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1080-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1082-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1059-azure.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1061-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1073-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1076-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1018-aws.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1020-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1096-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1097-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1091-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1092-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-84-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-86-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-44-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-45-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-139-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-140-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1057-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1058-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1110-azure.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1111-azure.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1035-azure.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1036-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-60-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-62-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-64-generic.btf

./archive/ubuntu/18.04/x86_64/5.4.0-1048-azure.btf
./archive/ubuntu/18.04/x86_64/5.4.0-1049-azure.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1011-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1016-aws.btf

./archive/ubuntu/18.04/x86_64/4.15.0-151-generic.btf
./archive/ubuntu/18.04/x86_64/4.15.0-153-generic.btf

./archive/ubuntu/18.04/x86_64/4.15.0-1099-aws.btf
./archive/ubuntu/18.04/x86_64/4.15.0-1101-aws.btf

./archive/ubuntu/18.04/x86_64/5.4.0-59-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-60-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-62-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-64-generic.btf
./archive/ubuntu/18.04/x86_64/5.4.0-65-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1020-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1022-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-26-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-28-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-29-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1045-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1047-aws.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1022-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1024-aws.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1015-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1017-aws.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1034-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1035-aws.btf

./archive/ubuntu/20.04/x86_64/5.4.0-40-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-42-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-45-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-47-generic.btf

./archive/ubuntu/20.04/x86_64/5.8.0-1040-azure.btf
./archive/ubuntu/20.04/x86_64/5.8.0-1041-azure.btf
./archive/ubuntu/20.04/x86_64/5.8.0-1042-azure.btf

./archive/ubuntu/20.04/x86_64/5.8.0-25-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-28-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-29-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-52-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-53-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-54-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1023-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1025-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-31-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-33-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1059-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1061-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1018-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1020-aws.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1021-aws.btf

./archive/ubuntu/20.04/x86_64/5.4.0-84-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-86-generic.btf

./archive/ubuntu/20.04/x86_64/5.8.0-34-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-36-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-38-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-40-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-41-generic.btf
./archive/ubuntu/20.04/x86_64/5.8.0-43-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1016-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1019-azure.btf

./archive/ubuntu/20.04/x86_64/5.11.0-1017-azure.btf
./archive/ubuntu/20.04/x86_64/5.11.0-1019-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1035-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1036-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-1048-azure.btf
./archive/ubuntu/20.04/x86_64/5.4.0-1049-azure.btf

./archive/ubuntu/20.04/x86_64/5.4.0-37-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-39-generic.btf

./archive/ubuntu/20.04/x86_64/5.4.0-59-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-60-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-62-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-64-generic.btf
./archive/ubuntu/20.04/x86_64/5.4.0-65-generic.btf

I think this could be optimized to reduce the storage needs by replacing the duplicated ones with a symlink. The feasibility and importance of this optimization is related to the adopted storage solution -> #12.
@rafaeldtinoco
Copy link
Contributor

I believe this can't currently be done because a symlink to the files would make the raw permlink not to work if someone tries to get it from github URL directly (which is a current use case), so I'm closing this, please do reopen if you see fit/need.

@itaysk
Copy link
Collaborator

itaysk commented Sep 7, 2022

@rafaeldtinoco are you sure about that? here's a test where the file bar is a symlink to the file foo and both work through GitHub raw urls:
https://github.com/itaysk/actionstest
https://raw.githubusercontent.com/itaysk/actionstest/main/foo
https://raw.githubusercontent.com/itaysk/actionstest/main/bar

@rafaeldtinoco rafaeldtinoco reopened this Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/bpf area/repository kind/chore kind/feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@itaysk @rafaeldtinoco @mauriciovasquezbernal