Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-trained weights for raft-large, fine-tuned on sintel #5079

Merged
merged 9 commits into from
Dec 9, 2021

Conversation

NicolasHug
Copy link
Member

@NicolasHug NicolasHug commented Dec 9, 2021

Towards #4644

This PR adds our own pre-trained weights as well as the original weights from the paper.

I evaluated these on Sintel test set, and we do slightly better than the original weights.

There is a limited amount of submissions we can make to sintel test, so I decided to just submit C_T_V2 after fine-tuning it on sintel with the "S+K+H" schedule (--train-dataset sintel_skh).

Results below are on sintel train, just to demonstrate that it works. They're not incredibly relevant since we trained on sintel train.

(raft) ➜  vision git:(raft_large_sintel) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset sintel --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_V1

Sintel val clean epe: 0.7647	1px: 0.9233	3px: 0.9687	5px: 0.9793	per_image_epe: 0.7626	f1: 2.8538
Sintel val final epe: 1.2172	1px: 0.8862	3px: 0.9461	5px: 0.9628	per_image_epe: 1.2111	f1: 4.9317


(raft) ➜  vision git:(raft_large_sintel) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset sintel --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_V2

Sintel val clean epe: 0.6482	1px: 0.9255	3px: 0.9701	5px: 0.9805	per_image_epe: 0.6470	f1: 2.6793
Sintel val final epe: 1.0197	1px: 0.8898	3px: 0.9493	5px: 0.9660	per_image_epe: 1.0151	f1: 4.5754

cc @datumbox

@facebook-github-bot
Copy link

facebook-github-bot commented Dec 9, 2021

💊 CI failures summary and remediations

As of commit 9b8db17 (more details on the Dr. CI page):


None of the CI failures appear to be your fault 💚



🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet.


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@NicolasHug NicolasHug added module: models other if you have no clue or if you will manually handle the PR in the release notes labels Dec 9, 2021
@NicolasHug NicolasHug mentioned this pull request Dec 9, 2021
12 tasks
Copy link
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@NicolasHug NicolasHug merged commit 954a117 into pytorch:main Dec 9, 2021
facebook-github-bot pushed a commit that referenced this pull request Dec 17, 2021
)

Reviewed By: fmassa

Differential Revision: D33184996

fbshipit-source-id: c19e25fd262529ae9059d94a51df1b3f688a8c78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/default cla signed module: models other if you have no clue or if you will manually handle the PR in the release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants