What is the relationship between samples_per_gpu and workers_per_gpu and sample_ratio？

[joeyslv]

I can only use the default

sample_ ratio=[1, 4]
samples_ per_ gpu=4
workers_ per_ gpu=4

But to increase the batchsize a bit,
samples_ per_ gpu=8,
when the program cannot run and a len() error will occur. Can you tell me the relationship between these three and how labeled and unlabeled data is sampled in the project? Thank you very much

[Adamdad]

There are three distinct concepts to understand:

1. sample_ratio=[1, 4] indicates the ratio of labeled to unlabeled samples within a single GPU. For instance, sample_ratio=[1, 4] means that there is 1 labeled and 4 unlabeled samples on each GPU.
2. samples_per_gpu=5 refers to the total number of samples per GPU, regardless of whether they are labeled or unlabeled. In fact, sum(sample_ratio) == samples_per_gpu.
3. workers_per_gpu=5 determines the number of threads that will be used to load the data. The optimal number of workers per GPU depends on your server setup. By default, we set workers_per_gpu equal to samples_per_gpu, but you can reduce this value if your server has limited CPU resources. If necessary, you can set it to 1 or 0.

ConsistentTeacher/configs/consistent-teacher/consistent_teacher_r50_fpn_coco_180k_10p.py

Lines 256 to 296 in 1fa6477

	data = dict(
	samples_per_gpu=5,
	workers_per_gpu=5,
	train=dict(
	_delete_=True,
	type="SemiDataset",
	sup=dict(
	type="CocoDataset",
	ann_file="data/coco_semi/semi_supervised/instances_train2017.${fold}@${percent}.json",
	img_prefix="data/coco/train2017/",
	pipeline=train_pipeline,
	),
	unsup=dict(
	type="CocoDataset",
	ann_file="data/coco_semi/semi_supervised/instances_train2017.${fold}@${percent}-unlabeled.json",
	img_prefix="data/coco/train2017/",
	pipeline=unsup_pipeline,
	filter_empty_gt=False,
	),
	),
	val=dict(
	img_prefix="data/coco/val2017/",
	ann_file='data/coco/annotations/instances_val2017.json',
	pipeline=test_pipeline
	),
	test=dict(
	pipeline=test_pipeline,
	img_prefix="data/coco/val2017/",
	ann_file='data/coco/annotations/instances_val2017.json'
	),
	sampler=dict(
	train=dict(
	type="SemiBalanceSampler",
	sample_ratio=[1, 4],
	by_prob=False,
	# at_least_one=True,
	epoch_length=7330,
	)
	),
	)

[X-KL]

Hello, I have been reproducing this paper recently and have encountered the issue you mentioned in another post. Loss=0. I trained on the Coco dataset, and before 10000, the unsup-loss and loss were both 0, but after 10000 the loss is noemal, but soon the loss became 0 again, and unsup_gmm_thr also became 0. Can we discuss this issue with you.

[huangnana1]

Hello, I have been reproducing this paper recently and have encountered the issue you mentioned in another post. Loss=0. I trained on the Coco dataset, and before 10000, the unsup-loss and loss were both 0, but after 10000 the loss is noemal, but soon the loss became 0 again, and unsup_gmm_thr also became 0. Can we discuss this issue with you.

请问您的该问题解决了吗，我也遇到这个问题

[Adamdad]

Hello @huangnana1 and @X-KL , how many GPUs are you using and did you change any values in the configs?

[X-KL]

hello, thanks for your reply, reference your previous reply in other issues,  the problem I have solved. thank you very much.

…

---Original--- From: "Xingyi ***@***.***> Date: Wed, Oct 9, 2024 14:21 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [Adamdad/ConsistentTeacher] What is the relationship between samples_per_gpu and workers_per_gpu and sample_ratio？ (Issue #19) Hello @huangnana1 and @X-KL , how many GPUs are you using and did you change any values in the configs? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>