Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on schedule.py #12

Open
qqqhhh-any opened this issue Jul 3, 2021 · 0 comments
Open

Error on schedule.py #12

qqqhhh-any opened this issue Jul 3, 2021 · 0 comments

Comments

@qqqhhh-any
Copy link

qqqhhh-any commented Jul 3, 2021

When running schedule and trying to use multithreading, there will be an error because of multithreading contention.

Modify the code according to the error prompt and add multiprocessing. Freeze_ Support () can solve this problem.

Error tips:
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

The revised code is as follows:

from save_to_csv import save_results
import logging
import sys
import utils.multiprocessing
from defaults import get_cfg_defaults
import os
import multiprocessing as mult

def f(setting):
    import train_AAE
    import novelty_detector

    fold_id = setting['fold']
    inliner_classes = setting['digit']
    cfg = setting['cfg']
    train_AAE.train(fold_id, [inliner_classes], inliner_classes, cfg=cfg)

    res = novelty_detector.main(fold_id, [inliner_classes], inliner_classes, classes_count, mul, cfg=cfg)
    return res

if __name__ == '__main__':
    # in main module use freeze_support() 
    mult.freeze_support()

    full_run = True

    logger = logging.getLogger("logger")
    logger.setLevel(logging.DEBUG)
    ch = logging.StreamHandler(stream=sys.stdout)
    ch.setLevel(logging.DEBUG)
    formatter = logging.Formatter("%(asctime)s %(name)s %(levelname)s: %(message)s")
    ch.setFormatter(formatter)
    logger.addHandler(ch)

    mul = 0.2

    settings = []

    classes_count = 10

    if len(sys.argv) > 1:
        cfg_file = 'configs/' + sys.argv[1]
    else:
        cfg_file = 'configs/' + input("Config file:")
    cfg = get_cfg_defaults()
    cfg.merge_from_file(cfg_file)
    cfg.freeze()

    for fold in range(5 if full_run else 1):
        for i in range(classes_count):
            settings.append(dict(fold=fold, digit=i,cfg=cfg))

    gpu_count = utils.multiprocessing.get_gpu_count()

    results = utils.multiprocessing.map(f, gpu_count, settings)

    save_results(results, os.path.join(cfg.OUTPUT_FOLDER, cfg.RESULTS_NAME))

translate by google Translator
source text:
关于schedule.py的错误
运行schedule,尝试使用多线程时,会出现错误,原因是多线程争用。
按照错误提示修改代码,增加multiprocessing.freeze_support()可以解决这个问题。
修改后的代码如下:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant