Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ml-service] Remote launch/request #415

Open
7 tasks
gichan-jang opened this issue Oct 30, 2023 · 2 comments
Open
7 tasks

[ml-service] Remote launch/request #415

gichan-jang opened this issue Oct 30, 2023 · 2 comments
Assignees

Comments

@gichan-jang
Copy link
Member

gichan-jang commented Oct 30, 2023

Currently, using ml-service-remote, registering models and pipelines on remote devices has been implemented.
Next, let's implement the function of launching the pipeline from the sender device (RTOS) or requesting interference using registered model on remote devices.

  • Pipeline launch: launch and set/get state of the pipeline
  • Inference request
    • Singleshot (send input data with model config file (in/out dim, type, etc.) if necessary)
    • Pipeline (using appsrc, get caps or model config file)
  • Permission: Who will get permission and how to manage it?
  • Scheduling: How to deal the many request? thread pool or load balancer, etc.
  • pull mode: Get model or pipeline from connected devices.
@taos-ci
Copy link
Collaborator

taos-ci commented Oct 30, 2023

:octocat: cibot: Thank you for posting issue #415. The person in charge will reply soon.

@gichan-jang gichan-jang self-assigned this Oct 30, 2023
@songgot
Copy link
Contributor

songgot commented Dec 21, 2023

How about creating the following APIs for remote services?

static void API_NAME (handle *handle,
  const char *pipeline_description, const char *file_name, const char *file_path,
  const char *file_desccription, const char *service_type, const char *service_key, const char *topic)
{
  int ret = ML_ERROR_NONE;
  gsize len = 0;

  ret = ml_option_create (&handle->option_h);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_option_set (handle->option_h, "node-type", "remote_sender", NULL);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_option_set (handle->option_h, "dest-host", handle->broker_ip, NULL);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_option_set (handle->option_h, "dest-port", &handle->broker_port, NULL);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_option_set (handle->option_h, "connect-type", handle->connect_type, NULL);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  if (g_strcmp0(handle->connect_type, "HYBRID") == 0) {
    ret = ml_option_set (handle->option_h, "host", handle->local_ip, NULL);
    if (ML_ERROR_NONE != ret) {
      return;
    }
  }

  ret = ml_option_set (handle->option_h, "topic", (void *)topic, NULL);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_service_remote_create (handle->option_h, NULL, NULL, &handle->service_h);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret = ml_option_create (&handle->service_option_h);
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret =
      ml_option_set (handle->service_option_h, "service-type", (void *)service_type, NULL); //service_type = model_raw
  if (ML_ERROR_NONE != ret) {
    return;
  }

  ret =
      ml_option_set (handle->service_option_h, "service-key", (void *)service_key, g_free); // service-key = model_registration_key
  if (ML_ERROR_NONE != ret) {
    return;
  }

  // ret = ml_option_set (handle->service_option_h, "activate", "true", NULL);
  // if (ML_ERROR_NONE != ret) {
  //   return;
  // }

  if (file_name && file_path && file_desccription) {
    ret = ml_option_set (handle->service_option_h, "name", (void *)file_name, NULL);
    if (ML_ERROR_NONE != ret) {
      return;
    }

    ret = ml_option_set (handle->service_option_h, "description", (void *)file_desccription, NULL);
    if (ML_ERROR_NONE != ret) {
      return;
    }

    dlog_print (DLOG_INFO, LOG_TAG, "transmission data:%s", file_path);

    ret = g_file_get_contents (file_path, &handle->contents, &len, NULL);
    if (FALSE == ret) {
      return;
    }
    ret = ml_service_remote_register (handle->service_h, handle->service_option_h, handle->contents, len);
    if (ML_ERROR_NONE != ret) {
      return;
    }
  } else if (pipeline_description) {
    ret =
      ml_service_remote_register (handle->service_h, handle->service_option_h, (void *)pipeline_description, strlen (pipeline_description) + 1);
    if (ML_ERROR_NONE != ret) {
      return;
    }

  } else {
    return;
  }
  return;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants