[NOTE] Notes on "POST /policies/replication" API

[moooofly]

This is a very complicated API.

by swagger file

The content body of this API consist of so many parameters. But which of them is required?

by Harbor UI

It seems that only three of them are really required

• Name
• Source project
• Endpoint

[moooofly]

Case 1

{
  "name": "sync-to-somewhere",
  "description": "",
  "trigger": {
    "kind": "Manual"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 10,
      "owner_id": 1,
      "name": "temp_9",
      "creation_time": "2018-11-02T06:58:29Z",
      "update_time": "2018-11-02T06:58:29Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}

[moooofly]

Case 2

{
  "name": "sync-again",
  "description": "",
  "trigger": {
    "kind": "Immediate"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 9,
      "owner_id": 1,
      "name": "temp_8",
      "creation_time": "2018-11-02T06:58:25Z",
      "update_time": "2018-11-02T06:58:25Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 2,
      "endpoint": "https://11.11.11.33",
      "name": "fake again",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": false,
      "creation_time": "2018-11-07T14:46:34.752062Z",
      "update_time": "2018-11-07T14:46:34.752062Z"
    }
  ]
}

[moooofly]

Case 3

{
  "name": "sync-third",
  "description": "",
  "trigger": {
    "kind": "Scheduled",
    "schedule_param": {
      "type": "Daily",
      "offtime": 68400
    }
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 8,
      "owner_id": 1,
      "name": "temp_7",
      "creation_time": "2018-11-02T06:58:22Z",
      "update_time": "2018-11-02T06:58:22Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 3,
      "endpoint": "http://11.11.11.44",
      "name": "fake third",
      "username": "moooofly",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T15:15:35.87736Z",
      "update_time": "2018-11-07T15:15:35.87736Z"
    }
  ]
}

[moooofly]

Case 4

{
  "name": "sync-fourth",
  "description": "444",
  "trigger": {
    "kind": "Manual"
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [
    {
      "kind": "repository",
      "value": "repo"
    },
    {
      "kind": "tag",
      "value": "v10086"
    },
    {
      "kind": "label",
      "value": 33
    },
    {
      "kind": "label",
      "value": 30
    },
    {
      "kind": "label",
      "value": 27
    }
  ],
  "projects": [
    {
      "project_id": 7,
      "owner_id": 1,
      "name": "temp_6",
      "creation_time": "2018-11-02T06:58:22Z",
      "update_time": "2018-11-02T06:58:22Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 0,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}

[moooofly]

Case 5

{
  "name": "sync-fifth",
  "description": "fifth",
  "trigger": {
    "kind": "Scheduled",
    "schedule_param": {
      "type": "Weekly",
      "weekday": 1,
      "offtime": 0
    }
  },
  "replicate_existing_image_now": true,
  "replicate_deletion": false,
  "filters": [],
  "projects": [
    {
      "project_id": 6,
      "owner_id": 1,
      "name": "temp_5",
      "creation_time": "2018-11-02T06:58:11Z",
      "update_time": "2018-11-02T06:58:11Z",
      "deleted": false,
      "owner_name": "",
      "togglable": true,
      "current_user_role_id": 1,
      "repo_count": 1,
      "metadata": {
        "public": "true"
      }
    }
  ],
  "targets": [
    {
      "id": 1,
      "endpoint": "http://11.11.11.22",
      "name": "fake endpoint",
      "username": "admin",
      "password": "",
      "type": 0,
      "insecure": true,
      "creation_time": "2018-11-07T14:42:31.876842Z",
      "update_time": "2018-11-07T14:42:31.876842Z"
    }
  ]
}

[moooofly]

Conclusion

Here are the major steps that Harbor UI does

1. By "GET /api/policies/replication?name=<xxx>" to check if the replication rule with name already exists.
2. By "GET /api/projects?name=<yyy>" to get source project info to be replicated.
3. By "GET /api/labels?scope=g" to get all global labels.
4. By "GET /api/labels?scope=p&project_id=" to get labels from source project.
5. By "GET /api/targets?name=<zzz>" to get the endpoint info to replicate to.
6. By "POST /api/policies/replication" to create the replication rule based on above info and some other info.

---

As you can see when the kind of trigger is "Scheduled":

• the offtime with "8:00 AM" is 0
• the offtime with "3:00 AM" is 68400

as per swagger file: The time offset with the UTC 00:00 in seconds.

---

The policy ID (parameter id defined in policyinfo) should not be included.

[moooofly]

by #30 (comment), you can see that

"filters": [
    {
      "kind": "repository",
      "value": "repo"
    },
    {
      "kind": "tag",
      "value": "v10086"
    },
    {
      "kind": "label",
      "value": 33
    },
    {
      "kind": "label",
      "value": 30
    },
    {
      "kind": "label",
      "value": 27
    }
  ],

the type of "value" can be of either string type or int type. This makes codes for json.Marshal a little bit inconvenient.

Currently, when setting label IDs with harborctl, you will get

$./harborctl replication policy create -e fake-remote -r sync-test -s test-1 -d "from cli" --label_ids_as_filter "1,2,3"
==> GET (with struct) https://localhost/api/projects?name=test-1
==> GET (with struct) https://localhost/api/targets?name=fake-remote


----- just for debug -----

===> {"name":"sync-test","description":"from cli","projects":[{"project_id":4,"owner_id":1,"name":"test-1","creation_time":"2019-02-10T04:48:17Z","update_time":"2019-02-10T04:48:17Z","deleted":false,"owner_name":"","togglable":true,"current_user_role_id":1,"repo_count":0,"chart_count":0,"metadata":{"public":"true","enable_content_trust":"","prevent_vul":"","severity":"","auto_scan":""}}],"targets":[{"id":1,"endpoint":"https://11.11.11.22","name":"fake-remote","username":"admin","password":"","type":0,"insecure":true,"creation_time":"2019-02-10T04:49:41.041147Z","update_time":"2019-02-10T04:49:41.041147Z"}],"trigger":{"kind":"Manual","schedule_param":{"type":"","weekday":0,"offtime":0}},"filters":[{"kind":"label","value":"1"},{"kind":"label","value":"2"},{"kind":"label","value":"3"}],"replicate_existing_image_now":true,"replicate_deletion":false}

----- just for debug -----

==> POST https://localhost/api/policies/replication
<==
<== Rsp Status: 400 Bad Request
<== Rsp Body: value the type of value should be integer for label filter
value the type of value should be integer for label filter
value the type of value should be integer for label filter

[#225#root@ubuntu-1604 /go/src/github.com/moooofly/harborctl]$

I think, making some adjustments from server side will be more reasonable.

@steven-zou PTAL