diff --git a/documentation/CLI-for-JFrog-Cloud-Transfer.md b/documentation/CLI-for-JFrog-Cloud-Transfer.md index 7b66eec6e..84d057368 100644 --- a/documentation/CLI-for-JFrog-Cloud-Transfer.md +++ b/documentation/CLI-for-JFrog-Cloud-Transfer.md @@ -88,32 +88,33 @@ By default, the target does not have the APIs required for the configuration tra Follow the below steps for enabling the configuration transfer. -1. Log in to [MyJFrog](https://my.jfrog.com/login/). +1: Log in to [MyJFrog](https://my.jfrog.com/login/). -2. Click on **Settings**. +2: Click on **Settings**. -3. Under the **Transfer Artifactory Configuration from Self-Hosted to Cloud** section, click on the **acknowledgment**checkbox. You cannot enable configuration transfer until you select the checkbox. +3: Under the **Transfer Artifactory Configuration from Self-Hosted to Cloud** section, click on the **acknowledgment**checkbox. You cannot enable configuration transfer until you select the checkbox. ![image](images/transfer-1.png) -4. If you have an Enterprise+ subscription with more than one Artifactory instance, select the target instance from the drop-down menu. +4: If you have an Enterprise+ subscription with more than one Artifactory instance, select the target instance from the drop-down menu. -5. Toggle **Enable Configuration Transfer** on to enable the transfer. The process may take a few minutes to complete. +5: Toggle **Enable Configuration Transfer** on to enable the transfer. The process may take a few minutes to complete. ![image](images/transfer-2.png) -6. The configuration transfer is now enabled and you can continue with the transfer process. +6: The configuration transfer is now enabled and you can continue with the transfer process. ### Step 2: Set up the source instance for pushing files to the target instance To set up the source instance, you must install the data-transfer user plugin in the primary node of the source instance. This section guides you through the installation steps. -1. Install JFrog CLI on the primary node machine of the source instance as described [here](#installing-jfrog-cli-on-the-source-instance-machine). +1: Install JFrog CLI on the primary node machine of the source instance as described [here](#installing-jfrog-cli-on-the-source-instance-machine). -2. Configure the connection details of the source Artifactory instance with your admin credentials by running the following command from the terminal. +2: Configure the connection details of the source Artifactory instance with your admin credentials by running the following command from the terminal. + ```sh jf c add source-server ``` -3. Ensure that the **JFROG\_HOME** environment variable is set and holds the value of JFrog installation directory. It usually points to the **/opt/jfrog** directory. In case the variable isn't set, set its value to point to the correct directory as described in the JFrog Product Directory Structure article.System Directories +3: Ensure that the **JFROG\_HOME** environment variable is set and holds the value of JFrog installation directory. It usually points to the **/opt/jfrog** directory. In case the variable isn't set, set its value to point to the correct directory as described in the JFrog Product Directory Structure article.System Directories If the source instance has internet access, follow this single step: @@ -125,15 +126,13 @@ jf rt transfer-plugin-install source-server If the source instance has no internet access, follow these steps instead. -1. Download the following two files from a machine that has internet access: - +1: Download the following two files from a machine that has internet access: Download **data-transfer.jar** from [https://releases.jfrog.io/artifactory/jfrog-releases/data-transfer/\[RELEASE\]/lib/data-transfer.jar](https://releases.jfrog.io/artifactory/jfrog-releases/data-transfer/%5BRELEASE%5D/lib/data-transfer.jar). - Download **dataTransfer.groovy** from [https://releases.jfrog.io/artifactory/jfrog-releases/data-transfer/\[RELEASE\]/dataTransfer.groovy](https://releases.jfrog.io/artifactory/jfrog-releases/data-transfer/%5BRELEASE%5D/dataTransfer.groovy). -2. Create a new directory on the primary node machine of the source instance and place the two files you downloaded inside this directory. +2: Create a new directory on the primary node machine of the source instance and place the two files you downloaded inside this directory. -3. Install the data-transfer user plugin by running the following command from the terminal. Replace the `` token with the full path to the directory which includes the plugin files you downloaded. +3: Install the data-transfer user plugin by running the following command from the terminal. Replace the `` token with the full path to the directory which includes the plugin files you downloaded. ```sh jf rt transfer-plugin-install source-server --dir "" @@ -145,41 +144,42 @@ If the source instance has no internet access, follow these steps instead. > The following process will wipe out the entire configuration of the target instance, and replace it with the configuration of the source instance. This includes repositories and users. --- -1. Install JFrog CLI on the source instance machine as described [here](#installing-jfrog-cli-on-the-source-instance-machine). +1: Install JFrog CLI on the source instance machine as described [here](#installing-jfrog-cli-on-the-source-instance-machine). -2. Configure the connection details of the source Artifactory instance with your admin credentials by running the following command from the terminal. +2: Configure the connection details of the source Artifactory instance with your admin credentials by running the following command from the terminal. ```sh jf c add source-server ``` -3. Configure the connection details of the target Artifactory server with your admin credentials by running the following command from the terminal. +3: Configure the connection details of the target Artifactory server with your admin credentials by running the following command from the terminal. ```sh jf c add target-server ``` -4. Run the following command to verify that the target URLs of all the remote repositories are accessible from the target. +4: Run the following command to verify that the target URLs of all the remote repositories are accessible from the target. ```sh jf rt transfer-config source-server target-server --prechecks ``` If the command output shows that a target URL isn't accessible for any of the repositories, you'll need to make the URL accessible before proceeding to transferring the config. You can then rerun the command to ensure that the URLs are accessible. - --- - **Note** - > The following process will wipe out the entire configuration of the target instance, and replace it with the configuration of the source instance. This includes repositories and users. - --- + +--- +**Note** +> The following process will wipe out the entire configuration of the target instance, and replace it with the configuration of the source instance. This includes repositories and users. +--- -5. Transfer the configuration from the source to the target by running the following command. +5: Transfer the configuration from the source to the target by running the following command. ```sh jf rt transfer-config source-server target-server ``` This command might take up to two minutes to run. - --- - **Note** - > * By default, the command will not transfer the configuration if it finds that the target instance isn't empty. This can happen for example if you ran the transfer-config command before. If you'd like to force the command run anyway, and overwrite the existing configuration on the target, run the command with the `--force` option. - > * In case you do not wish to transfer all repositories, you can use the `--include-repos` and `--exclude-repos` command options. Run the following command to see the usage of these options. - > ```jf rt transfer-config -h``` - --- - -6. View the command output in the terminal to verify that there are no errors. The command output is divided in to the following four phases: +--- +**Note** +> * By default, the command will not transfer the configuration if it finds that the target instance isn't empty. This can happen for example if you ran the transfer-config command before. If you'd like to force the command run anyway, and overwrite the existing configuration on the target, run the command with the `--force` option. +> * In case you do not wish to transfer all repositories, you can use the `--include-repos` and `--exclude-repos` command options. Run the following command to see the usage of these options. +> ```jf rt transfer-config -h``` +--- + +6: View the command output in the terminal to verify that there are no errors. The command output is divided in to the following four phases: ```sh ========== Phase 1/4 - Preparations ========== ========== Phase 2/4 - Export configuration from the source Artifactory ========== @@ -187,7 +187,7 @@ If the source instance has no internet access, follow these steps instead. ========== Phase 4/4 - Import configuration to the target Artifactory ========== ``` -7. View the log to verify there are no errors. +7: View the log to verify there are no errors. ![image](images/transfer-3.png) @@ -197,19 +197,19 @@ The target instance should now be accessible with the admin credentials of the s Once the configuration transfer is successful, you need to disable the configuration transfer on the target instance. This is important both for security reasons and the target server is set to be low on resources while configuration transfer is enabled. -1. Login to [MyJFrog](https://my.jfrog.com/login/) +1: Login to [MyJFrog](https://my.jfrog.com/login/) -2. Under the Actions menu, choose **Enable Configuration Transfer**. +2: Under the Actions menu, choose **Enable Configuration Transfer**. -3. Toggle **Enable Configuration Transfer** to **off** to disable configuration transfer. +3: Toggle **Enable Configuration Transfer** to **off** to disable configuration transfer. Disabling the configuration transfer might take some time. ### Step 5: Push the files from the source to the target instance -1. Install JFrog CLI on any machine that has access to both the source and the target JFrog instances. To do this, follow the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). +1: Install JFrog CLI on any machine that has access to both the source and the target JFrog instances. To do this, follow the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). -2. Run the following command to start pushing the files from all the repositories in source instance to the target instance. +2: Run the following command to start pushing the files from all the repositories in source instance to the target instance. ```sh jf rt transfer-files source-server target-server ``` @@ -245,9 +245,9 @@ The process will continue from the point it stopped when you re-run the command. While the file transfer is running, monitor the load on your source instance, and if needed, reduce the transfer speed or increase it for better performance. For more information, see the [Controlling the file transfer speed](#controlling-the-file-transfer-speed). -1. A path to an errors summary file will be printed at the end of the run, referring to a generated CSV file. Each line on the summary CSV represents an error log of a file that failed to to be transferred. On subsequent executions of the `jf rt transfer-files`command, JFrog CLI will attempt to transfer these files again. +1: A path to an errors summary file will be printed at the end of the run, referring to a generated CSV file. Each line on the summary CSV represents an error log of a file that failed to to be transferred. On subsequent executions of the `jf rt transfer-files`command, JFrog CLI will attempt to transfer these files again. -2. Once the`jf rt transfer-files`command finishes transferring the files, you can run it again to transfer files which were created or modified while the transfer. You can run the command as many times as needed. Subsequent executions of the command will also attempt to transfer files that failed to be transferred during previous executions of the command. +2: Once the`jf rt transfer-files`command finishes transferring the files, you can run it again to transfer files which were created or modified while the transfer. You can run the command as many times as needed. Subsequent executions of the command will also attempt to transfer files that failed to be transferred during previous executions of the command. --- **Note** @@ -267,11 +267,11 @@ The **jf rt transfer-config** command transfers all the config entities (users, > These steps trigger the transfer of the projects and repositories only. Other configuration entities like users are currently not supported. --- -1. Ensure that you have admin access tokens for both the source and target instances. You'll have to use an admin access token and not an Admin username and password. +1: Ensure that you have admin access tokens for both the source and target instances. You'll have to use an admin access token and not an Admin username and password. -2. Install JFrog CLI on any machine that has access to both the source and the target instances using the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). Make sure to use the admin access tokens and not an admin username and password when configuring the connection details of the source and the target. +2: Install JFrog CLI on any machine that has access to both the source and the target instances using the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). Make sure to use the admin access tokens and not an admin username and password when configuring the connection details of the source and the target. -3. Run the following command to merge all the projects and repositories from the source to the target instance. +3: Run the following command to merge all the projects and repositories from the source to the target instance. ```sh jf rt transfer-config-merge source-server target-server ``` @@ -323,19 +323,19 @@ Unlike the transfer-config command, which should be run from the primary note ma Follows these steps to installing JFrog CLI on that machine. -1. Install JFrog CLI by using one of the [JFrog CLI Installers](https://jfrog.com/getcli/). For example: +1: Install JFrog CLI by using one of the [JFrog CLI Installers](https://jfrog.com/getcli/). For example: ```sh curl -fL https://install-cli.jfrog.io | sh ``` -2. If your source instance is accessible only through an HTTP/HTTPS proxy, set the proxy environment variable as described [here](https://jfrog.com/help/r/jfrog-cli/proxy-support). +2: If your source instance is accessible only through an HTTP/HTTPS proxy, set the proxy environment variable as described [here](https://jfrog.com/help/r/jfrog-cli/proxy-support). -3. Configure the connection details of the source Artifactory instance with your admin credentials. Run the following command and follow the instructions. +3: Configure the connection details of the source Artifactory instance with your admin credentials. Run the following command and follow the instructions. ```sh jf c add source-server ``` -4. Configure the connection details of the target Artifactory instance. +4: Configure the connection details of the target Artifactory instance. ```sh jf c add target-server ``` @@ -352,19 +352,19 @@ curl -fL https://install-cli.jfrog.io | sh > If the source instance is running as a docker container, and you're not able to install JFrog CLI while inside the container, follow these steps. --- -1. Connect to the host machine through the terminal. +1: Connect to the host machine through the terminal. -2. Download the JFrog CLI executable into the correct directory by running this command. +2: Download the JFrog CLI executable into the correct directory by running this command. ```sh curl -fL https://getcli.jfrog.io/v2-jf | sh ``` -3. Copy the JFrog CLI executable you've justdownloadedto the container, by running the following docker command. Make sure to replace``with the name of the container. +3: Copy the JFrog CLI executable you've justdownloadedto the container, by running the following docker command. Make sure to replace``with the name of the container. ```sh docker cp jf :/usr/bin/jf ``` -4. Connect to the container and run the following command to ensure JFrog CLI is installed +4: Connect to the container and run the following command to ensure JFrog CLI is installed ```sh jf -v ``` @@ -397,15 +397,15 @@ To run the files transfer after you copy the filestore, add the `--filestore` co Before initiating the files transfer process, we highly recommend running pre-checks, to identify issues that can affect the transfer. You trigger the pre-checks by running a JFrog CLI command on your terminal. The pre-checks will verify the following: -1. There's network connectivity between the source and target instances. +1: There's network connectivity between the source and target instances. -2. The source instance does not include artifacts with properties with values longer than 2.4K characters. This is important, because values longer than 2.4K characters are not supported in JFrog Cloud, and those properties are skipped during the transfer process. +2: The source instance does not include artifacts with properties with values longer than 2.4K characters. This is important, because values longer than 2.4K characters are not supported in JFrog Cloud, and those properties are skipped during the transfer process. To run the pre-checks, follow these steps: -1. Install JFrog CLI on any machine that has access to both the source and the target JFrog instances. To do this, follow the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). +1: Install JFrog CLI on any machine that has access to both the source and the target JFrog instances. To do this, follow the steps described [here](#installing-jfrog-cli-on-a-machine-with-network-access-to-the-source-and-target-machines). -2. Run the following command: +2: Run the following command: ```sh jf rt transfer-files source-server target-server --prechecks ``` @@ -419,12 +419,12 @@ To run the pre-checks, follow these steps: By default, files that are larger than 25 GB will be blocked by the JFrog Cloud infrastructure during the files transfer. To check whether your source Artifactory instance hosts files larger than that size, do the following. -1. Run the following curl command from your terminal, after replacing the ``, `` and `` tokens with your source instance details. The command execution may take a few minutes, depending on the number of files hosted by Artifactory. +1: Run the following curl command from your terminal, after replacing the ``, `` and `` tokens with your source instance details. The command execution may take a few minutes, depending on the number of files hosted by Artifactory. ```sh curl -X POST /artifactory/api/search/aql -H "Content-Type: text/plain" -d 'items.find({"name" : {"$match":"\*"}}).include("size").sort({"$desc" : \["size"\]}).limit(1)' -u : ``` -2. You should get a result that looks like the following. +2: You should get a result that looks like the following. ``` { "results":[ @@ -443,15 +443,15 @@ By default, files that are larger than 25 GB will be blocked by the JFrog Cloud The value of **size** represents the largest file size hosted by your source Artifactory instance. -3. If the size value you received is larger than 25000000000, please avoid initiating the files transfer before contacting JFrog Support, to check whether this size limit can be increased for you. You can contact Support by sending an email to [support@jfrog.com](mailto:support@jfrog.com) +3: If the size value you received is larger than 25000000000, please avoid initiating the files transfer before contacting JFrog Support, to check whether this size limit can be increased for you. You can contact Support by sending an email to [support@jfrog.com](mailto:support@jfrog.com) ## Routing the traffic from the source to the target through an HTTPS proxy The `jf rt transfer-files` command pushes the files directly from the source to the target instance over the network. In case the traffic from the source instance needs to be routed through an HTTPS proxy, follow these steps. -1. Define the proxy details in the source instance UI as described in the Managing ProxiesManaging Proxies documentation. +1: Define the proxy details in the source instance UI as described in the Managing ProxiesManaging Proxies documentation. -2. When running the `jf rt transfer-files` command, add the `--proxy-key` option to the command, with Proxy Key you configured in the UI as the option value. For example, if the Proxy Key you configured is **my-proxy-key**, run the command as follows: +2: When running the `jf rt transfer-files` command, add the `--proxy-key` option to the command, with Proxy Key you configured in the UI as the option value. For example, if the Proxy Key you configured is **my-proxy-key**, run the command as follows: ```sh jf rt transfer-files my-source my-target --proxy-key my-proxy-key ```