diff --git a/readmes/README.macOS.md b/readmes/README.macOS.md index fd6de742e66..eb4d0898b25 100644 --- a/readmes/README.macOS.md +++ b/readmes/README.macOS.md @@ -1,153 +1,139 @@ -We've confirmed that these steps work on a brand new installation of macOS -Sierra. +# Compile and Install Cloudberry Database on macOS -## 1. Install needed dependencies +This document shares how to build, compile, and install Cloudberry Database on macOS for development purposes. Follow the steps below. -Execute the following, this script will start by caching the `sudo` password: +According to our test, these steps work well on macOS Ventura 13.4+ with both Intel and Apple silicon processors (M1 or M2). If you have an older version of macOS, upgrading is recommended. -```bash -./README.macOS.bash -source ~/.bash_profile -``` - -Note: This will install Homebrew if missing - -## 2. Allow ssh to use the version of python in path, not the system python - -Execute the following in your terminal: - -```bash -mkdir -p "$HOME/.ssh" - -cat >> ~/.bash_profile << EOF - -# Allow ssh to use the version of python in path, not the system python -# BEGIN SSH agent -# from http://stackoverflow.com/questions/18880024/start-ssh-agent-on-login/18915067#18915067 -SSH_ENV="\$HOME/.ssh/environment" -# Refresh the PATH per new session -sed -i .bak '/^PATH/d' \${SSH_ENV} -echo "PATH=\$PATH" >> \${SSH_ENV} - -function start_agent { - echo "Initialising new SSH agent..." - /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "\${SSH_ENV}" - echo succeeded - chmod 600 "\${SSH_ENV}" - source "\${SSH_ENV}" > /dev/null - /usr/bin/ssh-add; -} - -# Source SSH settings, if applicable -if [ -f "\${SSH_ENV}" ]; then - . "\${SSH_ENV}" > /dev/null - ps -ef | grep \${SSH_AGENT_PID} 2>/dev/null | grep ssh-agent$ > /dev/null || { - start_agent; - } -else - start_agent; -fi - -[ -f ~/.bashrc ] && source ~/.bashrc -# END SSH agent -EOF -``` - -Then bring `.bash_profile` into effect and allow ssh to use the version of python in path, not the system python. - -```bash -source ~/.bash_profile - -sudo tee -a /etc/ssh/sshd_config << EOF -# Allow ssh to use the version of python in path, not the system python -PermitUserEnvironment yes -EOF -``` - -## 3. Enable ssh to localhost without requiring a password - -Currently, the GPDB utilities require that it be possible to `ssh` to localhost -without using a password. To verify this, the following command should succeed -without erroring, or requiring you to enter a password. - -```bash -ssh # e.g., ssh briarwood (You can use `hostname` to get the hostname of your machine.) -``` - -However, if it is the first time that you are `ssh`'ing to localhost, you may -need to accept the trust on first use prompt. - -```bash -The authenticity of host '' can't be established. -ECDSA key fingerprint is SHA256:. -Are you sure you want to continue connecting (yes/no)? -``` - -If the hostname does not resolve, try adding your machine name to `/etc/hosts`, -like so: - -```bash -echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts -``` +## Step 1. Install needed dependencies -If you see `ssh: connect to host <> port 22: Connection refused`, enable remote -login in: +Run the following command to install the needed dependencies. You will be asked to enter the `sudo` password of your macOS system. ```bash -System Preferences -> Sharing -> Remote Login +source readmes/README.macOS.bash ``` -If you see a password prompt, add your SSH key to the `authorized_keys` file, -like so: +> [!NOTE] +> This will install [Homebrew](https://brew.sh/) if missing. + +## Step 2. Enable password-free SSH connection to localhost + +1. Enable **Remote Login** on your macOS system by navigating to **System Preferences** \> **Sharing** \> **Remote Login**. +2. Run the following command to verify whether password-free SSH connection to localhost has been enabled on your operating system. + + ```bash + ssh $(hostname) + ``` + + - If this command runs without error or requiring you to enter a password, run `exit` and go to [Step 3. Configure, compile, and install](#step-3-configure-compile-and-install). + - If you are required to enter a password, take the following steps to set up password-free SSH connection. + + 1. Run `ssh-keygen` and then `cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys`. + 2. Run `ssh $(hostname)` again to check whether password-free connection is ready. + 3. If ready, run `exit` and go to [Step 3. Configure, compile, and install](#step-3-configure-compile-and-install). + +> [!NOTE] +> +> - If it is the first time you are using `ssh` connection to localhost, you might need to accept the trust on the first use prompt: +> +> ```bash +> The authenticity of host '' can't be established. +> ECDSA key fingerprint is SHA256:. +> Are you sure you want to continue connecting (yes/no)? +> ``` +> +> - If your hostname does not resolve, try adding your machine name to `/etc/hosts`: +> +> ```bash +> echo -e "127.0.0.1\t$HOSTNAME" | sudo tee -a /etc/hosts +> ``` + +## Step 3. Configure, compile, and install ```bash -mkdir -p ~/.ssh -ssh-keygen -t rsa -b 4096 -C "your_email@example.com" -cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys -``` +# Run the following commands under the `cloudberrydb/` dir. +# 1. Configure the build environment. -After troubleshooting, verify one more time that you can ssh without a -password and that `~/.ssh/environment` exists, by running: +BREWPREFIX=$(brew --prefix); export PATH="$BREWPREFIX/opt/gnu-sed/libexec/gnubin:$BREWPREFIX/opt/apr/bin:$PATH"; CXXFLAGS="-I $BREWPREFIX/include" CFLAGS="-ggdb -Og -g3 -fno-omit-frame-pointer -I $BREWPREFIX/include" LDFLAGS="-L $BREWPREFIX/lib" CC=$(which gcc-13) CXX=$(which g++-13) ./configure --enable-debug --prefix=$(cd ~; pwd)/install/cbdb; -```bash -ssh -ls ~/.ssh/environment -``` +# 2. Compile and install Cloudberry Database. -If the ssh succeeded with out asking for a password, and that file exists, you -are good to go. Remember to exit all of your ssh session before moving on. +make -j8 +make -j8 install -## 4. Configure, compile, and install +# 3. Bring in Greenplum environment for Cloudberry Database into your running shell. -At this point head back the main [README.md](./README.md#build-the-database), -and continue from __Build the Database__. +source $(cd ~; pwd)/install/cbdb/greenplum_path.sh -## Troubleshooting macOS specific issues: +# 4. Install the Python dependencies. -### Workaround for libreadline / libxml2 on OSX 10.11 (El Capitan) +pip3 install --user -r readmes/python-dependencies.txt -Symptoms: -* After Running `./configure`, -* You see output - `configure: error: readline library not found`, and -* in `config.log` you see - `ld: file not found: /usr/lib/system/libsystem_symptoms.dylib for architecture x86_64` +# 5. Start a demo cluster. -There is an issue with Xcode 8.1 on El Capitan. Here's a workaround: +PORT_BASE=8000 make create-demo-cluster -```bash -brew install libxml2 -brew link libxml2 --force +# (gpdemo-env.sh contains __PGPORT__ and __MASTER_DATA_DIRECTORY__ values) ``` -Other workarounds may include downgrading to Xcode 7.3, or installing the CLT -package from Xcode 7.3. - -For more info, [this seems to be the best thread](https://github.com/Homebrew/brew/issues/972) - -### Disable System Integrity Protection +## Step 4. Verify the cluster + +1. You can verify whether the cluster has started successfully by running the following command. You will see many active `postgres` processes with ports ranging from `8000` to `8007`. + + ```bash + ps -ef | grep postgres + ``` + +2. Connect to the Cloudberry Database and see the active segment information by querying the system table `gp_segement_configuration`. For detailed description of this table, see the Greenplum document [here](https://docs.vmware.com/en/VMware-Greenplum/6/greenplum-database/ref_guide-system_catalogs-gp_segment_configuration.html). + + ```sql + $ psql -p 8000 postgres + postgres=# select version(); + postgres=# select * from gp_segment_configuration; + ``` + + Example output: + + ```shell + $ psql -p 8000 postgres + psql (14.4, server 14.4) + Type "help" for help. + + postgres=# select version(); + version + ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + PostgreSQL 14.4 (Cloudberry Database 1.0.0+1c0d6e2224 build dev) on x86_64-apple-darwin22.4.0, compiled by gcc-13 (Homebrew GCC 13.2.0) 13.2.0, 64-bit compiled on Sep 22 2023 10:56:01 + (1 row) + + postgres=# select * from gp_segment_configuration; + dbid | content | role | preferred_role | mode | status | port | hostname | address | datadir | warehouseid + ------+---------+------+----------------+------+--------+------+-----------------------------+-----------------------------+----------------------------------------------------------------------------------------------------------+------------- + 1 | -1 | p | p | n | u | 8000 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/qddir/demoDataDir-1 | 0 + 8 | -1 | m | m | s | u | 8001 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/standby | 0 + 3 | 1 | p | p | s | u | 8003 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast2/demoDataDir1 | 0 + 6 | 1 | m | m | s | u | 8006 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast_mirror2/demoDataDir1 | 0 + 2 | 0 | p | p | s | u | 8002 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast1/demoDataDir0 | 0 + 5 | 0 | m | m | s | u | 8005 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast_mirror1/demoDataDir0 | 0 + 4 | 2 | p | p | s | u | 8004 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast3/demoDataDir2 | 0 + 7 | 2 | m | m | s | u | 8007 | hashdatas-MacBook-Pro.local | hashdatas-MacBook-Pro.local | /Users/hashdata/Documents/GitHub/upstream/cloudberrydb/gpAux/gpdemo/datadirs/dbfast_mirror3/demoDataDir2 | 0 + (8 rows) + + postgres=# + ``` + + > [!NOTE] + > If the following problem occurs when connecting to the master: + > + > psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory + > the server running locally and accepting connections on that socket? + > + > You can solve it by running `export PGPORT=8000`. Run `export PGPORT=8000` every time a shell process is started. You can also write this in the `.bashrc` file by running `echo "export PGPORT=8000" >> ~/.bashrc`. + +3. Now we can finally run `installcheck-world` to test everything works fine: + + ```bash + # In the folder where your cloned the source code + make installcheck-world + ``` + +Congratulations 🎉! You've successfully installed and created a CloudberryDB cluster. Happy Hacking! 😉 -Note that you may need to disable System Integrity Protection in order to bring -up the gpdemo cluster. Without doing this, psql commands run in child processes -spawned by gpinitsystem may have the DYLD_* environment variables removed from -their environments.