Skip to content

Commit 42f61ee

Browse files
swap-awsswap-aws
authored
fix: (docs): updated getting started guide (#137)
[fix]: docs - updated getting started guide
1 parent ff20eaa commit 42f61ee

14 files changed

+345
-118
lines changed

‎docs/content/developer-guide/models/bedrock-models.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
1-
Amazon Bedrock users need to request access to models before they are available for use. You need to request access to models in Amazon Bedrock. To request access to additional models, select the Model access link in the left side navigation panel in the Amazon Bedrock console. See the [Bedrock Model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) user guide for more details.
1+
Before using the models, you will need to request access to specific models via Amazon Bedrock.
2+
3+
To request access to additional models, select the **Model access** link from the left side navigation panel in the Amazon Bedrock console. For detailed instructions, refer to the [Bedrock Model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html) user guide.

‎docs/content/getting-started/.pages.yml

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
nav:
3+
- Getting started: index.md
4+
- Prerequisites: prerequisites.md
5+
- Deploy Galileo: deploying-galileo.md
6+
- Validate and review the deployment: validate-deployment.md
7+
- Set up Cloud9 as Galileo development environment: cloud9-ide.md

‎docs/content/getting-started/cloud9-ide.md

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Setting up Cloud9 as your development environment
2+
3+
In your Galileo development account, select:
4+
5+
- Cloud9 → Create Environment
6+
- New EC2 Instance → m5.large + Ubuntu 22.04 + 4h Timeout + AWS Systems Manager
7+
- Open Cloud 9 IDE for the new environment
8+
- Follow the [resize instructions](https://ec2spotworkshops.com/ecs-spot-capacity-providers/workshopsetup/resize_ebs.html), giving your instance EBS volume 300GB
9+
- Reboot the instance before using Cloud9.
10+
11+
Using the Cloud9 instance, install these prerequisites:
12+
13+
```bash
14+
sudo apt update && sudo apt upgrade -y
15+
nvm install lts/iron
16+
nvm use lts/iron
17+
npm i -g pnpm
18+
wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add -
19+
sudo add-apt-repository 'deb https://apt.corretto.aws stable main'
20+
sudo apt-get update; sudo apt-get install -y java-18-amazon-corretto-jdk
21+
22+
sudo apt install -y python3.11
23+
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 110
24+
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 100
25+
# NOTE: select python3.11 from the list
26+
sudo update-alternatives --config python3
27+
curl -sSL https://install.python-poetry.org | python3 -
28+
```

‎docs/content/getting-started/deploying-galileo.md

+179
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,179 @@
1+
## Deploying Galileo
2+
3+
Before you start the deployment, make sure:
4+
5+
- Docker is running, with sufficient virtual disk space.
6+
- Your AWS credentials are set up and available in the shell.
7+
- You have reviewed the EULA before requesting access to Bedrock models.
8+
9+
### Using the CLI
10+
11+
!!! tip
12+
We recommend using the CLI for individuals, developer account, trials, and demos.
13+
14+
To deploy Galileo:
15+
16+
1. Open a CLI terminal and navigate to the Galileo directory.
17+
2. Run these commands:
18+
19+
```
20+
export AWS_REGION={current aws region you're in}
21+
export AWS_PROFILE=default
22+
pnpm bootstrap-account
23+
pnpm galileo-cli deploy
24+
```
25+
26+
This will display a guided CLI workflow for input.
27+
28+
!!! note
29+
30+
If you get a `(node:12100) [EACCES] Error: spawn galileo-cli EACCES` message, ignore it.
31+
32+
The following options are displayed for selecting a foundation model.
33+
34+
![CLI select prompt](images/cli-select.png)
35+
36+
3. To navigate these prompts:
37+
38+
- The circle filled green is the currently *selected* option. All other options are *unselected*.
39+
- The underlined and blue text option is the currently *highlighted* option.
40+
- Use the keyboard arrows **up** and **down** to move the currently highlighted option.
41+
- Use the **spacebar** to select/deselect the currently highlighted option.
42+
- To submit your final answer, press **Enter**.
43+
44+
4. Select the following options using the CLI:
45+
46+
- AWS Profile: **default**
47+
- AWS Region: **(press Enter, the filled in region should be the correct region. If not, fill in the region code)**
48+
- Administrator email address: **(enter your email address)**
49+
- Administrator username: **admin**
50+
- Deploy main application stack?: **Y**
51+
- Choose the foundation models to suq:qpport: **(unselect all then press enter)**
52+
- Foundation model region?: **us-west-2**
53+
- Enable Bedrock?: **Y**
54+
- Bedrock Region: **us-west-2**
55+
- Bedrock model ids: **Anthropic Claude (anthropic.claude-v2)**
56+
- Bedrock endpoint url (optional): **(press Enter, should be blank)**
57+
- Choose the default foundation model: **bedrock::anthropic.claude-v2**
58+
- Press Enter for the rest of the prompts.
59+
60+
Your terminal displays this information:
61+
62+
```
63+
____ __ _ _ _ _ _
64+
/ __ \ __ _ __ __ ___ / /__ _ __ _ | |(_)| | ___ ___ ___ | |(_)
65+
/ / _` | / _` |\ \ /\ / // __| / // _` | / _` || || || | / _ \ / _ \ _____ / __|| || |
66+
| | (_| || (_| | \ V V / \__ \ / /| (_| || (_| || || || || __/| (_) ||_____|| (__ | || |
67+
\ \__,_| \__,_| \_/\_/ |___//_/ \__, | \__,_||_||_||_| \___| \___/ \___||_||_|
68+
\____/ |___/
69+
✔ Config file name? … config.json
70+
✔ Application Name (stack/resource naming) … Galileo
71+
✔ AWS Profile … default
72+
✔ AWS Region (app) … us-west-2
73+
✔ Administrator email address Enter email address to automatically create Cognito admin user, otherwise leave blank
74+
… someone@somewhere.com
75+
✔ Administrator username … yourusername
76+
✔ Choose the foundation models to support ›
77+
✔ Foundation model region? … us-west-2
78+
✔ Enable Bedrock? … yes
79+
✔ Bedrock region … us-west-2
80+
✔ Loading available Bedrock models
81+
✔ Bedrock model ids › Anthropic Claude (anthropic.claude-v2)
82+
✔ Bedrock endpoint url (optional) …
83+
✔ Choose the default foundation model › bedrock::anthropic.claude-v2
84+
✔ Embedding model
85+
Enter the model id to use for embeddings, supports any AutoML model
86+
87+
Example: sentence-transformers/all-mpnet-base-v2, intfloat/multilingual-e5-large, sentence-transformers/all-MiniLM-L6-v2
88+
… sentence-transformers/all-mpnet-base-v2
89+
✔ Embedding Vector Size
90+
Enter the vector size for the chosen embedding model
91+
… 768
92+
✔ Embedding model instance type
93+
Enable autoscaling the embedding instance capacity based
94+
95+
Recommend "ml.g4dn.xlarge" for smaller datasets, and "ml.g4dn.2xlarge" for larger datasets
96+
… ml.g4dn.xlarge
97+
✔ Embedding model max capacity (autoscaling)
98+
Enable autoscaling the embedding instance capacity based
99+
100+
Ensure adequate Service Quota limit for SageMaker > "ml.g4dn.xlarge for endpoint usage"
101+
… 1
102+
✔ Indexing Pipeline instance type
103+
Instance type used for processing dataset files and indexing to vector store
104+
… ml.t3.large
105+
✔ Indexing Pipeline max containers
106+
Number of containers used for indexing files to vector store
107+
108+
Ensure adequate Service Quota limit for SageMaker > "ml.t3.large for processing job"
109+
… 5
110+
✔ Create vector store "index"?
111+
If enabled, will create a database index for the data to improve search over large datasets
112+
113+
Recommended for very large datasets
114+
… no
115+
✔ Deploy sample dataset? ›
116+
✔ Enable tooling in dev stage (SageMaker Studio, PgAdmin)? ›
117+
Synthesizing project repository...
118+
? [CDK DEPLOY] Execute the following command in 615092085770?
119+
cdk deploy --require-approval never --region us-west-2 --profile default -c "configPath=config.json" Dev/Galileo
120+
… yes
121+
```
122+
123+
!!! info
124+
125+
It takes about 40 minutes to build and deploy everything. While we wait, continue to the next page to have a look at how this project was built and how to extend it.
126+
127+
#### Updating configuration settings
128+
129+
The CLI will generate an application configuration file in demo/infra/config.json, which will persist your configuration. You can modify this file and redeploy to change the configuration, or use the CLI.
130+
131+
`pnpm run galileo-cli --help` for cli help info
132+
133+
For more details on CLI operations, refer to the [CLI page](../../developer-guide/cli).
134+
135+
!!! info "Cross-Region deployments"
136+
Galileo CLI allows you to deploy your LLM stack and application stack into different Regions.
137+
138+
### Using a CI/CD pipeline
139+
140+
!!! tip
141+
We recommend using the CI/CD pipeline deployment method for live services and for shared team accounts.
142+
143+
**Note**: Make sure your AWS credentials in your shell are correct.
144+
145+
1. Create a CodeCommit repository in your target account/Region name "galileo".
146+
2. Push this git repository to the `mainline` branch
147+
3. Run `pnpm run deploy:pipeline`
148+
149+
### Deploying manually
150+
151+
!!! tip
152+
We recommend using a manual deployment method only if you need to have full control and want to modify the application.
153+
154+
```sh
155+
pnpm install
156+
pnpm build
157+
158+
cd demo/infra
159+
pnpm exec cdk deploy --app cdk.out --require-approval never Dev/Galileo
160+
pnpm exec cdk deploy --app cdk.out --require-approval never Dev/Galileo-SampleDataset # (optional)
161+
```
162+
163+
## What is deployed?
164+
165+
As part of the deployment, the following services are deployed in your AWS account:
166+
167+
- A pre-built conversational UI that enables contextual conversation with memory,
168+
- An optimized embeddings vector store based on RDS Postgres and `pgvector`,
169+
- A scalable and elastic data ingestion pipeline,
170+
- A low latency text embeddings inference engine,
171+
- Retrieval augmented generation (RAG) features, and
172+
- A choice of open source large language models.
173+
174+
![Galileo High-level Architecture](../assets/images/galileo-arch.png)
175+
176+
## Next steps
177+
178+
- [Validate the deployment](validate-deployment.md)
179+
- [Set up Cloud9 as your development environment](cloud9-ide.md)

‎docs/content/getting-started/images/cfn-galileo-stacks.png

175 KB
Loading

‎docs/content/getting-started/images/cli-select.png

19.2 KB
Loading

‎docs/content/getting-started/images/galileo-chat.png

43.1 KB
Loading

‎docs/content/getting-started/images/galileo-login-screen.png

20.2 KB
Loading

‎docs/content/getting-started/images/galileo-stack-outputs.png

159 KB
Loading

‎docs/content/getting-started/index.md

+16-116
Original file line numberDiff line numberDiff line change
@@ -1,123 +1,23 @@
1-
# Getting Started
1+
# Quickstart
22

3-
## Prerequisites
3+
## What's in this guide?
44

5-
*Development Environment:*
5+
Galileo is a framework to help you rapidly experiment with, deploy, and launch Generative AI powered products and services on the AWS platform. This guide describes how to install and configure the Galileo Generative AI Reference Sample.
66

7-
| Tool | Version | Recommendation |
8-
| --------------------- | ----------- | ---------------------------- |
9-
| pnpm | >=8 <9 | <https://pnpm.io/installation> |
10-
| NodeJS | >=18 | Use Node Version Manager ([nvm](https://github.com/nvm-sh/nvm)) |
11-
| Python | >=3.10,<4 | Use Python Version Manager ([pyenv](https://github.com/pyenv/pyenv)) |
12-
| Poetry | >=1.5,<2 | <https://python-poetry.org/docs/> |
13-
| AWS CLI | v2 | <https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html> |
14-
| Docker[^1] | v20+ | <https://docs.docker.com/desktop/> |
15-
| JDK | v17+ | [Amazon Corretto 17](https://docs.aws.amazon.com/corretto/latest/corretto-17-ug/downloads-list.html) |
7+
After deployment, you can access all of these components:
168

17-
[^1]: Docker virtual disk space should have at least 30GB of free space. If you see `no space left on device` error during build, free up space by running `docker system prune -f` and/or increasing the virtual disk size.
9+
- A pre-built conversational UI that enables contextual conversation with memory,
10+
- An optimized embeddings vector store based on RDS Postgres and `pgvector`,
11+
- A scalable and elastic data ingestion pipeline,
12+
- A low latency text embeddings inference engine,
13+
- Retrieval augmented generation (RAG) features, and
14+
- A choice of open source large language models.
1815

19-
### AWS Service Quotas
16+
(Optional) You can deploy Galileo with a sample dataset of US Supreme Court decision documents, providing content users can interact with immediately.
2017

21-
Ensure the necessary service quota limits are increased *based on your configuration* before deploying. The deployment performs a check and will fail early if limits are not met.
18+
## Next steps
2219

23-
!!! warning "Minimum Service Quota Requirements"
24-
The embedding model usage is required for all deployments at this time, and must be 5 unless configured different in the code.
25-
26-
SageMaker processing job quota [ml.g4dn.2xlarge for processing job usage](https://console.aws.amazon.com/servicequotas/home/services/sagemaker/quotas) must be `>= 5`. This is required for current bulk processing of dataset into vectorstore.
27-
28-
For predefined models, check the instance type from the follow table to determine the quota limits you need to increase.
29-
30-
??? abstract "Predefined Model"
31-
--8<-- "development/models/predefined-models.md"
32-
33-
Example, if you only deploy the *Falcon Lite* predefined model, then you only need to ensure `ml.g5.12xlarge for endpoint usage >= 1`, while the other quotas of *X for endpoint usage* can remain 0. With the exception of below minimum requirements.
34-
35-
!!! tip "Cross-Region Deployments"
36-
Galileo CLI enables you to deploy your LLM stack and application stack into different regions.
37-
38-
---
39-
40-
## Quick Start
41-
42-
!!! tip "Bedrock Model access"
43-
--8<-- "development/models/bedrock-models.md"
44-
45-
Quickly deploy the full solution using the following:
46-
47-
> * Make sure docker is running! And with sufficient virtual disk space.
48-
> * Make sure your AWS credentials are setup and available in the shell.
49-
50-
### CLI
51-
52-
!!! tip "Recommended basic development for individuals, developer account, trials, and demos"
53-
54-
Use the companion cli for deploying the cdk infra
55-
56-
```sh
57-
pnpm install
58-
pnpm run galileo-cli
59-
# Follow the prompt from the cli to build and deploy what you want
60-
```
61-
62-
The cli will generate an application configuration file in demo/infra/config.json, which will persist your configuration. You can modify this file and redeploy to change the configuration, or use the cli again.
63-
64-
> `pnpm run galileo-cli --help` for cli help info
65-
66-
Check out the [CLI page](../../development/cli/) for more details on CLI operations.
67-
68-
### Manually
69-
70-
!!! tip "Recommended for full control and modification"
71-
72-
```sh
73-
pnpm install
74-
pnpm build
75-
76-
cd demo/infra
77-
pnpm exec cdk deploy --app cdk.out --require-approval never Dev/Galileo
78-
pnpm exec cdk deploy --app cdk.out --require-approval never Dev/Galileo-SampleDataset # (optional)
79-
```
80-
81-
### CI/CD pipeline
82-
83-
!!! tip "Recommended for live services and for shared team accounts"
84-
85-
1. Create a CodeCommit repository in your target account/region name "galileo"
86-
1. Push this git repository to the `mainline` branch
87-
1. Run `pnpm run deploy:pipeline`
88-
89-
> Make sure your AWS credentials in your shell are correct
90-
91-
## What is deployed?
92-
93-
![Galileo High-level Architecture](../assets/images/galileo-arch.png)
94-
95-
---
96-
97-
## Using Cloud9 as your Galileo Development Environment
98-
99-
In your Galileo development account:
100-
101-
* Cloud9 → Create Environment
102-
* New EC2 Instance → m5.large + Ubuntu 22.04 + 4h Timeout + AWS Systems Manager
103-
* Open Cloud 9 IDE for this new environment
104-
* Follow the resize instructions at <https://ec2spotworkshops.com/ecs-spot-capacity-providers/workshopsetup/resize_ebs.html>, giving your instance EBS volume 300GB (followed by a reboot)
105-
106-
In the Cloud9 instance shell, install some prerequisites:
107-
108-
```bash
109-
sudo apt update && sudo apt upgrade -y
110-
nvm install lts/iron
111-
nvm use lts/iron
112-
npm i -g pnpm
113-
wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add -
114-
sudo add-apt-repository 'deb https://apt.corretto.aws stable main'
115-
sudo apt-get update; sudo apt-get install -y java-18-amazon-corretto-jdk
116-
117-
sudo apt install -y python3.11
118-
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 110
119-
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 100
120-
# NOTE: select python3.11 from the list
121-
sudo update-alternatives --config python3
122-
curl -sSL https://install.python-poetry.org | python3 -
123-
```
20+
- [Review the prerequisites before deploying Galileo](prerequisites.md)
21+
- [Deploy the Galileo Generative AI Reference Sample](deploying-galileo.md)
22+
- [Validate the deployment](validate-deployment.md)
23+
- [Set up Cloud9 as your development environment](cloud9-ide.md)

0 commit comments

Comments
 (0)