Skip to content

Commit e2f2030

Browse files
nwoolmernwoolmer
authored
Merge branch 'main' into feat-pivot
2 parents cafbefd + b98254f commit e2f2030

16 files changed

+492
-263
lines changed

‎documentation/deployment/aws-official-ami.md renamed to ‎documentation/deployment/aws.md

+66-12
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,82 @@
11
---
2-
title: Launch the official QuestDB AMI via the AWS Marketplace
3-
sidebar_label: AWS Marketplace AMI
2+
title: Deploying to Amazon Web Services (AWS)
3+
sidebar_label: AWS
44
description:
5-
This document describes how to launch the official AWS Marketplace AMI with
6-
QuestDB installed and how to access and secure the instance on Amazon Web
7-
Services
5+
This document explains what to hardware to use, and how to provision QuestDB on Amazon Web Services (AWS).
86
---
97

8+
import FileSystemChoice from "../../src/components/DRY/_questdb_file_system_choice.mdx"
9+
import MinimumHardware from "../../src/components/DRY/_questdb_production_hardware-minimums.mdx"
1010
import InterpolateReleaseData from "../../src/components/InterpolateReleaseData"
1111
import CodeBlock from "@theme/CodeBlock"
1212

13-
AWS Marketplace is a digital catalog with software listings from independent
13+
14+
## Hardware recommendations
15+
16+
<MinimumHardware />
17+
18+
### Elastic Compute Cloud (EC2) with Elastic Block Storage (EBS)
19+
20+
We recommend starting with `M6` instances, with an upgrade to
21+
`R6` instances if extra RAM is needed. You can use either `i` (Intel) or `a` (AMD) instsance.
22+
23+
These should be deployed with an `x86_64` Linux distribution, such as Ubuntu.
24+
25+
For storage, we recommend using `gp3` disks, as these provide a better price-to-performance
26+
ratio compared to `gp2` or `io1` offerings.`5000 IOPS/300 MBps` is a good starting point until
27+
you have tested your workload.
28+
29+
<FileSystemChoice />
30+
31+
### Elastic File System (EFS)
32+
33+
QuestDB **does not** support `EFS` for its primary storage. Do not use it instead of `EBS`.
34+
35+
You may be able to use it as an object store, but we would recommend using `S3` instead, as a simpler,
36+
and cheaper, alternative.
37+
38+
### Simple Storage Service (S3)
39+
40+
QuestDB supports `S3` as its replication object-store in the Enterprise edition.
41+
42+
This requires very little provisioning - simply create a bucket or virtual subdirectory and follow
43+
the [Enterprise Quick Start](/docs/guides/enterprise-quick-start/) steps to configure replication.
44+
45+
### Minimum specification
46+
47+
- **Instance**: `m6i.xlarge` or `m6a.xlarge` `(4 vCPUs, 16 GiB RAM)`
48+
- **Storage**
49+
- **OS disk**: `gp3 (30 GiB)` volume provisioned with `3000 IOPS/125 MBps`.
50+
- **Data disk**: `gp3 (100 GiB)` volume provisioned with `3000 IOPS/125 MBps`.
51+
- **Operating System**: `Linux Ubuntu 24.04 LTS x86_64`.
52+
- **File System**: `ext4`
53+
54+
### Better specification
55+
56+
- **Instance**: `r6i.2xlarge` or `r6a.xlarge` `(8 vCPUs, 64 GiB RAM)`
57+
- **Storage**
58+
- **OS disk**: `gp3 (30 GiB)` volume provisioned with `5000 IOPS/300 MBps`.
59+
- **Data disk**: `gp3 (300 GiB)` volume provisioned with `5000 IOPS/300 MBps`.
60+
- **Operating System**: `Linux Ubuntu 24.04 LTS x86_64`.
61+
- **File System**: `zfs` with `lz4` compression.
62+
63+
## Launching QuestDB on EC2
64+
65+
Once you have provisioned your `EC2` instance with attached `EBS` storage, you can simply
66+
follow the setup instructions for a [Docker](docker.md) or [systemd](systemd.md) installation.
67+
68+
You can also keep it simple - just [download](https://questdb.com/download/) the binary and run it directly.
69+
QuestDB is a single self-contained binary and easy to deploy.
70+
71+
## Launching QuestDB on the AWS Marketplace
72+
73+
[AWS Marketplace](https://aws.amazon.com/marketplace) is a digital catalog with software listings from independent
1474
software vendors that runs on AWS. This guide describes how to launch QuestDB
1575
via the AWS Marketplace using the official listing. This document also describes
1676
usage instructions after you have launched the instance, including hints for
1777
authentication, the available interfaces, and tips for accessing the REST API
1878
and [Web Console](/docs/web-console/).
1979

20-
## Prerequisites
21-
22-
- An [Amazon Web Services](https://console.aws.amazon.com) account
23-
24-
## Launching QuestDB on the AWS Marketplace
25-
2680
The QuestDB listing can be found in the AWS Marketplace under the databases
2781
category. To launch a QuestDB instance:
2882

‎documentation/deployment/azure.md

+230
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
---
2+
title: Deploying to Microsoft Azure
3+
sidebar_label: Azure
4+
description:
5+
This document explains what to hardware to use, and how to provision QuestDB on Microsoft Azure.
6+
---
7+
8+
import FileSystemChoice from "../../src/components/DRY/_questdb_file_system_choice.mdx"
9+
import MinimumHardware from "../../src/components/DRY/_questdb_production_hardware-minimums.mdx"
10+
11+
## Hardware recommendations
12+
13+
<MinimumHardware />
14+
15+
### Azure Virtual Machines with Azure Managed Disk
16+
17+
Azure Virtual Machines have a naming convention that is handy for finding compatible instances.
18+
19+
**Do not** use instances with the letter `p`. These are `ARM` architecture instances, usually running
20+
on `Cobalt` chips.
21+
22+
**Do** use instances with the letter `s`. This indicates that it is compatible with `Premium SSD` storage,
23+
preferred for QuestDB.
24+
25+
Either `AMD EPYC` CPUs (`a` letter) or `Intel Xeon` (no letter) are appropriate for `x86_64` deployments.
26+
27+
We recommend starting with `D-series` instances, and then later upgrading to `E-series` if necessary i.e. for more RAM.
28+
29+
You should deploy using an `x86_64` Linux distribution, such as Ubuntu.
30+
31+
For storage, we recommend using [Premium SSD v2](https://learn.microsoft.com/en-us/azure/virtual-machines/disks-types#premium-ssd-v2) disks,
32+
and provisioning them at `5000 IOPS/300 MBps` until you have tested your workload.
33+
34+
:::note
35+
36+
`Premium SSD v2` disks only support locally-redundant storage (LRS). For Enterprise users, this
37+
is not an issue, as your data is secured using replication over Azure Blob Storage.
38+
39+
For open-source users, you may want to:
40+
41+
- downgrade to `Premium SSD` storage, which supports zone-redundant storage (ZRS).
42+
- or publish to multiple instances
43+
- or take frequent ZRS snapshots of your LRS disk.
44+
45+
:::
46+
47+
<FileSystemChoice />
48+
49+
:::warning
50+
51+
QuestDB does **not** support `blobfuse2`. Please use the above recommendations, or refer to [capacity planning](/docs/operations/capacity-planning/)
52+
53+
:::
54+
55+
### Azure NetApp Files
56+
57+
Azure NetAppFiles is a volume-as-a-service (VaaS) offering from Microsoft, supporting an NFS API.
58+
59+
This should **not** be used as primary storage for QuestDB, but could be used as an object store for Enterprise replication.
60+
61+
We would recommend using `Azure Blob Storage` instead as a simpler, and cheaper, alternative.
62+
63+
### Azure Blob Storage
64+
65+
QuestDB supports `Azure Blob Storage` as its replication object-store in the Enterprise edition.
66+
67+
To get started, use `Azure Storage Explorer` to create new `Blob Container`, and then follow the
68+
[Enterprise Quick Start](/docs/guides/enterprise-quick-start/) steps to create a connection string and
69+
configure QuestDB.
70+
71+
### Minimum specification
72+
73+
- **Instance**: `D4as v5` or `D4s v5` `(4 vCPUs, 16 GiB RAM)`
74+
- **Storage**
75+
- **OS disk**: `Premium SSD v2 (30 GiB)` volume provisioned with `3000 IOPS/125 MBps`.
76+
- **Data disk**: `Premium SSD v2 (100 GiB)` volume provisioned with `3000 IOPS/125 MBps`.
77+
- **Operating System**: `Linux Ubuntu 24.04 LTS x86_64`.
78+
- **File System**: `ext4`
79+
80+
:::note
81+
82+
If you use `Premium SSD` instead of `Premium SSD v2`, you should start with a `P20` size (`512 GiB`).
83+
This offers `2300 IOPS/150 MBps` which should be enough for basic workloads.
84+
85+
:::
86+
87+
### Better specification
88+
89+
- **Instance**: `E8as v5`or `E8s v5` `(8 vCPUs, 64 GiB RAM)`
90+
- **Storage**
91+
- **OS disk**: `Premium SSD v2 (30 GiB)` volume provisioned with `5000 IOPS/300 MBps`.
92+
- **Data disk**: `Premium SSD v2 (300 GiB)` volume provisioned with `5000 IOPS/300 MBps`.
93+
- **Operating System**: `Linux Ubuntu 24.04 LTS x86_64`.
94+
- **File System**: `zfs`
95+
96+
:::note
97+
98+
If you use `Premium SSD` instead of `Premium SSD v2`, you should upgrade to a `P30` size disk (`1 TiB`).
99+
This offers `5000 IOPS/200 MBps` which should be enough for higher workloads.
100+
101+
:::
102+
103+
## Launching QuestDB on Azure Virtual Machines
104+
105+
This guide demonstrates how to spin up a Microsoft Azure Virtual Machine that is
106+
running QuestDB on Ubuntu. This will help get you comfortable with Azure VM
107+
basics.
108+
109+
### Prerequisites
110+
111+
- A [Microsoft Azure account](https://azure.microsoft.com/) with billing
112+
enabled. Adding a credit card is required to create an account, but this demo
113+
will only use resources in the free tier.
114+
115+
### Create an Azure VM
116+
117+
1. In the Azure console, navigate to the **Virtual Machines** page. Once you are
118+
on this page, click the **Create** dropdown in the top left-hand corner of
119+
the screen and select the **Azure virtual machine** option.
120+
121+
2. From here, fill out the required options. If you don't already have a
122+
**Resource group**, you can create one on this page. We made a "default"
123+
group for this example, but you are free to choose any name you like. Enter
124+
the name of your new virtual machine, as well as its desired Region and
125+
Availability Zone. Your dialog should look something like this:
126+
127+
<Screenshot
128+
alt="The Create Instance dialog on Microsoft Azure"
129+
src="images/guides/microsoft-azure-ubuntu/create-vm.webp"
130+
width={450}
131+
title="Click to zoom"
132+
/>
133+
134+
3. Scroll down and select your desired instance type. In this case, we used a
135+
`Standard_B1s` to take advantage of Azure's free tier.
136+
4. If you don't already have one, create a new SSH key pair to securely connect
137+
to the instance once it has been created.
138+
139+
<Screenshot
140+
alt="The Create Instance dialog on Microsoft Azure, continued"
141+
src="images/guides/microsoft-azure-ubuntu/ssh-setup.webp"
142+
width={450}
143+
title="Click to zoom"
144+
/>
145+
146+
5. We will use Azure defaults for the rest of the VM's settings. Click
147+
**Review + create** to confirm your settings, then **Create** to download
148+
your new key pair and launch the instance.
149+
150+
<Screenshot
151+
alt="Deployment Complete"
152+
src="images/guides/microsoft-azure-ubuntu/deployment-complete.webp"
153+
width={450}
154+
title="Click to zoom"
155+
/>
156+
157+
Once you see this screen, click the **Go to resource** button and move on to the
158+
next section
159+
160+
### Set up networking
161+
162+
We now need to set up the appropriate firewall rules which will allow you to
163+
connect to your new QuestDB instance over the several protocols that we support.
164+
165+
1. In the **Settings** sidebar, click the **Networking** button. This will lead
166+
you to a page with all firewall rules for your instance. To open up the
167+
required ports, click the **Add inbound port rule** on the right-hand side.
168+
2. Change the **Destination port ranges** to the `8812,9000,9003`, set the
169+
**Protocol** to `TCP`, change the name to `questdb`, and click the **Add**
170+
button. This will add the appropriate ingress rules to your instance's
171+
firewall. It may take a few seconds, and possibly a page refresh, but you
172+
should see your new firewall rule in the list. Port 8812 is used for the
173+
postgresql protocol, port 9000 is used for the web interface, the REST API,
174+
and ILP ingestion over HTTP. Port 9003 is used for metrics and health check.
175+
176+
<Screenshot
177+
alt="Firewall rules for your Azure VM"
178+
src="images/guides/microsoft-azure-ubuntu/firewall-rules.webp"
179+
width={450}
180+
title="Click to zoom"
181+
/>
182+
183+
### Install QuestDB
184+
185+
Now that you've opened up the required ports, it's time to install and run
186+
QuestDB. To do this, you first need to connect to your instance over SSH. Since
187+
we named our SSH key `questdb_key`, this is the filename that the commands below
188+
use. You should substitute this with your own key name that you downloaded in
189+
the previous step. You also need to use your VM's external IP address instead of
190+
the placeholder that we have provided.
191+
192+
We first need to adjust the permissions on the downloaded file, and then use it
193+
to ssh into your instance.
194+
195+
```bash
196+
export YOUR_INSTANCE_IP=172.xxx.xxx.xxx
197+
chmod 400 ~/download/questdb_key.pem
198+
ssh -i ~/download/questdb_key.pem azureuser@$YOUR_INSTANCE_IP
199+
```
200+
201+
Once we've connected to the instance, we will use `wget`
202+
to download the QuestDB binary, extract it, and run the start script. Please visit
203+
the Ubuntu section at the [binary installation page](/download/) to make sure you are using the latest
204+
version of the binary package and replace the URL below as appropriate.
205+
206+
<InterpolateReleaseData
207+
renderText={(release) => (
208+
<CodeBlock className="language-bash">
209+
{`wget https://github.com/questdb/questdb/releases/download/${release.name}/questdb-${release.name}-rt-linux-x86-64.tar.gz
210+
tar -xvf questdb-${release.name}-rt-linux-x86-64.tar.gz
211+
cd questdb-${release.name}-rt-linux-x86-64/bin
212+
./questdb.sh start`}
213+
</CodeBlock>
214+
)}
215+
/>
216+
217+
Once you've run these commands, you should be able to navigate to your instance
218+
at its IP on port 9000: `http://$YOUR_INSTANCE_IP:9000`
219+
220+
<Screenshot
221+
alt="Firewall rules for your Azure VM"
222+
src="images/guides/microsoft-azure-ubuntu/web-console.webp"
223+
width={450}
224+
title="Click to zoom"
225+
/>
226+
227+
## Single Sign On with EntraID
228+
229+
If you are using EntraID to manage users, [QuestDB enterprise](/enterprise/) offers the possibility to do Single Sign On and manage your database permissions.
230+
See more information at the [Microsoft EntraID OIDC guide](/docs/guides/microsoft-entraid-oidc/).

‎documentation/deployment/digitalocean.md renamed to ‎documentation/deployment/digital-ocean.md

+15-11
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
---
2-
title: Launch QuestDB on DigitalOcean
3-
sidebar_label: DigitalOcean Droplet
4-
description: This document describes how to launch DigitalOcean droplet with QuestDB
2+
title: Deploying to Digital Ocean
3+
sidebar_label: Digital Ocean
4+
description:
5+
This document describes how to launch the official AWS Marketplace AMI with
6+
QuestDB installed and how to access and secure the instance on Amazon Web
7+
Services
58
---
69

10+
711
DigitalOcean is a platform with software listings from independent vendors that
812
run on cloud resources. This guide describes how to launch QuestDB via the
913
DigitalOcean marketplace using the official listing. This document also
@@ -35,10 +39,10 @@ method is preferred:
3539
import Screenshot from "@theme/Screenshot"
3640

3741
<Screenshot
38-
alt="Choosing the RAM and CPU capacity for a QuestDB DigitalOcean Droplet"
39-
height={591}
40-
src="images/blog/2021-07-09/choosing-droplet.webp"
41-
width={770}
42+
alt="Choosing the RAM and CPU capacity for a QuestDB DigitalOcean Droplet"
43+
height={591}
44+
src="images/blog/2021-07-09/choosing-droplet.webp"
45+
width={770}
4246
/>
4347

4448
4. Choose a region closest to you
@@ -49,10 +53,10 @@ import Screenshot from "@theme/Screenshot"
4953
the bottom of the page
5054

5155
<Screenshot
52-
alt="Finalizing the creation step of a DigitalOcean Droplet running QuestDB"
53-
height={591}
54-
src="images/blog/2021-07-09/questdb-droplet.webp"
55-
width={770}
56+
alt="Finalizing the creation step of a DigitalOcean Droplet running QuestDB"
57+
height={591}
58+
src="images/blog/2021-07-09/questdb-droplet.webp"
59+
width={770}
5660
/>
5761

5862
After 30 seconds, QuestDB should be ready to use. To validate that we set

0 commit comments

Comments
 (0)