Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

docs: Update Installation, overview, quickstart and Cortex version commands #200

Merged
merged 6 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/basic-usage/command-line.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ slug: "command-line"
Cortex has a [Docker](https://docs.docker.com/engine/reference/commandline/cli/) and [Ollama](https://ollama.com/)-inspired [CLI syntax](/docs/cli) for running model operations.

## How It Works
Cortex’s CLI invokes the Cortex Engine’s API, which runs in the background on port `1337`.
Cortex’s CLI invokes the Cortex Engine’s API, which runs in the background on port `3928`.


## Basic Usage
### [Start Cortex Server](/docs/cli)
```bash
# By default the server will be started on port `1337`
# By default the server will be started on port `3928`
cortex
```
### [Run Model](/docs/cli/run)
Expand Down
12 changes: 12 additions & 0 deletions docs/basic-usage/cortexrc.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
title: .cortexrc
description: .cortexrc Overview.
slug: "cortexrc"
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::
140 changes: 140 additions & 0 deletions docs/basic-usage/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
---
title: Overview
description: Overview.
slug: "basic-usage"
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex has an [API server](https://cortex.so/api-reference) that runs at `localhost:3928`.


## Usage
### Start Cortex.cpp Server
<Tabs>
<TabItem value="MacOs/Linux" label="MacOs/Linux">
```sh
# Stable
cortex start

# Beta
cortex-beta start

# Nightly
cortex-nightly start
```
</TabItem>
<TabItem value="Windows" label="Windows">
```sh
# Stable
cortex.exe start

# Beta
cortex-beta.exe start

# Nightly
cortex-nightly.exe start
```
</TabItem>
</Tabs>
### Run Model
```bash
# Pull a model
curl --request POST \
--url http://localhost:3928/v1/models/mistral/pull
# Start the model
curl --request POST \
--url http://localhost:3928/v1/models/mistral/start \
--header 'Content-Type: application/json' \
--data '{
"prompt_template": "system\n{system_message}\nuser\n{prompt}\nassistant",
"stop": [],
"ngl": 4096,
"ctx_len": 4096,
"cpu_threads": 10,
"n_batch": 2048,
"caching_enabled": true,
"grp_attn_n": 1,
"grp_attn_w": 512,
"mlock": false,
"flash_attn": true,
"cache_type": "f16",
"use_mmap": true,
"engine": "llamacpp"
}'
```
### Show the Model State
```bash
# Check the model status
curl --request GET \
--url http://localhost:3928/v1/system/events/model
```
### Chat with Model
```bash
# Invoke the chat completions endpoint
curl http://localhost:3928/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "",
"messages": [
{
"role": "user",
"content": "Hello"
},
],
"model": "mistral",
"stream": true,
"max_tokens": 1,
"stop": [
null
],
"frequency_penalty": 1,
"presence_penalty": 1,
"temperature": 1,
"top_p": 1
}'
```
### Stop Model
```bash
# Stop a model
curl --request POST \
--url http://localhost:3928/v1/models/mistral/stop
```
### Pull Model
```bash
# Pull a model
curl --request POST \
--url http://localhost:3928/v1/models/mistral/pull
```
### Stop Cortex.cpp Server
<Tabs>
<TabItem value="MacOs/Linux" label="MacOs/Linux">
```sh
# Stable
cortex stop

# Beta
cortex-beta stop

# Nightly
cortex-nightly stop
```
</TabItem>
<TabItem value="Windows" label="Windows">
```sh
# Stable
cortex.exe stop

# Beta
cortex-beta.exe stop

# Nightly
cortex-nightly.exe stop
```
</TabItem>
</Tabs>
2 changes: 1 addition & 1 deletion docs/basic-usage/server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Cortex has an [API server](https://cortex.so/api-reference) that runs at `localh
## Usage
### Start Cortex Server
```bash
# By default the server will be started on port `1337`
# By default the server will be started on port `3928`
cortex
# Start a server with different port number
cortex -a <address> -p <port_number>
Expand Down
35 changes: 32 additions & 3 deletions docs/cli/chat.md → docs/cli/chat.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ description: Cortex chat command.
slug: "chat"
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::
Expand All @@ -18,10 +21,36 @@ This CLI command calls the following API endpoint:
This command starts a chat session with a specified model, allowing you to interact directly with it through an interactive chat interface.

## Usage
:::info
You can use the `--verbose` flag to display more detailed output of the internal processes. To apply this flag, use the following format: `cortex --verbose [subcommand]`.
:::
<Tabs>
<TabItem value="MacOs/Linux" label="MacOs/Linux">
```sh
# Stable
cortex chat [options] <model_id> -m <message>

# Beta
cortex-beta chat [options] <model_id> -m <message>

# Nightly
cortex-nightly chat [options] <model_id> -m <message>
```
</TabItem>
<TabItem value="Windows" label="Windows">
```sh
# Stable
cortex.exe chat [options] <model_id> -m <message>

# Beta
cortex-beta.exe chat [options] <model_id> -m <message>

# Nightly
cortex-nightly.exe chat [options] <model_id> -m <message>
```
</TabItem>
</Tabs>

```bash
cortex chat [options] <model_id> <message>
```
:::info
This command uses a `model_id` from the model that you have downloaded or available in your file system.
:::
Expand Down
53 changes: 0 additions & 53 deletions docs/cli/cortex.md

This file was deleted.

68 changes: 68 additions & 0 deletions docs/cli/cortex.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: Cortex
description: Cortex CLI.
slug: /cli
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# Cortex
This command list all the available commands within the Cortex.cpp commands.

## Usage
:::info
You can use the `--verbose` flag to display more detailed output of the internal processes. To apply this flag, use the following format: `cortex --verbose [subcommand]`.
:::
<Tabs>
<TabItem value="MacOs/Linux" label="MacOs/Linux">
```sh
# Stable
cortex

# Beta
cortex-beta

# Nightly
cortex-nightly
```
</TabItem>
<TabItem value="Windows" label="Windows">
```sh
# Stable
cortex.exe

# Beta
cortex-beta.exe

# Nightly
cortex-nightly.exe
```
</TabItem>
</Tabs>


## Command Chaining
Cortex CLI's command chaining support allows multiple commands to be executed in sequence with a simplified syntax.

For example:

- [cortex run](/docs/cli/run)
- [cortex chat](/docs/cli/chat)

## Sub Commands

- [cortex models](/docs/cli/models): Manage and configure models.
- [cortex chat](/docs/cli/chat): Send a chat request to a model.
- [cortex ps](/docs/cli/ps): Display active models and their operational status.
- [cortex embeddings](/docs/cli/embeddings): Create an embedding vector representing the input text.
- [cortex engines](/docs/cli/engines): Manage Cortex.cpp engines.
- [cortex pull|download](/docs/cli/pull): Download a model.
- [cortex run](/docs/cli/run): Shortcut to start a model and chat.
- [cortex update](/docs/cli/update): Update the Cortex.cpp version.
- [cortex start](/docs/cli/start): Start the Cortex.cpp API server.
- [cortex stop](/docs/cli/stop): Stop the Cortex.cpp API server.
Loading
Loading