Skip to content

Commit

Permalink
feat: README.md (#11)
Browse files Browse the repository at this point in the history
* feat: README.md

* feat: update repo structure

* feat: run server on mac os and linux

* feat: use make

---------

Co-authored-by: vansangpfiev <sang@jan.ai>
  • Loading branch information
vansangpfiev and sangjanai authored May 7, 2024
1 parent c30a259 commit 3e7fd66
Showing 1 changed file with 88 additions and 1 deletion.
89 changes: 88 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,88 @@
# cortex.llamacpp
# cortex.llamacpp
cortex.llamacpp is a high-efficiency C++ inference engine for edge computing.

# Repo Structure
```
.
├── base -> Engine interface
├── examples -> Server example to integrate engine
├── llama.cpp -> Upstream llama C++
├── src -> Engine implementation
├── third-party -> Dependencies of the cortex.llamacpp project
```

## Build from source

This guide provides step-by-step instructions for building cortex.llamacpp from source on Linux, macOS, and Windows systems.

## Clone the Repository

First, you need to clone the cortex.llamacpp repository:

```bash
git clone --recurse https://github.com/janhq/cortex.llamacpp.git
```

If you don't have git, you can download the source code as a file archive from [cortex.llamacpp GitHub](https://github.com/janhq/cortex.llamacpp).

## Build library with server example
- **On Windows**
Install choco
Install make
```
choco install make -y
```

- **On Linux, and Windows:**

```bash
make build-example-server
```

- **On MacOS with Apple Silicon:**

```bash
make build-example-server CMAKE_EXTRA_FLAGS="-DLLAMA_METAL_EMBED_LIBRARY=ON"
```

- **On MacOS with Intel processors:**

```bash
make build-example-server CMAKE_EXTRA_FLAGS="-DLLAMA_METAL=OFF"
```

- **On Linux with CUDA:**

```bash
make build-example-server CMAKE_EXTRA_FLAGS="-DLLAMA_CUDA=ON"
```

## Start process

Finally, let's start Server.
- **On MacOS and Linux:**
```bash
mkdir engines\cortex.llamacpp\
cp ../../build/libengine.dylib engines/cortex.llamacpp/
./server
```
- **On MacOS and Linux:**
```bash
mkdir engines\cortex.llamacpp\
cp ../../build/libengine.so engines/cortex.llamacpp/
./server
```
- **On Windows:**
```bash
cd Release
mkdir engines\cortex.llamacpp\
copy ..\..\..\build\Release\engine.dll engines\cortex.llamacpp\
server.exe
```
# Quickstart
// TODO

0 comments on commit 3e7fd66

Please sign in to comment.