Feature: Rust Server Implementation (w/ Proposed Implementation Approach)

[joshmossas]

I'm open to any feedback regarding this proposal. Would be useful to get some discussion from other rust devs about the server development experience they want.

Proposal for Potential Rust Server Implementation

Of the high performance backend languages the Rust ecosystem provides tooling that makes creating a code-first implementation of an Arri server really straightforward (at least in concept it's straightforward). My proposed rust implementation suggests the usage of Procedural Macros to automatically create the App Definition and automatically handle parsing and serialization.

Table of Contents

• Defining Types
• Defining Procedures
• Registering Types and Procedures
• Creating a Rust Server Plugin For Arri
• Additional Notes

Defining Types

Types can be marked with a custom derive macro which will add functions for

• outputting an Arri Type Definition (ATD)
• parsing from JSON according to the ATD specification
• serializing from JSON according to the ATD specification

The macro could look something like this

#[derive(ArriModel)]
pub struct User {
  pub id: String,
  pub name: String,
  pub created_at: DateTime<FixedOffset>,
  pub updated_at: DateTime<FixedOffset>,
}

Which would implement a to_type_def() method that outputs the following ATD:

{
  "properties": {
    "id": {
      "type": "string"
    },
    "name": {
      "type": "string"
    },
    "created_at": {
      "type": "timestamp"
    },
    "updated_at": {
      "type": "timestamp"
    }
  },
  "metadata": {
    "id": "User"
  }
}

We could also provide an additional annotation to denote the casing of the JSON keys:

#[derive(ArriModel)]
pub struct User {
  pub id: String,
  pub name: String,
  #[arri(json_key = "createdAt")]
  pub created_at: DateTime<FixedOffset>,
  #[arri(json_key = "updatedAt")]
  pub updated_at: DateTime<FixedOffset>,
}

The mapping of Rust primitive types to ATD primitive types would be as follows:

ATD Type	Rust Type
"string"	String
"boolean"	bool
"timestamp"	DateTime
"float32"	f32
"float64"	f64
"int8"	i8
"uint8"	u8
"int16"	i16
"uint16"	u16
"int32"	i32
"uint32"	u32
"int64"	i64
"uint64"	u64

Handling Optional Fields

Option types will automatically be marked as an optional property when outputting to JTD. So this,

#[derive(ArriModel)]
pub struct User {
    pub id: String,
    pub name: String,
    pub is_admin: Option<bool>,
}

Will become,

{
  "properties": {
    "id": {
      "type": "string"
    },
    "name": {
      "type": "string"
    }
  },
  "optionalProperties": {
    "is_admin": "boolean"
  },
  "metadata": {
    "id": "User"
  }
}

An additional annotation could be added for instances where you would prefer to serialize the type as null instead of omitting it from the response

#[derive(ArriModel)]
pub struct User {
    pub id: String,
    pub name: String,
    #[arri(nullable = true)]
    pub is_admin: Option<bool>,
}

Enums and Discriminated Unions

For discriminated unions we will need to add some restrictions. Either none of the enums have properties or all of the enums have properties.

Basic Enum

A basic enum would look like this

#[derive(ArriModel)]
enum UserRole {
  Standard,
  Admin,
  Moderator,
}

Which would become the following in ATD

{
  "enum": ["STANDARD", "ADMIN", "MODERATOR"],
  "metadata": {
    "id": "UserRole"
  }
}

Tagged Unions

A tagged union would require all enum variants to have at least 1 property

#[derive(ArriModel)]
enum Shape {
  Rectangle { width: f64, height: f64 },
  Circle { radius: f64 },
}

Would output the following ATD:

{
  "discriminator": "type",
  "mapping": {
    "RECTANGLE": {
      "properties": {
        "width": {
          "type": "float64"
        },
        "height": {
          "type": "float64"
        }
      }
    },
    "CIRCLE": {
      "properties": {
        "radius": {
          "type": "float64"
        }
      }
    }
  },
  "metadata": {
    "id": "Shape"
  }
}

Defining Procedures

Procedures can be marked with another proc macro, which will act as syntaxtic sugar for creating an HTTP endpoint the "Arri" way. We can use the Leptos Server Functions as a potential reference for this. We can also enforce that the first parameter must be a struct that has derived ArriModel or whatever we end up calling it, and also enforce that Rpc functions must return a result type of Result<T, ArriError> where T is an ArriModel

So for example

#[rpc]
pub async fn get_user(params: UserParams) -> Result<User, ArriError> {
    // TODO
}

Becomes

POST /get-user

And get's represented as the following in the App Definition

{
  "procedures": {
    "getUser": {
      "transport": "http",
      "path": "/get-user",
      "method": "post",
      "params": "UserParams",
      "response": "User"
    }
  }
}

Additional annotation inputs could be use to manually override things like the endpoint path, http method, etc.

#[rpc(path = "/get_user", method = "get")]
pub async fn get_user(params: UserParams) -> Result<User, ArriError> {
    // TODO
}

Registering Types and Procedures

I haven't quite figured out the API for this part yet, but we need to register the procedures on the application itself. I know that somehow Leptos is able to do this automatically (i.e. you don't need to import your server functions and register them on the app instance). So it's worth looking into that. Ideally this would be done in such as way that:

• The inputs and outputs of procedures automatically get registered in the App
• Procedures can be grouped into services similar to the typescript implementation (ex: users.getUser)
• We can implement this in such a way that we don't force Rust users to use a specific HTTP framework. (I.e. users can use Axum or Actix or whatever)

Creating a Rust Server Plugin For Arri

That last thing would be to create an arri rust server plugin that knows how to run the rust server and knows where the rust app will output the App Definition file.

// The following is all psuedo-code

const rustServer = defineServerConfig({
  devFn(_, generators) {
    // run the server
    let rustProcess = spawn("cargo run", {
      stdio: "inherit",
    });

    // start watching wherever we expect the rust app to output the App Definition for changes
    const appDefWatcher = startFileWatcher(".arri/app_definition.json");
    appDefWatcher.on("change", async () => {
      // reread the app definition
      const appDef = JSON.parse(
        fs.readFileSync(".arri/app_definition.json", "utf8")
      ) as AppDefinition;
      // pass it to the generators
      await Promise.all(generators.map((item) => item.generator(appDef)));
    });

    // starting watching our rust source code
    const srcCodeWatcher = startFileWatcher("./src/**/*.rs");
    srcCodeWatcher.on("all", async () => {
      // if a change is made to the source code kill the rust process and restart it
      rustProcess.kill();
      rustProcess = spawn("cargo run", {
        stdio: "inherit",
      });
    });
  },
  buildFn(_, generators) {
    // build the server
    execSync("cargo build", {
      stdio: "inherit",
    });

    // run the app with some kind of flag indicating we don't want the server to start
    // but rather than we just want the app definition
    // this flag is something we can make every rust arri app check for maybe?
    execSync("./path/to/executable --codegen-only");

    // parse the app definition and pass it to the generators
    const appDef = JSON.parse(
      fs.readFileSync(".arri/app_definition.json", "utf8")
    ) as AppDefinition;
    await Promise.all(generators.map((item) => item.generator(appDef)));
  },
});

Anyway feedback welcome. Conceptually it's straightforward, realistically it's a lot to implement - particularly creating the proc macros for generating type definitions.

Additional Notes

We probably need to avoid using Serde because it doesn't support DateTime, but also we already need to create our own derive macros so we might as well handle parsing and serialization too. This will also give us more control over how things get parsed and serialized.

Some thought also needs to be given to how we will allow for passing of the request "context". Basically we should be able to access the user session, headers, and other stuff that might be relevant.