feat(ui): add support for custom field types

Node authors may now create their own arbitrary/custom field types. Any pydantic model is supported.

Two notes:
1. Your field type's class name must be unique.

Suggest prefixing fields with something related to the node pack as a kind of namespace.

2. Custom field types function as connection-only fields.

For example, if your custom field has string attributes, you will not get a text input for that attribute when you give a node a field with your custom type.

This is the same behaviour as other complex fields that don't have custom UIs in the workflow editor - like, say, a string collection.

feat(ui): fix tooltips for custom types

We need to hold onto the original type of the field so they don't all just show up as "Unknown".

fix(ui): fix ts error with custom fields

feat(ui): custom field types connection validation

In the initial commit, a custom field's original type was added to the *field templates* only as `originalType`. Custom fields' `type` property was `"Custom"`*. This allowed for type safety throughout the UI logic.

*Actually, it was `"Unknown"`, but I changed it to custom for clarity.

Connection validation logic, however, uses the *field instance* of the node/field. Like the templates, *field instances* with custom types have their `type` set to `"Custom"`, but they didn't have an `originalType` property. As a result, all custom fields could be connected to all other custom fields.

To resolve this, we need to add `originalType` to the *field instances*, then switch the validation logic to use this instead of `type`.

This ended up needing a bit of fanagling:

- If we make `originalType` a required property on field instances, existing workflows will break during connection validation, because they won't have this property. We'd need a new layer of logic to migrate the workflows, adding the new `originalType` property.

While this layer is probably needed anyways, typing `originalType` as optional is much simpler. Workflow migration logic can come layer.

(Technically, we could remove all references to field types from the workflow files, and let the templates hold all this information. This feels like a significant change and I'm reluctant to do it now.)

- Because `originalType` is optional, anywhere we care about the type of a field, we need to use it over `type`. So there are a number of `field.originalType ?? field.type` expressions. This is a bit of a gotcha, we'll need to remember this in the future.

- We use `Array.prototype.includes()` often in the workflow editor, e.g. `COLLECTION_TYPES.includes(type)`. In these cases, the const array is of type `FieldType[]`, and `type` is is `FieldType`.

Because we now support custom types, the arg `type` is now widened from `FieldType` to `string`.

This causes a TS error. This behaviour is somewhat controversial (see microsoft/TypeScript#14520). These expressions are now rewritten as `COLLECTION_TYPES.some((t) => t === type)` to satisfy TS. It's logically equivalent.

fix(ui): typo

feat(ui): add CustomCollection and CustomPolymorphic field types

feat(ui): add validation for CustomCollection & CustomPolymorphic types

- Update connection validation for custom types
- Use simple string parsing to determine if a field is a collection or polymorphic type.
- No longer need to keep a list of collection and polymorphic types.
- Added runtime checks in `baseinvocation.py` to ensure no fields are named in such a way that it could mess up the new parsing

chore(ui): remove errant console.log

fix(ui): rename 'nodes.currentConnectionFieldType' -> 'nodes.connectionStartFieldType'

This was confusingly named and kept tripping me up. Renamed to be consistent with the `reactflow` `ConnectionStartParams` type.

fix(ui): fix ts error

feat(nodes): add runtime check for custom field names

"Custom", "CustomCollection" and "CustomPolymorphic" are reserved field names.

chore(ui): add TODO for revising field type names

wip refactor fieldtype structured

wip refactor field types

wip refactor types

wip refactor types

fix node layout

refactor field types

chore: mypy

organisation

organisation

organisation

fix(nodes): fix field orig_required, field_kind and input statuses

feat(nodes): remove broken implementation of default_factory on InputField

Use of this could break connection validation due to the difference in node schemas required fields and invoke() required args.

Removed entirely for now. It wasn't ever actually used by the system, because all graphs always had values provided for fields where default_factory was used.

Also, pydantic is smart enough to not reuse the same object when specifying a default value - it clones the object first. So, the common pattern of `default_factory=list` is extraneous. It can just be `default=[]`.

fix(nodes): fix InputField name validation

workflow validation

validation

chore: ruff

feat(nodes): fix up baseinvocation comments

fix(ui): improve typing & logic of buildFieldInputTemplate

improved error handling in parseFieldType

fix: back compat for deprecated default_factory and UIType

feat(nodes): do not show node packs loaded log if none loaded

invokeai/app/api_app.py

@@ -1,7 +1,3 @@
-from typing import Any
-
-from fastapi.responses import HTMLResponse
-
 from .services.config import InvokeAIAppConfig
 
 # parse_args() must be called before any other imports. if it is not called first, consumers of the config
@@ -16,14 +12,15 @@
  import socket
  from inspect import signature
  from pathlib import Path
+ from typing import Any
 
  import uvicorn
  from fastapi import FastAPI
  from fastapi.middleware.cors import CORSMiddleware
  from fastapi.middleware.gzip import GZipMiddleware
  from fastapi.openapi.docs import get_redoc_html, get_swagger_ui_html
  from fastapi.openapi.utils import get_openapi
- from fastapi.responses import FileResponse
+ from fastapi.responses import FileResponse, HTMLResponse
  from fastapi.staticfiles import StaticFiles
  from fastapi_events.handlers.local import local_handler
  from fastapi_events.middleware import EventHandlerASGIMiddleware
@@ -51,7 +48,12 @@
  workflows,
  )
  from .api.sockets import SocketIO
- from .invocations.baseinvocation import BaseInvocation, UIConfigBase, _InputField, _OutputField
+ from .invocations.baseinvocation import (
+ BaseInvocation,
+ InputFieldJSONSchemaExtra,
+ OutputFieldJSONSchemaExtra,
+ UIConfigBase,
+ )
 
  if is_mps_available():
  import invokeai.backend.util.mps_fixes # noqa: F401 (monkeypatching on import)
@@ -147,15 +149,19 @@ def custom_openapi() -> dict[str, Any]:
 
  # Add Node Editor UI helper schemas
  ui_config_schemas = models_json_schema(
- [(UIConfigBase, "serialization"), (_InputField, "serialization"), (_OutputField, "serialization")],
+ [
+ (UIConfigBase, "serialization"),
+ (InputFieldJSONSchemaExtra, "serialization"),
+ (OutputFieldJSONSchemaExtra, "serialization"),
+ ],
  ref_template="#/components/schemas/{model}",
  )
  for schema_key, ui_config_schema in ui_config_schemas[1]["$defs"].items():
  openapi_schema["components"]["schemas"][schema_key] = ui_config_schema
 
  # Add a reference to the output type to additionalProperties of the invoker schema
  for invoker in all_invocations:
- invoker_name = invoker.__name__
+ invoker_name = invoker.__name__ # type: ignore [attr-defined] # this is a valid attribute
  output_type = signature(obj=invoker.invoke).return_annotation
  output_type_title = output_type_titles[output_type.__name__]
  invoker_schema = openapi_schema["components"]["schemas"][f"{invoker_name}"]