It's time for a paradigm shift! The future of software is in plain English, French or Latin.
During the computer revolution, we have seen multiple generations of computer languages, from the physical rewiring of the vacuum tubes through low-level machine code to the high-level languages like Python or JavaScript. And now, we're on the edge of the next revolution!
It's a revolution of writing software in plain human language that is understandable and executable by both humans and machines – and it's going to change everything!
The incredible growth in power of microprocessors and the Moore's Law have been the driving force behind the ever-more powerful languages, and it's been an amazing journey! Similarly, the large language models (like GPT or Claude) are the next big thing in language technology, and they're set to transform the way we interact with computers.
This shift is going to happen, whether we are ready for it or not. Our mission is to make it excellently, not just good.
Join us in this journey!
Promptbook project is ecosystem of multiple projects and tools, following is a list of most important pieces of the project:
| Project | Description | Link |
|---|---|---|
| Core | Promptbook Core is a description and documentation of the basic concepts, ideas and inner workings of how Promptbook should be implemented, and defines what features must be describable by book language. | https://ptbk.io https://github.com/webgptorg/book |
| Book language | Book is a markdown-like language to define core entities like projects, pipelines, knowledge,.... It is designed to be understandable by non-programmers and non-technical people | |
| Promptbook typescript project | Promptbook implementation in TypeScript released as multiple NPM packages | https://github.com/webgptorg/promptbook + Multiple packages on NPM |
| Promptbook studio | Studio to write Books and instantly publish them as miniapps |
https://promptbook.studio https://github.com/hejny/promptbook-studio |
| Hello World | Simple starter kit with Books integrated into the sample applications |
https://github.com/webgptorg/hello-world https://github.com/webgptorg/hello-world-node-js https://github.com/webgptorg/hello-world-next-js |
Organization groups together workflows, workforce, knowledge, instruments, and actions into one package. Entities in one organization can share resources (i.e. import workflows, teams, personas, knowledge, instruments and actions from each other).
Each organization has a unique URL; for example, https://promptbook.studio/my-cool-project/.
A workflow represents a piece of work that has specific input and output.
Private workflows can be imported within the organization, while public workflows can be imported inside the organization or used everywhere through their unique URLs. Each workflow has a unique URL; for example, https://promptbook.studio/my-cool-project/workflows/generate-website.
As a programmer, you can imagine a workflow as an async function that can be used inside your code. It exposes an interface with a record of input and output parameters but hides the internal implementation.
You can use workflows in other workflows, use it in classic programming languages as async functions that can be called, run workflow in CLI, or use Promptbook Studio to create instant miniapps.
A task is one step in a workflow. Each task is divided into two parts: the actual task job and a check that the result of the job is correct.
Each task can use results from previous tasks. Tasks in a workflow form a directed acyclic graph.
A task job is the actual work that needs to be done. It can be:
- Asking a persona to do a job; this is a higher abstraction of the calling model
- Asking the user to do a job
- Searching the knowledge base for information
- Using an action to perform external work and get the result
- Using an instrument, for example, a calculator
- A simple template to concatenate parameters to a hardcoded template
- Script execution to run custom code (Python/JavaScript/TypeScript/...)
- Calling a model directly to perform a job, bypassing the organization's workforce—for example, calling GPT's Assistant
After the task job is done, the result can be checked to see if it is correct. If it is not correct, the task is repeated a certain number of times.
You can expect:
- Result is in the expected format; for example, an available domain name
- Result is in the expected range; for example, between 1 sentence and 2 paragraphs
- Adversarial check by another persona to approve the result
The workforce is an abstraction above LLM models, tokens, temperature, top-k, top-p, and other model parameters. You can describe what you desire in human language like Jane, creative writer with a sense of sharp humor instead of gpt-4-2024-13-31, temperature 1.2, top-k 40, STOP token ".\n",....
A persona is the basic unit of the workforce. It is defined by its description; for example, Jane, creative writer with a sense of sharp humor.
This persona description is used to select the best model and parameters for the job. If the persona has conflicting requirements, Promptbook will try to find the best compromise or even combine multiple models to achieve the best result.
For example,
Josh, lawyer with perfect language and logic capabilities and a strong sense of privacyis not possible to achieve with one model. Big models likeGPT-4orClaude-3.5are great for language and logic, but they send data to the cloud. On the other hand,LLAMA-3is great for privacy but not as strong in language and logic. Therefore, Promptbook will create an ad-hoc meta-model usingLLAMA-3to strip all data,GPT-4for language and logic, and then back toLLAMA-3to reintroduce sensitive data.
Each persona can have access to different knowledge, instruments, and actions.
A team groups personas together. A team can also group other teams to form a complex responsibility hierarchy. Each team can have access to different knowledge, instruments, and actions.
A role is an ad-hoc modification of a persona. A role can be defined for a specific task; for example, Jane (email writer). Roles have access to the same knowledge, instruments, and actions as their parent persona.
Knowledge is external information that is used in task jobs. Knowledge can be:
Explicitas a text directly in the workflow, team, or personaFilewhich is referenced from the workflow, team, or persona We support various file types likepdf,docx,txt,md,odt,doc,rtf, and it's possible to easily add support for other file types. The file is parsed and stored in the knowledge baseWebsitewhich is referenced from the workflow, team, or persona The website is scraped and stored in the knowledge base
A knowledge piece is the smallest unit of knowledge that makes sense on its own. Every type of knowledge—explicit text, file, or website—is parsed/scraped and divided into knowledge pieces. These pieces are indexed, put into the knowledge base, and can be used in task jobs via techniques like retrieval-augmented generation.
Instruments are external information that cannot be pre-scraped and need to be fetched at the moment of the task job. For example:
- Current Time and date
- User's Location
- Searching the internet
- Computing some mathematical expression
- Weather in some location
- Stock price of some company
- Availability of some Domain name
- Calling a GET endpoint of an API
This is an abstraction above function calling and API calling in models.
Actions are similar to instruments but can change the state of the world. For example:
- Sending email
- Creating a file
- Ending a workflow
- Calling a POST endpoint of an API
Following is the documentation and blueprint of the Book language.
# 🌟 My first Book
- PERSONA Jane, marketing specialist with prior experience in writing articles about technology and artificial intelligence
- KNOWLEDGE https://ptbk.io
- KNOWLEDGE ./promptbook.pdf
- EXPECT MIN 1 Sentence
- EXPECT MAX 1 Paragraph
> Write an article about the future of artificial intelligence in the next 10 years and how metalanguages will change the way AI is used in the world.
> Look specifically at the impact of Promptbook on the AI industry.
-> {article}File is designed to be easy to read and write. It is strict subset of markdown. It is designed to be understandable by both humans and machines and without specific knowledge of the language.
It has file with .book.md or .book extension with UTF-8 non BOM encoding.
As it is source code, it can leverage all the features of version control systems like git and does not suffer from the problems of binary formats, proprietary formats, or no-code solutions.
But unlike programming languages, it is designed to be understandable by non-programmers and non-technical people.
Book is divided into sections. Each section starts with heading. The language itself is not sensitive to the type of heading (h1, h2, h3, ...) but it is recommended to use h1 for header section and h2 for other sections.
Header is the first section of the book. It contains metadata about the pipeline. It is recommended to use h1 heading for header section but it is not required.
Foo bar
Reserved words:
- each command like
PERSONA,EXPECT,KNOWLEDGE, etc. contentcontextknowledgeexamplesmodelNamecurrentDate
Todo todo
Todo todo
Todo todo
The following glossary is used to clarify certain concepts:
- Prompt drift is a phenomenon where the AI model starts to generate outputs that are not aligned with the original prompt. This can happen due to the model's training data, the prompt's wording, or the model's architecture.
- Pipeline, workflow or chain is a sequence of tasks that are executed in a specific order. In the context of AI, a pipeline can refer to a sequence of AI models that are used to process data.
- Fine-tuning is a process where a pre-trained AI model is further trained on a specific dataset to improve its performance on a specific task.
- Zero-shot learning is a machine learning paradigm where a model is trained to perform a task without any labeled examples. Instead, the model is provided with a description of the task and is expected to generate the correct output.
- Few-shot learning is a machine learning paradigm where a model is trained to perform a task with only a few labeled examples. This is in contrast to traditional machine learning, where models are trained on large datasets.
- Meta-learning is a machine learning paradigm where a model is trained on a variety of tasks and is able to learn new tasks with minimal additional training. This is achieved by learning a set of meta-parameters that can be quickly adapted to new tasks.
- Retrieval-augmented generation is a machine learning paradigm where a model generates text by retrieving relevant information from a large database of text. This approach combines the benefits of generative models and retrieval models.
- Longtail refers to non-common or rare events, items, or entities that are not well-represented in the training data of machine learning models. Longtail items are often challenging for models to predict accurately.
Note: Thos section is not complete dictionary, more list of general AI / LLM terms that has connection with Promptbook
- Organization (legacy name collection) group jobs, workforce, knowledge, instruments, and actions into one package. Entities in one organization can share resources (= import resources from each other).
- Jobs
- Task
- Subtask
- Workforce
- Persona
- Team
- Role
- Knowledge
- Public
- Private
- Protected
- Instruments
- Actions
- Jobs
- Book file
- Section
- Heading
- Description
- Command
- Block
- Return statement
- Comment
- Import
- Scope
- Section
- 📚 Collection of pipelines
- 📯 Pipeline
- 🙇♂️ Tasks and pipeline sections
- 🤼 Personas
- ⭕ Parameters
- 🚀 Pipeline execution
- 🧪 Expectations
- ✂️ Postprocessing
- 🔣 Words not tokens
- ☯ Separation of concerns
- 📚 Knowledge (Retrieval-augmented generation)
- 🌏 Remote server
- 🃏 Jokers (conditions)
- 🔳 Metaprompting
- 🌏 Linguistically typed languages
- 🌍 Auto-Translations
- 📽 Images, audio, video, spreadsheets
- 🔙 Expectation-aware generation
- ⏳ Just-in-time fine-tuning
- 🔴 Anomaly detection
- 👮 Agent adversary expectations
- view more
