Why a domain specific language?

[keawade]

Hey, first off, I love the vision of Bruno as a local-first HTTP client to replace Postman, Insomnia, and co. Thanks so much for building this fantastic tool!

I'm trying to understand what the motivation is for the Bru Lang DSL. I've browsed the Bru Lang sections of the documentation and I haven't found a clear statement for the goals of the language and, particularly, how those goals necessitate building a new language / markup format.

With me not currently seeing the point of Bru Lang, I wonder if it might be better to consider using an existing data format like JSON or YAML with JSON Schema instead of a custom language. In my mind, at least, that approach might be significantly less complicated as it means you wouldn't need to take on the overhead of maintaining a DSL and the associated tooling like parsers, formatters, linters, language servers, etc.

Edit: To add more context, I started thinking about this while trying to apply a bunch of automated modifications to a large collection of request files. I found myself wishing I could work with the data as JSON in JS or even YAML with an AST library instead of needing to do direct text manipulation in my script.

[helloanoop]

Update (7 Oct 2023): Keep a watchout for this experiment for supporting multiple languages on how the request gets stored. If it succeeds, then Bruno can offer flexibility and freedom of choice. Users should never feel constrained and compelled to use only the bru lang

I'll perhaps write a more detailed blog about this decision in a lot more detail later. But here is a decent summary.

I can assure you that I thought through this for months, because I knew that once you design a DSL, its pretty to hard to migrate stuff since we directly write to filesystem. Data format migrations are easier if its a hosted solution where your data is sitting in a database

How complex can it get?

At first glance, when you just think about url, http method, headers, request body, response body, it seems like a simpler problem. But as you expand your needs to store tests, assertions, request variables, response variables, docs it starts getting complicated. See this full blown request example where an api can accept data in multiple formats. Now imagine to represent this in JSON/YAML and keep it readable, I hope you are getting my viewpoint here.

Easy pr reviews via git for collaboration

The goal was not just to write to a file, but to be able to do a PR review flow in git. This meant that the plaint-text representation of the request had to be simple and easily readable enough to be able to do that.Also, simple enough so that you can open the collection folder in a editor (like vscode) and still be able to make changes.

I look forward to the day where it becomes a norm for developers start storing api collection in their source code itself. So that anyone who clones the repo can bring the service up and start playing exploring the api. And athen continue to maintain the api collections just like code.

Lang iterations

The language itself went through many iterations.

• First was break a giant collection json and save it as individual files
• Second attempt was to convert into a dsl (was a bit xmlishy)
• The third attempt that is used today to made it even simpler (jsonishy/groovishy)

@ajaishankar played a crucial role in helping me design the lang.

Why not JSON/YAML ?

Json didn't support multiline strings. Yaml was indentation hell. I wanted something neat, no need for quotes around strings. And lot of nuances, like "how do you represent a key-val paid as disabled in a single line".

Like it was quite nice to go from

headers {
  content-type: application/json
  accept-language: en-US,en;q=0.9,hi;q=0.8
}

headers:disabled {
  transaction-id: {{transactionId}}
}

To

headers {
  content-type: application/json
  accept-language: en-US,en;q=0.9,hi;q=0.8
  ~transaction-id: {{transactionId}}
}

Parser

Bruno DSL is built on top of Ohm
For v1 I used arcsecond parser combinator library by @francisrstokes
For V2, I used ohmjs library by @pdubroy The entire parser that translates .bru files to its json representation is just one single file

Once you wrap you head around it, Ohm is really powerful.

Github issue for reference: #82

[helloanoop]

@fractalf Sure.

I think we should be able allow to specify the DSL of choice (bru/json/yaml) as I mentioned here

If anyone in the community in the bandwidth has some time to take this up and experiment with this, we would like to support this option as a part of v1 itself.

[keawade]

I've created #407 to track a version of this experiment. I'll see what I can put together over the weekend.

[fractalf]

@helloanoop Nice! I like that idea

[Jiehong]

The format is not supposed to be read by humans so I don't agree with the "it was nice to" etc.

For me, it's the opposite: when using the bru cli + the text files, human-readable is extremely important.

Looks close to HCL to me, so not that weird or hard to learn.

[mrleblanc101]

Doesn't JSON5 support multiline string ?

[helloanoop]

Another cool thing Bruno's architecture is that the entire frontend app does not know anything about Bru Lang. There is beautiful abstraction built where the frontend app just knows about the JSON representation of the collection.

This tiny file is responsible for the bru<->json conversion. Meaning if we decide to support an entire new format (to represent a request), we just need to write a new parser and a new serializer. That would be it.

And then have a field in bruno.json which could be the parser to use for serialization/deserialization.

{
  "version": "1",
  "type": "collection",
  "lang-parser": "della"
}

We likely won't have to, but its cool to know that we can.
This architecture was not planned, but just happened to be that way.

I realised it when I was able to quickly move from brulang v1 to v2 to v3 without touching the frontend app. I just had to write a new parser and a new serializer.

Wait Wait Wait

As I think now, we could support option for people to choose either json / yaml / bru. It should be relatively easier to achieve this. We can think about it post v1 launch when the app is mostly stable with all necessary features.

PS: Della is Bruno's girlfriend :P.

[ajaishankar]

Other than a tilde (a UI concern) that has crept in, the DSL is nothing but some named sections which are either

1. A collection of key value pairs (hash)
2. Content blobs (json/text)

One of the goals was to avoid quoted strings and make it easy on the eyes.

I appreciate the fact that @helloanoop created a parser for this and it would help in a syntax highlighted editing experience.

But a hand written one for this format is also really tiny and simple.

For an alternate format, would love to see how complex requests look in TOML

[ajaishankar]

I wish 😄 Was trying out Rust, and cargo packages are specified in toml

[lcptrs]

I'd love to see TOML support! I've tried my hands on your example you've provided in the screenshot so we have something to discuss on:

[meta]
name = "Send Bulk SMS"
type = "http"
seq = 1

[get]
url = "http://api..."

[params]
apiKey = "secret"
numbers = 91234
message = "hello"

[headers]
content-type = "application/json"
accept-language = "en-US,en:"

[headers.disabled]
transaction-id = "{{transactionId}}"

[body]
apiKey = "secret"
numbers = "+013123123"
data = { sender = "TXLJL" }
# inline-table have currently a limitation on multi-line. Will be fixed with TOML 1.1 
# https://github.com/toml-lang/toml/pull/904
messages = [{ numbers = "+91082731", message = "asdfas" }]

# alternatively the body can be passed as a raw string
body = '''
{
  "apiKey": "secret",
  "numbers": "+013123123",
  "data": { 
    "sender": "TXLJL"
   },
  "messages": [{
    "numbers": "+91082731",
    "message": "asdfas" 
  }]
}
'''

[lizyChy0329]

I love TOML and I am trying to replace the JSON editor with TOML in my low-code platform

[Napam]

I really like the suggestion of using TOML!

I personally believe going with TOML would be better overall as most editors have tooling to deal with TOML files. Personally I imagined myself just using my text editor to manage Bruno through the bru files, but it was tiring to have no editor support such as

1. syntax highlighting
2. formatting
3. LSP

As of now there is no tooling to deal with the bru files that I know of, and switching to TOML would just give us all of that "for free".

Don't get me wrong, the bru syntax is nice! It just doesn't have all the tooling that comes with a well established format such as TOML.

EDIT: It doesn't necessarily have to be TOML, but the idea of leveraging something existing give you editor support for free, which is something I care more about the config being "as minimal as possible"

[helloanoop]

Seriously exploring TOML support #1303

I should be able to have a experimental build ready in another 3-4 days and look forward to gather community feedback in our monthly community call at 5pm GMT on Wed Jan 3rd 2024

[4lejandrito]

I really like Bruno, but when I was evaluating it I also felt the DSL too limiting for more complex use cases.

For this very reason I am building my own tool: https://github.com/4lejandrito/fetchbook which uses TypeScript as the DSL (see this). Bruno did inspire me to build it.

Maybe Bruno can take some inspiration from my little project too :)

[blaggacao]

To provide another data point, I have a feeling that the ideas are similar to those behind https://kdl.dev/ ( but no ~ there :-( ).

[helloanoop]

@keawade @Jiehong @4lejandrito @dustin-engstrom @lcptrs @blaggacao @lizy-Leeeeee @Napam @ajaishankar @martinsefcik @Its-treason

looking for feedback for some updates in .bru syntax. Please see #1290

PS: Currently improving .bru syntax. TOML support is very much in the pipeline, Will be focusing on TOML in Jan-Feb 2024

[Napam]

Thanks for considering our feedback @helloanoop! Here is my elaboration on my choice:

Spoiler warning - Elaborating my opinion

• I liked the proposed syntax more.
• The left ("current syntax") looks very neat and clean, but in my head it made less semantic sense. More specifically, the "get/post/put/patch... etc" block kinda throws me off a little. It took me a bit longer to grasp the concept (not that it was hard, but I grasped the proposed syntax faster)
• The proposed syntax is more "standard" and closer to what I would I expect in general, and I could understand it faster.

[Its-treason]

I like the new syntax, I feel like it makes everything a bit more structured.

I'm not sure if this will be Bru-Lang version 3, but if so, I think we should look into making better serializable/deserializable. Meaning that a JavaScript object can be converted into Bru-Lang V3 and back, without any extra helpers, and for every use case (No more envToJson, bruToJson etc. just one jsonToBru and bruToJson) like JSON or YAML.

[tringenbach]

Have you thought about supporting the Jetbrains IntelliJ's Http Client format?

Some links if you're not familiar with it:

• https://www.jetbrains.com/help/idea/http-client-in-product-code-editor.html
• https://www.jetbrains.com/help/idea/exploring-http-syntax.html
• The spec: https://github.com/JetBrains/http-request-in-editor-spec/
• The CLI client https://www.jetbrains.com/help/idea/http-client-cli.html

The spec is licensed as CC-BY-4.0 license. The rest of it is proprietary and closed sourced. I believe the IDE Plugin requires purchasing a license, but the CLI client, while still under a proprietary license, seems to be free as in beer.

Since they're also using a plain text file, for a lot of the same reasons that Bruno is, I think that's a hint that you're on the right track.

I'm not saying you should just use the IntelliJ format as your format. But it would be cool if it was an option.

The use case I have in mind, is let's say I'm on a team that uses IntelliJ and that plugin, and has the files committed to the repo. I'd want to be able to open it in Bruno and just use Bruno. But without "importing" and saving the collection in .bru. I'd rather that it "just work".

[blackthornedk]

The format is more widely supported, than just JetBrains. It's also supported in VS Code, and from a set of pipeline tools.
https://devblogs.microsoft.com/ise/api-testing-using-http-files/

Adding it as an storage option for selected collections, would make it possible, to collaborate with the same files, as other developers use in VS Code and JetBrains. It would also enable the pipeline tools to run the same request files. I would very much like this as a feature, where the storage backend can be selected on collection level.

[kinland]

Why not just use javascript files instead of reinventing the wheel? I agree JSON/Yaml have limitations that would be painful in this context, but JS itself doesn't have those problems. Seems like the Bru Markup language could have been avoided just by having exports with expected names (i.e.: insert export {meta, get, tests} if not present), or functions.

For example, the sample repo has:

meta {
  name: User Info
  type: http
  seq: 1
}

get {
  url: {{baseUrl}}/users/usebruno
  body: none
}

tests {
  test("status must be 200", function() {
    expect(res.status).to.eql(201);
  });
}

This could have been

const meta = {
  name: 'User Info'
  type: 'http'
  seq: 1
}

const get = {
  url: `${baseUrl}/users/usebruno`
  body: null
}

const tests = [
  test("status must be 200", function() {
    expect(res.status).to.eql(201);
  });
]

Or as functions:

meta({
  name: 'User Info'
  type: 'http'
  seq: 1
})

get({
  url: `${baseUrl}/users/usebruno`
  body: null
})

// honestly, why not just top-level 'test' without the collection?
tests([
  test("status must be 200", function() {
    expect(res.status).to.eql(201);
  });
})

That js function format is nearly identical to the DSL without reinventing the wheel or introducing limitations. A whole DSL to avoid quotes just seems... unnecessary.

[huongg9x]

My company are using Thor Client at work, but I would love to use Bruno at home.
Is it possible to support JSON Postman format like this example:
https://github.com/kapa24dev/OptimizelyAPICollections/blob/main/ServiceAPI/Entry/Get%20Entry.thorclient

Then I could change the extension file to .bru and use it directly in Bruno app