Support base64 image upload via page POST/PUT markdown content

[horshack]

Describe the feature you'd like

I would like to have an example in the API-documentation how to create a new page with a picture included in the middle of the text.

In

https://bookstack.mydomain.local/api/docs#pages-create

I find:

Any images included via base64 data URIs will be extracted and saved as gallery images against the page during upload.

While reading issues in github Dan Brown told that it's not a good idea to include pictures with base64 in the markdown-files because it is not good style to include blobs in the database-tables.

Describe the benefits this feature would bring to BookStack users

Transfering webpages with Bookstack-API to transfer many markdown-files would be possible.

[ssddanbrown]

Hi @horshack,
Base64 within content is the correct way to go when uploading HTML page content. These images are extracted into file-based references upon upload negating any of my old advisories not to upload base64 content.

Updating page content via markdown does not currently have the same ability. If desired I could update the title of this issue to re-focus it on adding support for base64 images to be uploaded within Markdown content.

[horshack]

Could you please show us an example how to include an image with the exact syntax to use in markdown please? I my experiments I was able to do this in the editor with copy and paste, but it was not shown in the resulting page. Thanks for your precious work. Richard

…

Am 28.08.2021 um 16:24 schrieb Dan Brown ***@***.***>:  Hi @horshack, Base64 within content is the correct way to go when uploading HTML page content. These images are extracted into file-based references upon upload negating any of my old advisories not to upload base64 content. Updating page content via markdown does not currently have the same ability. If desired I could update the title of this issue to re-focus it on adding support for base64 images to be uploaded within Markdown content. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[horshack]

Sorry, I did not read your answer careful enough:

• Uploading HTML with POST api/pages extracts included base64-images into separate files and shows the image in the end for the user
• Uploading markdown with POST api/pages does NOT extract included base64-images. It simply ignores them in the end. So the user later sees nothing when viewing the page

Is there a reason for this split of behaviour?

I think this behaviour should be documented at least with one sentence in

https://bookstack.mydomain.local/api/docs#pages-create

Thanks,
Richard

[horshack]

In HTML the part with the image has to look like this:

<img src="data:image/jpeg;base64,/9j/4AAQSkZJRgAB....H//Z">

I took the approach like this in my perl-programm to upload the markdown via API:

• markdowntext
• convert2html
• img src="file.jpg"
• convert realtive filename to full filename: img src="/complete/path/file.jpg"
• convert full filename to base64: img src="data:image/jpeg;base64,/9j/4AA...."

[horshack]

After experimenting today with this solution I found out that converting the document from Markdown to HTML before uploading was not a good idea. It is hard to change texts in HTML by the user later, simply because html is much harder to edit than markdown.

The images are now in Bookstack but for the price of having documents which are very hard to edit. I am not able to go this way. I wish there would be a way to upload the markdown-documents and images somehow directly via the API.

[ssddanbrown]

Hi @horshack,
The base64 image upload within page content is a fairly new feature, and it has not yet been rolled out nor requested for markdown content. As per my message above:

If desired I could update the title of this issue to re-focus it on adding support for base64 images to be uploaded within Markdown content.

Since it sounds like you do want that feature I'll update this issue accordingly.

[horshack]

Yes, please update the subject to make it a request for a new feature in the bookstack software. Thanks for your help, Richard

…

Am 28.08.2021 um 22:43 schrieb Dan Brown ***@***.***>:  Hi @horshack, The base64 image upload within page content is a fairly new feature, and it has not yet been rolled out nor requested for markdown content. As per my message above: If desired I could update the title of this issue to re-focus it on adding support for base64 images to be uploaded within Markdown content. Since it sounds like you do want that feature I'll update this issue accordingly. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[Kaligule]

Disclaimer: I have not worked with this API, nor do I know anything about API design.

Is this really the easiest way this API could work with images?

It looks unnatural to me:

• have you ever seen a markdown file with an base64 encoded image in the wild? Markdown is all about readability, embedded images are not.
• It is not very accessible for people who want their images in the site. They would have to do the encoding (perhaps in an environment where this is not easy), just for Bookstack to de the decoding right away.
• this doesn't solve the problem of other attachments that could be linked within the markdownfile (perhaps a csv, a movie, a pdf).
• is it clear how you would handle metadata?

So I would like to suggest 2 alternative ways on how images in pages could be handled by the api.

alternatives

I assume that people who want to create a page have a content as a markdown/html file which references image files with the typical markdown/html syntax (![alt text](path/to/imagefile.png) / <img src="path/to/imagefile.png" alt="alt text"> ) and the images for that. Both the content and the images would be local files:

• page_content.md
• path/to/imagefile.png

Idea: pull out the encoded file into it's own parameter

The api parameters of a post/put request could look like is:

{
	"book_id": 1,
	"name": "My API Page",
	"markdown": "# title\nHere is an image:\n![alt text](path/to/imagefile.png)\nThat was the image",
	"tags": [],
        "assets": {
                "path/to/imagefile.png" : {
                        "mime": "image/png",
                        "data": "R0lGODdhAQABAPAAAP8AAAAAACwAAAAAAQABAAACAkQBADs="
                 }
        }
}

BockStack could then save the assets and change the link to the image in the markdown/html to whatever bookstack uses internally.
If the images referenced in the content and the assets provided don't match Bookstack could return a status code that tells the user about that.

This solution is cool because it doesn't require the user to mesh together assets and content himself.

Idea: Let attachments be their own api object

Just like shelves, books, chapters, pages, you could have an /api/assets endpoint that deals specifically with images and the like. You post the assets and get back a link to the asset. The pages would then just reference the assets as urls.

This would allow great flexibility and extensibility in the work with assets. It might be a bit overengineered though. Perhaps Bookstack doesn't want to be that kind of cms? Or it does?

[horshack]

In fact, the API is not yet ready. The extension of the API about "attachment" are good. Attachments like pictures (directly shown in the page) or attachments linked to in pages.

I would like to create pages from directories with Markdown-files I own. And I have to program (in perl, my language) to manipulate my markdown-files to include base64-contents. This would be ok with me.

In the end the base64-content is getting separated from bookstack and changed to links to external file/image-objects. So when getting the article back from the frontend there will be a link to a separate object in bookstack.

[ssddanbrown]

@Kaligule

As @horshack has mentioned above, base64 would not remain in pushed markdown, It'll get extracted out to uploaded files by BookStack. This is more of a convenience feature and adding markdown support just aligns it with the HTML option.

I'd expect we'd still add a set of focused, specific, image API endpoints at some point in the future.

[ssddanbrown]

Thanks again for raising @horshack,
This has now been implemented in cb45c53 and will be part of the next feature release.

Just a note on parsing:

The implementation is quite simplistic and will generally look for the pattern:

![*](data:image/*)

Where the base64 image data parsing is terminated by a closing round bracket, quote mark or whitespace. Therefore the base64 content is expected to all be on a single line without any whitespace. This is to keep the implementation simple & performant while respecting certain markdown image syntax options (Such as including title text). This does not support markdown reference links.

[horshack]

Dan, thank you very much for your great work. It’s a pleasure for me to support you. Sincerly from Bavaria Richard Von meinen Handy gesendet. Rechtschreibfehler und die Kürze der Nachricht sind der Situation geschuldet.

…

Am 18.10.2021 um 12:51 schrieb Dan Brown ***@***.***>:  Thanks again for raising @horshack, This has now been implemented in cb45c53 and will be part of the next feature release. Just a note on parsing: The implementation is quite simplistic and will generally look for the pattern:  Where the base64 image data parsing is terminated by a closing round bracket, quote mark or whitespace. Therefore the base64 content is expected to all be on a single line without any whitespace. This is to keep the implementation simple & performant while respecting certain markdown image syntax options (Such as including title text). This does not support markdown reference links. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[SteveDinn]

I'm trying this today in v21.12.4 and it's not working for me. My markdown containing the base64 text is remaining in the body of the page and is NOT being converted to a normal image link.

My markdown consists of something similar to this:

![b1fb7471dbff12b2.jpg](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQE...[a few megabytes later]...Y+YAk7R7e9RJpIcVqf//Z)

Other page text here.

The page is created, but when I view it, it shows no image, and when I edit it, it shows the full multi-megabyte base64 string. It actually renders the image after nearly a minute of sitting there, but I was under the impression that it would automatically parse the base64 data out and store it as an image in the gallery.

Should I open a new issue for this?

[ssddanbrown]

@SteveDinn Yeah, done some testing based upon your comment. It would indeed fail with larger images due to hitting limits. Have opened #3249 with my findings to address. Apologies for this, My implementation for markdown was a little too simplistic.