Auto-Chart: Data display / Intelligent graphs / Plotting

[saurabhgaur]

It'd be great if big-agi can (optionally) include a chart using common charting libraries like d3js, plotly etc. incase the response returned from the bot includes data.

[enricoros]

Great request. In some sense, one would detect that there's an underline grid data (markdown grid, csv data, etc), and activate either a Table view (already provided in Markdown mode, in Settings), or Charts.

What would be the top-priority charts for your use case? Would be great to find a simple and easy and lightweight library for charts, and wire it in a way that as much data as possible is plotted.

Ideas for Libraries, Data output parsing, Top priority charts to do, and how do handle multi-columns data are very welcome.

[saurabhgaur]

Thanks @enricoros, I did a very hacky implementation to support Bar Charts and Line Charts for a POC

I used ChartJS (react-chartjs-2) and that seemed to work well for these chart types, with slight animation, and ability to turn on and off chart series etc.
I think something like ChartJS or the new https://observablehq.com/plot/marks/auto "The magic ✨ auto mark automatically selects a mark type that best represents the given dimensions of the data according to some simple heuristics. " might be a good addition.
We can also let the LLM suggest the chart type and use that to instantiate the suggested chart type

[enricoros]

This looks incredibly good!!

Would you be open to sending a PR? If you do, please develop on top of the next branch which has a better architecture and will replace the mainline in a few weeks.

Great project, that charting seems like magic.

I can see a vast utility for charting, especially if the model is injected with a tools pattern which pushes it to create more models, aka: a statistician or analyst persona.

I'm very open to picking up a PR, and if you so choose, make sure the charting library is not so heavy (for people that don't use charts).

[enricoros]

P.S.: formerly I used:

• Victory: great lib for professional looking results, but requires lots of code. Good to create well customized visual experiences (e.g. newspapers) but not flexible
• ApexCharts: quick and easy, less controllable and configurable, may miss features or chart types
• D3: OG, but requires quite some boilerplate and deep react knowledge

I like that Magic lib you pointed out above.

[saurabhgaur]

I can give it a try, any documentation to help get started would be super helpful, or else I'll try to look at the "next" branch and give it a go. - Thanks!

…

On Wed, May 24, 2023 at 4:08 PM Enrico Ros ***@***.***> wrote: P.S.: formerly I used: - Victory: great lib for professional looking results, but requires lots of code. Good to create well customized visual experiences (e.g. newspapers) but not flexible - ApexCharts: quick and easy, less controllable and configurable, may miss features or chart types - D3: OG, but requires quite some boilerplate and deep react knowledge I like that Magic lib you pointed out above. — Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABST6RO6BX5FRJXWBSTCVTXHXQKZANCNFSM6AAAAAAYLR23HE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[enricoros]

@saurabhgaur next branch is merged, and supports the latest models from Today's OpenAI announcement.
Did you experiment on this?

[enricoros]

Thanks @enricoros, I did a very hacky implementation to support Bar Charts and Line Charts for a POC I used ChartJS (react-chartjs-2) and that seemed to work well for these chart types, with slight animation, and ability to turn on and off chart series etc. I think something like ChartJS or the new https://observablehq.com/plot/marks/auto "The magic ✨ auto mark automatically selects a mark type that best represents the given dimensions of the data according to some simple heuristics. " might be a good addition. We can also let the LLM suggest the chart type and use that to instantiate the suggested chart type

Back to this - could you share the code for it? Looks pretty sweet and I'd love to productize it.

[vaibhavard]

Hello All!
I Tried mermaid for this , and it works as expected(though not able to create bargraphs)(Had to enable mermaid rendering )

prompt-You are a mermaid diagram creator.Write code for mermaid diagram as per the users request and always output the code in a codeblock.

This can also create innovative mindmaps:

Systemprompt:
Create a mermaid mindmap based on user input like these examples:
(Output code in code block like below)

mindmap
\t\troot(("leisure activities weekend"))
\t\t\t\t["spend time with friends"]
\t\t\t\t::icon(fafa fa-users)
\t\t\t\t\t\t("action activities")
\t\t\t\t\t\t::icon(fafa fa-play)
\t\t\t\t\t\t\t\t("dancing at night club")
\t\t\t\t\t\t\t\t("going to a restaurant")
\t\t\t\t\t\t\t\t("go to the theater")
\t\t\t\t["spend time your self"]
\t\t\t\t::icon(fa fa-fa-user)
\t\t\t\t\t\t("meditation")
\t\t\t\t\t\t::icon(fa fa-om)
\t\t\t\t\t\t("\`take a sunbath ☀️\`")
\t\t\t\t\t\t("reading a book")
\t\t\t\t\t\t::icon(fa fa-book)
text summary mindmap:
Barack Obama (born August 4, 1961) is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, he was the first African-American president of the United States.
mindmap
\troot("Barack Obama")
\t\t("Born August 4, 1961")
\t\t::icon(fa fa-baby-carriage)
\t\t("American Politician")
\t\t\t::icon(fa fa-flag)
\t\t\t\t("44th President of the United States")
\t\t\t\t\t("2009 - 2017")
\t\t("Democratic Party")
\t\t\t::icon(fa fa-democrat)
\t\t("First African-American President")
cause and effects mindmap:
mindmap
\troot("Landlord sells apartment")
\t\t::icon(fa fa-sell)
\t\t("Renter must be notified of sale")
\t\t::icon(fa fa-envelope)
\t\t\t("Tenants may feel some uncertainty")
\t\t\t::icon(fa fa-question-circle)
\t\t("Notice periods must be observed")
\t\t::icon(fa fa-calendar)
\t\t\t("Landlord can submit notice of termination for personal use")
\t\t\t::icon(fa fa-home)
\t\t\t\t("Tenant has to look for a new apartment")
\t\t\t\t::icon(fa fa-search)
\t\t("New owner")
\t\t::icon(fa fa-user)
\t\t\t\t("New owner takes over existing rental agreement")
\t\t\t\t::icon(fa fa-file-contract)
\t\t\t\t\t\t("Tenant keeps previous apartment")
\t\t\t\t\t\t::icon(fa fa-handshake)
\t\t\t\t("New owner terminates newly concluded lease")
\t\t\t\t::icon(fa fa-ban)
\t\t\t\t\t\t("Tenant has to look for a new apartment")
\t\t\t\t\t\t::icon(fa fa-search)

[enricoros]

I Tried mermaid for this , and it works as expected(though not able to create bargraphs)(Had to enable mermaid rendering )

This is really cool - do you have any code that we should productize?

[edmondop]

Maybe I can pick that up myself if no-one is working on this

[enricoros]

Update on this issue.

What we have:

• Render of Mermaid code (```mermaid ... ```)
• Rendering of PlantUML code (@startuml ... @enduml, @startmindmap ... @endmindmap, etc.)
• Render of Markdown tables
• A Visualize ... menu to turn Text into a chart (by using prompts to write code, which is then rendered)

For both of them, the code must be within the body of a message, or the message itself.

What we don't have

• Table extraction: identification of markdown tables, or CSV fragments and treating them like table blocks
• Auto-plotting of data: whenever a table is encountered - this ticket Auto-Chart: Data display / Intelligent graphs / Plotting #118
• Custom render of Tables and data therein: with sorting, filtering, coloring, download of CSV, etc.
• Data prompt switches*: when customizing a persona, add -mixin instructions to explain to generate visualization code when dealing with data and tables

Note that these 4 could as well be separate issues, so feel free to break them down.

[enricoros]

• Table extraction: identification of markdown tables, or CSV fragments and treating them like table blocks

With #392 merged, which features tabular data extraction, this becomes very feasible now.

What we need to make this happen is:

• detection of when Auto-Charting can be applied
• adding a button besides the "Download table as .csv" button
• detection of data types by column, to be able to (SAFELY) parse numerical data
• choice of the right charting library: https://observablehq.com/plot/marks/auto, ChartJS,
• formatting the data and options for the charting library
• injecting the chart when the button is toggled