[p5.js 2.0 RFC Proposal]: Using a separate math library for handling p5.js's math operations

[RandomGamingDev]

Increasing access

As discussed in #6527:
Using another library would mean a better documented and managed library with more features for p5.js, but it'll also be more performant, and we could even choose a library that can utilize the GPU for especially heavy operations. It would also make p5.js much more powerful and easier to deal with for those who want to use it for things like 4d scenes or other areas involving math which includes things from basic sites to fully fledged games and demos.

Which types of changes would be made?

• Breaking change (Add-on libraries or sketches will work differently even if their code stays the same.)
• Systemic change (Many features or contributor workflows will be affected.)
• Overdue change (Modifications will be made that have been desirable for a long time.)
• Unsure (The community can help to determine the type of change.)

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

What's the problem?

The current p5.js math component is heavily cluttered as discussed at #6527 & #6527 (comment) and isn't nearly as optimized as it could be which makes it hard to not only use if you're trying to understand and interact with the code, but also to contribute to, especially for a library designed to increase accessibility to beginners.
Plus, it's hard to integrate p5.js with other math libraries when you need any math that's more complex, for instance if someone wanted to do something that involved linear algebra, or calculus that isn't included in p5.js, like 4d scenes.

What's the solution?

The solution would be utilizing another math library using another library would mean a better documented and managed library with more features for p5.js, but it'll also be more performant, and we could even choose a library that can utilize the GPU for especially heavy operations.

Pros (updated based on community comments)

Example list:

• Consistency: This proposal increases API consistency by removing the old cluttered code that's long been an issue with some examples discussed here
• Readability: This proposal makes the source code more readable by utilizing another library which will have better documentation as well as be more consistent in its usage and features as well as their relationship to the documentation. It also means removal of the cluttered p5.Math code.
• Performance: Not only could this mean great performance enchantments, but even GPU computations for especially large tensor operations!
• Reliability: p5.Math clearly isn't reliable due to how easily it became a hard to read mess to deal with due to its numerous bad design choices as well as misdocumented and misimplemented features. Both of these makes sense to some degree considering the fact that neither the math nor the reprogramming from scratch are the main goal of p5.js, but rather understandability and openness to beginners as well as experts. Using a math library would perfectly cater to both parties by providing a stable, customizable, powerful, efficient, well documented base for us to build p5.js off of.
• Openness: As discussed previously, the messiness of p5.js can dissuade many who want to, not only contribute to the p5.js source code, but even just use p5.js for anything slightly involving math to a higher degree. With all of the previously mentioned pros fixed, this will mean that p5.js will thus be a lot more open to those who need to use it.

Cons (updated based on community comments)

• Time: This proposal will take quite some time since it will require replacing the majority of p5.Math and its documentation in order to improve p5.js.

Proposal status

Under review

[limzykenneth]

I'm generally hesitant to use an external math library for p5.js for a few reasons, mainly surrounding the use of external dependencies in p5.js. Where possible I would like to avoid external dependencies. We have external dependency on Opentype.js and it is definitely giving quite a bit of problems, including those listed below.

1. Larger bundle size - A lot of the bundle size of p5.js is taked up by Opentype.js and introducing another external dependency will likely make this worse. One of the goals of p5.js 2.0 is to optimize bundle size and even things like Opentype.js I'm thinking of refactoring it out or moving it into an optional component to save on size.
2. Longer term support risk - There are risk that whatever external library we end up using may fall out of support in the future, hamstringing us in terms of keeping things up to date. Many of the dependencies we use already have this problem which p5.js 2.0 is an opportunity to address them.

Most math libraries probably include things that we may not need for p5.js, at least on a basic level. I think at this stage it would be better if we can figure out what are the math features as well as interfaces that p5.js 2.0 needs first, whether it is multidimentional vectors and matrices, in-depth linear algebra, hardware acceleration, etc.

I suspect for a basic level we don't need full linear algebra functionalities and we can consider code implementation in other libraries for the simpler stuff that we do need and implementing it ourselves, eg. taking vector implementation code from Three.js and implementing it in our own module without explicitly including Three.js as a dependency.

To copy and paste from the current RFC:
A new math module will be implemented. The requirements of this new math module are:

• Unified vector and matrices support with compatible API with ml5.js.
• Explore reasonable performance optimizations through the use of specific algorithms, GPU implementation, or others.
• Possibility for addon libraries to provide own implementation (so that any external libraries can be made compatible without being implemented internally by p5.js).

Especially the last point, if we standardize the end user API of how math works in p5.js 2.0, addon libraries can be created to act as a bridge between any external math libraries and p5.js by overwriting the exsting math implementation or adding additional APIs.

Let me know if that made sense.

[nickmcintyre]

@RandomGamingDev thanks for bringing this up. You've pointed out a clear need for a refactor/rewrite, but adding a separate math library feels like a disproportionate solution. +1 to pretty much everything @limzykenneth said.

Functions such as cos() are generally fine as-is. p5.Matrix and p5.Vector could use some TLC as you've mentioned. Overall, the p5.js core only needs a few classes with a few dozen methods. I believe it's actually more accessible to contributors for those implementations to live in the p5.js repo. We should definitely aim to make them clean and performant.

@RandomGamingDev what do you think about opening a couple of related issues?

• Add a contributor code style guide (e.g., Airbnb unless it's a performance optimization)
• Update the math module starting with the requirements @limzykenneth shared

[RandomGamingDev]

Overall, the p5.js core only needs a few classes with a few dozen methods. I believe it's actually more accessible to contributors for those implementations to live in the p5.js repo. We should definitely aim to make them clean and performant.

I disagree with this point since I don't see how an already implemented library with proper documentation would be any less accessible compared to a custom solution built from the ground up and even more so if wrap the API using p5.js.

Larger bundle size - A lot of the bundle size of p5.js is taked up by Opentype.js and introducing another external dependency will likely make this worse. One of the goals of p5.js 2.0 is to optimize bundle size and even things like Opentype.js I'm thinking of refactoring it out or moving it into an optional component to save on size.

Math libraries like NumJs have tiny sizes like 720kb unpacked for NumJs in particular: https://www.npmjs.com/package/numjs

Longer term support risk - There are risk that whatever external library we end up using may fall out of support in the future, hamstringing us in terms of keeping things up to date. Many of the dependencies we use already have this problem which p5.js 2.0 is an opportunity to address them.

It should be safe to rely on libraries like NumJs since they should be supported for quite some time, not to mention that even if long term support is stopped, we will most likely already have all the features we need to not to mention that forks will most likely pop up considering its size as well as the fact that the plan would be to wrap the entire API with p5.js meaning that it'd be much easier to switch between math libraries.

Most math libraries probably include things that we may not need for p5.js, at least on a basic level. I think at this stage it would be better if we can figure out what are the math features as well as interfaces that p5.js 2.0 needs first, whether it is multidimentional vectors and matrices, in-depth linear algebra, hardware acceleration, etc.

I suspect for a basic level we don't need full linear algebra functionalities and we can consider code implementation in other libraries for the simpler stuff that we do need and implementing it ourselves, eg. taking vector implementation code from Three.js and implementing it in our own module without explicitly including Three.js as a dependency.

To copy and paste from the current RFC:
A new math module will be implemented. The requirements of this new math module are:

• Unified vector and matrices support with compatible API with ml5.js.
• Explore reasonable performance optimizations through the use of specific algorithms, GPU implementation, or others.
• Possibility for addon libraries to provide own implementation (so that any external libraries can be made compatible without being implemented internally by p5.js).

Especially the last point, if we standardize the end user API of how math works in p5.js 2.0, addon libraries can be created to act as a bridge between any external math libraries and p5.js by overwriting the exsting math implementation or adding additional APIs.

Honestly, I don't see an issue with implementing more complex and math heavy features if it can interact with external math libraries properly instead of the messy system currently being employed. However, this is assuming that it can be properly maintained and is worth doing so. One of the main reasons why I'm pushing for incorporating an external math library into p5.js is due to the issue of whether or not it's worth it to get people working on a largely graphics focused library to redo the wheel by rewriting core math code that's already been written before costing a lot of manpower as well as not guaranteeing a satisfactory level of code quality. The current p5.Math code, for me shows that this, if attempted, will be quite difficult and might not even work which would leave p5.js in an even worse off position.

@RandomGamingDev what do you think about opening a couple of related issues?

• Add a contributor code style guide (e.g., Airbnb unless it's a performance optimization)
• Update the math module starting with the requirements @limzykenneth shared

I agree that if we don't add a math library, this would probably be our best bet and I'd be fine with adding these issues. However, I do still feel like reimplementing the math implemented in other libraries and trying to get a largely graphics focused group of contributors to have to work on the math library alongside graphics code at the same time that they code everything will cost a lot of unnecessary manpower as well as not guaranteeing a satisfactory level of code quality.

[limzykenneth]

It should be safe to rely on libraries like NumJs since they should be supported for quite some time, not to mention that even if long term support is stopped, we will most likely already have all the features we need to not to mention that forks will most likely pop up considering its size as well as the fact that the plan would be to wrap the entire API with p5.js meaning that it'd be much easier to switch between math libraries.

I think for me Numjs more or less made the point of why I'm hesitant to incorporate external libraries. 720kb is somewhat significant in terms of file size, and to give some perspective, p5.js unminified is currently 4.34mb which 720kb will be about 16% of the overall size (granted not all 720kb will be packaged in and is likely an overestimate, it may be worth figuring out what actual impact it has on size). Numjs also appears already not under active maintenance for at least 2 years with last commit 3 years ago and last published on NPM 2 years ago. This is quite significant in the JavaScript ecosystem, and also means any issue we have that can be traced back to Numjs we don't have reasonable means to fix them.

The main point I'm trying to make is not to implement all the complex and heavy math features all by ourselves but rather to identify what actual math features we actually need first. Eg. we don't need math expression evaluation or expression derivative (so that's one large part of Math.js we don't need for example), we may or may not need full matrix operations, etc. Once we have an idea what we actually need then we can evaluate how we actually implement it: use available browser implementations, take existing implementations from other libraries, or if it is really necessary use external libraries. Without figuring out what we need at this stage we risk bloating the library with features that is never or very seldom used.

[RandomGamingDev]

I think for me Numjs more or less made the point of why I'm hesitant to incorporate external libraries. 720kb is somewhat significant in terms of file size, and to give some perspective, p5.js unminified is currently 4.34mb which 720kb will be about 16% of the overall size (granted not all 720kb will be packaged in and is likely an overestimate, it may be worth figuring out what actual impact it has on size). Numjs also appears already not under active maintenance for at least 2 years with last commit 3 years ago and last published on NPM 2 years ago. This is quite significant in the JavaScript ecosystem, and also means any issue we have that can be traced back to Numjs we don't have reasonable means to fix them.

While size should be a concern I don't think that numjs, especially with the limited amount of features we'll be using will pose an issue. Also, your point about NumJs while partially valid doesn't really apply since the API would be implemented in a way where it would all be under the p5.js namespace and because we might very likely do something similar to three.js where choose take certain parts of it. Either that, or we import it as normal.

The main point I'm trying to make is not to implement all the complex and heavy math features all by ourselves but rather to identify what actual math features we actually need first. Eg. we don't need math expression evaluation or expression derivative (so that's one large part of Math.js we don't need for example), we may or may not need full matrix operations, etc. Once we have an idea what we actually need then we can evaluate how we actually implement it: use available browser implementations, take existing implementations from other libraries, or if it is really necessary use external libraries. Without figuring out what we need at this stage we risk bloating the library with features that is never or very seldom used.

I'm fine with this part if the standardized interface for interacting with NumJs idea is implemented, but my main concern with that, again, is the issue of the unnecessary expenditure of manpower and issues with maintaining code quality.

[limzykenneth]

@RandomGamingDev There is ultimately going to be some level of engineering involve unless we don't even implement interfaces to any math operations and let users use external math libraries directly. I wouldn't worry too much about time and effort spent on this in this case as long as it is providing value (which I believe it will). For code quality, you can always be our check when things feel like it is going too out of hand, and I would really appreciate that extra perspective as well.

I'm fine with this part if the standardized interface for interacting with NumJs idea is implemented

General compatibility with say NumJs or Math.js is something I would like to aim for as well so it can be straightforward to bring in any external library, even without additional glue code in the form of an addon library, and have it be usable with most if not all p5.js functions. This should definitely be looked into.

[aferriss]

Setting aside the performance and code organization arguments, what are the specific use cases that bringing a separate math library in would enable. For example, what would a user be able to do with linear algebra or calculus functions that they cannot do now?

Second, do we have any way of knowing how much demand exists for the kinds of advanced math features that would come with a separate library? Anecdotally, are these features requested often by students or other people on forums / github / discord?

Lastly, can we benchmark data from different browsers / devices that can tell us specifically which parts of the p5 math library are slow and what needs improving? Then also some comparison with the library / optimization being proposed. It would be good to know what is in most need of help.

In general I think it would be good to get more specifics about the current perf and usage to help come to a decision.

[RandomGamingDev]

Setting aside the performance and code organization arguments, what are the specific use cases that bringing a separate math library in would enable. For example, what would a user be able to do with linear algebra or calculus functions that they cannot do now?

It wouldn't make anything currently impossible possible, but it'd increase the productivity and ease of contribution for those contributing to p5.js as well as the ease of use for anyone who just wants to do something involving linear algebra with p5.js

Second, do we have any way of knowing how much demand exists for the kinds of advanced math features that would come with a separate library? Anecdotally, are these features requested often by students or other people on forums / github / discord?

While I haven't done an interview, I know that p5.Math is oftentimes used for mathematical visualizations, as can be seen from the popular series "The Coding Train" which largely focuses on p5.js and, alongside many other mathematical displays would benefit massively. The performance, while it'd be good should be basically negligible so we can ignore that part for now.

Lastly, can we benchmark data from different browsers / devices that can tell us specifically which parts of the p5 math library are slow and what needs improving? Then also some comparison with the library / optimization being proposed. It would be good to know what is in most need of help.

I didn't do it for multiple browsers, but it should be around the same for most. The benchmarks can be found in this comment alongside a description and explanation for them:
#6527 (comment)

[davepagurek]

While not addressing every point in the proposal, an option brought up in #6527 is to basically keep our same API, same friendly errors and such, but swap out our vector/matrix implementations with a library to reduce some of the complexity of our own codebase and the scope of things we need to test, while also getting some performance gains (@RandomGamingDev got some good metrics on what those gains could be in the previously linked comment.) This feels like a sort of lower-risk change that doesn't affect p5's offerings to users, and costs less engineering time by keeping the same API. It might be a good GSoC-sized project too, to take some load off of maintainers.

[mohmoh412]

Hello Community,

My name is Mohini, and I am a self taught programmer. I've taken the time to thoroughly read the discussion and the related issues including the performance comparison provided.

I do feel that the integration of an external library like glMatrix for vector and matrix calculations is a good strategic step towards enhancing the mathematical capabilities of p5.js. Based on the performance benchmarks shared, it's clear that glMatrix could significantly boost the efficiency for intensive applications such as 3D graphics rendering, without compromising too much on memory usage.

I also believe that the adoption of glmatrix, or a similar high-performance library, will help in providing readability and ease of use for beginners while also offering advanced mathematical functionalities to seasoned programmers for complex projects.

Integrating an external library for vector and matrix operations will be a good first step, and in terms of scalability and long term goals, it would also provide an opportunity to reevaluate and possibly reorganise the existing math library structure, by consolidating related functionalities. This would ensure that the library remains intuitive and accessible, all while keeping the core values of simplicity and accessibility uncompromised.

I would like to add that having comprehensive guidelines and testing would allow us to identify any inefficiencies and set benchmarks for the implementations to help maintain the standard and efficiency of the math library in the long run.

I started my programming journey with p5.js and I am more than happy to have this as my first open source contribution .I'm also planning to apply for GSoC 2024, and would like to incorporate this project in my proposal. If there are any additional issues or improvements I haven't considered, I'm all ears and would really like to hear your inputs and suggestions.

[Qianqianye]

Adding p5.js math module contributor @holomorfo to this discussion

[holomorfo]

Hi community.
I want to share that I will be collaborating on p5.js 2.0 Math Module Revamp Project with the integration higher dimensional vector support and matrix support by moving p5.Matrix from WebGL module to the math module, as well as finding ways to integrate P5.js to external math libraries.

As a first part of this analysis we compared different of the most popular and accessible math and matrix operation libraries. The points of comparison were bundle size, complexity of implementation and wethere the library is actively maintained or not.

As reference the P5.js bundle size is aroound 1017 kB

glMatrix minimizes

52.2 kB
Last commit 5 months ago
Elementwise, similar to what we have now.
operations are static methods and does not have n-dimentional matrix

• mat2
• mat2d
• mat3
• mat4
• quat
• quat2
• vec2
• vec3
• vec4

Math js

670kB
Last commit last week
https://github.com/josdejong/mathjs
Lots of other functionality that we don't want/need at the moment
Operations are a separate function not a method of the class.

Num JS

174 kB minified
Last change 3 years ago
This looks like the best N dimentional
Has several dependencies in order to do optimization via cwise
ndarray-gemm
ndarray-ops
cwise-compiler
https://github.com/scijs/cwise?tab=readme-ov-file

Numbers js

6 years ago last commit
has N dimentional matices
elementwise multiplication without optimization

ML matrix

https://github.com/mljs/matrix
Unpacked size 412 kB
Last change 3 months ago+6
https://www.npmjs.com/package/ml-matrix
this one has general size matrices with for loops

Webgp blas

Alpha version
https://github.com/milhidaka/webgpu-blas
Multiplicaitons using webgl
The WebGPU standard is still in the process of being established and will not work in normal web browsers.
This one is not ready for production

Webassembly

https://dev.to/ndesmic/fast-matrix-math-in-js-2-wasm-3mbn
Webassembly migth be good in the future but at the moment it carries significatn overhead.

What we've observed this time is that WASM by itself does not yield great performance gains until it actually overcomes the overhead of memory copy, which in our case is around the 64x64 element mark. In these cases using typed arrays can still beat it by not introducing that complexity, the compiler seems smart enough to optimize these. However, once we start introducing SIMD to the picture WASM can overtake plain JS starting around 64x64 and pulling away as sizes get bigger.

Webgpu in safari

https://jott.live/markdown/webgpu_safari

Tensorflow JS?

No direct matrix multiplication found, it can only run models aleardy exported from regular Tensorflow library

Three.js

Does matrix multiplications elementwise 2x2, 3x3 4x4
Vectors also non N dimentional, similar to the current P5.js implementaiton

Matrix.js

matrix js
https://github.com/RandomGamingDev/MatrixJs
Dependencies
https://github.com/RandomGamingDev/VecJs
N dimentinal Vectors and matrices, couldn't find a concrete bundle size.

Conclusion

I find that In real production code it would be best to satrt by integrating optimized functions like i glMatrix
https://dev.to/ndesmic/fast-matrix-math-in-js-11cj
We will start with defining our desired API for Vector and Matrix, keeping the existing functionality and then we will add an Adapter component to flexibly integrate with different libraries while we define clearly what is the best balance between functionality and performance the p5 math module needs to maintain flexibility and accessibility.

This document will be updated with new information as it becomes available
CC: @limzykenneth @Qianqianye @davepagurek

[nickmcintyre]

@holomorfo glMatrix is a solid choice and I'm excited to see your work progress!

It turns out that TensorFlow.js has a linear algebra engine that's overkill for p5.js' core math module. I use it in an addon library called número that I'd love to make compatible with the new API. Feel free to tag me when you're ready to test the adapter.

[davepagurek]

Thanks for all the research @holomorfo! The adapter API is a good idea for being able to cover more use cases via addons.

[holomorfo]

As a first step, I'm updating the Vector class to have a values array to handle generic n-dimension arrays, while maintaining the existing API by adding some set and get accessors for x, y, z. Some methods might not be compatible directly with n-dimension for example:
- cross product only defined in 3 dimensions, dimension needs determinant
- heading method? only defined by 2D
- setHeading rotates 2D vector
- rotate also 2D
- angle between needs ND cross product
- lerp cannot be made n dimensional while assuming the last value is the amount and can be missing.
- rem has poly-morfism behaviour depending on arguments, how to extrapolate this case.

Draft PR

CC: @limzykenneth @davepagurek

[holomorfo]

Here I share the PR ready for review for the first version of the nD vectors, with most of the methods updated.
I'm starting to see some of the challenges of creating the adapter/facade pattern to integrate with some other third party libraries for both vector and matrix:

• The current processing API for both vector and matrix, which I'm adhering strictly to, depends on many modifications in place. Modification in place for most operations: add, mult, div, rem
• Also some specific behavior of methods like lerp and slerp assume that the vectors will be 3D and the 4th parameter will dedicated to the ammount. When addapting this for nD we can't assume that the last element will be the ammount and not just the last coordinate.

Issues like these might not be compatible with other libraries, but will research as a next step, which methods and how they can be adapted.

#7277

CC: @limzykenneth @davepagurek @Qianqianye