Skip to content

Multi-dimensional batch calculation: list of update batch dataset as a cartesian product #1201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
TonyXiang8787 wants to merge 66 commits into
base: main
Choose a base branch
from
Draft

Multi-dimensional batch calculation: list of update batch dataset as a cartesian product #1201

TonyXiang8787 wants to merge 66 commits into from

Conversation

@TonyXiang8787
Copy link
Member

@TonyXiang8787 TonyXiang8787 commented Nov 25, 2025 •
edited by mgovers
Loading

Closes #1176

Let user to provide a list of update batch datasets, which are used as cartesian product to produce a multi-dimentional batch calculation.

The implementation is a cartesian product over a linked list.

check list

  • C-API and core
  • C-API test
  • C-API in-code documentation
  • Python wrapper
  • Python test
  • Python in-code documentation
  • Adjustment on general documentation and examples

@TonyXiang8787
start slice dataset
8085e5b 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
add slice scenario
73d85c5 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
Merge branch 'main' into experimental/multi-dimension-batch
234f38d
@TonyXiang8787
batch dimension
be44d80 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
calculation implementation
1fd7ed3 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
error handling still needs to be done
45f4d72 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
error handling
06cb330 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
add batch dimensions
a4fc01e 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
batch dimension
118655f 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
start test
3b41f28 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
start test
d88db97 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
api will not work as intended
1b1190d 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
api will not work as intended
21c31c5 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
adjust md dataset
6e8c081 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
add dataset
0b989b4 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
crash yet
14b4039 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
fix bounds checking
3338881 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
remove span
01c86a9 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
fix clang tidy
7786b0c 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
format|
5a3f394 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
[skip ci] add cfunc in python
2650968 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
force nullptr
9aaa6bd 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
add options
e4aa439 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
proxy for multidimensional in python
237681f 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
modify main calculate input
b602f2e 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
type annotation
3e79237 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
[skip ci] not working yet
8d8d80c 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
fix dimensions
61cfd57 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
fix mypy
a1331ba 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
Merge branch 'main' into experimental/multi-dimension-batch
36707d9
figueroa1395
figueroa1395 reviewed Dec 12, 2025
View reviewed changes
Copy link
Member

@figueroa1395 figueroa1395 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just some nitpicks.

@TonyXiang8787
refactor stride size
4306491 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
change set/get next in cpp
6f1b5a1 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
change get/set next in capi
b472331 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
modify python side set next
3185c93 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
add c-api docs
b18d06c 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
try python docs
5967452 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
try space
e203ff1 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
adjust python docs
d222d3c 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
Copy link
Member Author

TonyXiang8787 commented Dec 12, 2025

@mgovers @figueroa1395 @nitbharambe,

The main part is almost finished. We still need to add general documentation and some notebook examples. I propose one of you to add this. This make sure we actually have the same understanding about how cartesian product of MD batch calculation works.

@TonyXiang8787
avoid cyclic chain
d1dc1cf 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
format
26f6ca4 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@mgovers mgovers changed the title Feature: support multi-dimensional batch calculation, list of update batch dataset as a cartesian product Multi-dimensional batch calculation: list of update batch dataset as a cartesian product Dec 15, 2025
@mgovers
sonar
f58e13f 
Signed-off-by: Martijn Govers <Martijn.Govers@Alliander.com>
TonyXiang8787 and others added 2 commits December 15, 2025 09:11
@TonyXiang8787 @mgovers
Update power_grid_model_c/power_grid_model_c/include/power_grid_model…
30ec19c 
…_c/model.h

Co-authored-by: Martijn Govers <martijn.govers@alliander.com>
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
do not use chain word
54a8d01 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@figueroa1395
Copy link
Member

figueroa1395 commented Dec 15, 2025 •
edited
Loading

@mgovers @figueroa1395 @nitbharambe,

The main part is almost finished. We still need to add general documentation and some notebook examples. I propose one of you to add this. This make sure we actually have the same understanding about how cartesian product of MD batch calculation works.

Good idea. I can probably pick this up. I'll bring it to Peter today so we allocate some time for it.

Edit: No further comments from me. It looks good to me at this stage besides the additional stuff you pointed out. I would only also silence the Sonar Cloud warnings as I deem them false positives.

mgovers
mgovers reviewed Dec 15, 2025
View reviewed changes
mgovers
mgovers reviewed Dec 15, 2025
View reviewed changes
TonyXiang8787 and others added 5 commits December 15, 2025 12:57
@TonyXiang8787 @mgovers
Update power_grid_model_c/power_grid_model_c/src/model.cpp
a1c0710 
Co-authored-by: Martijn Govers <martijn.govers@alliander.com>
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787 @mgovers
Update power_grid_model_c/power_grid_model_c/include/power_grid_model…
76dec6e 
…_c/dataset.h

Co-authored-by: Martijn Govers <martijn.govers@alliander.com>
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787 @mgovers
Update src/power_grid_model/_core/power_grid_model.py
f5e69df 
Co-authored-by: Martijn Govers <martijn.govers@alliander.com>
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
modify test case
7e2c865 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@TonyXiang8787
modify test case
c208af1 
Signed-off-by: Tony Xiang <19280867+TonyXiang8787@users.noreply.github.com>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 15, 2025

Quality Gate Passed Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
83.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@figueroa1395
Copy link
Member

figueroa1395 commented Dec 15, 2025

@mgovers @figueroa1395 @nitbharambe,
The main part is almost finished. We still need to add general documentation and some notebook examples. I propose one of you to add this. This make sure we actually have the same understanding about how cartesian product of MD batch calculation works.

Good idea. I can probably pick this up. I'll bring it to Peter today so we allocate some time for it.

Edit: No further comments from me. It looks good to me at this stage besides the additional stuff you pointed out. I would only also silence the Sonar Cloud warnings as I deem them false positives.

@TonyXiang8787 I can pick this up whenever you give me the green light and finish it off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgovers mgovers mgovers left review comments

@figueroa1395 figueroa1395 figueroa1395 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

@TonyXiang8787 TonyXiang8787

Labels

do-not-merge This should not be merged feature New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[FEATURE] 2-D or cross-join of batch updates

4 participants

@TonyXiang8787 @figueroa1395 @mgovers