Select link analysis #260
Select link analysis #260
Conversation
|
@pedrocamargo basic functionality for MSA and FW is here. I need to
|
| # - some assertions as per constructor comments | ||
| # - demand matrix operations do not take cores into account (i.e. we assume we have a 2d demand matrix per class) | ||
| # - tests | ||
|
|
There was a problem hiding this comment.
We could add a TODO here for later that is to have the code to find the matrices that were used from the assignment report.
There was a problem hiding this comment.
yes I'd like that. at the moment we're passing in the demand, but doing this automatically would be much better
aequilibrae/paths/select_link.py
Outdated
| } | ||
| return select_link_matrices | ||
|
|
||
| def run_select_link_analysis(self, link_ids: List[int]) -> None: |
There was a problem hiding this comment.
Should the annotation be " --> AequilibraeMatrix"?
There was a problem hiding this comment.
it's a dict of dict (mode and link id) of np.array, but should probably make it an Aequlibraematrix instead of np.array
Sure, it's just not as straight forward with having to read in files, and the resulting path-link arrays and indexes not being fixed length. I didn't have much time at the end of the day (middle of the night really :) ) so I decided to go for the quickest win |
Do you mean that the reading would have to happen in each thread separately? |
|
I think this could be handled much more efficiently with duckdb - just need to optimise the on-disk path file storage (currently many small parquet files because it is set up to save one per origin and traffic class and iteration - could be as simple as combining these) and then merge demand and query. I won't have time to do this anytime soon but will come back to it at some point. |
|
@jamiecook , I had forgotten about this PR Jan had prepared a long time ago. Let's discuss this. We either incorporate it scrap it |
|
I'm happy for this to be deleted given that there is a proper implementation during assignment now. This was meant as a quick fix to enable the functionality without spending too much time on it. It's also old - if I'd do this today I'd just process the parquet files with duckdb and most of the code above would go away. |
|
@pedrocamargo, don't know what has been decided for this PR but when calibrating a model, a solution to do link analysis without re-running a full assignment can save a lot of time (with a menu like flow bundle in VISUM) even with the needed storage. Being able to do both (save for later or direct link analysis during assignment like Jake commits) would be awesome! If we keep both solutions, I would be happy to try to work on the use of path_file. But I don't think I'll be really relevant about the creation of path_file itslef (structure, duckdb or feather, ...). Maybe somebody could have a brillant idea for a quick-win based on this PR for path file savinf ? (@janzill, @Jake-Moss or maybe @djfrancesco ?) |
I think we already have all the pieces for this, but there are some improvements that would help make this more useable.
|
Select link analysis via on-disk path files. This is slower than an implementation during assignment, however it has the advantage that it works for any link(s) as long as path files have been stored during assignment.
The main reason to go down this path is that path file saving is now available and therefore this implementation is a lot less work.