bug: motis doesn't import updated gtfs files without deleting data/

[nicolaswurtz]

In beta.21, when importing new/fresh gtfs files but with same name, motis import doesn't do anything (data/isn't updated).

Steps to reproduce:

• create data/and input/, plage gtfs file(s) in input/
• put gtfs paths & conf in config file
• run motis import -c config.yml and motis server: everything is ok at this point
• put updated gtfs files with same name
• run motis import -c config.yml: nothing is done (no output in cli)
• run motis server: motis uses the same old datas

I have to completely delete data/ folder to get new files to be imported.

Thx!

[felixguendling]

What does change with the GTFS update?

We currently hash the first 50MB + the file size:

motis/src/import.cc

Lines 117 to 121 in 72fe34b

	return cista::hash_combine(
	cista::hash(mmap.view().substr(
	0U, std::min(mmap.size(),
	static_cast<std::size_t>(50U * 1024U * 1024U)))),
	mmap.size());

... over all datasets:
https://github.com/motis-project/motis/blob/master/src/import.cc#L145-L149

This should be reliable as an indicator of changed data.

Do you have a link to both datasets?

You can delete single files from data/meta to force the according import step to rerun. This way, you don't have to import OSM data again. In case the timetable changed, you have to delete all files from steps that rely on the timetable (e.g. adr_extend, osr_footpaths).

[nicolaswurtz]

I thought there was this kind of check. My use case is a bit special: I'm trying to complete shapes.txt for all my gtfs files.
And in this case, the size changes obviously.
Mmm, I'll try to reproduce it with specific files and join them to this issue.

Thx a lot for your response!

[felixguendling]

Mmm, I'll try to reproduce it with specific files and join them to this issue.

If you have any example files that trigger this issue this would be very helpful to fix it :-)