Data Delivered Timestamp

[lhammerbh]

Hi Bert

Is it possible add a timestamp (datetime) in each table/file, where we can see, when these data were delivered to the lake house?

E.g. it could be the timestamp on the delta files that is included in a DeliveredDateTime column in each table.

Yours
Lars Hammer

[Bertverbeek4PS]

Hi @lhammerbh it is possible.
But wondering why you can to add it. What kind of scenario are you using it for?

[lhammerbh]

Hi @Bertverbeek4PS
It is a common practice in BI to have timestamp on a record, that says, when the data was extracted from the source system.
Usually there will be a set of columns on a record called something like DWCreatedDatetime and DWModifiedDatetime, that says when a record was first seen in the data warehouse (or lake house) or last modified in the data warehouse. This can be good for seeing how long time it takes from the time data are in the data warehouse till they are available for the users in reports or semantic models.
It is also used in cases, where there is no created/modified timestamp on the record itself as an estimate for these 2 timestamps. The created/modified timestamp is available in Business Central on all records for the later versions of BC, so this is not relevant here.
/Lars

[Bertverbeek4PS]

But in that case we must create an extra column for each table in BC and that will be filled in when the export is runned which will be called DeliveredDateTime.
Then the notebook will also take care of that column and will show it in the table in the lakehouse.

[lhammerbh]

Yes, I perfectly agree. That will be the consequence.
Maybe the column name should be $DeliveredDateTime, just as you use $Company, as both of these columns are a kind of metadata

[Bertverbeek4PS]

@lhammerbh I have created a pull request:
#103
Could you test this?

[lhammerbh]

Hi @Bertverbeek4PS
My good colleague, Birthe will validate this. Hopefully next week.
We'll get back to you, when validated