Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Export to Azure Purview #8

Closed
4 tasks done
Mimetis opened this issue Apr 28, 2021 · 0 comments · Fixed by #22
Closed
4 tasks done

Export to Azure Purview #8

Mimetis opened this issue Apr 28, 2021 · 0 comments · Fixed by #22
Assignees
@valeryjacobs @elenaterenzi
Labels
Azure Genera Azure knowledge enhancement New feature or request .Net Core language

Comments

@Mimetis
Copy link
Owner

Mimetis commented Apr 28, 2021 •
edited by elenaterenzi
Loading

Idea

Since all the engine has the same oupput format (Delta lake) we can imagine to export the dataset to Purview

Today

For now, there is no way to have a clear overview of all landing zones from a single endpoint.

image

Expectation

Once a delta lake is created / updated after a successful pipeline, or after creating an engine, having a reference from pureview

image

Assumptions

  • Assuming Purview already is deployed
  • SP config in Purview has been done (manually)
  • SP has been added as a 'Storage data reader'
  • SP secret has been added to the general Keyvault

Tasks

  • Setup storage account scan for each LZ and each Engine
    • setting up storage as a data source
    • setting up a scan
  • Connect ADF from each LZ in Purview
  • [ ]
  • [ ]
@Mimetis Mimetis added enhancement New feature or request .Net Core language Azure Genera Azure knowledge labels Apr 28, 2021
@Mimetis Mimetis changed the title Export to Azure Pureview Export to Azure Purview May 10, 2021
@elenaterenzi elenaterenzi linked a pull request May 21, 2021 that will close this issue
Purview integration: Sources and Source Scans #22
Merged
@elenaterenzi elenaterenzi mentioned this issue May 31, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@valeryjacobs valeryjacobs

@elenaterenzi elenaterenzi

Labels
Azure Genera Azure knowledge enhancement New feature or request .Net Core language
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Purview integration: Sources and Source Scans Mimetis/ProjectY
3 participants
@valeryjacobs @Mimetis @elenaterenzi