Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul of Adapter Development Contributing Docs #1727

Merged
merged 44 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
6584608
overview table
dataders Jul 14, 2022
5744fd3
update source URLs for macros
dataders Jul 14, 2022
7d904d4
update for grants and and new doc pages
dataders Jul 14, 2022
bef6519
Merge branch 'current' into outlining-and-monospacing-of-code
dataders Jul 14, 2022
326fdbd
cordon into new location
dataders Jul 15, 2022
bb5a186
initialize "what are adapters" page
dataders Jul 15, 2022
fa60adb
adding in that special @jasnonaz spice!
dataders Jul 15, 2022
5a22978
add diagram
dataders Jul 15, 2022
9d33209
now in next section
dataders Jul 15, 2022
e2dfe28
wrong doc ids
dataders Jul 15, 2022
a12e28a
correctly reference static images
dataders Jul 18, 2022
0901dcd
Update website/sidebars.js
dataders Jul 18, 2022
53a82dc
Merge branch 'current' into cordoning-adapter-docs
dataders Jul 18, 2022
7833d3d
Merge branch 'current' of github.com:dbt-labs/docs.getdbt.com into co…
dataders Aug 31, 2022
6576780
add adapter verification
dataders Aug 31, 2022
23d8d55
draft adapter overview
dataders Sep 1, 2022
9e87fe6
Merge branch 'current' into cordoning-adapter-docs
dataders Sep 1, 2022
289e5c1
Update website/sidebars.js
dataders Sep 1, 2022
cbecb45
docusaurus ref instead of relative dir paths
dataders Sep 6, 2022
a963e7c
close tags
dataders Sep 6, 2022
361c48d
Merge branch 'current' of github.com:dbt-labs/docs.getdbt.com into co…
dataders Sep 6, 2022
4d7c805
escape accidental HTML tag
dataders Sep 6, 2022
5f58414
url link fixes
dataders Sep 6, 2022
cca9a19
restructure
dataders Sep 6, 2022
8267fce
Merge branch 'outlining-and-monospacing-of-code' of github.com:dbt-la…
dataders Sep 6, 2022
4bc1965
Update website/docs/docs/contributing/adapter-development/1-what-are-…
dataders Sep 7, 2022
caefa58
cleaner language
dataders Sep 7, 2022
15a102c
typo cleanups h/t @sdurry
dataders Sep 7, 2022
3078de7
Merge branch 'cordoning-adapter-docs' of github.com:dbt-labs/docs.get…
dataders Sep 7, 2022
f3c8882
code formatting
dataders Sep 7, 2022
a7e4965
Merge branch 'current' of github.com:dbt-labs/docs.getdbt.com into co…
dataders Sep 7, 2022
00836b0
Update 1-what-are-adapters.md
matthewshaver Sep 8, 2022
d46566b
Update 5-promoting-a-new-adapter.md
matthewshaver Sep 9, 2022
be8d755
Update 5-promoting-a-new-adapter.md
matthewshaver Sep 9, 2022
edfb101
thanks @jtcohen6
dataders Sep 12, 2022
d68b154
Update website/docs/docs/contributing/adapter-development/6-verifying…
dataders Sep 12, 2022
add9a08
it's got the juice!
dataders Sep 12, 2022
65554be
Merge branch 'cordoning-adapter-docs' of github.com:dbt-labs/docs.get…
dataders Sep 12, 2022
9ef6795
more note responses
dataders Sep 13, 2022
4d138de
index page and prereq page
dataders Sep 13, 2022
63b2b79
Merge branch 'current' into cordoning-adapter-docs
dataders Sep 13, 2022
08dcff3
Update website/docs/docs/contributing/adapter-development/6-promoting…
dataders Sep 13, 2022
2562f51
actually introduce topic
dataders Sep 13, 2022
cccbdcc
Merge branch 'cordoning-adapter-docs' of github.com:dbt-labs/docs.get…
dataders Sep 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions _redirects
Original file line number Diff line number Diff line change
Expand Up @@ -324,3 +324,6 @@ https://tutorial.getdbt.com/* https://docs.getdbt.com/:splat 301!
/docs/guides/migration-guide/* /guides/migration/versions/:splat 301!
/docs/guides/best-practices /guides/best-practices
/docs/guides/* /guides/legacy/:splat 301!
docs/contributing/building-a-new-adapter /docs/contributing/adapter-development/2-building-a-new-adapter 302
docs/contributing/testing-a-new-adapter /docs/contributing/adapter-development/3-testing-a-new-adapter 302
docs/contributing/documenting-a-new-adapter /docs/contributing/adapter-development/4-documenting-a-new-adapter 302
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
title: "What are adapters? Why do we need them?"
id: "1-what-are-adapters"
---

## What are adapters?

dbt "adapters" are responsible for _adapting_ dbt's functionality to a given database. If you want to make dbt work with a new database, you'll probably need to build a new adapter, or extend an existing one. Adapters are comprised of three layers:

1. At the lowest level: An *adapter class* implementing all the methods responsible for connecting to a database and issuing queries.
2. In the middle: A set of *macros* responsible for generating SQL that is compliant with the target database.
3. (Optional) At the highest level: A set of *<Term id="materialization">materializations</Term>* that tell dbt how to turn model files into persisted objects in the database.

For any questions you may have, don't hesitate to ask in the [#adapter-ecosystem](https://getdbt.slack.com/archives/C030A0UF5LM) Slack channel. The community is very helpful and likely has experienced a similar issue as you.

Below is a diagram of how dbt-postgres, the adapter at the center of dbt-core, works.

![adapter architecture diagram](/img/adapter-guide/adapter architecture - postgres.png)
dataders marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,17 +1,11 @@
---
title: "Building a new adapter"
id: "building-a-new-adapter"
id: "2-building-a-new-adapter"
---

## What are adapters?
To learn what an adapter is and they role they serve, see [What are adapters?](1-what-are-adapters.md)

dbt "adapters" are responsible for _adapting_ dbt's functionality to a given database. If you want to make dbt work with a new database, you'll probably need to build a new adapter, or extend an existing one. Adapters are comprised of three layers:

1. At the lowest level: An *adapter class* implementing all the methods responsible for connecting to a database and issuing queries.
2. In the middle: A set of *macros* responsible for generating SQL that is compliant with the target database.
3. (Optional) At the highest level: A set of *<Term id="materialization">materializations</Term>* that tell dbt how to turn model files into persisted objects in the database.

This guide will walk you through the first two steps, and provide some resources to help you validate that your new adapter is working correctly. Once the adapter is passing most of the functional tests (see ["Testing a new adapter"](testing-a-new-adapter)
This guide will walk you through the first creating the necessary adapter classes and macros, and provide some resources to help you validate that your new adapter is working correctly. Once the adapter is passing most of the functional tests (see ["Testing a new adapter"](testing-a-new-adapter)
), please let the community know that is available to use by adding the adapter to the [Available Adapters](docs/available-adapters) page by following the steps given in [Documenting your adapter](docs/contributing/documenting-a-new-adapter).

For any questions you may have, don't hesitate to ask in the [#adapter-ecosystem](https://getdbt.slack.com/archives/C030A0UF5LM) Slack channel. The community is very helpful and likely has experienced a similar issue as you.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Testing a new adapter"
id: "testing-a-new-adapter"
id: "3-testing-a-new-adapter"
---

:::info
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Documenting a new adapter"
id: "documenting-a-new-adapter"
id: "4-documenting-a-new-adapter"
---

If you've already [built](/docs/contributing/building-a-new-adapter.md), and [tested](/docs/contributing/testing-a-new-adapter.md) your adapter, it's time to document it so the dbt community will know that it exists and how to use it!
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
---
title: "Promoting a new adapter"
id: "5-promoting-a-new-adapter"
---

## What is the model for engagement in the dbt Community?

- The most important thing here is recognizing that people are successful in the community when they join, first and foremost, to authentically engage. What does authentic engagement look like? It’s hard to have hard and fast rules but a good rule of thumb is to be a good Kantian about it - contributors to the community should think of contribution *as the end itself,* not a means towards other business KPIs (leads, members of their community etc). Profits are exhaust. Some ways to know if you’re authentically engaging:
- Is the *primary* purpose of an engagement to share knowledge and resources or to build brand engagement?
- Imagine you didn’t work at the org you do - can you imagine yourself still writing this?
- Is it written in formal / marketing language or does it sound like you, the human?

## **Who should join the dbt community slack:**

### **People that have insight into what it means to do hands on [analytics engineering](https://www.getdbt.com/analytics-engineering/) work**

The dbt Community Slack workspace is fundamentally a place for analytics practitioners to interact with each other - the closer the folks are in the community to actual data / analytics engineering work the more natural their engagement will be (and as such, the better outcomes for partners + community)

### **DevRel folks who have a strong practitioner focus**

DevRel folks often have a strong analytics background and good understanding of the community. It’s important to be sure that they are focused on *contributing,* not on driving community metrics for partner org (ie sign people up for their slack or events - although this will come naturally through authentic engagement)

### **Founder/execs who are interested in directly engaging in the community**

This is either incredibly successful or not at all depending on the profile of the founder. Typically

### **Software Engineers at partner products that are building and supporting integrations with either dbt Core or dbt Cloud
**
This is successful when the engineers are familiar with dbt as a product or at least have taken our training course

### **Who might struggle in the dbt community:**
- Folks in marketing roles:
- dbt Slack is not a marketing channel and attempts to use it as such invariable fall flat and can even lead to people being turned off of your product. Again, this doesn’t mean that dbt can’t serve marketing objectives, but the only proven method to do this sustainably is by along term commitment to engaging
- (*sometimes*) Folks in product roles:
- The dbt Community can be an invaluable source of feedback on your product and how to improve it. There are two main ways this can happen - organically (community members proactively suggesting a new feature) and via direct calls for feedback and users research. Direct calls for engagement must be done in your dedicated #tools channel - in general we’d recommend doing this relatively infrequently and with intentionality as they can overwhelm more organic discussions and feedback.

## What is the audience for an adapter release?

A new adapter is likely to drive huge community interest from several groups of people:
- Folks that currently use the database that the adapter is supporting
- Folks that are *interested* in potentially using that database/adapter
- Folks that are interested in dbt development in general.

If possible - it’s best to get the adapter in front of each of these groups. The first two will hopefully be found in a dedicated channel for your database. If one does not exist already, reach out in #channel-requests and we will get one made for you and include it in the next announcement about new channels.

The third group - is where the non-slack community comes into play. Twitter and LinkedIn are both great places to engage with the community and a well-orchestrated adapter release can generate some great authentic engagement here.

## How should we message our initial rollout and follow-up content?

The goal here is to tell the a story that is engaging to your users and the dbt Community. This means highlighting new use cases / functionality unlocked by this adapter in a way that will particularly resonate with for each segment. Note that there is a slight tension between what each of these two audiences may find interesting.

### For existing users of your technology new to dbt:
- A more general overview of what the value of dbt is and why it’s useful to your users. This can lean on general dbt messaging and talking points which are laid out in the [dbt viewpoint.](https://docs.getdbt.com/docs/about/viewpoint)
- Example of a rollout that speaks to the overall value of dbt + your product well

### For folks that are already familiar with dbt / the dbt Community:
- What unique use cases or advantages does your adapter provide over existing adapters? Who might be excited for this?
- How are you contributing to the dbt Community and ensuring that dbt users on your adapter will be well supported (tutorial content, packages, documentation, etc)
- Example of a rollout that is compelling for those familiar with dbt: [Firebolt](https://www.linkedin.com/feed/update/urn:li:activity:6879090752459182080/)

## How do we tactically manage distribution of content about new or existing adapters?

There are some tactical pieces on how and where to share that can help ensure success.

### On slack:
- #i-made-this channel - note that this channel has a policy against “marketing” and “content marketing” posts but if you write your content with the above guidelines in mind it should be able to be successful. Even with that it’s important to post here sparingly.
- Your own database / tool channel - this is where the folks that have opted in to receive communications from you and always a great place to share things that are relevant to them.

### On social media:
- Twitter
- LinkedIn
- We find that social media posts *from the author* or an individual connected to the project tend to have better engagement than posts from a company or organization account.
- Ask your partner representative about:
- Retweets / shares from the official dbt Labs accounts
- Flagging posts internally at dbt Labs to get individual employees to share

## How do we know if we have enough engagement for a dedicated channel in the dbt Slack?

You don’t need 1000 people in a channel to make it successful - but you do need at least a few active participants who can make it feel lived in. If you’re comfortable working in public this could be members of your own team - or it can be a few folks who you know that are highly engaged and would be interested in participating. Having even 2 or 3 regulars hanging out in a channel is really all that’s needed for a successful start and is in fact much more impactful than 250 people that never post.

## Is there a boilerplate announcement message that we can use for our initial adapter release?

This is a tough one - in general we’d recommend *against* boilerplate announcements and encourage finding a unique voice. That being said there are a couple things that we’d want to include:

- A summary of the value prop of your database / technology for folks that aren’t familiar
- The personas that might be interested in this news
- A description of what an adapter *is* ie:
> with the release of our new dbt adapter, you’ll be able to to use dbt to model and transform your data in [name-of-your-org]
- Particular or unique use cases or functionality unlocked by this adapter
- Plans for future / ongoing support / development
- The link to the documentation for using the adapter on the dbt Labs docs site
- An announcement blog

## Is there an example of how to announce new releases of an existing adapter?

This can vary substantially depending on the nature of the release but a good baseline is the types of release messages that [we put out in the #dbt-releases](https://getdbt.slack.com/archives/C37J8BQEL/p1651242161526509) channel.

![Full Release Post](/img/adapter-guide/0-full-release-notes.png)

Breaking this down:

- Visually distinctive announcement - make it clear this is a release
![title](/img/adapter-guide/1-announcement.png)
- Short written description of what is in the release
![description](/img/adapter-guide/2-short-description.png)
- Links to additional resources
![more resources](/img/adapter-guide/3-additional-resources.png)
- Implementation instructions:
![installation](/img/adapter-guide/4-installation.png)
- Future plans
![coming soon](/img/adapter-guide/5-coming-up.png)
- Contributor recognition (if applicable)
![thank yous](/img/adapter-guide/6-thank-contribs.png)
dataders marked this conversation as resolved.
Show resolved Hide resolved
16 changes: 13 additions & 3 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,21 @@ const sidebarSettings = {
items: [
"docs/contributing/oss-expectations",
"docs/contributing/contributor-license-agreements",
"docs/contributing/building-a-new-adapter",
"docs/contributing/testing-a-new-adapter",
"docs/contributing/documenting-a-new-adapter",
"docs/contributing/slack-rules-of-the-road",
"docs/contributing/long-lived-discussions-guidelines",
{
type: "category",
label: "Adapter development",
link: {
type: 'doc',
id: 'docs/contributing/adapter-development/1-what-are-adapters',
},
items: [
'docs/contributing/adapter-development/2-building-a-new-adapter',
'docs/contributing/adapter-development/3-testing-a-new-adapter',
'docs/contributing/adapter-development/4-documenting-a-new-adapter'
dataders marked this conversation as resolved.
Show resolved Hide resolved
]
}
],
},
{
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.