Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emmyoop patch 3 #857

Merged
merged 11 commits into from
Oct 19, 2021
3 changes: 3 additions & 0 deletions _redirects
Original file line number Diff line number Diff line change
Expand Up @@ -274,3 +274,6 @@ https://tutorial.getdbt.com/* https://docs.getdbt.com/:splat 301!
/reference/model-selection-syntax/#test-selection-examples /reference/node-selection/test-selection-examples 302
/docs/building-a-dbt-project/building-models/using-custom-database /docs/building-a-dbt-project/building-models/using-custom-databases 302
/dbt-cloud/api /dbt-cloud/api-v2 302
/reference/project-configs/source-paths /reference/project-configs/model-paths 302
/reference/project-configs/data-paths /reference/project-configs/seed-paths 302
/reference/project-configs/modules-paths /reference/project-configs/packages-install-path 302
4 changes: 2 additions & 2 deletions website/docs/docs/building-a-dbt-project/analyses.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ id: "analyses"

dbt's notion of `models` makes it easy for data teams to version control and collaborate on data transformations. Sometimes though, a certain sql statement doesn't quite fit into the mold of a dbt model. These more "analytical" sql files can be versioned inside of your dbt project using the `analysis` functionality of dbt.

Any `.sql` files found in the `analysis/` directory of a dbt project will be compiled, but not executed. This means that analysts can use dbt functionality like `{{ ref(...) }}` to select from models in an environment-agnostic way.
Any `.sql` files found in the `analyses/` directory of a dbt project will be compiled, but not executed. This means that analysts can use dbt functionality like `{{ ref(...) }}` to select from models in an environment-agnostic way.

In practice, an analysis file might look like this (via the [open source Quickbooks models](https://github.com/dbt-labs/quickbooks)):

Expand Down Expand Up @@ -46,4 +46,4 @@ To compile this analysis into runnable sql, run:
dbt compile
```

Then, look for the compiled SQL file in `target/compiled/{project name}/analysis/running_total_by_account.sql`. This sql can then be pasted into a data visualization tool, for instance. Note that no `running_total_by_account` relation will be materialized in the database as this is an `analysis`, not a `model`.
Then, look for the compiled SQL file in `target/compiled/{project name}/analyses/running_total_by_account.sql`. This sql can then be pasted into a data visualization tool, for instance. Note that no `running_total_by_account` relation will be materialized in the database as this is an `analysis`, not a `model`.
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ The events in this table are recorded by \[Snowplow](http://github.com/snowplow/
In the above example, a docs block named `table_events` is defined with some descriptive markdown contents. There is nothing significant about the name `table_events` — docs blocks can be named however you like, as long as the name only contains alphanumeric and underscore characters.

### Placement
Docs blocks should be placed in files with a `.md` file extension. By default, dbt will search in all resource paths for docs blocks (i.e. the combined list of [source-paths](source-paths), [data-paths](data-paths), [analysis-paths](analysis-paths), [macro-paths](macro-paths) and [snapshot-paths](snapshot-paths)) — you can adjust this behavior using the [docs-paths](docs-paths) config.
Docs blocks should be placed in files with a `.md` file extension. By default, dbt will search in all resource paths for docs blocks (i.e. the combined list of [model-paths](model-paths), [seed-paths](seed-paths), [analysis-paths](analysis-paths), [macro-paths](macro-paths) and [snapshot-paths](snapshot-paths)) — you can adjust this behavior using the [docs-paths](docs-paths) config.


### Usage
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/building-a-dbt-project/exposures.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Exposures make it possible to define and describe a downstream use of your dbt p

### Declaring an exposure

Exposures are defined in `.yml` files in your `models` directory (as defined by the [`source-paths` config](source-paths)), nested under an `exposures:` key.
Exposures are defined in `.yml` files nested under an `exposures:` key.

<File name='models/<filename>.yml'>

Expand Down
12 changes: 9 additions & 3 deletions website/docs/docs/building-a-dbt-project/package-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,13 @@ packages:

</File>

3. Run `dbt deps` to install the package(s). Packages get installed in the `dbt_modules` directory – by default this directory is ignored by git, to avoid duplicating the source code for the package.
<Changelog>

- **v1.0.0:** The default [`packages-install-path`](packages-install-path) has been updated to be `dbt_packages` instead of `dbt_modules`.

</Changelog>

3. Run `dbt deps` to install the package(s). Packages get installed in the `dbt_packages` directory – by default this directory is ignored by git, to avoid duplicating the source code for the package.

## How do I specify a package?
You can specify a package using one of the following methods, depending on where your package is stored.
Expand Down Expand Up @@ -247,8 +253,8 @@ Check out [dbt Hub](https://hub.getdbt.com) to see the library of published dbt
When you update a version or revision in your `packages.yml` file, it isn't automatically updated in your dbt project. You should run `dbt deps` to update the package. You may also need to run a [full refresh](run) of the models in this package.

### Uninstalling a package
When you remove a package from your `packages.yml` file, it isn't automatically deleted from your dbt project, as it still exists in your `dbt_modules/` directory. If you want to completely uninstall a package, you should either:
* delete the package directory in `dbt_modules/`; or
When you remove a package from your `packages.yml` file, it isn't automatically deleted from your dbt project, as it still exists in your `dbt_packages/` directory. If you want to completely uninstall a package, you should either:
* delete the package directory in `dbt_packages/`; or
* run `dbt clean` to delete _all_ packages (and any compiled models), followed by `dbt deps`.

### Configuring packages
Expand Down
8 changes: 4 additions & 4 deletions website/docs/docs/building-a-dbt-project/seeds.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ id: "seeds"
* [`seed` command](seed)

## Getting started
Seeds are CSV files in your dbt project (typically in your `data` directory), that dbt can load into your data warehouse using the `dbt seed` command.
Seeds are CSV files in your dbt project (typically in your `seeds` directory), that dbt can load into your data warehouse using the `dbt seed` command.

Seeds can be referenced in downstream models the same way as referencing models — by using the `ref` [function](dbt-jinja-functions/ref).

Expand All @@ -25,9 +25,9 @@ Poor use-cases of dbt seeds:

## Example
To load a seed file in your dbt project:
1. Add the file to your `data` directory, with a `.csv` file extension, e.g. `data/country_codes.csv`
1. Add the file to your `seeds` directory, with a `.csv` file extension, e.g. `seeds/country_codes.csv`

<File name='data/country_codes.csv'>
<File name='seeds/country_codes.csv'>

```text
country_code,country_name
Expand Down Expand Up @@ -62,7 +62,7 @@ Done. PASS=1 ERROR=0 SKIP=0 TOTAL=1
<File name='models/orders.csv'>

```sql
-- This refers to the table created from data/country_codes.csv
-- This refers to the table created from seeds/country_codes.csv
select * from {{ ref('country_codes') }}
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Sources make it possible to name and describe the data loaded into your warehous

### Declaring a source

Sources are defined in `.yml` files in your `models` directory (as defined by the [`source-paths` config](source-paths)), nested under a `sources:` key.
Sources are defined in `.yml` files nested under a `sources:` key.

<File name='models/<filename>.yml'>

Expand Down
12 changes: 12 additions & 0 deletions website/docs/docs/guides/migration-guide/upgrading-to-1-0-0.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,21 @@ dbt v1.0.0-b1 is currently available. If you have questions or encounter bugs, p
## Breaking changes

- The two type of test definitions are now "singular" and "generic" (instead of "data" and "schema", respectively). The `test_type:` selection method accepts `test_type:singular` and `test_type:generic`. (It will also accept test_type:schema and test_type:data for backwards compatibility.) **Not backwards compatible:** The `--data` and `--schema` flags to dbt test are no longer supported, and tests no longer have the tags `'data'` and `'schema'` automatically applied.
- The `packages` argument of [dispatch](dispatch) has been deprecated and will raise an exception when used.
- The "adapter_macro" macro has been deprecated. Instead, use the [dispatch](dispatch) method to find a macro and call the result.
- The `release` arg has been removed from the `execute_macro` method.

## New and changed documentation

- [Tests](building-a-dbt-project/tests), [test selection](test-selection-examples), [selection methods](node-selection/methods): Update `test_type:` selection method. Remove references to `--schema` and `--data` flags.
- [Parsing](parsing): partial parsing and static parsing have been turned on by default.
- [Global configs](global-configs) have been standardized. Related updates to [global CLI flags](global-cli-flags) and [`profiles.yml`](profiles.yml).

### Elsewhere in Core
- [model-paths](model-paths) have replaced `source-paths` in `dbt-project.yml.
- [seed-paths](seed-paths) have replaced `data-paths` in `dbt-project.yml with a default value of `seeds`.
- The default value of [test-paths](test-paths) has been updated to be the plural `tests`.
- The default value of [analysis-paths](analysis-paths) has been updated to be the plural `analyses`.
- The [packages-install-path](packages-install-path) was updated from `modules-path`. Additionally the default value is now `dbt-packages` instead of `dbt-modules`. You may need to update this value in [`clean-targets`](clean-targets).
- Default for `quote-columns` is now `True` for all adapters other than Snowflake.

13 changes: 9 additions & 4 deletions website/docs/faqs/configurable-data-path.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
---
title: Can I store my seeds in a directory other than the `data` directory in my project?
title: Can I store my seeds in a directory other than the `seeds` directory in my project?
---
By default, dbt expects your seed files to be located in the `data` subdirectory
<Changelog>

- **v1.0.0:** The config 'data-paths' has been deprecated in favor of [`seed-paths`](seed-paths).

</Changelog>
By default, dbt expects your seed files to be located in the `seeds` subdirectory
of your project.

To change this, update the [data-paths](reference/project-configs/data-paths.md) configuration in your `dbt_project.yml`
To change this, update the [seed-paths](reference/project-configs/seed-paths.md) configuration in your `dbt_project.yml`
file, like so:

<File name='dbt_project.yml'>

```yml
data-paths: ["seeds"]
seed-paths: ["custom_seeds"]
```

</File>
2 changes: 1 addition & 1 deletion website/docs/faqs/configurable-data-test-path.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Can I store my data tests in a directory other than the `test` directory in my project?
title: Can I store my data tests in a directory other than the `tests` directory in my project?
---
By default, dbt expects your singular test files to be located in the `tests` subdirectory of your project.

Expand Down
6 changes: 4 additions & 2 deletions website/docs/faqs/configurable-model-path.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
---
title: Can I store my models in a directory other than the `models` directory in my project?
---
- **v1.0.0:** The config 'source-path' has been deprecated in favor of [`model-paths`](model-paths).

By default, dbt expects your seed files to be located in the `models` subdirectory of your project.

To change this, update the [source-paths](reference/project-configs/source-paths.md) configuration in your `dbt_project.yml`
To change this, update the [model-paths](reference/project-configs/model-paths.md) configuration in your `dbt_project.yml`
file, like so:

<File name='dbt_project.yml'>

```yml
source-paths: ["transformations"]
model-paths: ["transformations"]
```

</File>
4 changes: 2 additions & 2 deletions website/docs/faqs/full-refresh-seed.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Found 0 models, 0 tests, 0 snapshots, 0 analyses, 130 macros, 0 operations, 1 se

Completed with 1 error and 0 warnings:

Database Error in seed country_codes (data/country_codes.csv)
Database Error in seed country_codes (seeds/country_codes.csv)
000904 (42000): SQL compilation error: error line 1 at position 62
invalid identifier 'COUNTRY_NAME'

Expand All @@ -51,7 +51,7 @@ Found 0 models, 0 tests, 0 snapshots, 0 analyses, 149 macros, 0 operations, 1 se

Completed with 1 error and 0 warnings:

Database Error in seed country_codes (data/country_codes.csv)
Database Error in seed country_codes (seeds/country_codes.csv)
column "country_name" of relation "country_codes" does not exist

Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1
Expand Down
2 changes: 1 addition & 1 deletion website/docs/faqs/schema-yml-name.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ No! You can name this file whatever you want (including `whatever_you_want.yml`)

Check out the [docs](configs-and-properties) for more information.

¹If you're declaring properties for seeds, snapshots, or macros, you can also place this file in the related directory — `data/`, `snapshots/` and `macros/` respectively.
¹If you're declaring properties for seeds, snapshots, or macros, you can also place this file in the related directory — `seeds/`, `snapshots/` and `macros/` respectively.
2 changes: 1 addition & 1 deletion website/docs/faqs/seed-custom-schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ seeds:
jaffle_shop:
schema: mappings # all seeds in this project will use the mapping schema by default
marketing:
schema: marketing # seeds in the `data/mapping/ subdirectory will use the marketing schema
schema: marketing # seeds in the `seeds/mapping/ subdirectory will use the marketing schema
```

</File>
2 changes: 1 addition & 1 deletion website/docs/faqs/testing-seeds.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ To test and document seeds, use a [schema file](configs-and-properties) and nest

## Example:

<File name='data/schema.yml'>
<File name='seeds/schema.yml'>

```yml
version: 2
Expand Down
4 changes: 1 addition & 3 deletions website/docs/reference/analysis-properties.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@
title: Analysis properties
---

Analysis properties can be declared in `.yml` files in:
- your `analyses/` directory (as defined by the [`analysis-paths` config](analysis-paths))
- your `models/` directory (as defined by the [`source-paths` config](source-paths))
We highly encourage you to define analysis properties in your `analyses/` directory (as defined by the [`analysis-paths` config](analysis-paths)).

You can name these files `whatever_you_want.yml`, and nest them arbitrarily deeply in subfolders within the `analyses/` or `models/` directory.

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/artifacts/manifest-json.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ All resources nested within `nodes`, `sources`, `exposures`, `macros`, and `docs
- `unique_id`: `<resource_type>.<package>.<resource_name>`, same as dictionary key
- `package_name`: Name of package that defines this resource.
- `root_path`: Absolute file path of this resource's package.
- `path`: Relative file path of this resource's definition within its "resource path" (`source-paths`, `data-paths`, etc.).
- `path`: Relative file path of this resource's definition within its "resource path" (`model-paths`, `seed-paths`, etc.).
- `original_file_path`: Relative file path of this resource's definition, including its resource path.

Each has several additional properties related to its resource type.
8 changes: 7 additions & 1 deletion website/docs/reference/commands/clean.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,10 @@ title: "clean"
id: "clean"
---

`dbt clean` is a utility function that deletes all folders specified in the `clean-targets` list specified in `dbt_project.yml`. This is useful for deleting the `dbt_modules` and `target` directories.
<Changelog>

- **v1.0.0:** `dbt_modules` has been replaced by `dbt_packages` by default for the [clean-target](clean-targets) for packages.

</Changelog>

`dbt clean` is a utility function that deletes all folders specified in the `clean-targets` list specified in `dbt_project.yml`. This is useful for deleting the `dbt_packages` and `target` directories.
9 changes: 8 additions & 1 deletion website/docs/reference/commands/seed.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,14 @@ title: "seed"
id: "seed"
---

The `dbt seed` command will load `csv` files located in the `data-paths` directory of your dbt project into your data warehouse.
<Changelog>

- **v1.0.0:** The default config for this command will now be `seed-paths` instead of `data-paths`.

</Changelog>


The `dbt seed` command will load `csv` files located in the `seed-paths` directory of your dbt project into your data warehouse.


### Selecting seeds to run
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/config-datatypes.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
description: A valid semantic version number, for example, 0.0.1

- name: directorypath
description: A valid path to a directory, relative to the `dbt_project.yml` file. For example, `data` would point to a subdirectory named `data`
description: A valid path to a directory, relative to the `dbt_project.yml` file. For example, `seeds` would point to a subdirectory named `seeds`

- name: [directorypath]
description: An array of directory paths
Expand Down
12 changes: 1 addition & 11 deletions website/docs/reference/configs-and-properties.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,19 +50,9 @@ Most configurations are "clobbered" when applied hierarchically. Whenever a more

In dbt, properties are declared in `.yml` files, in the same directory as your resources.

| Resource | Default directory | Defined by |
|-----------|---------------------------|------------------------------|
| models | `models/` | [source-paths](source-paths) |
| sources | `models/` | [source-paths](source-paths) |
| exposures | `models/` | [source-paths](source-paths) |
| seeds | `data/` or `models/` | [data-paths](data-paths) |
| snapshots | `snapshots/` or `models/` | [snapshot-paths](snapshot-paths) |
| analyses | `analyses/` or `models/` | [analysis-paths](analysis-paths) |
| macros | `macros/` or `models/` | [macro-paths](macro-paths) |

You can name these files `whatever_you_want.yml` and nest them arbitrarily deeply in subfolders within each directory.

(For backwards compatibility, the `models/` directory also supports seed, snapshot, analysis, and macro properties. We highly recommend that you define properties in dedicated paths alongside the resources they're describing.)
We highly recommend that you define properties in dedicated paths alongside the resources they're describing.

:::info

Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/dbt-jinja-functions/dispatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ title: "dispatch"
- **v0.19.2:** Limited rendering context for `dispatch` arguments. Includes backwards compatibility for widely used packages.
- **v0.20.0:** Parent adapters' macro implementations are included in search order. Formalized supported arguments.
- **v0.21.0:** All dispatched macros in the dbt global project include `dbt` namespace
- **v1.0.0:** The 'packages' argument is fully deprecated. Use `macro_namespace` and project-level `dispatch` config instead.

</Changelog>

Expand All @@ -19,7 +20,6 @@ __Args__:

* `macro_name` [required]: Name of macro to dispatch. Must be a string literal.
* `macro_namespace` [optional]: Namespace (package) of macro to dispatch. Must be a string literal.
* `packages` [DEPRECATED]: List of packages to search for implementations. Use `macro_namespace` and project-level `dispatch` config instead.

__Usage__:

Expand Down
12 changes: 9 additions & 3 deletions website/docs/reference/dbt_project.yml.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
<Changelog>

- **v1.0.0:** The default config name for `data-paths` is now [`seed-paths`](seed-paths), `source-paths` is now [`model-paths`](model-paths) and `modules-path` is now [`packages-install-path`](packages-install-path).

</Changelog>

Every [dbt project](projects) needs a `dbt_project.yml` file — this is how dbt knows a directory is a dbt project. It also contains important information that tells dbt how to operate on your project.

The following is a list of all available configurations in the `dbt_project.yml` file.
Expand All @@ -16,8 +22,8 @@ dbt uses YAML in a few different places. If you're new to YAML, it would be wort

[profile](project-configs/profile): profilename

[source-paths](project-configs/source-paths): [directorypath]
[data-paths](project-configs/data-paths): [directorypath]
[model-paths](project-configs/model-paths): [directorypath]
[seed-paths](project-configs/seed-paths): [directorypath]
[test-paths](project-configs/test-paths): [directorypath]
[analysis-paths](project-configs/analysis-paths): [directorypath]
[macro-paths](project-configs/macro-paths): [directorypath]
Expand All @@ -27,7 +33,7 @@ dbt uses YAML in a few different places. If you're new to YAML, it would be wort

[target-path](project-configs/target-path): directorypath
[log-path](project-configs/log-path): directorypath
[modules-path](project-configs/modules-path): directorypath
[packages-install-path](project-configs/packages-install-path): directorypath

[clean-targets](project-configs/clean-targets): [directorypath]

Expand Down
Loading