Dexy User Guide

Table of Contents

Introduction
Telling Dexy What To Do
Running Dexy
Filters
- Workspaces
- Templating Filters
Reporters
Colophon

{% macro select_soup(file, selector) -%} {% set soup_result = d[file].select(selector) -%} {% if len(soup_result) == 0 %} {{ debug(d[file]) }} {{ throw("No results for selector %s in file %s" % (selector, file)) }} {% endif -%} {{ print_soup(soup_result[0].contents) }} {% endmacro %}

{% macro print_soup(soup) %}

{% for line in soup %}{{ line }}{% endfor %}

{% endmacro %}

{% macro docsissue(issue_number) %} {% set issue_info = d['docs-issues.json'].from_json()[str(issue_number)] -%} {% if issue_info['state'] == "closed" -%} WARN: Issue {{ issue_number }} has been closed. Replace issue block with test. {{ debug("Issue %s has been closed but is still referenced in user-guide.adoc." % issue_number) }} {% endif -%}

[github] TODO: {{ issue_info.title }}

Please feel free to ask questions or add comments about this issue {{ issue_info.html_url }}[on github].

{% endmacro %}

{% macro issue(issue_number) %} {% set issue_info = d['issues.json'].from_json()[str(issue_number)] -%} {% if issue_info['state'] == "closed" -%} WARN: Issue {{ issue_number }} has been closed. Replace issue block with test. {{ debug("Issue %s has been closed but is still referenced in user-guide.adoc." % issue_number) }} {% endif -%}

[github] Desired Feature: {{ issue_info.title }}

Updated at: {{ issue_info.updated_at }}
Assigned to: {{ issue_info.assignee }}
Milestone: {{ issue_info.milestone }}

{% endmacro %}

{% macro source(qualified_name) -%} [code]

{{ d['modules.txt'][qualified_name + ':source'] }}

{% endmacro %}

{% macro example(qualified_test_name) -%} {% set file_name, test_name = qualified_test_name.split(":") -%} [shield]

{{ d['tests/' + file_name + '.py|pydoc'][test_name + '__test:source'] }}

{% endmacro %}

Introduction

{% set exs = "Clicking this won’t do anything, it’s just an example of what the icon looks like." -%}

This is the User Guide for dexy. It is intended to be a thorough guide to dexy features, but not an introductory tutorial or complete reference documentation. For links to other dexy documentation, please visit http://dexy.it/docs/

The User Guide is not complete yet, but it covers many topics and we have tried to focus on the most frustrating and confusing aspects of dexy as a priority (and hopefully we have also made them a little less frustrating in the process). In many places there are links to unit tests related to the content in this guide, you can click the shield icon to show/hide these sections. There are also embedded tickets which represent future planned development.

Telling Dexy What To Do

Dexy is an automation tool which will follow your instructions for what automated operations you would like done on your project. You give Dexy instructions about how to run your project using configuration files written using YAML syntax.

Configuration files should be named dexy.yaml.

The basic elements you can specify are:

file names/patterns and filters to apply to them
dependencies among documents (inputs are indented relative to parents)
settings for documents or filters (settings are indented under document)

Here’s a simple YAML config file demonstrating each of these: {{ example("configuration_test:basic_yaml") }}

The format for a document specification is a file name or file pattern followed by zero or more filters. Here are a few examples:

{{ d['examples/list-dict.yaml|idio|asciisyn']['document-specifications'] }}

You can leave off the initial asterisk if your file pattern starts with a dot, i.e. if you are matching all files with a given file extension. This is done for convenience and because if you want to start a string with an asterisk in YAML you need to wrap it in quotes.

File Names and Patterns

foo.txt instructs dexy to create a document named foo.txt where the source is a file named foo.txt in the project root. {{ example("configuration_test:config_txt_single_file") }}

bar/foo.txt instructs dexy to create a document named bar/foo.txt where the source is a file named foo.txt in the bar directory under the project root. {{ example("configuration_test:config_txt_single_file_in_subdir") }}

.txt instructs dexy to add all available files with ".txt" extension. You could also write this as "*.txt" (note the quotes) but dexy assumes entries starting with a dot are wildcard expressions and adds the asterisk for you. {{ example("configuration_test:config_txt_ext") }}

"*foo.txt" instructs dexy to add all files named foo.txt in any directory. Note that we need to wrap the expression in quotes because we are starting with an asterisk and can’t use the usual shortcut because we are matching a file name, not just an extension. {{ example("configuration_test:config_txt_wildcard") }}

Virtual Files

You can make dexy process a file which doesn’t really exist on the file system by using the contents setting: {{ example("configuration_test:virtual_file_contents") }}

contents is one of the available [_document_settings].

Custom Name Settings

You can change the name with which a file will be output by specifying the output-name document setting.

If the name starts with a slash / or contains any slashes it is assumed to be a full local (within the project) path to the desired destination of the file. If the name doesn’t contain a slash it is assumed you want to rename a file but keep it in its original directory.

Dexy will automatically apply string interpolation or string formatting to the name if it sees a % or { character. The environment for formatting will be populated with: - all document settings - name corresponding to the original document name - dirname corresponding to the original document directory - any local environment variables poopulated from a dexy-env.json file

Available variables are written to the dexy log at DEBUG level.

This example was run with these environment variables defined:

{{ d['examples/custom-names/dexy-env.json|idio|t'] }}

Here is the dexy.yaml:

{{ d['examples/custom-names/dexy.yaml|idio|t'] }}

And here are the input and output files:

{{ d['examples/templates.sh|idio|shint|asciisyn']['input-files'] }}

{{ d['examples/templates.sh|idio|shint|asciisyn']['output-files'] }}

Document Settings

A key: value pair indented under a document specification is a document setting. For example, output: True.

To see available document settings, you can use the dexy nodes command.

Files Without Filters

When no filters are specified for a document, the original contents of the file are made available to other documents, and the document will appear in Dexy’s final output unless the output setting has been set to False.

Filters

To specify filters, follow the filename/pattern with a pipe symbol | and a filter alias. You can chain as many filters as you want, in order, by adding more pipes and aliases. The first filter operates on the original contents of the source file, subsequent filters operate on the output from the previous filter.

You can run a source file through different filter combinations, and each will be a separate document in dexy.

{{ d['examples/list-dict.yaml|idio|asciisyn']['different-filters'] }}

Filter Settings

To customize filter settings, you need to first give the filter alias, and then a dictionary of the desired settings for that filter alias.

{{ d['examples/list-dict.yaml|idio|asciisyn']['filter-settings'] }}

To see available settings for a filter, you can use the dexy filters command with the -alias option. See [_filter_documentation].

Aliases

Document keys consist of the file name plus the filters. Document keys must be unique in Dexy. This poses a problem when you want to run a file through the same filters with different combinations of settings.

{{ d['examples/list-dict.yaml|idio|asciisyn']['without-aliases'] }}

To differentiate, you can place an alias filter at the end of your document key. This just needs to start with a hyphen, and then can optionally have some descriptive text.

Lists vs. Dicts

The syntax of the dexy.yaml file mixes list entries with dictionary (key: value) entries. When you have deep nesting of settings, such as in this example:

It helps to use a more dictionary-like syntax with curly braces and commas. This makes it more clear to read and also prevents compiler errors.

You can also try to reduce the amount of nesting by using named bundles.

Bundles

You can gather collections of documents together in named bundles and then refer to these bundles in other locations. This is helpful to make a more readable config file, reduce deep nesting and to re-use bundles of dependencies in different places.

You can use the dexy nodes command to view more information about the bundle node type:

{{ d['examples/filters.sh|idio|shint|asciisyn']['node-bundle'] }}

Scripts

Dexy guarantees that inputs are run before the documents which depend on them, but it doesn’t make any guarantees about the order in which sibling documents run. If you want to force dexy to run documents in a certain order, you do so by placing them in a bundle whose name is preceded by the script: prefix.

{{ d['examples/list-dict.yaml|idio|asciisyn']['script-bundles'] }}

The script: prefix instructs Dexy to construct a special kind of node which ensures its children are run in sequential order.

You can use the dexy nodes command to view more information about the script node type:

{{ d['examples/filters.sh|idio|shint|asciisyn']['node-script'] }}

Running Dexy

Command Line Help

Dexy’s command-line interface uses python-modargs to process commands and arguments. All arguments can take any number of dashes, so -r and --r and ---r all do the same thing.

The dexy help command gives you access to information about dexy commands:

The --all flag will print out all the available dexy commands:

You can get help on a particular command using the -on flag:

Setting Up and Running Dexy

It can be inconvenient if you accidentally run the dexy command somewhere you didn’t mean to, like in your home directory, so dexy won’t run unless it finds a .dexy directory in the current working directory. If you try to run dexy by accident, you’ll see a message like this:

{{ d['examples/run.sh|idio|shint|asciisyn']['dexy-without-setup'] }}

Running dexy setup creates the .dexy directory:

And now you can run dexy:

The .dexy directory is used to store working files, cached files, the dexy.log and some dexy reports:

{{ d['examples/run.sh|idio|shint|asciisyn']['show-hidden-files'] }}

Searching Dexy Batches

You can search the generated documents in the previous run via the dexy grep command, and you can get more detailed information about a document via the dexy info command.

There are some examples of using dexy info in this document, use control+F to search and find them.

Serving Generated Files

The dexy serve command runs a simple web server to serve the static assets generated by dexy. This command first looks for the output-site directory generated by the ws reporter, and if it doesn’t find this it looks for the output directory generated by the output reporter. It launches a static server and prints out the port on which the files will be served.

Cache Management

Dexy stores cached files in the .dexy directory to help speed up subsequent runs. You shouldn’t have to manage this manually, but if you want to force dexy to re-run everything you can empty the cache by running dexy with the -r option or running the dexy reset command.

Dexy might also create a .trash directory although it should remove this automatically.

Filter Documentation

The filters command lets you list all available dexy filters:

{{ d['examples/filters.sh|idio|shint|asciisyn']['list-filters'] }}

To print the full docstring and available settings for a particular filter, use the -alias option:

{{ d['examples/filters.sh|idio|shint|asciisyn']['filter-detail'] }}

Node Documentation

You use nodes (often without knowing it) when you write dexy.yaml files. Dexy guesses the node type you want, for example a PatternNode when you use a wildcard or implicit wildcard, a Doc when you specify an individual file. You can force a node to be of particular type by prefixing its name with the node type alias and a colon, as when you create a script node via script:screenshots.

The nodes command lets you list available node types:

To print the full docstring and available settings for a particular node, use the -alias option:

Template Env Documentation

The dexy env command gives you information about the template environment elements present. See the [_templating_filters] section.

Data Types Documentation

When Dexy processes a file and applies filters, each stage of processing is stored in a Data instance. There are different types of Data based on what sort of information you are storing.

The dexy datas command prints out a list of all data types:

By default, documents start out using the Generic data type and subsequent filters may change this depending on how the filters alter the data. You can see which data type is being used for a particular document by running the dexy info command:

You can then get more information about methods defined on the data type by running the dexy datas command, as suggested in the output of dexy info:

Custom data types are a way of exposing custom methods on data. For example the bs4 data type lets you run BeautifulSoup queries on HTML content of a document.

Additional Commands

Filters

This section deals with important concepts and features which are shared by all filters or groups of similar filters.

Workspaces

Many filters create a temporary workspace within the .dexy directory when they run. This workspace will mimic the directory structure of the main project and will be populated with the desired input files in their correct states (i.e. run through any applicable filters).

This provides a limited amount of isolation, in that processes are not changing files in the main project repository (unless there is a malicious or poorly-designed script), and any files generated as side effects do not clutter up the main project space.

In this example, a bash script is being run through the shint filter, and running the pwd command allows us to see the working directory where the code is being executed:

Check the filter documentation for each filter to see which of these workspace-related options are supported.

Using Working Directories

The use-wd boolean setting controls whether or not to create and populate a working directory and to set the process’s cwd to the working directory. The setting defaults to True. {{ example("process_filters_test:use_wd_option_defaults_to_true") }}

When use-wd is True (the default case), then a working directory is created within the .dexy/work directory. {{ example("process_filters_test:if_use_wd_true_code_runs_in_work_dir") }}

When use-wd is set to False, the code runs directly in the project root. {{ example("process_filters_test:if_use_wd_false_code_runs_in_project_home") }}

Including and Excluding Inputs

Working directories can be populated with the documents specified as dependencies or inputs. This can end up being a lot of files, and sometimes we want to control more precisely which files are copied. Several settings help to manage which files are copied. {{ source('dexy.filter.Filter.include_input_in_workspace') }}

workspace-exclude-filters A list of filter aliases. Input files which had these filters applied will be excluded.
override-workspace-exclude-filters A boolean specified on an input file. This input file will be included in working directories regardless of the parent’s workspace-exclude-filters setting.
workspace-include A list of filenames or wildcard patterns. These and only these will be written to the workspace. When this is set, workspace-exclude-filters and override-workspace-exclude-filters are ignored.

The workspace-exclude-filters setting takes a list of filter aliases and it doesn’t populate the working directory with any documents which include any of these filter aliases. So if jinja is in workspace-exclude-filters then a document named hello.txt|jinja will not be written to the working directory. {{ example("process_filters_test:workspace_exclude_filters_excluding_jinja") }}

To include all input files, set workspace-exclude-filters to an empty list. {{ example("process_filters_test:workspace_exclude_filters_no_excludes") }}

The workspace-exclude-filters setting defaults to ['pyg'] since usually syntax highlighted content is included in documents via templating, not via the file system. When pyg outputs image files or stylesheets, these have override-workspace-exclude-filters set to True by the filter. {{ example("process_filters_test:workspace_exclude_filters_pyg_defaults") }}

Making Extra Directories

Sometimes a tool expects a certain directory structure to exist when it runs, but this may not correspond to the directory structure of your project.

The mkdir and mkdirs settings let you specify extra directories which will be created in the working directory before the filter is run.

The mkdir setting creates a single directory based on a string. {{ example("process_filters_test:mkdir_creates_extra_directory_in_work_dir") }}

The mkdirs setting creates multiple directories based on a list. {{ example("process_filters_test:mkdirs_creates_extra_directories_in_work_dir") }}

Adding New Files

One of the reasons we tend to run scripts in their own working directories is because they generate extra files. LaTeX is notorious for generating .log, .aux, .bbl and a host of other files you usually aren’t interested in unless you need to debug somtehing. So by default Dexy just ignores any extra files which are created in working directories. If you need to do debugging, you can look in the working directory.

Sometimes, though, these extra files are useful and may even be the whole point of running a script. We may be generating a PNG file containing a graph, or a JSON or CSV file containing data.

The add-new-files setting controls how dexy treats these additional files. {{ source('dexy.filters.process.SubprocessFilter.add_new_files') }}

By default, add-new-files is False so Dexy ignores any new files which appear in the working directory. {{ example("process_filters_test:process_filters_have_add_new_files_false_by_default") }}

Some filters like casperjs which are almost always invoked for side effects will have add-new-files be True by default, so check the individual filter documentation. {{ example("process_filters_test:casperjs_has_add_new_files_true_by_default") }}

When add-new-files is False, no new files are added to dexy. {{ example("process_filters_test:if_add_new_files_false_new_files_not_added") }}

When add-new-files is True, new files are added to dexy. {{ example("process_filters_test:if_add_new_files_true_new_files_are_added") }}

The add-new-files setting can also be a list of expressions to match.

Entries in the list can be file extensions which should be added. {{ example("process_filters_test:add_new_files_list") }}

They can also be glob-style file patterns to match. {{ example("process_filters_test:add_new_files_pattern") }}

There is also an exclude-add-new-files setting which lets you list exceptions so you can skip directories, file names or patterns which otherwise would be included. {{ example("process_filters_test:exclude_add_new_files") }}

Additional Documents

Sometimes running a filter will cause extra documents to be added to the Dexy run. The split filter, for example, takes a HTML file and splits it into multiple files, each of which becomes an extra independent document. Extra documents may also be added as a result of the add-new-files setting (see the [_adding_new_files] section).

When new documents are added, you may wish to customize some of their settings or specify additional filters which should be applied to the new documents. You can do this via additional-doc-filters and additional-doc-settings.

The additional-doc-filters setting can be a string listing a single filter or single filter chain (a sequence of filters separated with pipes just as you would write in a dexy file) in which case every new document has these additional filters applied. {{ example("process_filters_test:additional_doc_filters") }}

If additional-doc-filters is a list, then separate new documents are created for each filter combination in the list. {{ example("process_filters_test:additional_doc_filters_list") }}

additional-doc-filters can also be a dictionary which maps file extensions to the filters which should be applied to those file extensions. If a file is found whose extension is not in the dictionary, then that file is added without any extra filters being applied. {{ example("process_filters_test:additional_doc_filters_dict") }}

The keep-originals boolean setting can be combined with additional-doc-filters and it instructs Dexy to also add the original files without any extra filters applied. {{ example("process_filters_test:additional_doc_filters_keep_originals") }}

The additional-doc-settings will apply extra settings to new documents. If this is a dictionary, then the entries in the dictionary are assumed to be setting names and values, and these will be applied to all new documents. {{ example("process_filters_test:additional_doc_settings") }}

additional-doc-settings can also be a list of lists where each element is a file extension and a dictionary of settings which will be applied to all files matching the extension. The ".*" extension can be used to provide default settings. {{ example("process_filters_test:additional_doc_settings_list") }}

Templating Filters

One of the most common things you will probably want do in dexy is to insert snippets of code into other documents using tags like {{ "{{ d['foo.py|pyg'] }}" }} using the jinja filter. The jinja filter is an example of a templating filter, and this chapter describes how these filters work and what elements are available for you to use in your documents.

A templating tool lets you insert content into a document template. Templating tools typically evaluate template tags like {{ "{{ foo }}" }} against an environment. An environment can be thought of as a hashmap like { "foo" : 123 }. The values in the hashmap can be simple values like 123, or they can be any type of object which is supported by the templating tool. Jinja2, for example, supports almost any kind of Python object including functions.

The TemplateFilter base class in Dexy prepares a giant hashmap containing various elements you might want to be able to refer to in your documents. It does so by running several Template Plugins, each of which returns a hashmap.

For example, the DexyVersion template plugin returns a hashmap with one entry, to let you refer to DEXY_VERSION (currently {{ DEXY_VERSION }}) in your documents. {{ source('dexy.filters.templating_plugins.DexyVersion') }}

All these individual hashmaps are combined together to generate the full environment.

Subclasses of TemplateFilter take this full environment and pass it to the templating system so it can be used to evalute template tags.

Choosing Template Plugins

By default, Dexy’s TemplateFilter includes all registered template plugins when it generates the template environment. (See the Cashew docs for details about how plugin registration works.) {{ example("template_plugins_test:by_default_dexy_runs_all_template_plugins") }}

If, instead, you want to specify which plugins to run, then you can use the plugins setting to specify a list of template plugin aliases to use. {{ example("template_plugins_test:use_plugins_attribute_to_specify_whitelist") }}

If you just want to exclude a few plugins, then you can use the skip-plugins filter setting to list template aliases you don’t want to be used. {{ example("template_plugins_test:use_skip_plugins_attribute_to_specify_blacklist") }}

The dexy env command prints all the environment elements which are available from running all the template plugins.

The Jinja Filter

The jinja filter is the recommended templating filter to use. It is the most widely tested and used. It uses the jinja2 templating system.

TODO verfiy this URL

Jinja has a lot of nice features, and you should familiarize yourself with the jinja template documentation.

One nice feature is jinja template filters. These are functions which you call by placing them after a pipe symbol like {{ "{{ 'foo' | indent }}" }}.

Yeah, we have a lot of "filters" and pipe symbols happening here, but that’s because both dexy and jinja are following similar conventions. Pipes are a classic unix concept and indicate that you take the output from one process and "pipe" it to the next process in a chain. It’s the same idea as when you write a pipe in a dexy.yaml file. I’ll refer to jinja’s implementation of filters as "jinja template filters" and dexy’s "jinja filter" as "the jinja dexy filter".

You can use all of the standard jinja template filters in dexy documents which you process through the jinja dexy filter. In addition to the built-in jinja template filters, dexy also implements some custom jinja template filters.

You can see the additional jinja template filters provided by dexy when you run dexy env. They are indicated by an asterisk.

Assertions in Documents

The jinja dexy filter allows you to make assertions about content you are including via jinja. For example, you may wish to assert that an error message does not appear in output, or you may wish to assert that a given HTML element is present, or that it contains certain text.

Here’s an example of a text file which includes output from a Python script:

{{ d['examples/assertions/docs-without-assertions.md'] }}

There’s no indication that anything is wrong when we run dexy, but the results show that there’s a problem:

{{ d['examples/assertions/docs-without-assertions-results.md'] | assert_contains("SyntaxError") }}

While Dexy does monitor for nonzero exit codes and notifies you (depending on the filter setting), not all errors will result in a nonzero exit code (for a variety of reasons). Making assertions is an alternative approach to ensuring that you are getting the content you expect from your inputs.

Now here’s the document with an assertion filter:

{{ d['examples/assertions/docs-with-assertions.md'] }}

When we run dexy, this is the result:

{{ d['examples/assertions.sh|idio|shint|asciisyn']['run-with-assertions'] }}

After fixing the file, we can run this again:

{{ d['examples/assertions/docs-with-assertions-results.md'] }}

The assertion filters return the original passed content if the assertion passes.

All available assertions can be seen by running the dexy env command:

{{ d['examples/assertions.sh|idio|shint|asciisyn']['list-assertions'] }}

Alternative Templating Filters

There are other subclasses of TemplateFilter available, although many of these are proofs of concept. If you have a reason not to use jinja then please get in touch to discuss alternatives.

Stubbing Out Dynamic Content

#cookbook #dummyfilter

Occasionally you may want to work on the prose of a document without worrying about the automation. For example, a technical writer may wish to concentrate on writing explanations which a developer will later help pair with examples. Or you may be working on a machine which doesn’t have everything configured for generating screenshots, which aren’t important to your work anyway. You want to be able to run subsequent filters like a markdown to HTML filter without having jinja crud get in the way.

This can be accomplished by adding an alternative configuration target which calls the dummyjinja filter instead of the jinja filter. The dummyjinja filter evaluates jinja tags but instead of using a real dexy environment, it just inserts an insert stub which allows subsequent filters to run without choking on curly braces.

Reporters

Reporters are what present the output from your dexy run.

To see the available reporters, run:

The default column indicates whether the reporter is enabled by default. You can control which reporters run via the reports argument to the main dexy command. If this is blank then each report’s default setting is used. If reports is not blank, it is interpreted as a string containing space-separated report aliases to be run in order.

To see full documentation and settings for an individual reporter, use the alias argument:

You can customize settings for a reporter using a dexyplugin.yaml file. An entry should start with reporter:alias: where alias is the reporter alias, and then have setting keys and new values listed in an indented block (YAML format, like dexy.yaml):

Output Reporter

The output reporter populates the output/ directory which contains short, canonical but potentially non-unique names for files. You can control whether a document appears in the output/ directory via the output setting. Setting this to True will ensure the document appears, setting this to False will ensure the document does not appear. The default behavior is determined by filters. If any filter’s default output setting is True then a document passed through that filter will default to a True setting. It’s easiest to start with defaults and then fine-tune to remove or add files from output/.

To change the name of the output/ directory, or to change any other report settings, requires writing a custom plugin, which can be in YAML or Python format.

The dir setting controls where the output is written. You can provide a different name for dir or even a different path, for example you can set the output to be in a gh-pages github branch to generate github pages content:

If you set the dir to be outside of the dexy project root then you will need to run dexy with the --writeanywhere setting, which you can pass on the command line or in a dexy.conf file:

Website Reporter

The Website reporter is similar to the Output reporter, and it uses the same criteria to determine whether it should include a file in output-site/ or not. The difference is that the Website reporter adds some features intended to help create a website-like output, such as applying HTML templates and help with creating navigation and links to other pages.

HTML Templates

By default the website reporter looks for templates named _template.html. For a given file foo/bar/baz.html the website reporter will look first in foo/bar/, then in foo/, then in the project root for a _template.html file, so you can override templates in subdirectories without needing to do any configuration.

You can change the default template name from _template.html to something else via the default-template setting.

You can also use the ws-filter setting on an individual file to specify an alternative template file. The ws-filter setting can also take boolean values to override the default behavior of whether a template should be applied or not. By default, templates are applied unless HTML header tags are already found in the HTML.

When writing a template, a {{ "{{ content }}" }} tag should indicate where the main content should go.

Here’s a simple HTML template:

{{ d['examples/website-simple/_template.html|idio|t'] }}

And here’s an index.html file containing some very simple content:

{{ d['examples/website-simple/index.html|idio|t'] }}

Here’s the index.html file in the output-site/ directory:

{{ d['examples/website-simple/output-site/index.html|idio|t'] }}

The project was configured with --reports setting including the ws reporter:

{{ d['examples/website-simple/dexy.conf|idio|t'] }}

Sectioned Documents

If HTML content has been split into sections then you can access section content via the content object, using either dictionary style access or jinja’s dot syntax (if the section name forms a valid python attribute name).

Here is an example of a HTML file with sections delimited using idio filter syntax:

{{ d['examples/website-sections/index.html'] }}

When this is put through the htmlsections filter the content will be split into sections:

{{ d['examples/website-sections/dexy.yaml|idio|t'] }}

The htmlsections filter is actually one of the aliases for the idio filter. It behaves a little differently than |idio: it splits content into sections following the idio rules, it doesn’t apply syntax highlighting, and then it sets output to True. It’s intended for use in just the scenario being described here.

The template can access these sections using either content.foo or content['foo']. The former is shorter, the latter works even when section names have spaces in them:

{{ d['examples/website-sections/_template.html|idio|t'] }}

Here’s the result:

{{ d['examples/website-sections/output-site/index.html|idio|t'] }}

You can also just call {{ "{{ content }}" }} on a sectioned document and dexy will insert all sections combined.

Navigation

The website template provides several elements to assist with creating website navigation.

page_title

The page_title element contains the title of the current page:

{{ select_soup('examples/website-navigation/_template.html', 'head') }}

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', 'head') }}

Current Page Location

The source element contains the canonical output filename of the current page. The current_dir and parent_dir elements refer to the directory containing the current page and its parent (if any), respectively:

{{ select_soup('examples/website-navigation/_template.html', '#pageinfo') }}

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', '#pageinfo') }}

Here’s the result for foo/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/index.html', '#pageinfo') }}

Here’s the result for foo/bar/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/bar/index.html', '#pageinfo') }}

Current Page’s Data Object

The s element contains the current page’s data object. (The same information is contained in the d element, you can use either one.)

{{ select_soup('examples/website-navigation/_template.html', '#s-object') }}

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', '#s-object') }}

Here’s the result for foo/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/index.html', '#s-object') }}

Here’s the result for foo/bar/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/bar/index.html', '#s-object') }}

Navigation & Node Objects

The Website reporter creates a Navigation object which represents the tree of directories in the project. Each directory is represented by a Node object.

Two node objects are made directly available within a website template, the root object representing the root of the project, and the nav object representing the node corresponding to the file being processed. The whole navigation tree is availalbe via the navtree objects.

Here are some basic methods and attributes of nodes, using the nav object representing the node for the document being processed:

{{ select_soup('examples/website-navigation/_template.html', '#nav') }}

The children attribute represents subdirectories. The docs attribute represents all documents found in the node’s directory. The index_page attribute corresponds to the index.html page, if there is one, in the node’s directory. The level attribute represents the number of directories above the node’s directory in the project.

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', '#nav') }}

Here’s the result for foo/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/index.html', '#nav') }}

Here’s the result for foo/bar/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/bar/index.html', '#nav') }}

Here’s these attributes for the root node:

{{ select_soup('examples/website-navigation/_template.html', '#root') }}

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', '#root') }}

It’s the same for foo/bar/index.html:

{{ select_soup('examples/website-navigation/output-site/foo/bar/index.html', '#root') }}

A common usage is to iterate over children to produce a list of subdirectories. You can create relative navigation on each page by using the nav object, or navgation for the entire site using the root object and iterating recursively over children.

There are some macros bundled with dexy which illustrate ways to use these elements to construct site navigation. You can use them as-is or copy them and modify them for your own needs.

The Navigation object is primarily used to generate the tree, but it can also be accessed via the navtree element. Its nodes attribute is a dictionary of all nodes accessed by path. The root attribute is the root node, which you can already access via root. There is a debug method which prints out all nodes and their attributes which you can include in a document.

{{ select_soup('examples/website-navigation/_template.html', '#navtree') }}

Here’s the result for index.html:

{{ select_soup('examples/website-navigation/output-site/index.html', '#navtree') }}

Links to Pages and Sections

The link() and section() methods allow you to create a HTML <a> link to a dexy page based on the page’s title or key and a section’s name.

The link() method links to a specified page, and optionally to a section on that page. The section() method lets you link to a specified section without needing to know which page it’s on.

Here’s an example of using these methods:

And here are the results:

{{ select_soup('examples/website-links/output-site/index.html', '#links') }}

The page entitled Home has its title specified in dexy.yaml:

{{ d['examples/website-links/dexy.yaml|idio|t']['home'] }}

The page entitled All About Foo, on which we link to the Foo and Welcome sections uses two filters wihch are useful to know about.

Yamlargs

The yamlargs filter lets you add some YAML metadata to the top of a page. The YAML is stripped off and the metadata is added to the page’s settings.

{{ d['examples/website-links/foo/index.html|idio|t'] }}

Soups

The soups filter uses BeautifulSoup 4’s HTML parser to look for any <h1> or <hn> tags and creates a section for each one it finds, and also creates an id attribute for each header element so they can be linked to.

{{ d['examples/website-links/output-site/foo/index.html'] }}

Listing Pages and Sections

You can use the dexy links command to print out a list of all valid keys which are possible to use as the target of the link() and section() commands:

{{ d['examples/website-links.sh|idio|shint|asciisyn']['links-command'] }}

Dexy will complain if you use a non-unique title or section name.

Other Plugins

In addiiton to the specific template elements covered above, the Website reporter makes use of many of dexy’s Template Plugins. You can configure this via the plugins setting.

Template Element Reference

The dexy info command provides information about indvidual documents in a dexy run, and this command takes a ws argument which adds customized documentation about the website template elements available, and their values, for this individual document.

Here’s an example. It includes website related tags, elements defined via template plugins, the contents of the Navigation tree, and methods available on nodes in the tree:

Run Reporter

The Run reporter generates an HTML file with information about the dexy run. It is enabled by default, you can find its output in .dexy/reports/run/index.html.

Colophon

This documentation was generated by Dexy and the Asciidoctor implementation of Asciidoc.

The source code lives on Github. If you have feedback or suggestions about this document please fell free to email info@dexy.it or open a github issue.

Here is the dexy.yaml file for this document:

Files

user-guide.adoc

Latest commit

History

user-guide.adoc

File metadata and controls

Dexy User Guide

Introduction

Telling Dexy What To Do

File Names and Patterns

Virtual Files

Custom Name Settings

Document Settings

Files Without Filters

Filters

Filter Settings

Aliases

Lists vs. Dicts

Bundles

Scripts

Running Dexy

Command Line Help

Setting Up and Running Dexy

Searching Dexy Batches

Serving Generated Files

Cache Management

Filter Documentation

Node Documentation

Template Env Documentation

Data Types Documentation

Additional Commands

Filters

Workspaces

Using Working Directories

Including and Excluding Inputs

Making Extra Directories

Adding New Files

Additional Documents

Templating Filters

Choosing Template Plugins

The Jinja Filter

Assertions in Documents

Alternative Templating Filters

Stubbing Out Dynamic Content

Reporters

Output Reporter

Website Reporter

HTML Templates

Sectioned Documents

Navigation

page_title

Current Page Location

Current Page’s Data Object

Navigation & Node Objects

Links to Pages and Sections

Yamlargs

Soups

Listing Pages and Sections

Other Plugins

Template Element Reference

Run Reporter

Colophon