Skip to content

Commit 91b0ac9

Browse files
KateVishnyaЕкатерина Московскаяholamgadol
authored
Added additional link checking. (#25)
* Updating processing the url and repo_url (#22) * update (includes.py): path processing * upgate (includes.py): update processing repo_url * update: add test * update (readme.md): add note for repo_url * update readme and test * resolve conflict * update: deleted print * fix: added error handling * fix: full links Co-authored-by: Екатерина Московская <ekaterina.markova@rt.ru> * update: bump version to 1.1.15 * fix (docs): minor fixes * fix: style errors * update includes.py * update: checking the included links * fix (includes.py): re, exceptions_characters * fix: add exceptions * update readme, add readme_ru * fix: includes.py * added test from_test_links.md * added test_include_internal_links * fix test_include_internal_links * fix tests * fix tests * fix tests * fix (Readme_ru.md): fixed typos * fix: correction of syntax errors * fix: correction of typos, added exception handling when opening url * techtechnical: rename file for test * fix: tests * fix: change timeout * fix: added flag allow_failure * fix (docs): style and grammar fixes * fix (tests): prepare for master branch * fix: typo in message --------- Co-authored-by: Екатерина Московская <ekaterina.markova@rt.ru> Co-authored-by: holamgadol <holamgadol@gmail.com>
1 parent 198d383 commit 91b0ac9

File tree

7 files changed

+581
-59
lines changed

7 files changed

+581
-59
lines changed

README.md

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ preprocessors:
2626
- includes:
2727
cache_dir: !path .includescache
2828
recursive: true
29+
stub_text: true
2930
extensions:
3031
- md
3132
- j2
@@ -43,8 +44,18 @@ preprocessors:
4344
`recursive`
4445
: Flag that defines whether includes in included documents should be processed.
4546

47+
`stub_text`
48+
: Flag that defines whether to insert an error message about adding content from the included file into the text of the document.
49+
Default `true`.
50+
51+
`allow_failure`
52+
: Flag that defines whether to raise warnings instead of errors.
53+
Default `true`.
54+
4655
`extensions`
47-
: List of file extensions that defines the types of files which should be processed looking for include statements. Might be useful if you need to include some content from third-party sources into non-Markdown files like configs, templates, reports, etc. Defaults to `[md]`.
56+
: List of file extensions that defines the types of files which should be processed looking for include statements.
57+
Might be useful if you need to include some content from third-party sources into non-Markdown files like configs, templates, reports, etc.
58+
Defaults to `[md]`.
4859

4960
`aliases`
5061
: Mapping from aliases to Git repository URLs. Once defined here, an alias can be used to refer to the repository instead of its full URL.
@@ -74,11 +85,13 @@ The preprocessor allows two syntax variants for include statements.
7485

7586
The **legacy** syntax is simpler and shorter but less flexible. There are no plans to extend it.
7687

77-
The **new** syntax introduced in version 1.1.0 is stricter and more flexible. It is more suitable for complex cases, and it can be easily extended in the future. This is the preferred syntax.
88+
The **new** syntax introduced in version 1.1.0 is stricter and more flexible.
89+
It is more suitable for complex cases, and it can be easily extended in the future. This is the preferred syntax.
7890

7991
Both variants of syntax use the `<include>...</include>` tags.
8092

81-
If the included file is specified between the tags, it’s the legacy syntax. If the file is referenced in the tag attributes (`src`, `repo_url`, `path`), it’s the new one.
93+
If the included file is specified between the tags, it’s the legacy syntax.
94+
If the file is referenced in the tag attributes (`src`, `repo_url`, `path`), it’s the new one.
8295

8396
### The New Syntax
8497

@@ -177,15 +190,19 @@ wrap_code="triple_backticks" code_language="yaml">
177190
: Full content of the ending heading when it’s necessary to include some part of the referenced file content. *The referenced heading will not be included.*
178191

179192
`from_id`
180-
: ID of the starting heading or starting anchor when it’s necessary to include some part of the referenced file content. The `from_id` attribute has higher priority than `from_heading`. If the `to_heading`, `to_id`, or `to_end` attribute is not specified, the preprocessor cuts the included content to the next heading of the same level. *The referenced id is included.*
193+
: ID of the starting heading or starting anchor when it’s necessary to include some part of the referenced file content.
194+
The `from_id` attribute has higher priority than `from_heading`. If the `to_heading`, `to_id`, or `to_end` attribute is not specified, the preprocessor cuts the included content to the next heading of the same level.
195+
*The referenced id is included.*
181196

182197
> **NOTE:** If you want `from_id` and `to_id` features to work with [anchors](https://foliant-docs.github.io/docs/preprocessors/anchors/), make sure that anchors preprocessor is listed *after* includes in foliant.yml.
183198

184199
`to_id`
185-
: ID of the ending heading or ending anchor when it’s necessary to include some part of the referenced file content. The `to_id` attribute has higher priority than `to_heading`. *The referenced id will not be included.*
200+
: ID of the ending heading or ending anchor when it’s necessary to include some part of the referenced file content.
201+
The `to_id` attribute has higher priority than `to_heading`. *The referenced id will not be included.*
186202

187203
`to_end`
188-
: Flag that tells the preprocessor to cut to the end of the included content. Otherwise, if `from_heading` or `from_id` is specified, the preprocessor cuts the included content to the next heading of the same level as the starting heading, or the heading that precedes the starting anchor.
204+
: Flag that tells the preprocessor to cut to the end of the included content.
205+
Otherwise, if `from_heading` or `from_id` is specified, the preprocessor cuts the included content to the next heading of the same level as the starting heading, or the heading that precedes the starting anchor.
189206

190207
Example:
191208

@@ -196,10 +213,15 @@ wrap_code="triple_backticks" code_language="yaml">
196213
Here `Some Heading {#custom_id}` is the full content of the heading, `custom_id` is its ID, and `one_more_custom_id` is the ID of the anchor.
197214

198215
`wrap_code`
199-
: Attribute that allows to mark up the included content as fence code block or inline code by wrapping the content with additional Markdown syntax constructions. Available values are: `triple_backticks`—to add triple backticks separated with newlines before and after the included content; `triple_tildas`—to do the same but using triple tildas; `single_backticks`—to add single backticks before and after the included content without adding extra newlines. Note that this attribute doesn’t affect the included content. So if the content consists of multiple lines, and the `wrap_code` attribute with the value `single_backticks` is set, all newlines within the content will be kept. To perform forced conversion of multiple lines into one, use the `inline` attribute.
216+
: Attribute that allows to mark up the included content as fence code block or inline code by wrapping the content with additional Markdown syntax constructions.
217+
Available values are: `triple_backticks`—to add triple backticks separated with newlines before and after the included content; `triple_tildas`—to do the same but using triple tildas; `single_backticks`—to add single backticks before and after the included content without adding extra newlines.
218+
Note that this attribute doesn’t affect the included content. So if the content consists of multiple lines, and the `wrap_code` attribute with the value `single_backticks` is set, all newlines within the content will be kept.
219+
To perform forced conversion of multiple lines into one, use the `inline` attribute.
200220

201221
`code_language`
202-
: Language of the included code snippet that should be additionally marked up as fence code block by using the `wrap_code` attribute with the value `triple_backticks` or `triple_tildas`. Note that the `code_language` attribute doesn’t take effect to inline code that is obtained when the `single_backticks` value is used. The value of this attribute should be a string without whitespace characters, usually in lowercase; examples: `python`, `bash`, `json`.
222+
: Language of the included code snippet that should be additionally marked up as fence code block by using the `wrap_code` attribute with the value `triple_backticks` or `triple_tildas`.
223+
Note that the `code_language` attribute doesn’t take effect to inline code that is obtained when the `single_backticks` value is used.
224+
The value of this attribute should be a string without whitespace characters, usually in lowercase; examples: `python`, `bash`, `json`.
203225

204226
### Optional Attributes Supported in Both Syntax Variants
205227

@@ -224,7 +246,8 @@ wrap_code="triple_backticks" code_language="yaml">
224246
By default, the starting heading is included to the output, and the ending heading is not. Starting and ending anchors are never included into the output.
225247

226248
`inline`
227-
: Flag that tells the preprocessor to replace sequences of whitespace characters of many kinds (including `\r`, `\n`, and `\t`) with single spaces (` `) in the included content, and then to strip leading and trailing spaces. It may be useful in single-line table cells. Default value is `false`.
249+
: Flag that tells the preprocessor to replace sequences of whitespace characters of many kinds (including `\r`, `\n`, and `\t`) with single spaces (` `) in the included content, and then to strip leading and trailing spaces.
250+
It may be useful in single-line table cells. Default value is `false`.
228251

229252
`project_root`
230253
: Path to the top-level (“root”) directory of Foliant project that the included file belongs to. This option may be needed to resolve the `!path` and `!project_path` modifiers in the included content properly.
@@ -243,9 +266,12 @@ Different options can be combined. For example, use both `sethead` and `nohead`
243266

244267
### The Legacy Syntax
245268

246-
This syntax was the only supported in the preprocessor up to version 1.0.11. It’s weird and cryptic, you had to memorize strange rules about `$`, `#` and stuff. The new syntax described above is much cleaner.
269+
This syntax was the only supported in the preprocessor up to version 1.0.11.
270+
It’s weird and cryptic, you had to memorize strange rules about `$`, `#` and stuff.
271+
The new syntax described above is much cleaner.
247272

248-
The legacy syntax is kept for backward compatibility. To use it, put the reference to the included file between `<include>...</include>` tags.
273+
The legacy syntax is kept for backward compatibility.
274+
To use it, put the reference to the included file between `<include>...</include>` tags.
249275

250276
Local path example:
251277

0 commit comments

Comments
 (0)