Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build script: no local biblio, use only Specref ids #3992

Merged
merged 10 commits into from
Aug 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 73 additions & 102 deletions scripts/md2html/md2html.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ complete control over formatting and syntax highlighting */
'use strict';

/**
@author Mike Ralphson <mike.ralphson@gmail.com>
**/
* @author Mike Ralphson <mike.ralphson@gmail.com>
**/

const fs = require('fs');
const path = require('path');
Expand Down Expand Up @@ -34,7 +34,7 @@ const md = require('markdown-it')({
linkify: true,
typographer: true,
highlight: function (str, lang) {
if (lang && hljs.getLanguage(lang)) { // && !argv.respec) {
if (lang && hljs.getLanguage(lang)) {
try {
return '<pre class="nohighlight"><code>' +
hljs.highlight(str, { language: lang }).value +
Expand Down Expand Up @@ -86,56 +86,9 @@ function preface(title,options) {
],
},
],
localBiblio: {
"OpenAPI-Learn": {
title: "OpenAPI - Getting started, and the specification explained",
href: "https://learn.openapis.org/",
publisher: "OpenAPI Initiative"
},
"OpenAPI-Registry": {
title: "OpenAPI Initiative Registry",
href: "https://spec.openapis.org/registry/index.html",
publisher: "OpenAPI Initiative"
},
//TODO: remove localBiblio once Specref PRs https://github.com/tobie/specref/pulls/ralfhandl are merged
"JSON-Schema-Validation-04": {
authors: [ "Kris Zyp", "Francis Galiegue", "Gary Court" ],
href: "https://datatracker.ietf.org/doc/html/draft-fge-json-schema-validation-00",
publisher: "Internet Engineering Task Force (IETF)",
status: "Internet-Draft",
title: "JSON Schema: interactive and non interactive validation. Draft 4",
date: "1 February 2013"
},
"JSON-Schema-05": {
authors: [ "Austin Wright" ],
href: "https://datatracker.ietf.org/doc/html/draft-wright-json-schema-00",
publisher: "Internet Engineering Task Force (IETF)",
status: "Internet-Draft",
title: "JSON Schema: A Media Type for Describing JSON Documents. Draft 5",
date: "13 October 2016"
},
"JSON-Schema-Validation-05": {
authors: [ "Austin Wright", "G. Luff" ],
href: "https://datatracker.ietf.org/doc/html/draft-wright-json-schema-validation-00",
publisher: "Internet Engineering Task Force (IETF)",
status: "Internet-Draft",
title: "JSON Schema Validation: A Vocabulary for Structural Validation of JSON. Draft 5",
date: "13 October 2016"
},
"JSON-Schema-Validation-2020-12": {
authors: [ "Austin Wright", "Henry Andrews", "Ben Hutton" ],
href: "https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00",
publisher: "Internet Engineering Task Force (IETF)",
status: "Internet-Draft",
title: "JSON Schema Validation: A Vocabulary for Structural Validation of JSON. Draft 2020-12",
date: "8 December 2020"
},
"SPDX": {
href: "https://spdx.org/licenses/",
title: "SPDX License List",
publisher: "Linux Foundation"
}
}
// localBiblio: {
// // add local bibliography entries here, add them to https://www.specref.org/, and remove them here once published
// }
};

let preface = `<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><title>${md.utils.escapeHtml(title)}</title>`;
Expand All @@ -154,9 +107,10 @@ function preface(title,options) {
preface += '</head><body>';
preface += '<style>';
preface += '#respec-ui { visibility: hidden; }';
preface += 'h1,h2,h3 { color: #629b34; }';
preface += '.dt-published { color: #629b34; } .dt-published::before { content: "Published "; }';
preface += 'a[href] { color: #45512c; }'; // third OAI colour is #8ad000
preface += '#title { color: #578000; } #subtitle { color: #578000; }';
preface += '.dt-published { color: #578000; } .dt-published::before { content: "Published "; }';
preface += 'h1,h2,h3,h4,h5,h6 { color: #578000; font-weight: normal; font-style: normal; }';
Comment on lines +110 to +112
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the new green: all headlines now use the same color, font-weight, and font-style and differ only in font-size.

preface += 'a[href] { color: #45512c; }';
preface += 'body:not(.toc-inline) #toc h2 { color: #45512c; }';
preface += 'table { display: block; width: 100%; overflow: auto; }';
preface += 'table th { font-weight: 600; }';
Expand All @@ -166,7 +120,7 @@ function preface(title,options) {
preface += 'pre { background-color: #f6f8fa !important; }';
preface += 'code { color: #c83500 } th code { color: inherit }';
preface += 'a.bibref { text-decoration: underline;}';
preface += fs.readFileSync(path.resolve(__dirname,'gist.css'),'utf8').split('\n').join(' ');
preface += fs.readFileSync(path.resolve(__dirname,'gist.css'),'utf8').split(/\r?\n/).join(' ');
preface += '</style>';
preface += `<h1 id="title">${title.split('|')[0]}</h1>`;
preface += `<p class="copyright">Copyright © ${options.publishDate.getFullYear()} the Linux Foundation</p>`;
Expand Down Expand Up @@ -247,68 +201,31 @@ let indents = [0];
for (let l in lines) {
let line = lines[l];

// remove TOC from older spec versions, respec will generate a new one
if (line.startsWith('## Table of Contents')) inTOC = true;
if (line.startsWith('<!-- /TOC')) inTOC = false;
if (inTOC) line = '';

// special formatting for Definitions section
if (line.startsWith('## Definitions')) {
inDefs = true;
bsFix = false;
}
else if (line.startsWith('## ')) inDefs = false;

// recognize code blocks
if (line.startsWith('```')) {
inCodeBlock = !inCodeBlock;
line += '\n'; // fixes formatting of first line of syntax-highlighted blocks
}

if (!inCodeBlock && line.startsWith('#')) {
let indent = 0;
while (line[indent] === '#') indent++;
let originalIndent = indent;

let prevIndent = indents[indents.length-1]; // peek
let delta = indent-prevIndent;

if (!argv.respec) {
if (delta===0) indent = lastIndent
else if (delta<0) indent = lastIndent-1
else if (delta>0) indent = lastIndent+1;
}

if (indent < 0) {
indent = 1;
}
if (argv.respec && (indent > 1)) {
indent--;
}
let newIndent = indent;
if (!argv.respec && (indent <= 2) && bsFix) {
newIndent++;
}

let title = line.split('# ')[1];
if (inDefs) title = '<dfn>'+title+'</dfn>';
line = ('#'.repeat(newIndent)+' '+title);

if (delta>0) indents.push(originalIndent);
if (delta<0) {
let d = Math.abs(delta);
while (d>0) {
indents.pop();
d--;
}
}
lastIndent = indent;
}

Comment on lines -265 to -304
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved unchanged down next to the other code block for headline processing

if (line.indexOf('<a name="')>=0) {
if (line.indexOf('<a name="parameterAllowEmptyValue"/>')>=0)
// fix syntax error in 2.0.md
line = line.replace('<a name="parameterAllowEmptyValue"/>','<span id="parameterAllowEmptyValue"></span>');
else {
line = line.replace('<a name=','<span id=');
line = line.replace('</a>','</span>');
// replace deprecated <a name="..."></a> with <span id="..."></span>
line = line.replace(/<a name="([^"]+)"><\/a>/g,'<span id="$1"></span>');
}
}

Expand Down Expand Up @@ -352,6 +269,7 @@ for (let l in lines) {
line = line.replace('consult http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4)','consult [[HTML401]] [Section 17.13.4](http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4)');
line = line.replace('[IANA Status Code Registry](https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml)','[[IANA-HTTP-STATUS-CODES|IANA Status Code Registry]]');
line = line.replace('[IANA Authentication Scheme registry](https://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml)','[[IANA-HTTP-AUTHSCHEMES]]');
line = line.replace('[JSON Reference](https://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03)','[[JSON-Reference|JSON Reference]]');
line = line.replace('[JSON Schema Specification Draft 4](https://json-schema.org/)','[[JSON-Schema-04|JSON Schema Specification Draft 4]]');
line = line.replace('[JSON Schema Core](https://tools.ietf.org/html/draft-zyp-json-schema-04)','[[JSON-Schema-04|JSON Schema Core]]');
line = line.replace('[JSON Schema Validation](https://tools.ietf.org/html/draft-fge-json-schema-validation-00)','[[JSON-Schema-Validation-04|JSON Schema Validation]]');
Expand All @@ -361,13 +279,15 @@ for (let l in lines) {
line = line.replace('[JSON Schema Specification Draft 2020-12](https://tools.ietf.org/html/draft-bhutton-json-schema-00)','[[JSON-Schema-2020-12|JSON Schema Specification Draft 2020-12]]');
line = line.replace('[JSON Schema Core](https://tools.ietf.org/html/draft-bhutton-json-schema-00)','[[JSON-Schema-2020-12|JSON Schema Core]]');
line = line.replace('[JSON Schema Validation](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00)','[[JSON-Schema-Validation-2020-12|JSON Schema Validation]]');
line = line.replace('[SPDX](https://spdx.org/licenses/)','[[SPDX]]');
line = line.replace('[SPDX](https://spdx.org/licenses/) license','[[SPDX-Licenses]]');
line = line.replace('[XML namespaces](https://www.w3.org/TR/xml-names11/)','[[xml-names11|XML namespaces]]');
line = line.replace('JSON standards. YAML,','[[RFC7159|JSON]] standards. [[YAML|YAML]],'); // 2.0.md only
line = line.replace('JSON or YAML format.','[[RFC7159|JSON]] or [[YAML|YAML]] format.');
line = line.replace(/YAML version \[1\.2\]\(https:\/\/(www\.)?yaml\.org\/spec\/1\.2\/spec\.html\)/,'[[YAML|YAML version 1.2]]');
}

// fix relative links (to examples)
//TODO: adjust when moving examples to a different repo
if (!inCodeBlock && line.indexOf('](../') >= 0) {
const regExp = /\((\.\.[^)]+)\)/g;
line = line.replace(regExp,function(match,group1){
Expand All @@ -376,6 +296,50 @@ for (let l in lines) {
});
}

// fix indentation of headings
// - make sure that each heading is at most one level deeper than the previous heading
// - reduce heading level by one if we're in respec mode except for h1
if (!inCodeBlock && line.startsWith('#')) {
let indent = 0;
while (line[indent] === '#') indent++;
let originalIndent = indent;

let prevIndent = indents[indents.length-1]; // peek
let delta = indent-prevIndent;

if (!argv.respec) {
if (delta===0) indent = lastIndent
else if (delta<0) indent = lastIndent-1
else if (delta>0) indent = lastIndent+1;
}

if (indent < 0) {
indent = 1;
}
if (argv.respec && (indent > 1)) {
indent--;
}
let newIndent = indent;
if (!argv.respec && (indent <= 2) && bsFix) {
newIndent++;
}

let title = line.split('# ')[1];
if (inDefs) title = '<dfn>'+title+'</dfn>';
line = ('#'.repeat(newIndent)+' '+title);

if (delta>0) indents.push(originalIndent);
if (delta<0) {
let d = Math.abs(delta);
while (d>0) {
indents.pop();
d--;
}
}
lastIndent = indent;
}

// wrap section text in <section>...</section> tags for respec
if (!inCodeBlock && argv.respec && line.startsWith('#')) {
let heading = 0;
while (line[heading] === '#') heading++;
Expand All @@ -384,16 +348,23 @@ for (let l in lines) {
if (delta>0) delta = 1;
let prefix = '';
let newSection = '<section>';
if (line.includes('## Version ')) {
const m = line.match(/# Version ([0-9.]+)$/);
if (m) {
// our conformance section is headlined with 'Version x.y.z'
// and respec needs a conformance section in a "formal" specification
newSection = '<section class="override" id="conformance">';
// adjust the heading to be at level 2 because respec insists on h2 here
// Note: older specs had this at h4, newer specs at h2, and all heading levels have been reduced by 1 in the preceding block
line = '#' + m[0];
delta = 1;
heading = 2;
}
if (line.includes('Appendix')) {
newSection = '<section class="appendix">';
}

// heading level delta is either 0 or is +1/-1, or we're in respec mode
/* respec insists on <section>...</section> breaks around headings */
// respec insists on <section>...</section> breaks around headings

if (delta === 0) {
prefix = '</section>'+newSection;
Expand Down
1 change: 1 addition & 0 deletions tests/md2html/fixtures/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.html text eol=lf
35 changes: 35 additions & 0 deletions tests/md2html/fixtures/basic-new.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><title>OpenAPI Specification v30.0.1 | Introduction, Definitions, &amp; More</title><meta name="description" content="The OpenAPI Specification (OAS) defines a standard, programming language-agnostic interface description for HTTP APIs."><link rel="canonical" href="https://spec.openapis.org/oas/latest.html" /><script src="../js/respec-w3c.js" class="remove"></script><script class="remove">var respecConfig = {"specStatus":"base","latestVersion":"https://spec.openapis.org/oas/latest.html","editors":[],"formerEditors":[],"publishDate":"3001-04-01T00:00:00.000Z","subtitle":"Version 30.0.1","edDraftURI":"https://github.com/OAI/OpenAPI-Specification/","shortName":"OAS","historyURI":null,"lint":false,"logos":[{"src":"https://raw.githubusercontent.com/OAI/OpenAPI-Style-Guide/master/graphics/bitmap/OpenAPI_Logo_Pantone.png","alt":"OpenAPI Initiative","height":48,"url":"https://openapis.org/"}],"otherLinks":[{"key":"Participate","data":[{"value":"GitHub OAI/OpenAPI-Specification","href":"https://github.com/OAI/OpenAPI-Specification/"},{"value":"File a bug","href":"https://github.com/OAI/OpenAPI-Specification/issues"},{"value":"Commit history","href":"https://github.com/OAI/OpenAPI-Specification/commits/main/versions/30.0.1.md"},{"value":"Pull requests","href":"https://github.com/OAI/OpenAPI-Specification/pulls"}]}]};</script></head><body><style>#respec-ui { visibility: hidden; }#title { color: #578000; } #subtitle { color: #578000; }.dt-published { color: #578000; } .dt-published::before { content: "Published "; }h1,h2,h3,h4,h5,h6 { color: #578000; font-weight: normal; font-style: normal; }a[href] { color: #45512c; }body:not(.toc-inline) #toc h2 { color: #45512c; }table { display: block; width: 100%; overflow: auto; }table th { font-weight: 600; }table th, table td { padding: 6px 13px; border: 1px solid #dfe2e5; }table tr { background-color: #fff; border-top: 1px solid #c6cbd1; }table tr:nth-child(2n) { background-color: #f6f8fa; }pre { background-color: #f6f8fa !important; }code { color: #c83500 } th code { color: inherit }a.bibref { text-decoration: underline;}/** * GitHub Gist Theme * Author : Louis Barranqueiro - https://github.com/LouisBarranqueiro */ .hljs { display: block; background: white; padding: 0.5em; color: #333333; overflow-x: auto; } .hljs-comment, .hljs-meta { color: #969896; } .hljs-string, .hljs-variable, .hljs-template-variable, .hljs-strong, .hljs-emphasis, .hljs-quote { color: #df5000; } .hljs-number { color: #008080; } .hljs-keyword, .hljs-selector-tag, .hljs-type { color: #a71d5d; } .hljs-literal, .hljs-symbol, .hljs-bullet, .hljs-attribute { color: #0086b3; } .hljs-section, .hljs-name { color: #63a35c; } .hljs-tag { color: #333333; } .hljs-title, .hljs-attr, .hljs-selector-id, .hljs-selector-class, .hljs-selector-attr, .hljs-selector-pseudo { color: #795da3; } .hljs-addition { color: #55a532; background-color: #eaffea; } .hljs-deletion { color: #bd2c00; background-color: #ffecec; } .hljs-link { text-decoration: underline; } </style><h1 id="title">OpenAPI Specification v30.0.1 </h1><p class="copyright">Copyright © 3001 the Linux Foundation</p><section class="notoc" id="abstract"><h2>What is the OpenAPI Specification?</h2>The OpenAPI Specification (OAS) defines a standard, programming language-agnostic interface description for HTTP APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic. When properly defined via OpenAPI, a consumer can understand and interact with the remote service with a minimal amount of implementation logic. Similar to what interface descriptions have done for lower-level programming, the OpenAPI Specification removes guesswork in calling a service.</section><section class="override" id="sotd" data-max-toc="0"><h2>Status of This Document</h2>The source-of-truth for the specification is the GitHub markdown file referenced above.</section>
<section><h1>Heading 1</h1>
<p>Text for first chapter</p>
<section class="override" id="conformance"><h2>Version 30.0.1</h2>
<p>This is the conformance section</p>
</section></section><section><h1>Heading 2</h1>
<p>Text for first section</p>
<section><h2>Heading 3</h2>
<p>Text for first subsection</p>
<pre class="nohighlight"><code>
<span class="hljs-punctuation">{</span>
<span class="hljs-attr">&quot;foo&quot;</span><span class="hljs-punctuation">:</span> <span class="hljs-literal"><span class="hljs-keyword">true</span></span>
<span class="hljs-punctuation">}</span>
</code></pre>
<pre class="nohighlight"><code>
text/plain
</code></pre>
<pre class="highlight "><code>
no language
</code></pre>
<table>
<thead>
<tr>
<th>Version</th>
<th>Date</th>
</tr>
</thead>
<tbody>
<tr>
<td>30.0.1</td>
<td>3001-04-01</td>
</tr>
</tbody>
</table>

33 changes: 33 additions & 0 deletions tests/md2html/fixtures/basic-new.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Heading 1

Text for first chapter

## Version 30.0.1

This is the conformance section

## Heading 2

Text for first section

### Heading 3

Text for first subsection

```json
{
"foo": true
}
```

```text
text/plain
```

```
no language
```

Version | Date
--------|-----------
30.0.1 | 3001-04-01
Loading