Missing files in remark-cli macOS pattern resolution #52
Comments
github-actions
bot
added
👋 phase/new
|
Hey @karlhorky! 👋 If you want to include MDX add the
I suspect this is globstar related. shopt -s globstarOn your mac shell.
This shouldn't be possible. In addition to address potential follow ups:
|
This comment has been minimized.
This comment has been minimized.
github-actions
bot
added
👎 phase/no
Nope, the PR only updated from
Been working for literally years before the bump to Can create a PR to my reproduction repo if you need hard proof. |
Looks like karlhorky/repro-remark-lint-no-dead-urls-folder-patterns#1 So it looks like, on macOS (without any further configuration or installing anything):
So on macOS, the quotes are important for |
@karlhorky your screenshot shows 8 files being updated, it's doing a lot more than just updating a dependency.
When the glob isn't quoted it is handled by your shell. |
Those are other files that I changed myself, also not related to updating any dependencies. It was me trying different options to fix the problem of files being missed. But I tested it first without those other unrelated changes. |
|
I still strongly suspect that |
|
It was in the previous But maybe something else caused a different Not in the diff of the PR though. |
|
I also tried out
/** @type {Omit<import('unified-engine').Options, 'processor'>} */
const config = {
extensions: ['mdx'],
plugins: [
'remark-mdx',
[ |
I'm glad it works.
The type for a config is |
Yeah, maybe what the old version of |
Thanks! I'll take a look at using this type instead. Does it also have the Edit: hm... /**
* Configuration passed to a Plugin or Processor
*/
interface Settings {
[key: string]: unknown
}Would be great to have some type checking for typos like /** @type {import('unified-engine').Preset} */
const config = {
settings: {
frail: true,
fraiil: true, // 💥 No error on typo
},
frail: true, // 💥 Object literal may only specify known properties, and 'frail' does not exist in type 'PresetSupportingSpecifiers'.Edit 2: I think that's why I went with the original type of So I think I'll stick with Edit 3: It seems that |
Settings are per-pipeline, and depend a lot on the parser and compiler used. |
|
Weird, so in my reproduction repo ( But then in our private monorepo on GitHub Actions (many more links), it succeeds silently without any output, although I know that there are broken links contained in some of the files: The broken links show up on my local macOS development environment though 🤔 I've tried patterns with I also thought it could be a path length issue, but that didn't cause the repro to stop working: Would be cool to be able to show the full list of URLs tested, even if they are passing - I tried Maybe that's a feature request for |
Even on ubuntu, this is controlled by your shell. shopt -s globstar
Check how you are running the script, and what the current wording directory
I'm pretty sure the issue is folder related, rather than URL. |
|
Interesting:
Run time: 5m 20s
Run time: 10m 27s Not sure why it would take twice as long though, since there is only the |
It is, if you want to turn that off for single files use
Usually when it takes significantly longer when using globs/patterns. |
|
Update: I have it working with the folder path now (the one like I did this by removing a large amount of the content in the (long) MDX file. So it appears that it's related to MDX content in the file - maybe something invalid or some link that is causing |
What indicates it is the MDX content specifically, over the length of the file? Another worth while question, how many URLs are in that document? The functional attempts to access all urls in the document simultaneously. remark-lint-no-dead-urls/lib/index.js Lines 148 to 149 in c1365e1 With a sufficiently large document you could be triggering your own rate limiting/DDoS protections.
Again, this plugin can literally only see a single file at a time remark-lint-no-dead-urls/lib/index.js Line 57 in c1365e1 |
I guess it could also be file length (485 lines), but I was not expecting that. To reiterate, the entire file content is not a problem when specifying a path directly to the file eg. It appears that the problematic bits are code blocks in the file.
There are 60 URLs in the file, and blocking is not a problem when I use a file path directly to the file.
Oh maybe I was misunderstanding earlier - are you saying that the directory path pattern I'm using above
Oh maybe I was misunderstanding earlier - are you saying that the directory path pattern I'm using above |
Do the code blocks contain a remark-lint-no-dead-urls/lib/index.js Line 101 in c1365e1 Could you share an anonymized/non-proprietary version of the file?
I'm saying that
@wooorm could that change the file meta data? remark-lint-no-dead-urls/lib/index.js Line 84 in c1365e1 @karlhorky are you testing any relative links which would depend on this? |
|
Interesting, it seems the behavior is "flaky" - when running the command below, sometimes The one Chrome extension link Here is an example of some of the HTML that we have in the code blocks (unusual HTML 3.2 code, teaching about legacy HTML): 16. Right click on the new `legacy` folder and create a new file called `index.html` with the following content:
```html
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<title>TodoMVC</title>
</head>
<body>
<script>
function rerender() {
frames[1].location.reload();
}
function click(id) {
for (var i = 0; i < state.numTodos; i++) {
if (state.todos[i].id == id) {
state.todos[i].complete = !state.todos[i].complete;
rerender();
return;
}
}
}
function addTodo(text) {
var nextId = state.nextId;
state.nextId++;
state.todos[state.numTodos] = makeTodo(nextId, text, false);
state.numTodos++;
rerender();
}
function clearCompleted() {
var newTodos = new Object();
var newNum = 0;
for (var i = 0; i < state.numTodos; i++) {
if (!state.todos[i].complete) {
newTodos[newNum] = state.todos[i];
newNum++;
}
}
state.todos = newTodos;
state.numTodos = newNum;
rerender();
}
function filter(newFilter) {
state.filter = newFilter;
rerender();
}
function makeTodo(id, label, complete) {
var todo = new Object();
todo.id = id;
todo.label = label;
todo.complete = complete;
return todo;
}
var state = new Object();
state.filter = 'all';
state.todos = new Object();
state.numTodos = 0;
state.nextId = 1;
</script>
<FRAMESET rows="75,*" border="0" frameborder="no" framespacing="0">
<FRAME src="header.html" scrolling="no">
<FRAME name=content src="todos.html" scrolling="no">
</FRAMESET></body>
</html>
```
Parts of the code are also highlighted red in the HTML highlighting in VS Code, because of the invalid / legacy code:
Maybe I should try running it again 5 or 10 times with the direct path to the file, with unchanged file content - maybe Edit: Since I saw that there were Edit 2: No, upgrading to |
|
Ok, that was actually easier than I thought to reproduce - just run the lint command multiple times in the GitHub Actions workflow and it immediately shows the intermittent false negatives / flaky behavior: |
Initial checklist
Affected packages and versions
remark-cli@12.0.1, remark-lint@10.0.0, remark-lint-no-dead-urls@2.0.0
Link to runnable example
https://github.com/karlhorky/repro-remark-lint-no-dead-urls-folder-patterns
Steps to reproduce
macos-latest(GitHub Actions workflow file) and observe the missed files in patternsremark . ...,remark ./src/pages/ ...,remark src/pages/**/*.mdx ...etc on macOS (only paths to singular files seem to be working on macOS)This doesn't happen on
ubuntu-latestThis didn't happen in versions before
remark-lint-no-dead-urls@2.0.0Runtime: Node.js latest LTS (20.17.0)
Expected behavior
Patterns are resolved to multiple files
Actual behavior
Patterns are resolved to a single file or zero files
Alternatives Considered
Maybe this is a general problem in
remark-cliupstream? (eg. maybe this issue should be transferred toremarkjs/remarkissues instead?)Runtime
Other (please specify in steps to reproduce)
Package manager
pnpm
OS
macOS
Build and bundle tools
No response
The text was updated successfully, but these errors were encountered: