Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Referencing a footnote multiple times lists the footnote once for each reference #241

Closed
martinhath opened this issue Mar 2, 2016 · 13 comments

Comments

@martinhath
Copy link

Originally posted here: gohugoio/hugo#1912

I don't know if this is intended or not (what would the return link do?), but I feel like duplicating the footnote is not the right way to go. The link even points to the uppermost footnote (instead of the one generated for this link).

An effect of this is if you try to reference the footnote itself in the footnote, Hugo crashes.

[^recursion]: See Recursion[^recursion]

output:

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0xc49b10, 0x16)
    /usr/lib/go/src/runtime/panic.go:527 +0x90
runtime.sysMap(0xc9cf300000, 0x1aec60000, 0x18e600, 0xfb8498)
    /usr/lib/go/src/runtime/mem_linux.go:203 +0x9b
runtime.mHeap_SysAlloc(0xf97920, 0x1aec60000, 0xc820194180)
    /usr/lib/go/src/runtime/malloc.go:426 +0x160
runtime.mHeap_Grow(0xf97920, 0xd7630, 0x0)
    /usr/lib/go/src/runtime/mheap.go:628 +0x63
runtime.mHeap_AllocSpanLocked(0xf97920, 0xd7630, 0x100)
    /usr/lib/go/src/runtime/mheap.go:532 +0x5f1
runtime.mHeap_Alloc_m(0xf97920, 0xd7630, 0xffffff0100000000, 0x7f3d43317e10)
    /usr/lib/go/src/runtime/mheap.go:425 +0x1ac
runtime.mHeap_Alloc.func1()
    /usr/lib/go/src/runtime/mheap.go:484 +0x41
runtime.systemstack(0x7f3d43317e28)
    /usr/lib/go/src/runtime/asm_amd64.s:278 +0xab
runtime.mHeap_Alloc(0xf97920, 0xd7630, 0x10100000000, 0x20)
    /usr/lib/go/src/runtime/mheap.go:485 +0x63
runtime.largeAlloc(0x1aec5e817, 0xc800000001, 0xc82066f5f8)
    /usr/lib/go/src/runtime/malloc.go:748 +0xb3
runtime.mallocgc.func3()
    /usr/lib/go/src/runtime/malloc.go:637 +0x33
runtime.systemstack(0xc820022000)
    /usr/lib/go/src/runtime/asm_amd64.s:262 +0x79
runtime.mstart()
    /usr/lib/go/src/runtime/proc1.go:668

goroutine 86 [running]:
runtime.systemstack_switch()
    /usr/lib/go/src/runtime/asm_amd64.s:216 fp=0xc82066f498 sp=0xc82066f490
runtime.mallocgc(0x1aec5e817, 0x9f2f00, 0xc800000001, 0xc83922c8e4)
    /usr/lib/go/src/runtime/malloc.go:638 +0x9c4 fp=0xc82066f568 sp=0xc82066f498
runtime.newarray(0x9f2f00, 0x1aec5e817, 0xc82066f608)
    /usr/lib/go/src/runtime/malloc.go:780 +0xc9 fp=0xc82066f5a8 sp=0xc82066f568
runtime.makeslice(0x9dda00, 0x1aec5e817, 0x1aec5e817, 0x0, 0x0, 0x0)
    /usr/lib/go/src/runtime/slice.go:32 +0x165 fp=0xc82066f5f8 sp=0xc82066f5a8
bytes.makeSlice(0x1aec5e817, 0x0, 0x0, 0x0)
    /usr/lib/go/src/bytes/buffer.go:195 +0x64 fp=0xc82066f648 sp=0xc82066f5f8
bytes.(*Buffer).grow(0xc8201ace00, 0x127, 0xbcaea0)
    /usr/lib/go/src/bytes/buffer.go:103 +0x282 fp=0xc82066f6f0 sp=0xc82066f648
bytes.(*Buffer).Write(0xc8201ace00, 0xc839228840, 0x127, 0x14e, 0x0, 0x0, 0x0)
    /usr/lib/go/src/bytes/buffer.go:131 +0x4b fp=0xc82066f748 sp=0xc82066f6f0
github.com/russross/blackfriday.(*Html).FootnoteItem(0xc82022e6c0, 0xc8201ace00, 0xc8202dbb54, 0xe, 0x3ac, 0xc839228840, 0x127, 0x14e, 0x0)
    /home/martin/go/src/github.com/russross/blackfriday/html.go:362 +0x17f fp=0xc82066f7a0 sp=0xc82066f748
github.com/spf13/hugo/helpers.(*HugoHtmlRenderer).FootnoteItem(0xc8202b9be0, 0xc8201ace00, 0xc8202dbb54, 0xe, 0x3ac, 0xc839228840, 0x127, 0x14e, 0x0)
    <autogenerated>:10 +0xb7 fp=0xc82066f800 sp=0xc82066f7a0
github.com/russross/blackfriday.secondPass.func1(0xc8201ace00)
    /home/martin/go/src/github.com/russross/blackfriday/markdown.go:468 +0x1bf fp=0xc82066f8c0 sp=0xc82066f800
github.com/russross/blackfriday.(*Html).List(0xc82022e6c0, 0xc8201ace00, 0xc8202b9e60, 0x1)
    /home/martin/go/src/github.com/russross/blackfriday/html.go:386 +0x78 fp=0xc82066f900 sp=0xc82066f8c0
github.com/russross/blackfriday.(*Html).Footnotes(0xc82022e6c0, 0xc8201ace00, 0xc8202b9e60)
    /home/martin/go/src/github.com/russross/blackfriday/html.go:348 +0x7d fp=0xc82066f938 sp=0xc82066f900
github.com/spf13/hugo/helpers.(*HugoHtmlRenderer).Footnotes(0xc8202b9be0, 0xc8201ace00, 0xc8202b9e60)
    <autogenerated>:12 +0x67 fp=0xc82066f968 sp=0xc82066f938
github.com/russross/blackfriday.secondPass(0xc8202c4900, 0xc8203e4000, 0x1152, 0x1335, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/russross/blackfriday/markdown.go:473 +0x174 fp=0xc82066f9c8 sp=0xc82066f968
github.com/russross/blackfriday.MarkdownOptions(0xc8202daa00, 0x14ac, 0x1500, 0x7f3d42ad8440, 0xc8202b9be0, 0xaa5f, 0x0, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/russross/blackfriday/markdown.go:384 +0x3a3 fp=0xc82066fa48 sp=0xc82066f9c8
github.com/russross/blackfriday.Markdown(0xc8202daa00, 0x14ac, 0x1500, 0x7f3d42ad8440, 0xc8202b9be0, 0xaa5f, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/russross/blackfriday/markdown.go:340 +0x8b fp=0xc82066fab0 sp=0xc82066fa48
github.com/spf13/hugo/helpers.markdownRenderWithTOC(0xc8202176e0, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/helpers/content.go:237 +0xa2 fp=0xc82066fb10 sp=0xc82066fab0
github.com/spf13/hugo/helpers.RenderBytesWithTOC(0xc8202176e0, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/helpers/content.go:360 +0x2eb fp=0xc82066fb58 sp=0xc82066fb10
github.com/spf13/hugo/hugolib.(*Page).renderContent(0xc820284000, 0xc8202daa00, 0x14ac, 0x1500, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/page.go:264 +0x27c fp=0xc82066fbe8 sp=0xc82066fb58
github.com/spf13/hugo/hugolib.commonConvert(0xc820284000, 0x7f3d42ad8000, 0xc8200155c0, 0x0, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/handler_page.go:116 +0xfc fp=0xc82066fd80 sp=0xc82066fbe8
github.com/spf13/hugo/hugolib.markdownHandler.PageConvert(0x0, 0x0, 0x0, 0xc820284000, 0x7f3d42ad8000, 0xc8200155c0, 0x0, 0x0, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/handler_page.go:59 +0x51 fp=0xc82066fdc0 sp=0xc82066fd80
github.com/spf13/hugo/hugolib.(*markdownHandler).PageConvert(0xc820165ec0, 0xc820284000, 0x7f3d42ad8000, 0xc8200155c0, 0x0, 0x0, 0x0, 0x0)
    <autogenerated>:107 +0xde fp=0xc82066fe18 sp=0xc82066fdc0
github.com/spf13/hugo/hugolib.(*MetaHandle).Convert(0xc8202b60c0, 0xbbf7a0, 0xc820284000, 0xc82011e000, 0xc820217080)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/handler_meta.go:77 +0x43d fp=0xc82066ff18 sp=0xc82066fe18
github.com/spf13/hugo/hugolib.pageConverter(0xc82011e000, 0xc8202170e0, 0xc820217080, 0xc8202772a0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:796 +0x108 fp=0xc82066ff80 sp=0xc82066ff18
runtime.goexit()
    /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc82066ff88 sp=0xc82066ff80
created by github.com/spf13/hugo/hugolib.(*Site).CreatePages
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:740 +0x63b

goroutine 1 [semacquire, 1 minutes]:
sync.runtime_Semacquire(0xc8202772ac)
    /usr/lib/go/src/runtime/sema.go:43 +0x26
sync.(*WaitGroup).Wait(0xc8202772a0)
    /usr/lib/go/src/sync/waitgroup.go:126 +0xb4
github.com/spf13/hugo/hugolib.(*Site).CreatePages(0xc82011e000, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:756 +0x801
github.com/spf13/hugo/hugolib.(*Site).Process(0xc82011e000, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:534 +0x3d7
github.com/spf13/hugo/hugolib.(*Site).Build(0xc82011e000, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:403 +0x47
github.com/spf13/hugo/commands.buildSite(0xc82066d8cf, 0x1, 0x1, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/commands/hugo.go:535 +0xcf
github.com/spf13/hugo/commands.build(0xc82066d9ee, 0x1, 0x1, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/commands/hugo.go:422 +0x2cd
github.com/spf13/hugo/commands.server(0xf88020, 0xc8201630f0, 0x0, 0x1, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/hugo/commands/server.go:155 +0x780
github.com/spf13/cobra.(*Command).execute(0xf88020, 0xc8201630a0, 0x1, 0x1, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/cobra/command.go:565 +0x63b
github.com/spf13/cobra.(*Command).ExecuteC(0xf86c20, 0xf88020, 0x0, 0x0)
    /home/martin/go/src/github.com/spf13/cobra/command.go:656 +0x56b
github.com/spf13/hugo/commands.Execute()
    /home/martin/go/src/github.com/spf13/hugo/commands/hugo.go:126 +0x71
main.main()
    /home/martin/go/src/github.com/spf13/hugo/main.go:24 +0x28

goroutine 17 [syscall, 1 minutes, locked to thread]:
runtime.goexit()
    /usr/lib/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 18 [runnable]:
gopkg.in/fsnotify%2ev1.(*Watcher).isClosed(0xc8200161e0, 0x0)
    /home/martin/go/src/gopkg.in/fsnotify.v1/inotify.go:64
gopkg.in/fsnotify%2ev1.(*Watcher).readEvents(0xc8200161e0)
    /home/martin/go/src/gopkg.in/fsnotify.v1/inotify.go:214 +0x39f
created by gopkg.in/fsnotify%2ev1.NewWatcher
    /home/martin/go/src/gopkg.in/fsnotify.v1/inotify.go:60 +0x3de

goroutine 7 [chan receive, 1 minutes]:
github.com/spf13/viper.(*Viper).WatchConfig.func1(0xc8200182a0)
    /home/martin/go/src/github.com/spf13/viper/viper.go:266 +0x24f
created by github.com/spf13/viper.(*Viper).WatchConfig
    /home/martin/go/src/github.com/spf13/viper/viper.go:267 +0x35

goroutine 19 [select, 1 minutes]:
github.com/spf13/viper.(*Viper).WatchConfig.func1.1(0xc8200161e0, 0xc820147b00, 0x2c, 0xc8200182a0)
    /home/martin/go/src/github.com/spf13/viper/viper.go:247 +0x539
created by github.com/spf13/viper.(*Viper).WatchConfig.func1
    /home/martin/go/src/github.com/spf13/viper/viper.go:263 +0x20a

goroutine 91 [chan receive, 1 minutes]:
github.com/spf13/hugo/hugolib.converterCollator(0xc82011e000, 0xc820217080, 0xc820216900)
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:813 +0xa9
created by github.com/spf13/hugo/hugolib.(*Site).CreatePages
    /home/martin/go/src/github.com/spf13/hugo/hugolib/site.go:743 +0x684
@ahogen
Copy link

ahogen commented Feb 14, 2017

It'd be really nice if this were fixed. I hate having a gazillion duplicate references, just because I chose to cite some info multiple times... Looks nasty and gross.

@arikroc
Copy link

arikroc commented May 31, 2017

@choueric What do you think?

@dmitshur
Copy link
Collaborator

dmitshur commented May 31, 2017

What do you think regarding #241 ?

This looks like a valid issue, but it's not completely clear to me where the responsibility of de-duplicating footnotes lies. It's likely within scope of blackfriday, but needs to be considered.

It doesn't seem anyone's actively working on it right now, so help is welcome.

Some initial questions are... How does CommonMark specification deal with this situation? Has anything been done about this in v2 (/cc @rtfb)?

@rdwatters
Copy link

From what I can tell, there is no proposed standardization for footnotes in the latest Commonmark spec: http://spec.commonmark.org/0.27/

That said, ordering and reuse of in-line references (i.e., footnotes) is critical in STM (scientific, technical, medical) publishing, so I hope this feature/fix is implemented soon. BTW, please consider this a belated thank you for putting together such a badass Markdown parser 😄

@rtfb
Copy link
Collaborator

rtfb commented Jun 1, 2017

I can reproduce the problem both in v1 and v2. Tried to come up with a quick fix, but failed so far; the problem is quite entrenched. I'll keep digging.

Here are the repro tests, in case anyone else wants to chime in:

func TestRepro241(t *testing.T) {
	tt := []string{
		`This [^citation] is a sentence [^citation].
[^citation]: https://google.com/`,
		"",
	}
	// TODO: test with HTML_FOOTNOTE_RETURN_LINKS
	doTestsInlineParam(t, tt, Options{Extensions: EXTENSION_FOOTNOTES}, 0, HtmlRendererParameters{})
}

func TestReproCrash(t *testing.T) {
	tt := []string{
		"This describes[^recursion]\n\n[^recursion]: See Recursion[^recursion]",
		"",
	}
	doTestsInlineParam(t, tt, Options{Extensions: EXTENSION_FOOTNOTES}, 0, HtmlRendererParameters{})
}

@arikroc
Copy link

arikroc commented Jun 6, 2017

@rtfb Any luck / are you still looking at it, or have you tabled it for now?

@rtfb
Copy link
Collaborator

rtfb commented Jun 6, 2017

Unfortunately, I couldn't find time for it. And I'm leaving for vacations in a few days, so unlikely to get back to it until end of June. If anyone's willing to give it a try, I'll be happy to review :-)

For a recursion case, start looking at infinite loop resulting from repetitive calls to inline() here, which eventually reaches link(), which in turn adds another copy to the p.notes array, ad nauseam.

@choueric
Copy link
Contributor

choueric commented Jun 7, 2017

I think there is a straightforward way to fix it, which is adding a stage variable in struct parser to distinguish the parsing and rendering process of footer from processes of header and block.

Because this infinite loop only happens when dealing with the footer, it can be avoided by not adding the existed link reference into p.notes if current process is footer according to stage.

There is another question: what's the proper text to replace the [^citation] in a footnote in footer, the note id of itself ?

@arikroc
Copy link

arikroc commented Jun 9, 2017

Nice @choueric. Looks like one of the tests/checks failed with your pull request #366

@choueric
Copy link
Contributor

choueric commented Jun 9, 2017

I checked the CI log and it is a failure of runing too long (10mins) for golang v1.2.2. The times of other versions are less than 10 minutes.

I suppose it's not a coding problem ?

rtfb added a commit that referenced this issue Jun 10, 2017
fix duplicate and recursive footnotes. (#241)
@rtfb
Copy link
Collaborator

rtfb commented Jun 10, 2017

Thanks to @choueric, this is now fixed. Give it a round of tests on v1 please (needs to be forward ported to v2).

@rtfb rtfb closed this as completed Jun 10, 2017
@dmitshur
Copy link
Collaborator

needs to be forward ported to v2

@rtfb, did you forget to add this issue to #348?

@rtfb
Copy link
Collaborator

rtfb commented Jun 25, 2017

No, I added the corresponding PR (#366). Thanks for keeping an eye on it!

willdollman pushed a commit to willdollman/blackfriday that referenced this issue Feb 27, 2021
Fix the infinite loop when there is a self-refer footnote by checking if
the reference is already added into parser.notes.

It also fixes of creating duplicate footnotes through this modification.

Add coresponding testcase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants