Build performance analysis #7949
Comments
|
FWIW, this is the change that seemed to make All that was was REMOVING the |
|
OK, reverting that change didn't affect the build time of |
|
Same results after a reboot on my side and a machine that isn't doing anything else, so I don't think the increase in time is related to that |
|
I suspect that this code (which exists on On every single, welcome, talk, speaker, speakers, and program page, that query gets executed twice just to build the One thing that could help (I don't quite understand how cumulative these things are) would be to combine all the partials in |
|
PR #7950 combines all the SEO templates into a single template; on my local machine this cut 30s off of the hugo build. If the PR passes, I'll merge it :) |
|
PR #7950 has been merged :) |
|
Just ran template metrics again, so these are the templates to continue to target: |
|
For the record, since these happen in parallel, it doesn't do a lot of good to optimize anything other than the program page first (it's the one that takes the longest). Even if we make One thing that is weird is that the build above took 157798 ms, which is 2.6 minutes, but the program pages report as taking 4 minutes. Go figure. |
|
Aha. The durations for template metrics are based on CPU time, not elapsed time. |
|
I had a thought on how we could improve the performance of the Right now, part of why it's so slow and intensive is this set of code: <div class="speaker-bio-talks">
<h3>{{ .Title }} at {{ $e.city }} {{$e.year}}</h3>
<ul class="list-group">
{{- $.Scratch.Set "speaker" .File.BaseFileName -}}
{{ range where (where $.Site.Pages "Type" "talk") ".File.Dir" "="
(print "events/" $e.name "/program/") }}
<!-- Now we can display stuff! -->
{{- range .Params.speakers -}}
{{- if eq . ($.Scratch.Get "speaker") -}}
{{- $.Scratch.Set "display" "true" -}}
{{- end -}}
{{- end -}}
{{- if eq ($.Scratch.Get "display") "true" -}}
<a href = "{{ .Permalink }}" class= "list-group-item
list-group-item-action">{{ .Title }}</a>
{{ $.Scratch.Set "display" "false" }}
{{- end -}}
{{- end -}} <!-- end range where -->
</ul>
</div>
</div>Let's walk through this and understand what it's doing (fyi, this is what shows the "Michael Bluth at Ponyville 2020" widget with the talks at that event from this speaker): $.Scratch.Set "speaker" .File.BaseFileNameThis takes the value of the range where (where $.Site.Pages "Type" "talk") ".File.Dir" "=" (print "events/"
$e.name "/program/")This executes a query against all the pages of type "talk" in the range .Params.speakersThis query executes on every "talk" page it finds in the event's program directory. It steps through all the values in the array {{- if eq . ($.Scratch.Get "speaker") -}}
{{- $.Scratch.Set "display" "true" -}}
{{- end -}}Stepping through each of the values in the array, if the current value of the array item matches what is in Scratch for "speaker", it sets a value of "true" to the Scratch variable "display". {{- if eq ($.Scratch.Get "display") "true" -}}
<a href = "{{ .Permalink }}" class= "list-group-item
list-group-item-action">{{ .Title }}</a>
{{ $.Scratch.Set "display" "false" }}
{{- end -}}If "display" is currently "true", it generates the HTML to create the link to the talk. It then sets "display" back to "false" So, there are a few things that are less than efficient. One thing that comes to mind is that the Scratch variable of "display" may no longer be needed. We can possibly just do this: {{- if eq . ($.Scratch.Get "speaker") -}}
<a href = "{{ .Permalink }}" class= "list-group-item
list-group-item-action">{{ .Title }}</a>
{{- end -}}I suspect there was a reason in the past to use the scratch variable for this (usually we use them because nesting That being said, the biggest problem is that for every speaker, we have to query and load all the talks from that event. It's a small thing but it can definitely add up, as there are a lot of Speaker pages. The possible solution is to add a bit of frontmatter to the This would basically embed the talks that this speaker is giving at that event inside the frontmatter of the speaker page. So let's say that Michael Bluth is giving two talks: one by himself and one with George Bluth. The frontmatter would look like this: In the template, we would no longer have to do the lookups at all on the pages of type But there's a problem. We need the title of the talk, and we won't have that from the frontmatter. So basically, I've just realized this potential solution doesn't work. I'm keeping this in this comment in case it spurs another idea. |
|
Spinning off older events to Here's what it used to be: now it looks like this: I have no idea why the average duration for |
|
I will say that thinking about this now and looking at the average duration being less than a second, archiving off to |
That page iterates over all the talks and all the speakers, I think? So pulling a bunch of old content out of the build should really reduce the length of those loops. |
|
Tim, you are a smart cookie. I had repressed the fact that I looped over ALL talks, not just the ones for that event :) |
|
After your edit this comment makes no sense. Serves me right for typing slowly 😀 |
|
NINJA EDIT! |
|
Comparing the last time this was ran to today (which includes more archiving and a hugo upgrade that is supposed to improve performance) Previous: Current: Definitely seems like the latest hugo upgrade improved performance? I am also wondering if things got better since the |
This might actually work - if we use |
Continuing the work @tgross did in devopsdays/devopsdays-theme#643
So yeah, that program page, wut.
Although the
partials/headstuff has gotten much worse somehow. This is what it looked like a year ago:So building the
headfor the pages used to take a total of 16 seconds, and now it takes over 2 minutes. Hmm.The text was updated successfully, but these errors were encountered: