Add support for multiple contentDirs

[tylerbutler]

I am filing this issue per the suggestion from @bep here: https://discourse.gohugo.io/t/multiple-contentdirs/7462/4

Here's the scenario for supporting multiple contentDirs:

Most site content is committed to the site's git repo, in the content directory, but I also write draft posts from a variety of different devices like my phone or iPad. I use Dropbox to sync the posts between devices.

What I'd like to do is configure Hugo to look for content in multiple places, so I can add my Dropbox/hugo_content/ folder to the list and get content from both locations during a site build.

[mpcabd]

@bep could you check 22e336d when you can? It might be far from perfect but I wanted to review before I write tests and update the documentation.

[bep]

@mpcabd your code looks clean and simple enough, but, in my head, the problems with this approach are

1. it isn't well defined which file will win when the same file lives in two locations. You may have a defined order in filepath.Walk (but we may decide to go concurrently at some point) at the moment, but we also do reloading of content based on file system events, which must be handled somehow.
2. Also, saving the new content file to n locations on hugo new wasn't the intention behind this issue -- that would lead to lots of confusion.

[mpcabd]

Thanks for the quick review!

1. My implementation takes the first instance it finds in order of directories defined in the config, I had already tested that.
2. hugo new creates the new content in the first folder always, see the screenshot

Where the config is

baseURL = "http://example.org/"
languageCode = "en-us"
title = "My New Hugo Site"
contentDirs = ["content1", "content2"]
theme = "ananke"

[tylerbutler]

@mpcabd I see in your proposal that you've added a new config property contentDirs. For what it's worth, I'd prefer a design where the existing property is used. I.e. if I set contentDir to a single path, then it pulls content from that path alone. If I specify a list of paths instead, then it pulls content from all of them.

I'm new to Hugo, though, so it's possible that that is against some design principles I'm not aware of, or has technical reasons preventing it from being implemented. But in general I find it better to have a flexible configuration property rather than multiple configuration properties that interplay.

Assuming you do have to keep them separate properties:

1. What happens if I specify both contentDir and contentDirs?
2. What happens if contentDirs is a string, not a list of strings?
3. What happens if contentDir is a list of strings, not a string?

[tylerbutler]

Sorry, I looked more closely at the PR. I'm also new to Go so I'm muddling my way through it. 😄

Anyway, I think the answer to all three of my questions is that the user will see an error during site build.

[mpcabd]

@bep I'll take care of point 1 that you mentioned, and will clarify point 2 more in code as it might not be very clear what the code is trying to do.

@tylerbutler I will try to change some bits to test having contentDir only, but it might be harder to test since it won't have a static type. In my current implementation contentDir is a string while contentDir is a []string.

Thanks both for the review, I'll come back with more changes.

[mpcabd]

I just pushed c218a18 that will, I hope, attend to your concerns @bep, let me know what you think.

[vassudanagunta]

I'm a big believer in the KISS principle. I think Hugo should stick to one function, that is to transform source content (provided in as simply specified as possible structure of files) into a set of static web pages.

Editing or syncing source content from multiple devices should be solved elsewhere. There are many other avenues to address this. For example:

• One each device use an editor that can write to the single contentDirs. For example there are editors on all devices that can edit GitHub files, or DropBox files. So pick one location and then choose the editors that work with that. For example on iOS you can use Editorial + WorkingCopy to edit and publish Markdown your mobile device.
• use git init --separate-git-dir or git clone --separate-git-dirso that your contentDir working tree is in DropBox but your repository (.git dir) is not (to avoid syncing all the repo data, and to avoid allowing DropBox sync to corrupt it.). Edits made via your mobile device will be automatically seen on your main computer, from which you can commit the changes into the repo (and push into GitHub).
• Us a syncing tool. For example, one that pushes new content from DropBox into GitHub.

I personally use the first two options. They both work well. The first option for when I have to get something published immediately from my mobile device, and the second for when I'm simply doing authoring on my mobile device that can be published later, via my laptop.

I also think the OP's specific use case is problematic and misguided. The whole point of using git or GitHub for the contentDir is so that the source content is all under source control with full history and change tracking.

[bep]

I was planning to handle this as part of #4027 -- but that got to be a little bit too big a task. Will have to put this on hold for a little.

[yordis]

Any update on it folks?

I would like to leverage Hugo in a mono-repo environment.

[jmooring]

Seems like this was resolved with mounts:
https://gohugo.io/hugo-modules/configuration/#module-config-mounts

Can we close this issue?

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.