Non-MD files in the content directory (like jpg's and png's) end up as html's

[RubenN]

When I put non-md files in the content directory, they should just be copied over (untouched!).

Now they end up as html's

[spf13]

This is a bug, but I don't think you'll want the fixed behavior either.

/content is for documents that need processing
/static is for content that doesn't need processing

I think the fix here should be to ignore any filetype hugo doesn't know how to process.

Does this make sense? Is there a reason for putting non parsed files like pngs in content vs static?

[RubenN]

I would say that content should be what the term says: a collection of info
like documents, pictures, etc.
Static should be for the interchangable parts like layouting, css, etc.

The most important reason is to keep content together (pictures in a blog
for example).

I agree that ignoring is the most sensible thing to do!

2013/12/3 Steve Francia notifications@github.com

This is a bug, but I don't think you'll want the fixed behavior either.

/content is for documents that need processing
/static is for content that doesn't need processing

I think the fix here should be to ignore any filetype hugo doesn't know
how to process.

Does this make sense? Is there a reason for putting non parsed files like
pngs in content vs static?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/147#issuecomment-29737091
.

[robotamer]

I think the way content is interpreted here is: Files that need processing.
Static files are js, css but also image, video, pdf
To keep organized you could do something like
/static/image/blog/...

[robotamer]

Also there is one more interesting thing.

There are is a static folder and it has a sub folder called static as well.

like this: static/static/js
if you place a static file in to the first static folder it shows up in the root of the public folder.
Haven't tested this but you could create a folder in there as well.

looking like this `static/blog/image/foo.jpeg

Then your public folder will be organized the way you are asking for..

[spf13]

Correct.

Best,
Steve

-- 
Steve Francia
http://stevefrancia.com
http://spf13.com
http://twitter.com/spf13

On December 4, 2013 at 2:23:10 PM, Dennis T Kaplan (notifications@github.com) wrote:

Also there is one more interesting thing.

There are is a static folder and it has a sub folder called static as well.

like this: static/static/js

if you place a static file in to the first static folder it shows up in the root of the public folder.
Haven't tested this but you could create a folder in there as well.

looking like this `static/blog/image/foo.jpeg

Then your public folder will be organized the way you are asking for..

—
Reply to this email directly or view it on GitHub.

[luckyrandom]

Is there any update on this issue? It's inconvenient to put all images in separate directory, especially when plots are generated by scripts supplementing the post.

On the other hand, care must be taken when images or other assets are put in local directory. It may encourage the use of relative path, which may link to the wrong path when partial rendering is included by paginations or other pages.

I checked a few popular static site generator. wintersmith seems to be the only one to support local assets and relative links. It patches markdown renderer to support that, and it may not be a easy job if multiple renderers need to be patched.

[cburgdorf]

I think separating strictly by file type is backwards. If I have a picture only for one post, I want it to be next to my post.md file.

[spf13]

I see your point, but I'd like to clarify the intent here.

We're not separating by file type, but by purpose.

/Content is meant to be compiled by hugo.

/static will be copied without any processing

HTML and md are welcome in either location. If in /static they will be copied without being touched.

I'm willing to concede that this separation may not be necessary and perhaps determining intent only by file type would be appropriate, but it is more limited and would make the watching capability less efficient and more complicated.

Steve Francia
spf13.com
@spf13

On Apr 11, 2014, at 4:18 PM, Christoph Burgdorf notifications@github.com wrote:

I think separating strictly by file type is backwards. If I have a picture only for one post, I want it to be next to my post.md file.

—
Reply to this email directly or view it on GitHub.

[cburgdorf]

Understood. Would still like to have that feature :) I'll investigate the possibilities/impact down the road once I got my feet wet with the code base. Still struggling to get my environment set up.

[jcw]

Splitting files by how Hugo treats them is a bit of a software-centric approach, to be honest. When adding a blog post, it really is more convenient to create the post and everything it needs in one place. Even systems which split things up (WordPress, MarsEdit) have moved towards making it easy to do everything in one spot (usually via drag and drop, which is not feasible with Hugo, since it doesn't deal with the editing process itself).

I vote a big +1 to support images next to the text of a post/article.
This should be all about the workflow of people, not how Hugo sees things.

[spf13]

@jcw I think that's a pretty compelling argument. Let's plan on improving this for v0.12.

[RubenN]

Excellent, I couldn't agree more!

Op maandag 19 mei 2014 heeft Steve Francia notifications@github.com het
volgende geschreven:

@jcw https://github.com/jcw I think that's a pretty compelling
argument. Let's plan on improving this for v0.12.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/147#issuecomment-43458988
.

[jcw]

Thanks for considering this, Steve. I'm really looking forward to be able to use this - would like to convert a large weblog filled with images over to hugo. The combination of markdown, speed, and flexibility (plus the fact that you're actively developing it further) makes it a no-brainer for writing just about anything for the web, IMO.

[oscar-b]

@spf13 Just wanted to give my +1 to this as well. I started looking at migrating a clients portfolio site to Hugo (currently in ZenPhoto), but I quickly realized that there is no way to dynamically resize images. I would want to put the original images in albums, and being able to generate thumbnails and appropriate sized versions from the originals (think <picture></picture> with @2x and @3x versions).

In this context (and your own roadmap which lists "Dynamic image resizing via shortcodes"), content images should go to /content, since they need processing by Hugo :)

[mohae]

Some thoughts on this.

Support a per fileType processor that would process content files based on their file extension.
If a processor for that fileType is not implemented, the file is not processed.

This would allow for current functionality, in regards to .md files, support other markup languages by allowing processors to be added for them, and various image processing, when implemented.

I'm sure I'm missing some obvious stuff here. Any thoughts?

[stou]

I aggree with @jcw that separating content text, and content images and other media in separate folders feels awkward.

Maybe someting like the processing chain used by docpad could be useful:
https://docpad.org/docs/begin#a-note-on-rendering

Just an idea

[aledalgrande]

+1 for separating by context and not application/processing based

[spf13]

@stou ,

I've done some work on this already and there's a bit more to do still. I'm a bit confused about your comment about docpad. Docpad takes the exact same approach as Hugo does now by placing all binary files (images) into the /static directory.

I understand the chaining abstraction they provide, but to me that seems like a novelty. Why would anyone what to process files from one markup into another into another? (md/asciidoc/??) -> html seems sufficient.

[stou]

@spf13
About the chaining
I think I got mixed up with some of the comment in issue #320

I still think the chaining could be a usefull way to configure processing of files e.g.:

file.html.md --> file.html
file.js.coffee --> file.js
file.html.jade --> file.html
file.xml.jade --> file.xml
file.css.scss --> file.css

if you add more preprocessors in the future.
Consider this a feature request, but maybe the file extension idea could be used as part of the solution to issue #320 instead of adding a lot of configuration options / extra settings in front matter.

[kot-behemoth]

Any updates on this? Being able to have each post contain all the assets (be it images or scripts) next to the post's text would be invaluable. I'm thinking of moving to Hugo, so had a look at the docs, but couldn't find a confirmation of being able to do this just yet.

Thanks!

[mrkishi]

Any updates on the custom processors, as well? It's an idea that also interests me. For instance, one could optimize images on build.

Honestly, I found the content folder somewhat misleading as it's not meant to hold arbitrary site content, but processable inputs.

Also, if we are to think them as raw, unprocessed documents, it makes sense that's the place we'd place related data, full resolution images and etc., as well. They would then be processed on build: charts generated, images resized and optimized, html generated. Of course, a pass-through processor is still a valid processor. :)

As not every file could (easily) have a front matter, @stou's file extensions idea seems like an acceptable solution for processor selection, if slightly magic. Some ideas of what may be possible:

• /content/articles/lengthy-research.md: Article.
• /content/articles/lengthy-research/chart1.pngchart.json: json to png charting.
• /content/articles/lengthy-research/image1.optim.png: image optimization.
• /content/articles/lengthy-research/image2.optim-600x150.png: image optimization and resizing.
• /content/articles/lengthy-research/image3.png: pass-through?

[spf13]

I like this idea, but @stou's file extension idea only allows for a single result from the process. What if I want to make a thumbnail, medium size and full size from a single image?

[mrkishi]

How about introducing the concept of external alternatives to front matter? Appendix, lol?

This could make things slightly more consistent: every input (to be processed) must have a front matter, either embedded (markdown's front matter) or external (stand alone yaml and toml files that select the processor?). On the other hand, it adds to the project complexitivity and, potentially, build time.

• /content/articles/test.md: embedded front matter.
• /content/articles/test/image.png: means nothing by itself.
• /content/articles/test/image.toml: what to do with image.png.

This could get annoying pretty fast, though. Another option would be to add the processing options to the pages' front matter:

• /content/articles/test/image-{1,2,3}.png: do nothing by themselves.
• /content/articles/test.md: embedded front matter saying hey, include images 1, 2 and 3, using preprocessor x with arguments y and z.

I think this could work, but it would limit our ability to process arbitrary files: they would have to "belong" to an article. Perhaps on top of that there could exist support for stand alone .toml and .yaml files, that simply contain instructions for other files, similarly to the first suggestion but without the 1:1 toml/file relation.

[bguiz]

Think we might be getting a little side tracked here, with the discussion on preprocessing files. That's a nice to have for non markdown files, but for a first cut, I will be quite happy in *.md files are the only ones that are processed, and everything else just gets copied as-is.

Creating a rails or docpad style asset pipeline based on multiple file extensions is a nice to have, but I think simply being able to put images referenced in a markdown file in the same folder as the markdown file will aid authoring in Hugo quite a lot.

[jcw]

+1

[bep]

+1 what bguiz says

[derekperkins]

This seems like a great place to hook in an image processing pipeline, which has two major stages I can think of:

• Compression & Optimization
  We could check for the availability of some commandline tools like optipng, jpegtran, etc and use them if they are available. If not, the user doesn't know the difference. The user could specify in their config file whether they want to enable processing.
• Resizing & Cropping
  As mentioned in Image Resizing and Cropping #1014, it would be nice to provide this as an option. Off the top of my head, you could also use the config file in a similar fashion to WordPress to register specific file sizes that can be referenced elsewhere in the code. https://codex.wordpress.org/Function_Reference/add_image_size

This is obviously much more intensive than just generating html files, so it'd have to come with some sort of caching mechanism so it didn't have to copy and regenerate all the image files every single time.

[derekperkins]

The more I think about it, the more that seems a little intense for Hugo. While it would be super nice if it could fit the workflow, it may be more suited to a CI environment to go through and build all the right sizes on deployment. The only thing you lose by doing that is not being able to set the right width / height attributes inside your html tags.

[eduncan911]

This is not to address only images next to md files, but all file types.

We all agree that Hugo:

1. should not try to process files it doesn't know about in the /content directory.
2. move all files it finds in /content over to that same directory in the generated site, keeping the exact same structure.

E.g., side-by-side files:

/content/post/summer-fun/what-i-did-last-summer.md
/content/post/summer-fun/jet-skiing.png
/content/post/summer-fun/jet-skiing.mkv
/content/post/summer-fun/jet-ski-for-sale.zip

What you mention is another idea: file handlers. One could create their own handler and process the files with an external script or something if it matches an extension. But that's a different discussion for another thread. :)

On Thu, May 14, 2015 at 8:19 PM, Derek Perkins notifications@github.com
wrote:

The more I think about it, the more that seems a little intense for Hugo.
While it would be super nice if it could fit the workflow, it may be more
suited to a CI environment to go through and build all the right sizes on
deployment. The only thing you lose by doing that is not being able to set
the right width / height attributes inside your html tags.

—
Reply to this email directly or view it on GitHub
#147 (comment).

[spf13]

To everyone. I committed a generic file handler that simply places the files in the same path in the destination directory. This will enable people to place images and other files along side their content. There's still a bit more to do around this, but the fundamental issue / request in these 3 tickets is solved. It does need a bit of testing so I'd love everyone to check out head and play with it.

[eduncan911]

Awesome Steve!

Will let know in the next few days!

-E
On May 20, 2015 7:06 PM, "Steve Francia" notifications@github.com wrote:

To everyone. I committed a generic file handler that simply places the
files in the same path in the destination directory. This will enable
people to place images and other files along side their content. There's
still a bit more to do around this, but the fundamental issue / request in
these 3 tickets is solved. It does need a bit of testing so I'd love
everyone to check out head and play with it.

—
Reply to this email directly or view it on GitHub
#147 (comment).

[bep]

Works for me.

[FooSoft]

Thank you very much for this change.

[FooSoft]

Running into an issue when I have HTML files (specifically reveal.js stuff) along with markdown files:

ERROR: 2015/05/21 Error rendering site: Error(s) rendering pages: template: __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html:188: function "socketId" not defined
Available templates:
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/examples/embedded-media.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/test-markdown-element-attributes.html
ERROR: 2015/05/21   _internal/shortcodes/test.html
ERROR: 2015/05/21   partials/header.html
ERROR: 2015/05/21   partials/photos.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/examples/barebones.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/plugin/notes/notes.html
ERROR: 2015/05/21   
ERROR: 2015/05/21   _internal/shortcodes/ref.html
ERROR: 2015/05/21   _internal/_default/sitemap.xml
ERROR: 2015/05/21   partials/navbar.html
ERROR: 2015/05/21   _internal/google_news.html
ERROR: 2015/05/21   404.html
ERROR: 2015/05/21   partials/article-contents.html
ERROR: 2015/05/21   partials/github.html
ERROR: 2015/05/21   _internal/shortcodes/highlight.html
ERROR: 2015/05/21   _internal/_default/rss.xml
ERROR: 2015/05/21   _internal/disqus.html
ERROR: 2015/05/21   _internal/twitter_cards.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/plugin/markdown/example.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/examples/slide-backgrounds.html
ERROR: 2015/05/21   _internal/pagination.html
ERROR: 2015/05/21   _default/single.html
ERROR: 2015/05/21   _internal/shortcodes/relref.html
ERROR: 2015/05/21   _internal/shortcodes/figure.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/test-pdf.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/test-markdown-slide-attributes.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/test.html
ERROR: 2015/05/21   __research/search/presentation/index.html
ERROR: 2015/05/21   _internal/schema.html
ERROR: 2015/05/21   partials/article-summary.html
ERROR: 2015/05/21   partials/footer.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/index.html
ERROR: 2015/05/21   _internal/opengraph.html
ERROR: 2015/05/21   _default/list.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/examples/math.html
ERROR: 2015/05/21   __research/search/presentation/bower_components/reveal.js/test/test-markdown.html
CRITICAL: 2015/05/21 Error(s) rendering pages: template: __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html:188: function "socketId" not defined

It appears that Hugo tries to interpret them as templates.

[spf13]

Html files are interpreted as content files and will be parsed as such when in the content directory. 

If you would like them to not be parsed then they need to be in the static directory. 

This error seems to indicate that they are not in the content directory but in layouts as that is the only place where templates are looked for. Can you provide more complete paths for these files?

Best,
Steve

-- 

Steve Francia
steve.francia@gmail.com

Docker Inc
http://spf13.com 

     

On May 21, 2015 at 12:56:46 AM, Alex Yatskov (notifications@github.com) wrote:

Running into an issue when I have HTML files (specifically reveal.js stuff) along with markdown files:

ERROR: 2015/05/21 Error rendering site: Error(s) rendering pages: template: __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html:188: function "socketId" not defined
Available templates:
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/examples/embedded-media.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/test-markdown-element-attributes.html
ERROR: 2015/05/21 _internal/shortcodes/test.html
ERROR: 2015/05/21 partials/header.html
ERROR: 2015/05/21 partials/photos.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/examples/barebones.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/plugin/notes/notes.html
ERROR: 2015/05/21
ERROR: 2015/05/21 _internal/shortcodes/ref.html
ERROR: 2015/05/21 _internal/_default/sitemap.xml
ERROR: 2015/05/21 partials/navbar.html
ERROR: 2015/05/21 _internal/google_news.html
ERROR: 2015/05/21 404.html
ERROR: 2015/05/21 partials/article-contents.html
ERROR: 2015/05/21 partials/github.html
ERROR: 2015/05/21 _internal/shortcodes/highlight.html
ERROR: 2015/05/21 _internal/_default/rss.xml
ERROR: 2015/05/21 _internal/disqus.html
ERROR: 2015/05/21 _internal/twitter_cards.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/plugin/markdown/example.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/examples/slide-backgrounds.html
ERROR: 2015/05/21 _internal/pagination.html
ERROR: 2015/05/21 _default/single.html
ERROR: 2015/05/21 _internal/shortcodes/relref.html
ERROR: 2015/05/21 _internal/shortcodes/figure.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/test-pdf.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/test-markdown-slide-attributes.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/test.html
ERROR: 2015/05/21 __research/search/presentation/index.html
ERROR: 2015/05/21 _internal/schema.html
ERROR: 2015/05/21 partials/article-summary.html
ERROR: 2015/05/21 partials/footer.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/index.html
ERROR: 2015/05/21 _internal/opengraph.html
ERROR: 2015/05/21 _default/list.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/examples/math.html
ERROR: 2015/05/21 __research/search/presentation/bower_components/reveal.js/test/test-markdown.html
CRITICAL: 2015/05/21 Error(s) rendering pages: template: __research/search/presentation/bower_components/reveal.js/plugin/notes-server/notes.html:188: function "socketId" not defined

It appears that Hugo tries to interpret them as templates.

—
Reply to this email directly or view it on GitHub.

[FooSoft]

I ended up moving the stuff out of content into static and it's all groovy now. I was just under the impression that with this new change, when processing the content directory, Hugo would convert markdown files to HTML, and pass everything else through without processing it (I thought that the layout directory is the only place that would be processed for templates). I'll post a repro in a few.

[spf13]

The intent of /content is things are in there to be processed. This change does permit files to live alongside the content and if a dedicated handler isn't found it will just copy them over. This isn't the safest thing to do as in the future a handler may be defined for the file type and change the behavior. For instance, we could (and probably will) add a handler that does some automatic optimization to images... a generally but not universally desirable feature. If you want the images untouched...

The intent of /static is that things in there are not to be processed and will be copied over exactly as they are.

[spf13]

It's also the case that hugo supports html, markdown, asciidoc and rst currently as content files.

[eduncan911]

There's the list... Was searching the docs for what is currently processes.

Maybe someone with a little more time can update the docs and submit a pull
request. (FYI, the Hugo docs is in github as well, generated by Hugo
itself - by Steve I am guessing).
On May 21, 2015 8:30 AM, "Steve Francia" notifications@github.com wrote:

It's also the case that hugo supports html, markdown, asciidoc and rst
currently as content files.

—
Reply to this email directly or view it on GitHub
#147 (comment).

[FooSoft]

In case there is interest, I put up the website that illustrates the issue I was having here (5mb download):

http://foosoft.net/dl/hugo-repro.tar.gz

[zachio]

Can you make hugo ignore desktop.ini files? Google Drive puts that file in there if youre using Google Drive sync on Windows. So you have to manually delete all the desktop.ini files to get Hugo to work.

[bep]

@zachdyer ini-files should be covered by fc946de . Have you tested this with Hugo 0.14?

[zachio]

Actually no but good job dude yes!

On Wed, May 27, 2015 at 3:48 PM, Bjørn Erik Pedersen <
notifications@github.com> wrote:

@zachdyer https://github.com/zachdyer ini-files should be covered by
fc946de
fc946de
. Have you tested this with Hugo 0.14?

—
Reply to this email directly or view it on GitHub
#147 (comment).

[image: ZachDyerDesignHeaderGoogle.png]
Zach Dyer
1471 W South St.
Ozark, MO 65721
417.582.5445
www.zachdyerdesign.com
hosting.zachdyerdesign.com

Breach of confidentiality & accidental breach of confidentiality

This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the system manager.
This message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and
delete this e-mail from your system. If you are not the intended recipient
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.

[gima]

Does not work with permalinks.

Files are laid out in a directory path matching the pre-permalink URL, thus effecively ignoring permalinks completely.

Should a new bug report be opened or is this sufficient?

[fazalmajid]

Interesting discussion. I am in the process of converting a WP blog to Hugo, and my last big hurdles are image gallery handling and ATOM support. To be a world-class platform, Hugo needs fully integrated media management.

I am experimenting with a different approach than what was suggested above: treat JPEG files in /content as an external media type (the same way asciidoc, mmark, ReST and HTML are treated), copy the file as-is, but process its metadata (EXIF/IPTC/XMP/ICC) as implied front matter, and provide templates with functions that can perform standard image operations like thumbnail generation, resizing, watermarking, cropping, and so on.

This would allow templates to become full-fledged gallery generators, e.g. automatically generate a photoblog or galleries from photos dropped into /content/photos/ by Lightroom et al. This seems to me like more powerful and extensible, and aligned with the design of Hugo (and the limitations of Go in terms of lacking dynamic linking or similar extensibility mechanisms to implement plugins).

I'll share my work if it turns out to be successful. Happy Thanksgiving!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.