_**Update:** This post is obsolete as of Hugo 0.43. See [Regis Philbert's great summary of best practices][rp] instead._ [rp]: https://regisphilibert.com/blog/2018/07/hugo-pipes-and-asset-processing-pipeline/ - - - - [Hugo][] is a great static site generator written in Go. I use it for this blog. Its advantages are that it's very fast, very easy to set up, and very flexible, but its disadvantage is that it doesn't have the mature community support that [Jekyll][] has. One example of that is that Hugo has no particular recommended route for managing a static asset pipeline. In this post, I'd like to explain how my personal pipeline works to see if it can help other Hugo users. [Hugo]: http://gohugo.io [Jekyll]: https://jekyllrb.com/docs/assets/   ## What is a pipeline? But first, what is a “static asset pipeline”? Static assets are bits of code for a webpage that don't change from one page to the next, such as CSS files, JavaScript files, webfonts, and icons. Often these files should be preprocessed somehow before being sent to a server: Sass files and ES6 files need to be converted to CSS and vanilla JS respectively, all files should be minimized, text files should be gzipped, cachebusting hashes should be added, and so on. "Cachebusting" is the name for giving a different file name to an asset whenever it changes. Let's say you were to change the CSS on your site but left the filename the same. A user who returns to your site might still have the CSS in their browser's cache and so see strange results due to a mismatch between the new HTML and the old CSS. By changing the filename every time you change an asset, you prevent this problem and have the further benefit that you can tell browsers to cache the asset [forever][], since it won't be changed while using the same name. One way to do this is manually, with names like "site-v1.css," "site-v2.css," and so on. The disadvantage here is that you have to remember to do it and to change both the filename and the template. Another way to do it is to use the date. This approach has the disadvantage that your users have to download new static files after every deploy even if you didn't change anything. The best practice for this problem is to use the "hash signature" of a file. With a hash, the same file input always produce the same hash output, and different file inputs are mathematically guaranteed to produce different outputs (except in the case of an astronomically unlikely "[collision][]"). For example, the MD5 hash of `Hello World!\n` is `8ddd8be4b179a529afa5f2ffae4b9858`, but `Hello, World!\n` is `bea8252ff4e80f41719ea13cdf007273`. Even simple changes in input produce radically different output. [forever]: http://bitsup.blogspot.de/2016/05/cache-control-immutable.html [collision]: https://en.wikipedia.org/wiki/Collision_attack ## How my pipeline works So, for my static site, I want several things to happen when I run the deploy command: - Clean out the results of prior builds - Recompile Sass files - Make hashes for static assets - Have Hugo build the site - Minify results from Hugo - Gzip results for Nginx - Rsync results to my server To run all of these commands, I use [Task][], a simple make-clone written in Go that uses YAML files to describe tasks to run. I put a gist of Github of [my current Taskfile.yml][gist] and related files. [Task]: https://github.com/go-task/task [gist]: https://gist.github.com/earthboundkid/e3716a0a89fab0484d0552a750c8cfb7 Let's talk about some of these steps in depth. ### Recompile Sass files My Sass isn't so special, but I do use the [autoprefixer][] postcss plugin, so that I don't have to manually write out prefixes like `-webkit` for older browsers. [autoprefixer]: https://github.com/postcss/autoprefixer sassc -m -t compressed src/scss/site.scss static/css/site.css postcss --use autoprefixer -o static/css/site.css static/css/site.css ### Making asset hashes To make asset hashes I use a tool I wrote called [scattered][]. The actual command is [scattered]: https://github.com/earthboundkid/scattered scattered -output data/assets.json -srcbasepath static '*.css' '*.png' '*.js' That command tells scattered to make a copy of any CSS, JS, or PNG file in the directory static to a new file with the hash of the asset included in its filename, and to output a file called assets.json into Hugo's data directory. For example, as I'm writing this, my assets.json has this bit of JSON (among its other lines): { "css/site.css": "css/site.b3173f80fd05f4d6729d733d076f42b5.css" } ### Having Hugo build the site Using Hugo's [data templates][] feature, I can refer to hashes listed in my assets.json in my templates. In a Hugo template, `{{ index .Site.Data.assets "css/site.css" }}` will be translated into the name of my latest hashed asset. [data templates]: https://gohugo.io/templates/data-templates/ (To keep from using the hashes in local development, I actually use a partial template that you can see in [my gist][gist].) ### Minify, Gzip, Rsync For minification, I use [minify][] from tdewolff, which can process HTML, CSS, JS, and XML (RSS). It strips out whitespace and other insignificant parts of a file so that users don't have to download any fluff. [minify]: https://github.com/tdewolff/minify To speed up my webserver a little bit more, I use Nginx's [gzip_static][] module. It works by having compressed files in a directory next to regular files and sending those to browsers that support [GZIP compression](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#Compressing_with_gzip). I create the gzip files with [monterey jack][], another tool that I've written in Go. [gzip_static]: http://nginx.org/en/docs/http/ngx_http_gzip_static_module.html [monterey jack]: https://github.com/carlmjohnson/monterey-jack Finally, I send the files to my server with rsync: rsync --verbose --progress --stats --compress \ --rsh=/usr/bin/ssh --recursive --times --perms --links \ --delete-during --exclude "*~" --exclude ".*" \ public/ mysite:/sites/epro ## Conclusion So, that's my pipeline. There's room for improvement. I don't use much JavaScript on my site, so I don't have an ES2017 transpiler in the pipeline, but it would be relatively simple to extend it to add that. I'm hosting my webfonts through Google Fonts, at least at the moment, so that's another simplification. If you have ideas for how to improve my pipeline or improve Hugo itself, let me know.