Reponsive Image Rabbit Hole – Part 3

Part of the Responsive Images series

It’s been a few days since I’ve written anything, and I’m going to make my tardiness up to you with some detailed science experiments. You can find part 1 and part 2 of this series here and here.

This article will be helpful for understanding responsive images in general, but also the @astrojs/image Astro component for generating responsive images in particular.1 I’m not the fastest learner and it generally requires me to poke around at things awhile before I understand how they work, so browser developer tools are a must in my process.

First off, I owe Tony Sull, author of Astro Image, a public apology. In the midst of converting some sites of mine from Eleventy to Astro, I found my images on my astro sites to look really low resolution compared to what I was getting in Eleventy with the eleventy-image plugin. Failing to understand how responsive images work (but thinking I did) and failing to perform the right tests and sanity checks made me think I was writing equivalent code and getting different results. I even went so far as to complain about the image quality from Astro Image in the Astro Discord a couple of times.

But I was wrong. Astro Image works just fine. It’s me, myself, and I that wasn’t working just fine.

When using Astro Image to generate responsive images, two factors have to be taken into consideration:

  1. The differences between Astro Image’s Picture component and Image component,
  2. How browsers choose responsive image sources for the user’s screen resolution/pixel-ratio and viewport size.

Astro Image’s Picture component

I showed in Part 2 of the Responsive Image Rabbit Hole how the Picture component can be used to generate multiple sources with their own srcsets for any given image, and what the resulting HTML would look like. I showed how the Image component would generate one width of image based on your input at whatever quality and format you specified. In contrast to Picture, you get one image width and one image format.

I also talked about needing to take screen resolution and pixel-ratio into account in order to generate images large enough to look good on high resolution displays when displayed at the desired width.

The result is that making images of high enough resolution on a high pixel-ratio screen with the Picture component is easy.

For example, here I want 600, 900, 1200, and 1500 pixel-wide versions of the image in avif, webp, and png, and I plan to display it at 200px wide.

<Picture
src={`/images/beer/${beer.image}.png`}
widths={[600, 900, 1200, 1500]}
aspectRatio="1:1"
sizes="200px"
formats={["avif", "webp", "png"]}
alt={`${beer.brewery} ${beer.name}`}
/>

Checking the result with Chrome developer tools, the image is displayed at 200px wide and the image downloaded is the 600px wide version.

600px wide image displayed at 200px width

Let’s additionally generate 200 and 400px wide images to see which one gets downloaded to display at my desired 200px width.

<Picture
src={`/images/beer/${beer.image}.png`}
widths={[200, 400, 600, 900, 1200, 1500]}
aspectRatio="1:1"
sizes="200px"
formats={["avif", "webp", "png"]}
alt={`${beer.brewery} ${beer.name}`}
/>

The answer is that the 400px image gets downloaded to display at 200px wide. Why? Simple… My iMac has a screen with a pixel-ratio of 2. It wants a 2x image for any given display width.

400px wide image displayed at 200px width

Now let’s use Chrome’s responsive device mode and look at what happens on an iPhone 12 Pro screen with the exact same Picture component parameters as above, still generating image widths at 200, 400, 600, 900, 1200, and 1500px wide.

600px wide image displayed at 200px width

Now we get the 600px wide image in order to display it nicely, because the iPhone 12 Pro has a screen pixel-ratio of 3.

You can use the console in your browser developer tools to show you the pixel-ratio for your computer’s screen as well as any simulated devices in responsive mode. Below are the pixel-ratios for my 27” 5K iMac screen and the screen of the iPhone 12 Pro that I used for the above tests.

Mac window device pixel ratio

iPhone 12 Pro window device pixel ratio

The Mac has a device pixel ratio of 2, so it needs and receives the 400px wide image to display at 200px, and the iPhone has a device pixel ratio of 3, so it wants and receives a 600px wide image to display nicely at 200px. If you have a picture element in your page and you are getting a larger image than you expect given what you set for the image size in the Picture sizes attribute, check your display window pixel ratio. Chances are it’s higher than 1.

Astro Image’s Image component

The Astro Image Picture component lets us easily overcome this, but it’s not so clear how to do so with the Image component. Image component accepts one width in its width attribute, and it has no concept of sizes. Setting the width (and height and aspect ratio if needed or desired) determines the size of image generated.

This means either one of two things: You’re going to get a fuzzy image on high resolution displays if you set width to the actual display width you want, or you’re going to set it higher than your desired display width and you’re just going to get an image displayed wider than desired and that is still fuzzy.

For example, let’s throw an Image component in the mix. Let’s give it a width of 200px because that’s what size we want our image to be. Let’s look at it next to the unchanged Picture component with the settings we last used above.

<Image
src={`images/beer/${beer.image}.png`}
width={200}
aspectRatio="1:1"
format="webp"
/>
<Picture
src={`/images/beer/${beer.image}.png`}
widths={[400, 600, 900, 1200]}
aspectRatio="1:1"
sizes="200px"
formats={["avif", "webp", "png"]}
alt={`${beer.brewery} ${beer.name}`}
/>

All these images are being displayed at 200 x 200px, as desired. But only the ones from the Picture component are nice and sharp, because it’s using 400px wide images for those. The Image component is generating a single 200px wide image and using that, and it’s noticeably blurrier than the 400px wide 2x image.

Image and Picture components with 200px width

Ok, fine, you say. Give image a width of 400 and call it done. That is in fact the correct answer (assuming your site never gets displayed on a screen with a higher pixel ratio than 2), but guess what? You’re also telling it to generate an img tag with a width attribute set to 400. I’m sure it’s not going to surprise you to see what happens next:

Image component with 400px width

To add insult to injury, not only is it displayed at twice the size you actually want, it’s still fuzzy by comparison because it’s not acting as a 2x image. It’s just a bigger 1x image on a screen with a pixel ratio of 2. Wah, wah.

Astro Image’s Image component doesn’t generate srcsets like the Picture component does, even though we saw in Part 1 of this Response Image Rabbit Hole that the HTML img tag does support srcsets.

If you use the Astro Image component, you will need to generate an image large enough to look good on screens with pixel ratios of 2 and 3, and then use CSS to control the display size.

In order to conduct these experiments for you, I got rid of any CSS related to image size. I’m going to put img class height and width properties back in, like this:

.beer-image img {
height: 200px;
width: 200px;
border: 3px solid var(--menu-surface);
border-radius: 10%;
aspect-ratio: 1;
margin-bottom: -0.6rem;
}

The result is the image from both the Image component and the Picture component are displayed at 200 x 200px, and both use 400 x 400px images as source files. Both now look equally sharp on my Retina display.

Image component with 400px width using CSS to set the display size

Lesson learned

The lesson I learned is very simple:

  • Use the Astro Image Picture component and make sure to use sizes to control the display size and use widths to generate sizes that will look good on high density displays, OR
  • Use the Image component, set the width that will look good on screens that want 2x and 3x images, and control the display size with CSS.

Footnotes

  1. For simplicity, I’ll reference it as Astro Image.

Reponsive Image Rabbit Hole – Part 2

Part of the Responsive Images series

In installment 1 of this responsive image topic, I talked about how the modern approach to giving site visitors the best combination of image file size and image quality comes down to generating a bunch of versions of the image and letting the browser choose. Further, the browser chooses by being given a choice of sources and/or srcset elements using the HTML picture or img. So the two step process for making image optimization possible for site visitors is: 1) Make a bunch of image files for each image you will display, 2) Create the HTML that allows the browser to know about and choose from the available options.

This sounds like a lot of work to do whenever you want to drop an image in a blog post. Who wants to do all this every time? The correct answer is no one. Anyone who does this manually for every image they want to inflict on their visitors doesn’t understand that the computer is there to work for them instead of the other way around. Fortunately, all modern web frameworks understand this and have solutions in place to tackle image optimization.

Astro Image plugin

In the case of Astro, the official answer to this is the @astrojs/image plugin. For simplicity I’ll just call it Astro Image from now on. To understand what image optimization plugins do, the Astro Image documentation says this:

Images play a big role in overall site performance and usability. Serving properly sized images makes all the difference but is often tricky to automate.

This integration provides <Image /> and <Picture> components as well as a basic image transformer, with full support for static sites and server-side rendering. The built-in image transformer is also replaceable, opening the door for future integrations that work with your favorite hosted image service.

There are a couple key points here. One is providing Image and Picture Astro components. That means you can generate all the html you need with a component like this:

<Picture
src={beerlatest}
widths={[800, 1200, 1800]}
sizes="(max-width: 800px) 95vw, 90vw"
formats={["webp"]}
alt="Latest episode beer list view"
/>

The resulting HTML will be the fully conceived HTML picture element with all the sources and srcsets you need. I added judicious use of carriage returns and tabs to make each of the elements more readable.

<picture class="astro-EI35XRNH">
<source
type="image/webp"
srcset="
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_1mD09L.webp 800w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z1gGQwg.webp 1200w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z22UqRY.webp 1800w
"
class="astro-EI35XRNH"
sizes="(max-width: 800px) 95vw, 90vw"
/>
<source
type="image/png"
srcset="
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_wgsuf.png 800w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z2b2h6I.png 1200w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_1yzNhO.png 1800w
"
class="astro-EI35XRNH"
sizes="(max-width: 800px) 95vw, 90vw"
/>
<img
src="/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z8LQlw.png"
class="astro-EI35XRNH"
loading="lazy"
decoding="async"
alt="Latest episode beer list view"
/>
</picture>

You may notice the different file names for each image resolution in the srcset for each of the two sources. If you guessed that the second part of what Astro Image does is generate the different image files for the browser to choose from, you win a virtual round of applause. For each of the widths you specify in the widths attribute of the Astro Image Picture component, Astro Image will generate an image of that width for that source’s file type. For local images, all heights will be calculated to keep the original aspect ratio, while for remote images, an aspect ratio must be provided for Picture to know what height to use.

Astro Image also has an Image component which you can use to create resized images in whatever format you desire. However, there are some limitations to the Image component in Astro Image. You can only generate one size (it does not make use of the HTML img srcset attribute) and one format. This means you need to remember my warning about high resolution screens at the bottom of part 1. This means if you use the Image component, you are going to certainly want to specify a width of 2-3x the pixel width you plan to display the image at.

I use this for my About page selfie image. Below is the Astro component code followed by the resultant HTML.

<Image
class="about-av"
src={av}
width={600}
format={"webp"}
alt="Scott Willsey"
quality={85}
/>
<img
class="about-av astro-AT6AUSG4 astro-UXNKDZ4E"
alt="Scott Willsey"
width="600"
height="600"
src="/assets/ScottLatest.cbf6b2e6_1ymKwq.webp"
loading="lazy"
decoding="async"
/>

I actually display the image at 300x300 (which I control in css) and it looks ok on high resolution screens because the image is 600x600.

The Retina wrinkle (again)

Remember last time when I said retina or high resolution displays throw a monkey in the wrench of displaying images? I fooled myself for a long time into thinking Astro Image wasn’t working correctly because I kept forgetting about it, even though I know very well about retina displays and their need for higher resolution images.

But now, because apparently I can’t quit using words, I’ll have to save that for part 3. I want to explain what I did on my Eleventy sites and what I was doing with my Astro sites, and how converting Friends with Beer from one to the other helped me understand my incredible ignorance about how all this works in the first place.

Stay tuned.

Reponsive Image Rabbit Hole – Part 1

Part of the Responsive Images series

Contents

Why image optimization?

Image optimization and how browsers can handle various methods of optimization is a pretty interesting topic. The basic idea is to give the browser options for any given image so that it can display them as intended by the site or article author, but with as little data transfer and image loading time as possible.

Browser variables that can affect image rendering efficiency are things like platform (mobile vs desktop-class browser), internet connection bandwidth, viewport size, screen resolution, and which image formats the browser supports. Ben Holmes wrote a great article on the topic of perfect image optimization that you should read which talks about some of these variables.1

How image optimization?

Ok, that’s not really proper English, but you get the point. Now we know we need to try not to send bigger images to the browser than necessary, but we still want them to look good. How do we do this?

The answer is: make multiple sizes and formats of each image and let the browser figure it out.

Modern browsers allow you to specify source sets for images. Given these image source sets, the browser can make a choice on which one it wants to request to perform the role of the image specified in the img tag.

Image source sets can be specified using either the HTML picture element or directly in the HTML img element itself.

HTML img element

With an HTML img, you can specify a srcset like this (example from MDN):

<img
srcset="elva-fairy-480w.jpg 480w, elva-fairy-800w.jpg 800w"
sizes="(max-width: 600px) 480px,
800px"
src="elva-fairy-800w.jpg"
alt="Elva dressed as a fairy"
/>

In the above example, there are two jpg versions of the same image available, one 480px wide and the other 800px wide. The browser will download the image size that makes sense for it given the viewport size and screen resolution. The value of the sizes attribute specifies that if the viewport is 600px or less, you’ll get a 480px wide image, otherwise you’ll get an 800px wide one.

HTML picture element

The Picture element is a little more involved but also more versatile (again from MDN):

<picture>
<source srcset="photo.avif" type="image/avif" />
<source srcset="photo.webp" type="image/webp" />
<img src="photo.jpg" alt="photo" />
</picture>

As you can see, instead of just one srcset, you can have multiple sources (one source for each image format option available), each with their own srcsets. These srcsets can in turn contain multiple image sizes. Here’s an example of this from my last post on this site:

<picture class="astro-EI35XRNH">
<source
type="image/webp"
srcset="
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_ZFUDaL.webp 300w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z2uWKfV.webp 600w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_1mD09L.webp 800w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z1gGQwg.webp 1200w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z22UqRY.webp 1800w
"
class="astro-EI35XRNH"
sizes="(max-width: 800px) 95vw, 90vw"
/>
<source
type="image/png"
srcset="
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z1MMor.png 300w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_13Es8j.png 600w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_wgsuf.png 800w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z2b2h6I.png 1200w,
/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_1yzNhO.png 1800w
"
class="astro-EI35XRNH"
sizes="(max-width: 800px) 95vw, 90vw"
/>
<img
src="/assets/BeerList-FCBA21C9-2F71-4051-B283-51452F68625D.d9a54970_Z8LQlw.png"
class="astro-EI35XRNH"
loading="lazy"
decoding="async"
alt="Latest episode beer list view"
/>
</picture>

This is an admittedly extreme example of generating 10 different images (5 sizes of webp, 5 sizes of png) just for one actual image on the website. I probably shouldn’t do this many resolutions in practice, and in fact I probably gain no benefit from doing this many. I probably only need 2 or 3 of those. But it does give you a good idea of the fact that each source in a picture element is a specific format of image, and inside that source, the srcset contains the different image sizes available for that format option.

The Sizes attribute works as it does with img, in this case specifying that up to 800px browser width, the image should be sized to take up 95% of the viewport width, and above 800px wide, only 90% of the viewport width. The context here is that my whole website content section is set to a maximum of 70ch or something like that, so even if you have your browser in fullscreen mode on a 5k iMac, the image will only be 90% of 70ch wide anyway.

The Retina wrinkle

Speaking of 5k iMac displays, there’s a wrinkle in this whole image optimization scheme: high-resolution displays (known as Retina displays in the Apple world). Basically for a given resolution, the screen uses double or triple the pixel density in order to display things sharp enough that the individual pixels can’t been seen by the human eye. What this means in terms of images on websites is that if you want to display a nice looking 800px wide image on a Retina display, you actually need a much higher resolution version of the image.

The image resolution issue was something I tripped over when fighting my image optimization strategy for both this site and the work-in-progress Astro version of Friends with Beer. I knew this fact but didn’t take it into account when I was looking at which size image was downloading for a given image. I thought the Astro Image component I was using was downloading a larger image than it should be given the size I wanted to display, but in fact the only thing that was broken was my understanding of how responsive images work.

I’ll tell that story in Part 2. In the meantime, here are some excellent links on image optimization.

Footnotes

  1. Ben now works for Astro, the framework that I use for this website and highly endorse.

Map Your Stuff

Part of the Astro series

One of the patterns you’ll see frequently in Astro is using the JavaScript array map function. Array.map() creates a new array that holds the results of performing whatever function you provide on each element of the original array.

Ok, that’s clear as mud.

But let’s say you have a podcast. Let’s say this podcast is called Friends with Beer, and let’s say you have a json file full of information about the beer you drink on your podcast. Let’s say it looks like this, repeated n number of times where n is the number of beers you’ve had on the podcast.

beer.json
[
{
"id": "OShp7ovkwb6F14mpRqFbw",
"name": "Hell or High Watermelon",
"brewery": "21st Amendment Brewery",
"image": "21stAmendmentBreweryHellOrHighWatermelon-EA669A2C-D404-422C-8495-AA268674CAA5",
"sortOrder": "0",
"episodes": ["14"],
"url": "https://www.21st-amendment.com/beers/hell-or-high-watermelon",
"rating": [
{
"host": "Scott",
"vote": "thumbs-up",
"description": "I wish it had more watermelon flavor, but it is a nice light wheat beer that's very pleasant."
}
]
},
...
]

Presumably you’d like to show the latest episode on your site’s home page with a little view featuring the beer that was consumed on that episode, like this:

Latest episode beer list view

First thing you need to do is grab the json file and find all beer associated with whatever episode is the latest. I have a file named beerlist.mjs that exports a beerList function this because I want to be able to get a beer list in other places on the site too.

beerList.mjs
import beer from "../../data/beer.json";
export function beerList(episode) {
const ep = episode ?? 0;
let beers = Array.from(beer);
return ep === 0
? beers
: beers.filter((beer) => beer.episodes.includes(String(episode)));
}

I can optionally pass in an episode number to filter the list by. If I do, I return an array of the episode-filtered beers. If no episode number is provided, I just return an array of the full list of beers.

Now I can create that view from the image above by importing my function and using it like this:

BeerList.astro
---
import { Icon } from "astro-icon/components";
import { Image } from "@astrojs/image/components";
import { beerList } from "./utilities/beerlist.mjs";
const { episode } = Astro.props;
const beers = beerList(episode);
---
<div class="beer-container">
{
beers.map((beer) => (
<div class="beer">
<div class="beer-image">
<a href={`/images/beer/${beer.image}.png`}>
<Image
src={`/images/beer/${beer.image}.png`}
width="300"
aspectRatio="1:1"
format="webp"
alt={`${beer.brewery} ${beer.name}`}
/>
</a>
</div>
<div class="beer-name">
<div class="brewery">{beer.brewery}</div>
<div>
<a href={`/bottle/${beer.id}`}>{beer.name}</a>
</div>
<div class="beer-details">
<span>
<Icon name="fluent:info-24-filled" />
</span>
<span>
<a href={`/bottle/${beer.id}`}>View Details</a>
</span>
</div>
</div>
</div>
))
}
</div>

The fun part is everything inside the map function. As you can see, my beerList function returns an array of beers. I map that so that for each beer in the array, I output the HTML inside the map function. This consists of an image of the beer, the brewery name, the beer name, and a link to view the page for that beer.

You can also make your maps more legible by creating a component to use inside the map. Here’s an example from the code for the paginated blog posts on this site that uses a Post component to do all the rendering of each post, just passing the individual mapped post to the component. It looks neater and is easier to understand, but it means creating another component. It just depends how much you want to break things down into separate components. If you need to show posts in a similar manner elsewhere besides the paginated list, you may want to do it by mapping your array items as props to a separate component, like below.

[page].astro
<Base title="test">
<section aria-label="Post list">
{
posts &&
posts.map((post) => {
return (
<Post content={post}>
<post.content />
</Post>
);
})
}
<Pager page={page} , pageSize={pageSize} />
</section>
</Base>

The Astro docs have a good example of using the map function in the “Converting markdown to MDX” guide and (more usefully for most people) the Astro.glob function documentation.

By the way, if you’re wondering why I treat episode number as a number sometimes and treat it like a string other times (inside beer.json, for example), rest assured you aren’t the only one wondering. I took that json file from my existing Eleventy site for Friends with Beer and didn’t think much about it. Refinements are certainly a valid consideration.

This Is a Blog

This is a blog.

After reading that, you may be thinking “Omg, he’s hit his head and now he’s reduced to touching everything he sees and saying its name out loud”, but here’s what I’m getting at: it’s not a wiki or a digital garden or a set of links for future reference. All of those things are useful, and they’re all things I’m considering as additions to this site. I’m just not sure yet what combination of these I’ll wind up with.

Wikipedia seems to define a personal wiki as something on a local computer or USB stick, to be carried around.1 But I reject their reality and substitute my own. There are many online personal wikis that are done very well and provide utility to both the site owner and anyone else who wanders into them. At some point, the line between personal and public wiki could get debatable, but it’s pretty easy to categorize a wiki made by an individual, for primarily that individual and anyone else who might care, containing information of interest to that specific individual, as a personal wiki.

I first started thinking about an online personal wiki when I was into Eleventy, and I came across Robb Knight’s personal wiki built in Eleventy and made public on GitHub. I was planning on taking his code and modifying it to suit my needs. But then more pressing projects happened, and then I started getting into Astro, and the Eleventy wiki never happened. However, I like the idea because I do want an always accessible categorized set of links I can rely on, and I like the wiki idea as a container for those links. Links + context, in other words. The result is that now I’m going to need to make my own in Astro, and if you think that sounds like a complaint, you don’t realize how much fun I’m having working with Astro. 😂

The Digital Garden concept is a little harder for me to decide what I want to do with. Digital gardens seem to be more like blogs than wikis in terms of post style, but more like wikis in terms of linking to concepts or being categorized in a certain way. It’s very nebulous. Also it seems like digital gardens are more about reading experience and less about finding information quickly or efficiently. I’ve never seen a digital garden yet where I could find what I thought was the root of a topic or category and dig into it. It’s more like wandering around finding things randomly, which doesn’t really meet my needs for linking to things in an organized manner nor for blogging things that are semi-easy to follow as a narrative.

And let’s be honest, there are only so many hours in the day and I already have put off making a wiki for months, at least. So I think right now I’ll pursue a strategy of making a personal wiki site subdomain as well as slowly adding in some of my posts from the previous incarnation of the blog to this one.

What say ye, dear reader?

Footnotes

  1. Strangely enough for a wiki site that’s dependent on being online… or maybe it’s not so strange at all.

RSS, Astro, and Me – Part 2

Part of the Astro series

As I mentioned in Part 1 of this installment, while trying to modify my site RSS feed to contain the full body of each post in my feed items, I ran into an inconvenient truth about how MDX exposes its file content as a component and how I could not use that component outside of an Astro component. JavaScript just doesn’t know what it is. Support for the MDX Content component has to be built into whatever framework you’re using.

Astro, of course, is built to use MDX and take full advantage of the MDX Content component, so Astro Discord member Chris Adiante proposed I simply use an Astro component to create the RSS (with full access to the MDX content) and then have it write the rss file to the file system. Since my site is Astro SSG (fully static, only changing when I rebuild the site) and not Astro SSR (server-rendered on demand), I can use this technique without any problems.1

By the way, Chris is the creator of the really amazing looking Astro M²DX remark plugins. If you’re using MDX with Astro, you should definitely give these a look!

The Strategy

In order to write an RSS feed using an Astro component, the Astro component has to get called – or to put it another way, it has to be used in a page. It can’t just sit in a folder in src. Also, it should be called once and once only. It can’t be put inside a layout file used by multiple pages because I want it written just once during the build. And finally, it has to have no effect on the rendering of the page that it’s in. Its purpose is to generate the RSS for all site posts and then write that RSS to a file called rss.xml. It has nothing that should be displayed, and it cannot alter the output of the page that hosts it.

To meet these requirements, I decided to use this Astro component in index.astro, the Astro page that gets built into index.html.

The Components

I use two components to create my RSS file, in the manner suggested by Chris Adiante: WriteFile.astro and RssXml.astro. RssXml.astro generates the RSS and WriteFile takes its output and dumps it to disk in the form a file called rss.xml.

RssXml.astro

The way RssXml.astro works is dictated by the fact that in order to render, it needs to directly output some html tags outside a javascript loop. This is because Astro components write html. It’s how they work.

The fact that Astro components write html should not be overlooked because it can also mess with the actual RSS XML generated, a truth that caused me much grief until I learned about Fragments and set:html. Using them in the combination <Fragment set:html="" /> outputs RSS XML that isn’t messed with by Astro trying to ruin the format of XML link elements, for example.

src/components/RssXml.astro
---
import config from "config";
import path from 'path';
import { rfc2822 } from "../components/utilities/DateFormat.js";
const { allPosts } = Astro.props;
const rssHeaderXml = `<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="/rss/styles.xsl" type="text/xsl"?>
<rss version="2.0">
<channel>
<title>${config.get("title")}</title>
<description><![CDATA[ ${config.get("description")} ]]></description>
<link>${config.get("url")}</link>`;
const rssFooterXml = ` </channel>
</rss>
`;
---
<Fragment set:html={rssHeaderXml} />
{allPosts.map(post =>
<Fragment set:html={`
<item>
<title>${post.frontmatter.title}</title>
<link>${new URL(
path.join("/", path.basename(post.file, path.extname(post.file))),
config.get("url")
)}</link>
<guid>${new URL(
path.join("/", path.basename(post.file, path.extname(post.file))),
config.get("url")
)}</guid>
<description><![CDATA[ ${post.frontmatter.description}]]></description>
<pubDate>${rfc2822(post.frontmatter.date)}</pubDate>
<content:encoded><![CDATA[`} />
<post.Content/>
<Fragment set:html={`]]></content:encoded>
</item>`} />)}
<Fragment set:html={rssFooterXml} />

This is not a complex component. All it has to do is generate and output XML. It doesn’t even have to get the posts to include, because those are passed in as a prop.

Some points of note:

  • I create a constant for the top of the RSS file above the items which holds all the channel tags, and a constant for the end of the file after the items (which is just the closing channel and RSS tags).
  • Secondly, the items are created in a JavaScript .map function which takes the array of posts and maps each item to create HTML fragments for them. It’s all very straightforward if you’ve looked at an RSS feed before.
  • The most interesting detail to note by far, and indeed the whole reason behind this custom RSS approach, is the <post.Content/> component inside the .map function. For each post being mapped, I have a <Fragment/> wrapping everything before the post.Content object, then I end it, reference the post.Content object, and then create another Fragment object to wrap up the item XML.

It’s important to understand that <post.Content/> is being accessed directly inside the Rss.Xml Astro component, it’s not inside a JavaScript string or any other wrapper.

It has to be directly written in Astro as a direct Astro component item, or it will be meaningless. It’s Astro that understands what to do with this Content component, not JavaScript or any other framework.

WriteFile.astro

If you’re hoping for a giant code listing that requires lots of explanation, WriteFile.astro will make you sad. It’s ridiculously simple:

src/components/WriteFile.astro
---
import fs from "node:fs/promises";
export interface Props {
fileUrl: URL;
}
const { fileUrl } = Astro.props;
if (Astro.slots.has("rss-writer")) {
const html = await Astro.slots.render("rss-writer");
await fs.writeFile(fileUrl, html);
}
---

It’s 100% frontmatter. There is no output. If this has you thinking “but you just told me you have to have output in an Astro component”, that’s only true for Astro components that have to output any text. This component does not have to, it takes the text generated by RssXml.astro and writes it to disk. I understand completely if that explanation doesn’t clarify things much, but if you start playing with Astro components or pages, you’ll find out what happens when you don’t write html tags or include other Astro components in the base output.2

The important point with this one is waiting for that slot to render before performing fs.writeFile. I never would have known to do this, or especially how to do this. This was all Chris Adiante. The reason it’s necessary to do this is because the page needs to be written up to that point for the RssXml.astro component to do its thing and have something for WriteFile.astro to actually write to disk.

The one thing I did differently to his suggestion is to not look and wait for default slot. Using default slot meant everything before RssXml.astro’s output also got written into the file. Not what I want. As a result, I created a named slot in my Base.astro layout template (which index.astro uses) and then target both WriteFile.astro and RssXml.astro to this slot inside of index.astro.

index.astro content section

This is just the content section of index.astro without any frontmatter, just to show you how I incorporated my WriteFile and RssXml Astro components so that they do their thing when the index page is built.

src/pages/index.astro
<Base title={title} , description={description}>
<section aria-label="Blog post list">
{
indexPosts.map((mdxpost) => {
return (
<Post content={mdxpost}>
<mdxpost.Content />
</Post>
);
})
}
<nav id="pager">
{
allPosts.length > pageSize ? (
<div>
<a href="/2">Older Posts</a>
</div>
) : null
}
</nav>
</section>
<WriteFile fileUrl={rssFileUrl} slot="rss-writer">
<RssXml allPosts={allPosts} slot="rss-writer" />
</WriteFile>
</Base>

The above is everything in index.astro except the frontmatter section. The part that writes the RSS file is at the bottom. One thing about named slots that tripped me up is that both components have to name the slot to use explicitly, not just the outer component. Anything you want to wind up in a named slot needs to name that slot.

Base.astro

The final piece of the puzzle is just the named slot at the bottom of Base.astro, my layout template used by index.astro and all my other pages.

src/layouts/Base.astro
---
import Header from "../components/Header.astro";
import Footer from "../components/Footer.astro";
import "../styles/sw2.css";
export interface Props {
title: string;
description: string;
url: string;
}
const { title, description } = Astro.props;
---
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>{title}</title>
<meta name="title" content={title} />
<meta name="description" content={description} />
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
<link
rel="preload"
href="/fonts/BlinkMacSystemFont-Medium.woff2"
as="font"
type="font/woff2"
crossorigin
/>
</head>
<body>
<Header />
<main>
<slot />
</main>
<Footer />
<script is:inline src="/scripts/barefoot.min.js"></script>
<script is:inline>
lf = new BareFoot();
lf.init();
</script>
</body>
<slot name="rss-writer" />
</html>

See that innocent looking slot down at almost the very bottom named “rss-writer”? That’s the named slot that allows index.astro to render the WriteFile and RssXml astro components. You might also notice the default <slot/>in between the html <main> tags. That’s where all the content generated by any page that uses Base.astro as its layout gets rendered.

TLDR; and Summary

There’s a lot to digest here. The key takeaways are:

  • Accessing MDX content is done through a Content component, which Astro understands, but JavaScript does not.
  • As a result, trying to access that content for a full-item-body RSS feed requires using Astro components to generate the feed.
  • Astro components only write HTML, not XML or an RSS feed file. Using Astro components to generate XML therefore means writing that to disk yourself.
  • This only works in SSG environments. You would not want to use this with an SSR site because it would write rss.xml over and over again. If I were going to use this on an SSR site, I’d want a page with Astro components that isn’t public facing, which would get called by a node script executed by a cron job or whenever a post is added to the site.

As always, I’m not a brilliant programmer and I’m sure there are better ways of doing everything I did. I also would never have figured out the syntax and layout requirements for this without the help of Chris in the Astro Discord.

Footnotes

  1. Astro currently forces a choice between a fully static site (SSG) or a server-rendered site (SSR). Future support for SSG with SSR routes as needed is on the roadmap (I think).

  2. Hint: Nothing. Nothing is what happens.

RSS, Astro, and Me – Part 1

Part of the Astro series

The first Astro site I put on the web was Siracusa Says, which went live on August 7th. The second Astro site was this site, on August 21st. If you think about how bare bones this site is, and that there’s a 3 week gap there, you might be tempted to think that Astro doesn’t allow for particularly quick development. The truth is, it does, but I also have a day job that was more demanding than normal during that time. In fact, this site was super simple to build. The thing that took me the longest was figuring out a look that I would only have to be 90% ashamed of.

I’m not a designer.

But it’s not all unicorns and fluffy kitties with Astro. Astro is a very new framework and it’s very much a work in progress. One of the late design decisions taken by the development team before Astro 1.0 was released was to stop developing customized markdown with component support, and make markdown just markdown, and use MDX for markdown with component support.

MDX is an interesting animal. If you create an MDX file, the MDX spec will give you access to parts of that file in different ways. For example, the body of the document (in other words, the actual content) is exposed as a component. It’s an object. And being able to access that object depends on whatever framework you’re using supporting MDX and providing that access for you.

Astro does provide this ability to access MDX content as an object. Let’s say you grab all your site’s posts, which happen to be mdx files, using Astro.glob, like this:

let allPosts = await Astro.glob("../content/*.mdx");
allPosts = allPosts.sort(
(a, b) =>
new Date(b.frontmatter.date).valueOf() -
new Date(a.frontmatter.date).valueOf(),
);

You now have an array of posts in the variable allPosts. You can use JavaScript’s map function to deal with them individually, like this:

{
allPosts.map((mdxpost) => {
return (
<Post content={mdxpost}>
<mdxpost.Content />
</Post>
);
});
}

Without worrying too much about the rest of the details here, just understand that <mdxpost.Content /> is a component object that exposes the content from one post. The map contains all the posts, and each mapped mdxpost has a .Content component that holds the content.

If it makes your head hurt and you find it weird, you’re not alone. I guess people coming from JavaScript frameworks like React might be used to things like this - I’m not really sure because I don’t know anything about React. At any rate, this type of JavaScript component is not an unknown thing, it’s just new to me.

Now that you have some of the backstory on how MDX works, let me just say that it created a bit of a problem for me with respect to my RSS feed. The reason for this is that Astro components output HTML. Only HMTL. They can’t output JSON or XML or anything other than HTML. This means the @astrojs/rss package that provides RSS support to Astro doesn’t use Astro components to create the RSS file, it uses JavaScript (most likely TypeScript). It therefore does not support the MDX file Content component object, and it therefore means that creating an RSS feed that Astro way limits me to a summary type feed, without full body content for each item in the feed.

Here’s my rss.xml.js for the Siracusa Says RSS feed as an example:

rss.xml.js
export const get = () =>
rss({
stylesheet: "/rss/styles.xsl",
title: config.get("title"),
description: config.get("description"),
site: config.get("url"),
items: Array.from(episodes)
.reverse()
.map((episode) => ({
title: episode.frontmatter.title,
link: new URL(
path.join(config.get("episodes.path"), episode.frontmatter.slug),
config.get("url"),
),
pubDate: rfc2822(episode.frontmatter.pubDate),
description: episode.frontmatter.description,
customData: `<enclosure url="${config.get("episodes.audioPrefix")}/${
episode.frontmatter.audiofile
}" length="${episode.frontmatter.bytes}" type="audio/mpeg" />`,
})),
});

Initially I thought I could use the customData property of the rss package to stuff my MDX file content into, but there is literally no way to get add the .Content component in a way that this JavaScript understands. The best I can do is see the function that returns it or see [object Object]. Not very useful.

To summarize all the above: using MDX as my post content files and @astrojs/rss to support rss feed creation in Astro resulted in my only being able to provide truncated RSS feed items. In order to solve this, I would have to take the advice of Astro Discord member Chris-Adiante and use an Astro component to render the RSS, allowing access to each posts .Content component, and then writing the rss to the filesystem as an rss.xml file.

That’s exactly what I did. I’ll show you how in Part 2.