As a small demo without any additional detail 😅, y’all can peek at https://fns-demo-cloud--11ty.netlify.app/authors/ all of the author pages there are On-demand Builders, while the rest of the site is Build-time rendered

@zachleat now I wonder how in Eleventy you would limit the number of pages generated by the build, to let others be built later on demand. Would it be with a dynamic permalink: false depending on an environment variable?

very nice; ya i can see how it might work for Eleventy and hopefully the modifications for the Hugo and Jekylls of the world will be equally easy.

The Big Boss™ for this approach is the SSGs that then hydrate into clientside routing - i imagine breaking routes up like that is a lot more tangly for Gatsby and perhaps vuepress/docusaurus too.

what can i say except... good luck team :) important problem you are working on here!

hopefully the modifications for the Hugo and Jekylls of the world will be equally easy

Unless something changes, Netlify Functions only support Go and JavaScript. So unless your SSG is written in Go or JavaScript and can easily be called to render a single page, nope.

FWIW, we're working on something like this for Bridgetown: bridgetownrb/bridgetown#276
(similar basic architecture to Jekyll, and like Jekyll, requires Ruby)

Yeah with this model you are making a trade off. Rather than the first request for a URL having a stale or fallback view for the first user to request it, that very first view be the very first visitor would instead take a slight performance hit. I'd expect that to be minimal and typically imperceptible to the user (big flashing caveat of "it's possible to make any render slow no matter the tech!").

I like this trade because it means that we can have confidence of what is being served as part of each deploy (or from the long tail of requests to ODB functions in which are part of any given deploy).

I'm wary of serving stale views as an initial default. Especially if they might include their own caching strategies in the browser which he developer then also needs to understand and manage.

Your suggestion of having some control over the TTL or other caching characteristics of the views being rendered by an DOB function is interesting. My personally feeling is that this is an area where caution is needed . Especially for the default behaviours. This RFC tries to express the model for DPR as being an extension to the rendering and persisting model which deploys already encapsulate. Cirrently, this is easy to reason about — my views are valid and consistent and will be updated when I do another deploy or publish a previous deploy. I'm in favour of them aligning with that as much as possible so as to not muddy the waters for developers when trying to reason about the state of the site. Indeed there are other ways to use serverless functions already instead of/ in addition to this new mechanic.

All good food for thought!

This is a good point — a lot of the discussion internally has been around using this for the "long tail" pages that make builds slow. For example, if you're building tens of thousands of pages and your build takes >10 minutes, could you build the 1,000 pages that receive the most traffic in ~1 minute and let the long tail traffic get generated via DPR?

Another strategy would be to send an async call as part of your post-build scripts (you could use a Netlify Build Plugin for this or do it through some other process) to hit all the pages that use DPR — that way your build script is the first hit in the vast majority of cases and users would always hit the cache. This approach is really interesting to me because it's a kind of "build what's important, deploy, then build the rest" approach. I haven't seen anyone try this yet but I'm really interested in how it will work.

Another approach could be to have a Lambda/Function etc. that hits the content that wasn't included

This is a great idea! I use this on the Learn With Jason API since the API results mirror my page content. You can see it in action here: https://www.learnwithjason.dev/api/episode/lets-learn-esbuild (this is using an early prototype that relies on CDN nodes, so depending on where you load this from you might see the first build perf hit, but repeat hits will use the cache)

Yeah, Lambda/Functions would be similar to what we have previously implemented to achieve better SEO a couple years back.

https://twitter.com/matijagrcic/status/1039855036942172161

We read the sitemap each day and then fired of WebDrivers, later used Puppeteers to get the HTML snapshot and stored that on a CDN

https://developers.google.com/search/docs/ajax-crawling/docs/specification#serving-the-html-snapshot-corresponding-to-the-dynamic-page

This solution was used on several major e-commerce sites and allowed us to concentrate on using latest tools while SEO was intact.

Wix uses similar approach with Puppeteers for testing
https://twitter.com/matijagrcic/status/1285340983832596488

Another strategy would be to send an async call as part of your post-build scripts (you could use a Netlify Build Plugin for this or do it through some other process) to hit all the pages that use DPR — that way your build script is the first hit in the vast majority of cases and users would always hit the cache.

We have something like this built into our Jamstack platform https://docs.layer0.co/guides/static_prerendering and talked about it at NextjsConf and JamstackConf https://youtu.be/cVJxZEyShs4?t=530

Just to emphasise, the performance cost we are talking about is for the very first request to a URL, and this is not per user. So the penalty here should be incredibly low and be outweighed by the confidence that the content being returned for requests to any given URL will be consistent and predictable.

Good discussion points in this part of the thread, and all very valuable.

Interesting to see other people's efforts in this same area, and in particular with pre-rendering specific paths or resources. There is nuance under the waterline about how those rendered views are propagated and persisted around the edge of a CDN which this RFC deliberately omits (so we can focus on the function and logical model it enables).

Good point @smoya! The RFC deliberately omits discussion of the complexity behind the scenes of how the rendered views are persisted around the CDN, but there is nothing to stop people us from exploring ways to additionally prime things post-deploy.

Personally I'd want to look for data on how well (or poorly) those very first requests perform before going down that route. the potential marginal gains for only the very first request for a view might not justify the additionally complexity.

Certainly, as you and @matijagrcic note, there are some potential strategies to mitigate this if it was deemed necessary.

I'm interested to see how the conversation about having control over cache develops, @reegodev. It certainly seems to be an area where lots of folks have thoughts (and by cache I'll assume we're talking about persisting an asset as the current, fresh resource in the CDN rather than controlling caching behaviour in the browser, which is already possible).

Of course this is always about the CDN persistency.

I do see value in being able to easily invalidate and therefor generate fresh renders of resources as required. Each deploy would do that and we may need to introduce another mechanism. But have reservations about granular cache control. Perhaps some sensible defaults might help here. Certainly something for thought and more discussion.

I'm particularly interested in this "another mechanism" you mention. Any ideas already in mind?
If managing caching granularity goes against atomic deploys, i can totally support another method of creating new deploys which does not trigger a build of the website.
I'm aware that you can already do something similar with either build hooks or API calls, but these methods always trigger a full build. I'm also aware now most frameworks can cache some of the build work, ie: they can skip the webpack build; but they still need to run the generators which could still take some time.
What i'm thinking is a method of copying all build artifacts to a new deploy hash, without the artifacts created through DPR / ODB.
This way when my content changes i have two choices:

1. Trigger a normal build which regenerates my static pages and invalidates DPR artifacts.
   This can take some time and can also be expensive if done very frequently.
2. Trigger a "soft" build that copies over my static pages and just invalidates DPR artifacts.
   This can be immediate and also quite cheap i guess

If this is something akin to what you can currently do in Netlify, where you can access the intra-build cache during your builds, then yes, it could be very beneficial. I've used this mechanism quite a bit to support incremental builds where the structure of the site lends itself well to the technique.

Any doc reference or link to share? Would love to experiment on this

Some thoughts @reegodev:

I recently heard that Vercel is evaluating to allow builds to bring over cached pages from past deploys, do you think is it something that is worth discussing in this RFC?

At the risk of being a sounding like I'm just tooting our own horn you can do that today on Layer0. We had a lot of debate internally about the feature because it does break the atomicity of standard jamstack and makes things harder to reason about but the feedback from our customers (who are very large sites trying to make the jamstack work) is what finally drove us to support it. Though it remains an "opt-in" only feature.

Of course this is always about the CDN persistency.

This is a very insightful statement. IMHO the CDN is really the lynchpin that makes the Jamstack work and the future of the architecture will be about redefining how developers use the CDN.

However, I'm still a bit unconvinced about the lack of customization of the cache duration...while it's fine for blog pages, it's not really great for a product page if you track stock availability (I'm sure there are also other examples of content that changes often and could benefit from a shorter cache).

I agree. We run into this a lot. IMHO on a large site the right solution is to allow specifying the cache duration and supporting targeted cache clears (both of which are things we support on Layer0). Otherwise you're rebuilding too much. I realize this is against "classic Jamstack" philosophically but actually think it strikes the right balance practically.

@ianand it's great to see that our issues are shared by others.
Mind if I ask how does your platform handle rollbacks when pages with different cache durations are involved? Do you just invalidate on-the-fly SSR cache and let the next request regenerate the page?

Mind if I ask how does your platform handle rollbacks when pages with different cache durations are involved? Do you just invalidate on-the-fly SSR cache and let the next request regenerate the page?

@reegodev Basically yes. Developers can specify the cache duration and can explicitly invalidate specific paths if need be in response to backend changes. If there's a full rollback, the entire cache is cleared (unless you enable a special option to turn that off), which can then be rebuilt by the platform if desired using https://docs.layer0.co/guides/static_prerendering .

It's true. There would the cost of a delay for the first user arriving at a DPR page that has yet to be rendered. But that feels like the right tradeoff to make & seems preferable to serving either a fallback page or a stale asset.

However, as mentioned elsewhere, there's also the possibility to essentially run through a queue of site URLs to trigger the DPR renders automatically. Doing this second async render pass would greatly reduce the likeliness of any page having to be rendered as a user waits in a browser.

What's great about this model is the power developers will have to decide when and how pages are rendered, balancing upfront build times with these considerations.

Thanks for replying! So, if I understood correctly I...

• would have control when and how to 'rebuild' some cached page
• would have control to queue async renderings after the critical paths of my website are rendered at build time

If that's the scenario, this is going to be awesome :)

Yes absolutely and this is exactly the model I think this could open up.

• Critical pages are prerendered as part of the build
• DPR functions are then called so that additional pages are rendered asynchronously
• If a user arrives at a DPR page ahead of being rendered, the page is generated on demand. This happens at the cost of a slight delay, but without concerns around fallback experiences or stale cache.

It will take a little coordination with site generators to walk the site map and fire off the DPR functions for each additional page, but triggering async building in this way helps control and narrow the window between the initial build and the time when all pages are finally rendered into persistent assets. This optional enhancement could address the concerns about builds continuing to be atomic when the data returned from APIs / content systems change over time.

@toddmorey @kylejrp

Is there any thought towards having the option to run the DPR functions as post-build background tasks? Then new deploys are still speedy, but you get the benefit of (eventually) having every page built instead of generating them on the fly.

At Layer0 we call this feature "Parallel Static Rendering" and it's a great approach in that it lets you decouple the build time overhead from the deployment. We spoke about it at NextJsConf and JamstackConf https://youtu.be/cVJxZEyShs4?t=530

Thought-provoking points! Thinking about 1., would it be cumbersome to separate such tasks from the main build pipeline (critical path)? I imagine you could maintain a slower script which does a lot of this type of groundwork without slowing deployments.

I suppose this applies to your second point as well, but in my mind letting through a redundant style (and thus an unnecessarily large build output) is not as critical as maintaining the validity of links.

Apologies if this whole reply is tangential, it just seemed intuitive to me that since a major goal of DPR is to optimize build speed, we should keep (what I would consider) extravaganza outside of the critical path.

Thanks @betabong. Some helpful thoughts in here!

we need to have access to the result of the initial build process. Let's say in the build process we gather data from different sources. Now to render a page we want to have access to this data. In my understanding we would have to gather that data on every page request, right?

I made a small proof of concept to explore this using the early work on Netlify's implementation (ODB). This PoC combines a number of different techniques to perform different types of render depending on what is required. I was particularly interested to see how well different serverless functions could be combined to provide a solution to a variety of functional requirements in one site, without losing the ability to reason about the state of the site after any given deploy.

Hopefully of interest will be this part of the ODB function which requires the data previously fetched from an external API by a build plugin ready for use during a deploy. I think that satisfies what you were describing.

• PoC: https://demo-odb-news.netlify.app/
• Repo: https://github.com/philhawksworth/demo-odb-news

... I don't see much difference to a conventional server rendered approach really. 👴

The key difference is in the way that the result of the first request to a URL served by an ODB function would be persisted in the CDN. This is different to the conventional server (or serverless 🥴) rendered response which would not get this persisting behaviour. The intent is to persist the first response in the CDN in the same way that you expect the results of each deploy to persist. And so let you reason about the state of your site, and consider the versioning, development workflow and publishing of deploys the same as you always could with a totally regenerated site.

Is this approach particularly vulnerable to DoS with bad actors making a ton of requests to new URLs?

Since this model persists the first request to a URL and serves subsequent request directly for what exists in the CD, I'd say less so that any traditional rendering model where all views were rendered on demand on a server for every request.

I think that your question might have been motivated by a desire to mitigate lots of "parallel builds". I think it's worth noting that this model is not suggesting that each first request would initiate an entire build or indeed anything more than is required to return the view for that single resource as per its URL. As you note, implementations will play a part, but I'd expect that each of these requests should be very fast and focussed. I'm personally not picturing these as initiating new builds so much as returning a single page view.

In my proof of concept implementations, the functions have been completing in between 25 and 47ms (for simple views, but still which fetched data from a remote API). I'd not anticipate the need to build queuing or other logic to manage that. Especially since in order to abuse it, one would need to coordinate the request to very large numbers of unique URLs which are serviced by serverlerss functions which are provided by infrastructure designed to automatically and elastically scale.

wouldn’t it be equally possible to address the build time problem with the same technology solution by massively parallelising the build process?

That's an interesting thought. It would require each site generators and frameworks to be designed and developed to work in this way, This proposal is trying to avoid tightly coupling the SSGs to the infrastructure or to require a complete re-architecture of the existing SSGs and frameworks. I wonder if any maintainers have explored such a compute model for their site generators.

I think that your question might have been motivated by a desire to mitigate lots of "parallel builds".

Yes, exactly that. If the build time can be kept as low as suggested then that would seem reasonable, though that strikes me as a difficult assumption/requirement to deliver or enforce from a developer experience perspective. 25ms from cold? As in, generator and site not already loaded and in memory?

25ms from cold? As in, generator and site not already loaded and in memory?

From cold, yes. But not in this instance, using a generator. Using a WiP of Netlify's ODB functions (which would be Netlify's implementation of the DPR architectural model) you can use an additive approach to enrich the output of an SSG with any tools you like to render a page. You don't have to use the same SSG (although there will be some nice DX advantages to doing so depending on the use case)

In this early PoC the on-demand views (the content pages in this section) are returned by a function which requires the data which was gathered during the most recent build, and a template provided by a javascript sting literal. Populating that view takes around 2ms. Another example, which I'll share soon when it is better documented, does the same thing except it fetches the data from a remote data API, and yes, that do so from cold in around 24 - 45ms in my testing so far. Here's a gist of that function so you can .. get the gist :)

these are all good questions!

Today without DPR, there is no tie in to Netlify as your only deploy solution. DPR would tie you into using Netlify.

the design here is specifically set up to degrade gracefully. the SSG can use any serverless system to render on demand, but if their serverless provider implements DPR support they'd be able to persist that response

SSGs could also support server environments, etc.

the general idea is that each SSG would only need to implement a "build one page" API, which many already have (though most currently have this as an internal and not a public API). the implementation wouldn't add any Netlify-specific code, but supporting "build one page in a serverless function" would allow devs using that SSG with Netlify (or any platform that supports it) to take advantage of DPR

SSG's should be handling the issues of slow builds and making it a priority already.

definitely! our hope is that this encourages a pattern of "build the most important stuff ahead of time and defer the rest", but the real dream is that tech like Rust and esbuild get us to a point where it takes a HUGE site to make this necessary

Using DPR for improving a build time, seems to just offset the build time. Are we just trading one slower build for another to give a perceived faster build (rhetorical)? One of our users is going to take the hit on that first DPR.

a pattern that I'm really excited about is "async builds" where the most important content gets built before deployment, and the rest of the content is immediately rendered by a post-deploy hook so that no one ever feels the slowdown unless they visit a page IMMEDIATELY after it's deployed

the async part of the build would also be parallelized since it's all serverless, so it should be MUCH faster than building all of that content synchronously

Are we moving toward a more complicated setup in our SSG's for larger sites?

like I mentioned, in a perfect world the SSG tech will get fast enough that it's rare to need DPR, but currently build times are enough of a bottleneck that teams are feeling the need to go to server rendered sites to get around it. we feel strongly that this will cause more pain for teams in the long run, so DPR is intended to solve for today's limitations without tying itself to a given framework or tool

does that all make sense?

Yeah, it all makes sense. I think about all these patterns a ton, of course. I think this is a great discussion to be having right now. It's definitely time.

a pattern that I'm really excited about is "async builds" where the most important content gets built before deployment

I like the idea here for sure. There's plenty of room for this type of setup and I could see it being really advantageous if we tackle it correctly in the community.

the real dream is that tech like Rust and esbuild get us to a point where it takes a HUGE site to make this necessary

I'm a huge proponent of the faster tooling camp. I've been pretty excited to see how we go further here also, which is where I was heading with these concerns also.

Thanks for the feedback also Mr Lengstorf. 😉

hey, @wardpeet! just realized we left you hanging here

the main challenge if we allow arbitrary changes to deployments is that it muddies the mental model and creates a lot of traps

for example: caching is easy if you have one layer of it — take some assets, put them on the CDN, replace them if there's a new site build. that's Jamstack! it's a clear workflow that you can explain without a whiteboard

however, if you add multiple layers of caching, it gets really hard to track. every large team I've worked with has at least one horror story of misconfigured caches breaking the production site. because of this, we want to be really wary of introducing a secondary caching layer into the standard architecture. that's why DPR is set up to build once, then go to the same caching layer as all other prebuilt assets until the next build — otherwise we have the potential for caches to depend on each other and get out of sync, and that's a recipe for a bad time

I do think there are ways we can design thoughtfully around partially regenerating sites, but we want to be really cautious about how we implement it — otherwise we'll hit all the original problems of e.g. Varnish and other caching layers that are so great... until something goes wrong and you spend your weekend trying to figure out which layer of the cache is out of sync 🙃

thanks!

Thanks for getting back to me 😄. It makes sense, Caching is great until it doesn't work as expected. I'm all on board with clearing the cache of all items. To me, an "atomic deploy" refers to upload all files over again. Many providers won't upload these new files if they haven't changed.

This is more an implementation detail but that's how I read the spec that every file needs to be re-deployed.

how would you handle data changes that affect more than one page? e.g. you change the title of a post and a bunch of your category overviews or taxonomy pages need to change?

Good question - Gatsby's incremental build engine handles that by maintaining a dependancy tree and updating all of pages that are affected by a content change. Tools like Next.js ISR handles this by using a cache timeout, which doesn't work for DPR immutability...

Notwithstanding the complexities, I think this is a limitation. We provide a way for content creators to generate their own landing pages, which means we don't control the content that is created. For the most part, this created content is static and doesn't change very often. The current DPR proposal would require us to redeploy our website or wait for the cache to be invalidated.

I think this relates to @wardpeet's comment: whether or not a file is re-uploaded is an implementation detail. If a CDN is able to determine that a file is unchanged then it should be considered safe and still not a violation of the DPR rules to not upload it again. The only way it would violate the expectations would be if there are pages that would change if rebuilt, but are not uploaded. Whether it's through the framework knowing that a page's data is unchanged, or a CDN knowing that the file hash is unchanged, I think it's reasonable to only upload deltas and for it to still be considered an atomic deploy.