An Honest Review of Gatsby

We decided to adopt Gatsby for Sentry’s customer-facing documentation - well, I should say that I decided. We were already using it successfully for a variety of static marketing content, and I knew it had a lot of hype, so after a brief proof-of-concept it seemed like a safe choice.

To help contextualize everything I’m about to say, it’s important to understand the scope of our usage. Sentry’s documentation is not as straightforward as you might think - in fact, there are over 3,000 pages as of writing. We have a large amount of templated content designed to render language-specific examples, as well as a variety of different types of documentation (user guides, help desk-y articles, code-rich technical docs). Originally we had extended Jekyll to support a lot of this, but Ruby isn’t widely used at Sentry (approximately 0% of the engineering team knows Ruby), and it had become a big mess of spaghetti code with slow build times.

I also want to note that while this blog post is primarily focusing on the flaws of Gatsby as a framework, I’m not here to tell you that it’s not good for your use case. That said, I was not able to discover many of these short comings easily when evaluating Gatsby, and many things you read on the internet don’t stem out of real-world usage. My hope here is that Gatsby continues to improve over time, and that, as a user, you can be more informed about if it’s the right choice for you.

Adopting Gatsby

So, enter Gatsby. It seemed fast, was built on React (we’re experts on that here, with our gigabyte-sized Sentry frontend app), and had a huge adoption (assumed future existence and stability). While we didn’t have the desire to use MDX, it also seemed like a positive outcome given we could more easily deal with some of the rich aspects of our docs site, without having to resort to 2010-era JavaScript. We assumed a bunch of the other features of Gatsby had value-add, but we didn’t have an immediate need. These were things like dynamic source data - thus the need for a GraphQL engine at all - as well as the large plug-in ecosystem.

We started by iteratively converting sections of the Jekyll site into Gatsby - running them side by side for a time. At one point we eventually bulk converted pages, and ripped off the band aid. At this point though it was becoming clear build times were a problem. You’d spend at least 5 minutes on image optimization alone, with no way to even disable that. Slowly but surely we were depleting the ozone later on re-optimizing images which had already been pre-optimized. Oh yeah, and we were crippling our iteration speed as well, since the build cache would invalidate under a variety of situations in early development.

Making this worse was how we deployed Gatsby. We started off leveraging what we had already done: deploying Jekyll with Docker onto our own infrastructure - effectively just proxied via a CDN. We continued that for a period of time, but deploy times were far too long - upwards of 30-40 minutes for everything to build. Eventually we moved over to Vercel which dropped it down closer to 10 minutes, but ultimately it can’t fix what it doesn’t control.

The build and deploy times were the first of many woes, and they represent what would become a continued frustration: a problem without a clear solution.

Enter MDX

Rewind time a little bit - this actually wasn’t our first project converting documentation to Gatsby. The proof-of-concept I mentioned earlier was actually our developer documentation, which I had migrated out of Notion to make public. While doing that we had gotten our hands dirty with some initial MDX usability and extensions - like our code samples which support toggling between different languages. This was one of the many things we needed to solve for, but MDX made it look like it’d be seemingly easy. No more jQuery DOM manipulation, just clean, encapsulated React components. Or so we thought.

Almost immediately we hit rough spots with MDX. We were coming from Jekyll - which was Liquid-rendered (a template engine) markdown - to MDX - a strange offspring of Markdown and JSX, attempting all of the benefits of both, but missing by a fairly large margin. Let’s illustrate the crux of the issue with what has got to be one of the most common needs in a documentation system: an alert (or callout) component:

<Alert level="info">You should know something important about this!</Alert>

At face value this looks great. Alert is just a React component, and JSX is close enough to HTML that non-technical folks are able to pick it up fairly easily. Now the problem comes into play when you actually want to do something in the real world. Here’s an example from our API docs:

<Alert level="warning" title="Note">
    **PUT/DELETE** methods only apply to updating/deleting issues.
Events in sentry are immutable and can only be deleted by deleting the whole issue.
</Alert>

How would you expect this to render? Both as an engineer and a non-engineer, I would expect - given this is markdown - that the “PUT/DELETE” text would be bold. It’s not. Because the MDX interpreter decides that once you enter a component block, it’s no longer markdown. So instead, we’re forced with this monstrosity everywhere in our documentation:

<Alert level="warning" title="Note"><markdown>

**PUT/DELETE** methods only apply to updating/deleting issues.
Events in sentry are immutable and can only be deleted by deleting the whole issue.

</markdown></Alert>

There’s two things you should note here: 1) we have to use this <markdown> tag, 2) we have to put empty new-lines to ensure paragraph tags render.

“But David”, you might say, “why don’t you just tell the Alert component to render the text as markdown?“. If only you could, or at least, if only I could have possibly found a way to achieve that as a user with minimal Gatsby or MDX internals knowledge.

To Gatsby, or at least to the MDX team’s credit, they recognize some of these problems and there is work underway on a 2.0 of the MDX dialect. While I’m confident they will improve things, I’m not confident MDX can ultimately succeed. It’s likely going to tradeoff one problem for another due to what it’s trying to achieve in the first place. It may get to a good place, but frankly, we need to step back and look at what we’re trying to solve, instead of creating a solution to a problem we don’t have. I don’t need JSX syntax in my markdown, I need a way to include JSX components. That might sound similar, but its quite a different thing.

As an example, there’s no reason I couldn’t simply use markdown syntax, and provide a way to achieve something akin to:

<a-valid-html-tag-because-markdown-allows-that a-valid-property="a-value">

This wouldn’t force us to work around quirks in a new language (or interpreter even), and could be solved in a much more sustainable way. There are other alternatives as well. A generic way to render extensions in markdown could simply call into a React component, and avoid even trying to hijack HTML in the first place. While I don’t know what this might look like in Markdown, in Sphinx’s use of reStructuredText this was solved early on with Directives:

.. my-directive:: some data
   :property-name: property-value

I will hold out for MDX 2.0 and hope that finds a nice minimal-compromise place, but if not, we’ll be looking for a way to extend native markdown.

A Broken DOM

While we were able to work around the kinks of MDX, there’s been some things not yet solved. One of those is the layer which Gatsby uses to apply diffs to the DOM. I’m going to caveat this section with I don’t know what the technical implementation is, but I can make some assumptions given what I know of the domain. The system itself is intended to apply deltas to the DOM. This is naively also how React works, and I imagine under the hood it’s relying on React at least for part of it. We’ve had issues with this identified in two places already:

• progressive image loading
• dynamic JSX components

While they might not be linked to the same issue, they smell like they are, so we’re going to roll with it. The problem exhibits itself when you have a bunch of DOM that to a naive robot might look the same:

<div>foo</div>
<div>bar</div>
<div>baz</div>
<div>foobizbar</div>

In React it uses the graph to identify which node is which - effectively creating a unique entity ID based on its location. In cases where that’s difficult, React will warn you to explicitly bind a key attribute on each element to ensure it can more accurately deal with updates. While I would assume Gatsby is at least partially using React’s DOM engine, what we see in production effectively takes the above example, and replaces some of the content with other subsets of content - meaning it’s unable to accurately identify which nodes need updated.

We’ve seen this where a progressive image is replaced with an entirely different image that’s present near it on the page. We’ve also seen this happen for a dynamically loaded section of content (our language-selector include tags). While we’ve yet to identify a fix for the image tags, our other issue was resolved by literally changing a div tag to a different tag, one which is less commonly used (in our case, section).

All of the cases happen after Gatsby’s initial static render and exist only when applying some form of delta.

Let’s Talk GraphQL

It’s a static website generator. It literally does not need GraphQL all over the place. While there are few instances in the real world where that is valuable, it shouldn’t require a GraphQL API to read objects that are already in memory.

I don’t want to spend the energy to hammer this in, but take a look at Jared Palmer’s Gatsby vs. Next.js as it echoes my thoughts.

So, let’s actually not talk about GraphQL, but all its done is create complexity for us.

Minor Gripes

There’s a number of other things we’ve found fairly frustrating at this point, but this post is already getting long, so I’m choosing to summarize them.

• Transform extensions are disconnected from source extensions. In the most naive cases this is fine, but in the real world I need to transform data entirely differently based on its source. Data is more than a mimetype. An example of this for us is we export some markdown rendered content as JSON. That content needs to render under entirely different rules than other markdown files. Well, it can’t (at least not easily), so we’re left with praying no one fucks up.
• Setting aside that we even need GraphQL, trying to get objects modeled into a format that works in a semi-correct way in Gatsby is terrifying. For example, Page objects generally represent your URLs, yet the docs, code samples, tutorials, etc, suggest you create a disconnected entity to actually render the content with the page. For example, instead of binding page-relevant context to page.. as context (a native concept), you’re pushed to doing GraphQL queries to pull in front-matter. Stop. Using. GraphQL.
• Given the disconnect between data and parsing, it’s very difficult to achieve templated content without again resorting to GraphQL. Due to the flaws with MDX we wanted to be able to inject something more akin to a traditional template variable (for our SDK package versions). Ideally this would have been able to read from a simple pre-existing data source, but because AST parsing happens wildly outside of the scope of the infinite list of things Gatsby does in its build process, this becomes needlessly complicated.
• The docs are lacking in a variety of key areas. You’ll begin stumbling as soon as you get into any of the configuration APIs (e.g. onCreateNode) where there is missing or very-incomplete documentation.
• Why does a static site use so much bandwidth. I’m sure it’s something we’ve done wrong, but the large amounts of data pulled back from page-data.json aren’t calming. Likely also connected to GraphQL. Blame it all on GraphQL.

There are certainly more, but consistently I’ve been asked about why I’m frustrated with Gatsby, so I wanted to share what was top of mind.

In Closing

While this post may paint Gatsby in a negative light, I would still say its a framework worth considering. There’s often not going to be a one-size-fits-all, and Gatsby has a lot of great qualities. This site for example is built on Gatsby, and it’s certainly more enjoyable to build with than Jeykll was. That said, I don’t touch GraphQL or MDX here, have effectively no images, and clearly don’t spend a lot of time working on it.

With that in mind, there are two pieces of advice I would give to the Gatsby team:

1. It should not be slow - it’s just HTML. If this doesn’t change I don’t see Gatsby succeeding.
2. Please for the love of anything make a functional markup language that doesn’t require 10 years of domain experience to make functional.

For those of you who you’re interested, you can browse the mess that has become our lives over on GitHub:

I’m sure there’s things we can do better, so if something seems wildly off given what I’m saying, maybe you can help teach us.

from Hacker News https://ift.tt/3n95b0x