FormAPI Blog

How Yak Shaving Helped Me Speed Up GitBook Development by 90 Percent

I’ve been using GitBook for the FormAPI documentation. It’s been working great in production, but the development server was really slow. The first time you change a file, it restarts pretty quickly, but eventually it would take over 45 seconds to restart and regenerate the files. It would keep getting stuck at Stopping server .... My workaround was to just write all the Markdown first, and then start GitBook and check the browser to make sure it looked ok. This was not an ideal workflow.

While I was writing my blog post about proxying requests through a Rails development server, I noticed that saving a Hugo blog post would cause GitBook to restart. This was very strange. I don’t usually run Hugo and GitBook at the same time, so I hadn’t noticed this before.

I had put my GitBook configuration file (book.json) in the root directory of my repo. I like having a “monorepo”, so the root directory includes the backend, frontend, blog, docs, etc.

My GitBook docs were in the ./docs directory, and I had set "root": "./docs" in book.json. (I set it up like this because I just thought it would be easier to run gitbook serve from the root directory.) I had assumed that GitBook would only look at the ./docs directory, but now I realized that the file watching code was looking through all of the files in my repo.

After reading through some of the source, I found out that it was only looking for files with the following patterns:

["book.json", "book.js", "_layouts/**", "**/*.md",
"**/*.markdown", "**/*.mdown", "**/*.adoc", "**/*.asciidoc"]

So it had been watching the *.md files in my Hugo blog. But it was also picking up a lot of *.md files from node_modules, and it needed to search through a ton of files to check if they matched.

I moved my book.json into the ./docs directory so that I could run GitBook from there. That was a great start, but it would still slow down after restarting a few times. This made me even more annoyed, because I was so sure that I had figured it out. Now I was on a mission to fix it.

I wasn’t sure why GitBook needed to stop and start the server whenever a file changed. Someone else had opened an issue for that on GitHub, and I forgot that I had commented on it almost a year ago. I wanted to see if removing the server restart would solve the problem.

I looked at the code in lib/cli/serve.js, and I figured out how to fix it. I got rid of the server restart, because this wasn’t necessary and it worked fine without it. I changed the file watching code so that it would only start once, instead of every time a file was changed. And I also excluded all the files in node_modules.

After making these changes, GitBook takes about ~3 seconds to regenerate all of the static pages, and it doesn’t slow down at all if you keep changing files. I tried running it from my root directory, but this was still extremely slow, and I assume it’s because it has to search through so many files (even though it shouldn’t be looking anywhere except ./docs).

I’ve sent a PR, but it’s not a very active project, so I don’t think anyone will look at it.

Conclusion

I’m glad I could finally track down this issue and fix it. I wasn’t expecting to fix that today. Sometimes it’s good to work on random little things that you find interesting, because you never know where you’ll end up.

GitBook is not being maintained anymore, but I still really like the API theme, and it works pretty well in general. I was thinking about porting this theme to Hugo to make it much faster, but that’s not so important any more. I might still switch to mkdocs in the future, and I’ve also heard good things about Gatsby.

If you’re using GitBook, you might want to try the FormAPI/gitbook fork for a better development experience.