Summary

We compare three modes of loading ZSH themes and plugins: with plain source, with zplugin in non-turbo mode, and with zplugin in turbo mode.

No performance difference is observed between the tested loading modes. With zplugin in turbo mode the perceived loading latency is higher because typed characters don’t appear on the screen until loading finishes.

Methodology

The measurement methodology is recorded in the screencast. It starts with a docker command [1] that creates slow-theme.zsh and slow-plugin.zsh that serve as stand-ins for a slow-to-load ZSH plugin and theme respectively. Each takes one second to load.

The same docker command creates ~/.zshrc that loads slow-theme.zsh and slow-plugin.zsh. It supports three modes of loading, controlled by MODE parameter.

• MODE=source: Load themes and plugins by sourcing them.
• MODE=zplugin-non-turbo: Load themes and plugins with zplugin in non-turbo mode.
• zplugin-turbo: Load themes and plugins with zplugin in turbo mode.

While plugins and themes are being loaded, a simple prompt is displayed. Eventually it gets replaced by the “real” prompt set by slow-theme.zsh.

Each mode of loading is tested in turn by launching ZSH and immediately starting to type echo hello world. Observations:

• MODE=source and MODE=zplugin-non-turbo:
  • the “loading” prompt appears instantly
  • the typed characters appears on the screen
  • two seconds later the real prompt appears; the part of the command that’s already been typed is there
• MODE=zplugin-turbo:
  • the same as above except that typed characters do not appear on the screen while plugins and themes are being loaded

Afterwards ZSH startup time is measured for different loading modes. This is the time between the moment zsh is started and plugins and themes finish loading. Evidently, ZSH startup time is slightly over 2 seconds and virtually identical for all loading modes. Note that all loading modes allow for commands to be typed before loading finishes. However, with zplugin in turbo mode keyboard input is hidden until the end of the loading process, which may make the perceived loading latency higher.

Discussion

The results of this study contradict the prevailing opinion that zplugin in turbo mode speeds up loading of plugins and themes. This requires explanation.

The recently published Performance test of some of Zsh plugin managers (PTOSOZPM from now on) measures ZSH startup time when loading 28 plugins and themes. Its results indicate that zplugin in turbo mode can reduce ZSH startup time by 85% compared to zplugin in non-turbo mode.

Looking at the source code, we can see that ZSH startup time is measured as the time it takes for the following command to execute:

zsh -i -c exit

This command measures the time to source global and user ZSH configuration files. However, when starting an interactive zsh, there is additional computation that has to happen before shell can be considered “started”. Notably, precmd hooks and select zle widgets must run.

In our setup, zsh -i -c exit completes in 22 ms when using zplugin in turbo mode. Recall that typed characters don’t appear on the screen for 2 seconds after starting zsh. Thus, PTOSOZPM and our study are using two different definitions of ZSH startup latency that yield 2 orders of magnitude different results.

Let’s step back and consider which events users can see happening once they type zsh.

1. A “loading” prompt appears.
2. Typed characters appear on the screen.
3. The final prompt appears.

PTOSOZPM measures different latencies for different loading modes. For zplugin in turbo mode it measures how long it takes for the “loading” prompt to appear, while for all other modes it measures the time to the final prompt. This mistake is easy to make because only zplugin in turbo mode has non-empty prompt in PTOSOZPM. If a non-empty “loading” prompt is added to all loading modes (the same way as in our methodology), this measurement error becomes apparent.

Once this mistake in PTOSOZPM is corrected, we can measure 3 latencies for different loading modes.

1. Time to “loading” prompt is 1 ms for all modes (if implemented the same way as in our methodology).
2. Time to see the first typed character on the screen is 1 ms for all modes with one exception. For zplugin in turbo mode it’s over 100 ms.
3. Time to see the final prompt is over 100 ms for all modes.

When PTOSOZPM is corrected to measure the same thing for all loading modes, its results confirm ours.

We also need to explain the widespread user reports that zplugin turbo mode results in significantly lower perceived ZSH startup latency. Our conjecture is that the appearance of the non-empty “loading” prompt gives this impression. To test it, one can load plugins and themes without turbo mode and display “loading” prompt with the code from our methodology. If our conjecture is correct, the perceived ZSH startup latency will be not higher than for zplugin in turbo mode.

All our tests with turbo mode involved zplugin ice with wait"0". With these parameters plugins and themes are loaded immediately after the first precmd hooks are run. We’ve concluded that there are no advantages to this kind of turbo mode: it doesn’t reduce ZSH startup latency and even makes the perceived latency higher. Perhaps there is value to wait"1", which loads plugins and themes 1 second after starting zsh? Unfortunately, we cannot recommend this mode either. Keyboard input will still be hidden while plugins and themes are loading. So, if a user starts typing a command immediately after launching zsh, the first few characters will appear on the screen, and then shell will appear to freeze. Once everything is loaded, prompt will refresh, all buffered characters will appear at once, and shell will unfreeze. Without turbo mode there is no shell freezing and everything will finish loading 1 second sooner, which is clearly better.

Conclusions

We conclude that zplugin turbo mode doesn’t offer any advantages (performance or otherwise) while having disadvantages.

Our confidence in this conclusion is paultry 10%. We are almost certain we’ve fucked it up somewhere.

[1] The docker command:

docker run -e LANG=en_US.utf8 -e TERM -it --rm archlinux/base bash -uec '
  pacman -Sy --noconfirm zsh git curl

  curl -fsSL https://raw.githubusercontent.com/zdharma/zplugin/master/doc/install.sh |
    sh >/dev/null

  >~/slow-theme.zsh cat <<END
repeat 10 sleep 0.1s                 # simulate 1 second of work
PROMPT="%1F%n%f in %4F%~%f %2F❯%f "  # set prompt
END

  >~/slow-plugin.zsh cat <<END
repeat 10 sleep 0.1s  # simulate 1 second of work
END

  >>~/.zshrc cat <<END
PROMPT="%3Floading>%f "  # prompt to display while loading plugins and themes
print -Pn \$PROMPT

case \$MODE in
  source)
    # Load themes and plugins by sourcing them.
    source ~/slow-theme.zsh
    source ~/slow-plugin.zsh
  ;;
  zplugin-non-turbo)
    # Load themes and plugins with zplugin in non-turbo mode.
    zplugin snippet ~/slow-theme.zsh
    zplugin snippet ~/slow-plugin.zsh
  ;;
  zplugin-turbo)
    # Load themes and plugins with zplugin in turbo mode.
    zplugin ice silent nocd wait"!"
    zplugin snippet ~/slow-theme.zsh
    zplugin ice silent nocd wait
    zplugin snippet ~/slow-plugin.zsh
  ;;
  *)
    PROMPT="%2F❯%f "
  ;;
esac

if [[ \$EXIT == yes ]]; then
  # Exit zsh as soon as all themes and plugins are loaded.
  zplugin ice lucid nocd wait atload"exit 0"
  zplugin snippet /dev/null
fi

TIMEFMT="
Elapsed: %mE"

# Once plugins and themes are loaded, clear the "loading" prompt.
_clear-prompt() {
  print -Pn "\r%E\r"
  unfunction _clear-prompt
  setopt prompt_cr prompt_sp
  precmd_functions=(\${(@)precmd_functions:#_clear-prompt})
}
unsetopt prompt_cr prompt_sp
precmd_functions=(\$precmd_functions _clear-prompt)
END

  MODE=zplugin-turbo EXIT=yes zsh &>/dev/null
  exec zsh'