GNU/Linux ◆ xterm ◆ bash 5076 views

Let’s take a look at the dynamics of cpu hotplugging on a Nexus 5 running Maru. Keep an eye on the right two panes, which show CPU utilization on the top and cpu states and cpuset limits on the bottom.

When the system is idle, note how all cores except cpu0 are offline. This is thanks to a hotplugging daemon called mpdecision that is found on Android devices running Qualcomm chipsets.

When we add load, mpdecision will start turning on cores. When the load drops off, mpdecision will start shutting off cores until only cpu0 is on. This is a great optimization for battery life.

Now, on Maru, the desktop is isolated within an LXC container, and (unknown to me when I built this…) is by default placed in a cpuset (a kernel resource controller for managing task scheduling across cores). Normally, this would be fine since LXC will just copy over the root cpuset configuration (all available cpus), but with hotplugging it is always throttled down to just cpu0 due to the fact that cpuset subgroups must be subsets of it’s parents. That is, all child cpusets shrink as the parent shrinks, but never grows as the parent grows. That means we always end up with the container cpuset being just cpu0!

To fix this temporarily, we can poke the cpuset sysfs tree and configure LXC and it’s children to access all cores…

…but after we idle again we end up in the same state as before with the LXC cpuset dropping to just 0.

To fix this permanently, we need to tell LXC to not use cpusets at all, which requires a build-time change, which I am committing shortly!

Cool, huh?

More context and info here: https://github.com/maruos/maruos/issues/58