VirtualBox: Reverse Caps Lock on the Guest

When testing VirtualBox with a Windows guest on a Linux host, I noticed that the Caps Lock state is inverted on the guest: when Caps Lock is disabled on the host Linux, it is activated on the Windows guest and vice versa. Press the Caps Lock key on the host to reverse its state on the host but not on the guest. Pressing the Caps Lock key on the guest reverses its state on the host and the guest. The current fix seems to be flipping once on the host and once on a guest. But what exactly is the problem here? Can I ask the guest to start with the correct state at startup?