This is a story/writeup of how I debugged UCW layout on XWayland. You may learn
+here how keyboard layouts work on Linux.
+
Background: The UCW layout
+
The UCW keyboard layout is one of the interesting methods of typing Czech
+letters on a rather American keyboard. The main idea is having a classic US
+(QWERTY) layout, but without the CapsLock, which serves as a key to add
+diacritics, so that e.g. Caps+s creates "š". It is naturally possible to
+combine Caps with Shift to create uppercase letters, and because not all
+letters can have diacritics in Czech (e.g. there is no "ǩ"), it also manages to
+cover all the German and Slovak letters. (Sometimes, there are multiple
+diacritics for a single letter, then the additional diacritics are nearby:
+Caps+e is "é", Caps+w is "ě".) The CapsLock can still be pressed by using
+Alt+Caps.
+
The layout has some nice features like avoiding deficiencies with the Czech
+layout (diacritics on number row are too far, both round parentheses are on the
+same key, other parentheses only with AltGr at random places, no asterisk, for
+some reason we have "§" though, …). Another nice feature is that it is rather
+interoperable: I am able to type on any computer with the "most standard"
+layout and when writing in Czech I can just not press the CapsLock key and only
+lose my diacritics, computers can have this layout system-wide even when
+foreign people use it,…
+
I would not say many people use the layout but at least several my friends do
+and I have come across several random people on the Internet who do also. More
+on this below :-)
+
The important part is how the layout is set up. Fortunately, it is contained in
+the xkeyboard-config database, so the following command just enables it in Xorg:
+
+setxkbmap us,cz -variant ,ucw -option grp:caps_switch
+
+
Technically, this sets two layouts (or groups), which are switched by
+pressing the CapsLock key. This has some disadvantages (I have managed a few
+times to have the order accidentally swapped and layout switchers (or their
+users) are sometimes confused), but actual issues are rare. Under X11, that is…
+
Corollary 1: I want UCW to work differently.
+
+
My custom keyboard layout?
+
I didn't like fact that UCW layout is in fact just an overlay and not a
+"proper" level 3, so I started studying how to create a custom keyboard layout.
+Few notable resources: XKB page on ArchWiki, manpages for
+xkeyboard-config(7), setxkbmap(1) and xkbcomp(1) and looking into
+/usr/share/X11/xkb/symbols/cz (found by grepping /usr/share/X11 for
+"ucw").
+
Another reason for creating my own layouts is various tweaks esp. on laptop
+keyboards. For example, the laptop I am typing this sentence on has broken the
+up arrow key, so I would like to map it somewhere. I use xmodmap for that,
+but if this could be contained in a single layout, it would make stuff simpler
+for me. (And as I will later learn, xmodmap does not work well with xkb
+layouts…)
+
Also, I have another tweak on my keyboards: Compose key. This really calls for
+the custom layout! And when at it, I should create one for the ttys as well…
+
But this had quite a low priority (the kbd.sh script in all my homes does
+the job well enough), so I didn't get to it in time. Luckily, maybe – if I were
+quicker I would not end up in this rabbit hole. At this point, I knew the basic
+stuff and was reasonably sure that I could hack it together, at least somehow;
+what was stopping me was lack of time and not being sure how I want to manage
+machine-specific tweaks in two different layout syntaxes (xkb for X11 and kbd
+for ttys).
+
+
+
Aside: A few notes about xkb layouts for completeness
+
This all has been described elsewhere in more detail, but it is useful to
+know when debugging layouts, so I will mention it here again.
+
There are two forms xkb layouts may be specified. The low-level one is called
+KcCGST (keycodes, compat, geometry, symbols, types) and describes very much
+everything that should happen on any keypress. These actions are described in
+subdirectories of /usr/share/X11/xkb/ of the same name.
+
The high-level descriptions are called RMLVO (rules, model, layout, variant,
+option) and are what you usually configure, either with setxkbmap or using
+GUI tools. Files in /usr/share/X11/xkb/rules/ describe the translation of
+RMLVO to KcCGST (the XML files add human-readable names for the GUIs and the
+like). Maybe confusingly, the RMLVO names of options and layouts are very
+similar to the names of KcCGST compat and symbols, but are generally a
+different thing AFAIK.
+
KcCGST also come in two forms: the readable one and the complete one. The
+readable one does not determine everything in one file, instead including
+others. The complete description is, well, complete. It is kind of similar to
+the C preprocessing.
+
The readable KcCGST are what is stored in the filesystem as well as what you
+get from setxkbmap -print and may serve as a good starting point for
+tweaks. To get the complete ones one can use xkbcomp $DISPLAY -.
+
The processing of these description is different in Xorg and in Wayland. If
+I were to bet where the bug is before digging into it, this sounds like a very
+likely culprit.
+
+
+
A wild friend has appeared
+
… and he had a problem and asked me if I would have a look into it. Apparently,
+group switching didn't work in XWayland. At first, I wanted to work it around
+by finally creating the Unified UCW keymap™, but I wanted to learn more about
+XKB (from TODO GUIDES), which took me quite long, again. (Studying, working,
+helping with camps and fixing layouts takes some time…)
+
Also, since the issue is apparently bigger, solving all group switches would be
+a better and more useful solution than just hacking up the Unified UCW keymap
+(even though I want my tweaks to eventually be xkb-based anyway).
+
I had very little experience with Wayland until then, mostly because X11 worked
+for me rather well. I tried running Sway few times before, but usually quickly
+reverted to using Xorg for various reasons (Nvidia GPU in laptop, very laggy
+mouse, maybe this exact issue with UCW layout) which I didn't feel like
+solving at that moment.
+
I wanted to help my friend, though, so I started looking more into using Sway
+on another laptop and reading about internals of input system on Peter
+Hutterer's blog TODO NAME SPELLING? Occasionally I would randomly google
+(with DuckDuckGo :-D) for the issue with group switching, just in case…
+
+
The nerd snipe issue
+
TODO SOMEONE on GitHub found out an interesting workaround: when they
+disabled the group switch, the UCW layout started working. Complete "huh?"
+moment, I knew I wanted to know more about why that worked. The
+xkeyboard-common(7) manpage says about more group switching options, so I
+randomly tried switching by Menu key in XWayland (using setxkbmap even
+though it complained and was supposed to do nothing), and ended up with both
+CapsLock and Menu adding diacritics.
+
After a bit of mathematic thinking (more in my comment of the main issue) I
+concluded that it was actually working fine, just that the SetGroup action
+was evaluated twice, which would overflow back to the first group when two
+layouts are set (probably the most common case). Mathematic term for this issue
+is "arithmetic in the Z_2 group" :-)
+
Corollary 2: A simple workaround is adding the UCW overlay twice, e.g.
+setxkbmap us,cz,cz -variant ,ucw,ucw (and thus changing the algebraic group
+to be Z_3).
+
I think that another workaround would be to change the compat rules for
+XWayland, but it feels nasty to have such a quirk in xkeyboard-config database.
+Changing them for everyone (in compat/evdev directly) might break other
+systems, so that also does not seem to be a good way.
+
Corollary 3: We are fixing XWayland (or the way it processes events from Wayland).
+
+
Learning what happens
+
I continued reading, this time mostly who-t's post series about custom
+layouts, core Wayland protocol as well as the source code of various
+tools (xev, wev, xkbcli interactive-wayland &c.) Given that I knew
+very little about the inner workings of the stack, I wanted to find some code
+that would be run and "enhance" it with so many debug prints that I would
+understand what was the state of various parts of the Xwayland's "stacked xkb".
+I kind of knew that one part of this is somewhere in the XWayland server,
+whose code felt intimidating, so I wanted to determine where to start nudging
+it from what APIs the clients use.
+
+
Tip
+
Downloading various packages in Linux is rather simple, as well as their
+rebuilding. In Arch, I can get the package's PKGBUILD just with yay -G
+package, build it with makepkg and install it using yay -U
+package-….pkg.tar. Makepkg also has some options which allow me to tweak
+the source code after downloading it and before building it.
+
Other distros are probably similar, e.g. for Debian-based distros one can
+use apt source, debuild and apt install to do the same.
+
+
One interesting observation is that contrary to my expectation, XWayland does
+not seem to use libxkbcommon (according to the /proc/PID/maps file). This
+can have several reasons, but its source code also contains a slightly tweaked
+version of (Xorg-style) xkb, which might mean I will be dealing with the ugly
+code :-/ (Actually, XWayland might not process key events itself, instead
+just passing them to clients, but this seems inconsistent with the issue – what
+else would be introducing the second group switch?)
+
+
+
+
+