diff --git a/content/ucw-keymap-on-xwayland.rst b/content/ucw-keymap-on-xwayland.rst new file mode 100644 index 0000000..8ff1bca --- /dev/null +++ b/content/ucw-keymap-on-xwayland.rst @@ -0,0 +1,210 @@ +UCW keyboard layout on XWayland +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ + +:slug: ucw-keymap-on-wayland +:date: 2023-12-31 18:23 +:tags: xkb, linux, xwayland +:category: random +:keywords: xkb, xorg, wayland, hack +:lang: en +:translation: false +:status: draft + +This is a story/writeup of how I debugged UCW layout on XWayland. You may learn +here how keyboard layouts work on Linux. + +Background: The UCW layout +========================== + +The UCW keyboard layout is one of the interesting methods of typing Czech +letters on a rather American keyboard. The main idea is having a classic US +(QWERTY) layout, but without the CapsLock, which serves as a key to add +diacritics, so that e.g. Caps+s creates "š". It is naturally possible to +combine Caps with Shift to create uppercase letters, and because not all +letters can have diacritics in Czech (e.g. there is no "ǩ"), it also manages to +cover all the German and Slovak letters. (Sometimes, there are multiple +diacritics for a single letter, then the additional diacritics are nearby: +Caps+e is "é", Caps+w is "ě".) The CapsLock can still be pressed by using +Alt+Caps. + +The layout has some nice features like avoiding deficiencies with the Czech +layout (diacritics on number row are too far, both round parentheses are on the +same key, other parentheses only with AltGr at random places, no asterisk, for +some reason we have "§" though, …). Another nice feature is that it is rather +interoperable: I am able to type on any computer with the "most standard" +layout and when writing in Czech I can just not press the CapsLock key and only +lose my diacritics, computers can have this layout system-wide even when +foreign people use it,… + +I would not say many people use the layout but at least several my friends do +and I have come across several random people on the Internet who do also. More +on this below :-) + +The important part is how the layout is set up. Fortunately, it is contained in +the xkeyboard-config database, so the following command just enables it in Xorg:: + + setxkbmap us,cz -variant ,ucw -option grp:caps_switch + +Technically, this sets *two* layouts (or *groups*), which are switched by +pressing the CapsLock key. This has some disadvantages (I have managed a few +times to have the order accidentally swapped and layout switchers (or their +users) are sometimes confused), but actual issues are rare. Under X11, that is… + +Corollary 1: I want UCW to work differently. + +My custom keyboard layout? +-------------------------- + +I didn't like fact that UCW layout is in fact just an overlay and not a +"proper" level 3, so I started studying how to create a custom keyboard layout. +Few notable resources: `XKB page on ArchWiki`__, manpages for +`xkeyboard-config(7)`__, `setxkbmap(1)`__ and `xkbcomp(1)`__ and looking into +``/usr/share/X11/xkb/symbols/cz`` (found by grepping ``/usr/share/X11`` for +"ucw"). + +Another reason for creating my own layouts is various tweaks esp. on laptop +keyboards. For example, the laptop I am typing this sentence on has broken the +up arrow key, so I would like to map it somewhere. I use ``xmodmap`` for that, +but if this could be contained in a single layout, it would make stuff simpler +for me. (And as I will later learn, ``xmodmap`` `does not work well`__ with xkb +layouts…) + +Also, I have another tweak on my keyboards: Compose key. This really calls for +the custom layout! And when at it, I should create one for the ttys as well… + +But this had quite a low priority (the ``kbd.sh`` script in *all* my homes does +the job well enough), so I didn't get to it in time. Luckily, maybe – if I were +quicker I would not end up in this rabbit hole. At this point, I knew the basic +stuff and was reasonably sure that I could hack it together, at least somehow; +what was stopping me was lack of time and not being sure how I want to manage +machine-specific tweaks in two different layout syntaxes (xkb for X11 and kbd +for ttys). + +Aside: A few notes about xkb layouts for completeness +----------------------------------------------------- + +This all has been described elsewhere in more detail, but it is useful to +know when debugging layouts, so I will mention it here again. + +There are two forms xkb layouts may be specified. The low-level one is called +KcCGST (keycodes, compat, geometry, symbols, types) and describes very much +everything that should happen on any keypress. These actions are described in +subdirectories of ``/usr/share/X11/xkb/`` of the same name. + +The high-level descriptions are called RMLVO (rules, model, layout, variant, +option) and are what you usually configure, either with ``setxkbmap`` or using +GUI tools. Files in ``/usr/share/X11/xkb/rules/`` describe the translation of +RMLVO to KcCGST (the XML files add human-readable names for the GUIs and the +like). Maybe confusingly, the RMLVO names of options and layouts are very +similar to the names of KcCGST compat and symbols, but are generally a +different thing AFAIK. + +KcCGST also come in two forms: the readable one and the complete one. The +readable one does not determine everything in one file, instead including +others. The complete description is, well, complete. It is kind of similar to +the C preprocessing. + +The readable KcCGST are what is stored in the filesystem as well as what you +get from ``setxkbmap -print`` and may serve as a good starting point for +tweaks. To get the complete ones one can use ``xkbcomp $DISPLAY -``. + +The processing of these description is `different`__ in Xorg and in Wayland. If +I were to bet where the bug is before digging into it, this sounds like a very +likely culprit. + +A wild friend has appeared +========================== + +… and he had a problem and asked me if I would have a look into it. Apparently, +`group switching didn't work in XWayland`__. At first, I wanted to work it around +by finally creating the Unified UCW keymap™, but I wanted to learn more about +XKB (from TODO GUIDES), which took me quite long, again. (Studying, working, +helping with camps *and* fixing layouts takes some time…) + +Also, since the issue is apparently bigger, solving all group switches would be +a better and more useful solution than just hacking up the Unified UCW keymap +(even though I want my tweaks to eventually be xkb-based anyway). + +I had very little experience with Wayland until then, mostly because X11 worked +for me rather well. I tried running Sway few times before, but usually quickly +reverted to using Xorg for various reasons (Nvidia GPU in laptop, very laggy +mouse, maybe *this exact* issue with UCW layout) which I didn't feel like +solving at that moment. + +I wanted to help my friend, though, so I started looking more into using Sway +on another laptop and reading about internals of input system on `Peter +Hutterer's blog`__ TODO NAME SPELLING? Occasionally I would randomly google +(`with DuckDuckGo`__ :-D) for the issue with group switching, just in case… + +The nerd snipe issue +==================== + +`TODO SOMEONE`__ on GitHub found out an interesting workaround: when they +*disabled* the group switch, the UCW layout *started working*. Complete "huh?" +moment, I knew I wanted to know more about why that worked. The +`xkeyboard-common(7) manpage`__ says about more group switching options, so I +randomly tried switching by Menu key in XWayland (using ``setxkbmap`` even +though it complained and was supposed to do nothing), and ended up with *both* +CapsLock and Menu adding diacritics. + +After a bit of mathematic thinking (more in my `comment of the main issue`__) I +concluded that it was actually working fine, just that the ``SetGroup`` action +was evaluated *twice*, which would overflow back to the first group when two +layouts are set (probably the most common case). Mathematic term for this issue +is "arithmetic in the Z_2 group" :-) + +Corollary 2: A simple workaround is adding the UCW overlay twice, e.g. +``setxkbmap us,cz,cz -variant ,ucw,ucw`` (and thus changing the algebraic group +to be Z_3). + +I think that another workaround would be to change the ``compat`` rules for +XWayland, but it feels nasty to have such a quirk in xkeyboard-config database. +Changing them for everyone (in ``compat/evdev`` directly) might break other +systems, so that also does not seem to be a good way. + +Corollary 3: We are fixing XWayland (or the way it processes events from Wayland). + +Learning what happens +===================== + +I continued reading, this time mostly `who-t's post series about custom +layouts`__, `core Wayland protocol`__ as well as the source code of various +tools (``xev``, ``wev``, ``xkbcli interactive-wayland`` &c.) Given that I knew +very little about the inner workings of the stack, I wanted to find some code +that would be run and "enhance" it with so many debug prints that I would +understand what was the state of various parts of the Xwayland's "stacked xkb". +I kind of knew that one part of this is *somewhere* in the XWayland server, +whose code felt intimidating, so I wanted to determine where to start nudging +it from what APIs the clients use. + +.. tip:: + + Downloading various packages in Linux is rather simple, as well as their + rebuilding. In Arch, I can get the package's PKGBUILD just with ``yay -G + package``, build it with ``makepkg`` and install it using ``yay -U + package-….pkg.tar``. Makepkg also has some options which allow me to tweak + the source code after downloading it and before building it. + + Other distros are probably similar, e.g. for Debian-based distros one can + use ``apt source``, ``debuild`` and ``apt install`` to do the same. + +One interesting observation is that contrary to my expectation, XWayland does +not seem to use libxkbcommon (according to the ``/proc/PID/maps`` file). This +can have several reasons, but its source code also contains a slightly tweaked +version of (Xorg-style) xkb, which might mean I will be dealing with `the ugly +code`__ :-/ (Actually, XWayland might not process key events itself, instead +just passing them to clients, but this seems inconsistent with the issue – what +else would be introducing the second group switch?) + + +.. understanding xev, XKeyEvents &c. +.. Note: neither xev is very barebones and does very little postprocessing of + events. (naturally.) +.. X11: state: 0x2000, sway: 0x4000 or 0x0. These state bits are *not* + mentioned in libX11 docs… (bits 13 and 14) + +.. TO-MENTION: + - nested compositor issues + - the IM resolves compose? (I could bisect it, but too lazy) + - copy my issue comment in case link rots. + - list of all the packages I had downloaded and compiled