A PoC Guide To Authenticating WiFi Clients Using Kerberos @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ :language: en :date: 2023-02-11 :tags: poc, network, guide :slug: kerberos-wifi-poc-rst .. FIXME: remove "-rst" from slug TODO: I hate myself for letting this indent using spaces. FIXME: - links with ``…`` - multiline links - anchors - footnotes I have been recently helping someone setting up a WiFi which would authenticate users using a centrally deployed Kerberos. It turned out to be a bit painful with lots of dead ends, so this is meant to save time anyone reading this.[^If anyone can even find this post, that is…] .. TODO: video link Disclaimer: I definitely cannot claim in-depth understanding of what is happening. Please take anything here with a grain of salt, I might be wrong. Also, as the title says, this is just a PoC setup. It works, but is very much hacked together, not nice, probably not secure, etc. I discuss security improvements at the end of the post. I also do not inherently say that you need all the steps written below. I only document, what ended up working for me in the end, so there might be some extraneous settings that are not really used. While this post contains a few storytelling parts and feels like being written chronologically, it was written post hoc and occasionally explains stuff I did not know when experimenting with this. I tried to help the reader undestand what is happening more than to describe how I got there.[^Also, I pretty much just randomly searched the web and tried snippets, the important part is how I eventually understood what I had done.] In other words, this should be understood as "RADIUS and Kerberos setup for the impatient by an impatient". I didn't find anything, so I wrote it myself. What we want in the end ======================= Obviously, the goal is to have some WiFi end devices[^I will try to avoid the word "client" in order not to confuse it with a RADIUS client, which is the WiFi access point.]. They connect to an access point (AP) and authenticate with their username and password using IEEE 802.1X (a.k.a WPA-Enterprise). The credentials then get verified using the central Kerberos, and if they are correct, the end device is allowed to use the network. The 802.1X standard (or maybe only its WPA variation) uses authentication using RADIUS, which is unfortunately different from Kerberos, so there is a RADIUS server in the middle which forwards the credentials to Kerberos. (This has an unfortunate security deficiency of RADIUS server handling cleartext passwords). .. TODO: overall architecture scheme I do not dive into the rest of the network configuration like IP addressing scheme and assignment, DNS etc. While in my testing a DHCP setup was involved, it is out of scope for this post. Terminology ----------- .. TODO: radius server, what secrets do we use, …, what authentication/links are between the components. Technologies used ----------------- - A laptop with a WiFi adapter which supports AP mode (look for "Supported interface modes" in ``iw phy``) - Arch Linux (various versions), Debian bullseye - hostapd for AP serving - FreeRADIUS for RADIUS server - MIT Kerberos (krb5) as a test kerberos setup - Random WiFi devices In my PoC deployment most of this runs on localhost, but it should not be hard to distribute services to different machines. .. this feels so wrong… .. _Setup: A note about setup and testing -------------------------------------------------- I decided to migrate to VM halfway through, when I realized I would need to deploy my own Kerberos realm. I had the hostapd + cleartext in RADIUS working on bare metal Arch and tested with a few devices, so that seemed trustworthy enough to do the rest of the testing only using ``radtest`` against FreeRADIUS and either cleartext passwords or Kerberos in the VM. I first set it up on Arch, but it did not work, so I tried whether Debian would have more-working configuration out-of-box. It didn't, but now I know the differences between the systems. So the final deployment is with a Debian VM. In the end, I ended up pointing the bare-metal hostapd on Arch to the virtualized RADIUS server to test it end-to-end. I am stripping my IP addresses, both for security reasons and because my network setup is actually more complicated than this, so it would be confusing. If you can run everything on localhost (either bare-metal, or with some kind of passing your WiFi adapter to the VM), I believe it should run fine. Hostapd: The easy part of the setup =================================== Installing hostapd is simple, it is probably just ``hostapd`` in the repositories. (Yes, one package even for Debian…) Setting up hostapd is quite straight forward: set WiFi SSID and let it forward request to the RADIUS server. Its `default configuration file `__ is quite well documented in comments, my only changes to ``hostapd.conf``. are:: ssid=testX auth_algs=3 ieee8021x=1 eapol_version=2 eap_message=hello wpa=2 wpa_key_mgmt=WPA-EAP wpa_pairwise=TKIP CCMP # RADIUS config auth_server_addr=127.0.0.1 auth_server_port=1812 auth_server_shared_secret=testing123 # Optionally also set acct_server_{addr,port,shared_secret} Arch has also changed some paths compared to the official tarball, but that seems unimportant. Side note: Authenticating against hostapd directly -------------------------------------------------- Hostapd is actually capable of performing basic authentication itself. While it AFAIK cannot forward any credentials to other services, for basic testing this is sufficient. The simpler option is classic pre-shared key:: auth_algs=1 wpa_key_mgmt=WPA-PSK wpa_passphrase=TheVerySecurePreSharedKey If we want to authenticate by name and password, it can also do that. Instead of setting ``auth_server_*`` settings, generate a CA key, a private key and a certificate for it (so that the CA certifies the server certificate), set paths and direct hostapd to the user file:: # This should probably be different file. I have no idea what I did and whether this is any secure. PoC stuff :-) ca_cert=/etc/hostapd/server.pem server_cert=/etc/hostapd/server.pem private_key=/etc/hostapd/server.key eap_user_file=/etc/hostapd/hostapd.eap_user The key generation is out of scope for this post, mostly because I struggled with it too much and then stole the keys from the FreeRADIUS anyway. You can try running various commands from `ArchWiki `__ and maybe be more successful.[^Learning OpenSSL syntax seemed like too much work to me, even though it is quite useful :-)] The ```hostapd.eap_user`` `__ file contains some pre-defined credentials, they probably work, I do not know half the authentication mechanisms. For authenticating as ``hello:world`` I ended up with this (I found this somewhere on the internet, sorry, I cannot probably find source):: "hello" PEAP [ver=0] "hello" MSCHAPV2 "world" [2] At this point, the WiFi should be running, visible and it should be possible to connect. FreeRADIUS basic deployment =========================== Installing the base of FreeRADIUS is also simple, in both distributions it is sufficient to install the ``freeradius`` package and it pulls any dependencies. You also need to bootstrap certificates, which Debian does during the installation, on Arch you need to run the ``/etc/raddb/certs/bootstrap`` script yourself (which is about `the only thing ArchWiki tells you `__). You could now run the bundled systemd service (and on Debian it runs by default), but for debugging it is `recommended `__ to run ``radiusd -X`` in console and see its output. (Also see `Caveat 1 `__.) To test RADIUS, you can just follow the `Getting Started `__ guide, but I'll try to sum it up: First, add your user in ``/etc/raddb/users`` (I put it somewhere around the line with the example "bob" user):: clear Cleartext-Password := "password" Again, this file is quite heavily commented, which helps a bit. Then we need to tell FreeRADIUS to allow connecting from the outside. The default configuration in ``/etc/raddb/clients.conf`` probably has it uncommented:: client localhost { ipaddr = 127.0.0.1 secret = testing123 # Possibly other settings, but e.g. localhost_ipv6 does not have anything more configured, so this might be sufficient. } Note that the secret is shared with the "client", i.e. the authenticator, hostapd for us. You can now test the setup with ``radtest clear password 127.0.0.1 0 testing123`` – it should end with the line "Received Access-Accept". (The "0" parameter should be a "nas-port-number", zero worked for me… Given the comments for port in ``/etc/raddb/clients.conf``, it might just look up the 1812 port in ``/etc/services``.) With this and hostapd setup, the ``clear:password`` credentials should also work for accessing the WiFi. What is happening ----------------- At this point, I think that it is reasonable to understand a few bits of ``radiusd -X`` output and configuration. Configuration basics ```````````````````` The configuration is split across multiple files, but really the main one is just ``radiusd.conf``, which contains core configuration and then includes the rest: - The configuration of each "site" (instance) is in ``sites-enabled/`` - Each module has its configuration in ``mods-enabled/``, but for some of them (e.g. files) it is split also to ``mods-config/`` - Known clients are defined in ``clients.conf`` - The ``users`` file is just a symlink to ``mods-config/files/authorize`` - I did not need to care for the other files included How a user gets authenticated depends on the site configuration, and is performed in several phases: 1. Authorize phase (and the ``authorize { }`` block) describes, which modules should try matching the request. If the request matches for some module, it gets an ``Auth-Type`` assigned, which seems to be an abstract way of describing a method of authentication. If no module matches, FreeRADIUS falls back to rejecting the client. (I do not know whether this can be changed.) In our case, we want ``files`` to try to match. 2. Depending on the Auth-Type, the relevant module from ``authenticate`` block is used to perform the authentication. Modules in both sections can return one of several results like ``ok``, ``notfound``, ``reject``, ``fail``, etc. (for example see `rlm_krb5 `__ description).[^Sometimes, the modules are prefixed with ``rlm_``. As I understand it, the module is really called like ``rlm_files`` and has a corresponding ``.so`` in ``/lib/freeradius``, but for most of the configuration the prefix is dropped.] Some modules do not implement ``authorize`` section (rlm_krb5), some do not implement ``authenticate`` (files). If I understand this correctly, some modules cannot match requests and some cannot verify the passwords. The files module is quite interesting in this manner, since it seems to only add the password as a hint and leaves the verification itself to PAP (or maybe some other module depending on authentication method, e.g. ``radtest -t …``) There are other sections relevant for proxying and accounting, I did not care about those. The configuration itself is written in "unlang" which seems to be a DSL to describe the sequences outlined above. By default, there are two sites enabled: the default one, and an "inner-tunnel". The latter is used for EAP and similar protocols, which tunnel the request to another server. I think the configuration should be quite similar for both sites, but it allows for differences. The radiusd output `````````````````` At the start, radiusd just reads its config and dumps it to console. The semi-important parts of the dumped configuration: - ``main`` → ``security`` contains user and group which FreeRADIUS setuid's into, so it does not run as root. - At one point, all the ``Auth-Type``'s are created. - Sometime later, the ``files`` module gets loaded:: # Loaded module rlm_files # Loading module "files" from file /etc/freeradius/3.0/mods-enabled/files files { filename = "/etc/freeradius/3.0/mods-config/files/authorize" acctusersfile = "/etc/freeradius/3.0/mods-config/files/accounting" preproxy_usersfile = "/etc/freeradius/3.0/mods-config/files/pre-proxy" } - And then the module gets "instantiated":: # Instantiating module "files" from file /etc/freeradius/3.0/mods-enabled/files reading pairlist file /etc/freeradius/3.0/mods-config/files/authorize reading pairlist file /etc/freeradius/3.0/mods-config/files/accounting reading pairlist file /etc/freeradius/3.0/mods-config/files/pre-proxy We will later see similar lines for the krb5 module, but it is not enabled yet. After the initialization, FreeRADIUS is "Ready to process requests". The handling of each request has its sequence number in parentheses at the start of each line. When handling the request, we can read: - The request itself:: (0) Received Access-Request Id 109 from 127.0.0.1:46187 to 127.0.0.1:1812 length 75 (0) User-Name = "clear" (0) User-Password = "password" (0) NAS-IP-Address = 127.0.1.1 (0) NAS-Port = 0 (0) Message-Authenticator = 0x01fc48c3e3a30442c018651db62ddcce - The processing of the ``authorize`` phase:: (0) # Executing section authorize from file /etc/freeradius/3.0/sites-enabled/default … (0) files: users: Matched entry clear at line 91 (0) [files] = ok … (0) [pap] = updated (0) } # authorize = updated (0) Found Auth-Type = PAP We see that the ``Auth-Type`` got set to PAP (Password authentication protocol) - The processing of ``authenticate`` (Why it is described as "group" I do not know):: (0) # Executing group from file /etc/freeradius/3.0/sites-enabled/default (0) Auth-Type PAP { (0) pap: Login attempt with password (0) pap: Comparing with "known good" Cleartext-Password (0) pap: User authenticated successfully (0) [pap] = ok (0) } # Auth-Type PAP = ok Auth-Type was PAP, so the relevant section matched, the only module there matched the password and we are in. - The response:: (0) Sent Access-Accept Id 109 from 127.0.0.1:1812 to 127.0.0.1:46187 length 0 - Inbetween some sections which I did not need to care about. If the credentials in our setup are not correct, either the PAP rejects a bad password, or files do not even match a bad name and an Auth-Type is not set. Kerberos ======== Here starts the fun and ugliness. I did not have any working Kerberos realm, nor was in a position to persuade admins of one to trust my machine with handling user's credentials. (I do not know, whether I really need to handle the passwords in cleartext, but given how Kerberos tickets are created, I think I do. And AFAIK EAP cannot tunnel Kerberos protocol.) So we end up deploying our custom realm. Installation: ``krb5`` package on Arch, ``krb5-kdc`` and ``krb5-admin-server`` on Debian. Debian's installation scripts ask some questions and then generate the config file, but I ended up following `ArchWiki `__ anyway. My ``/etc/krb5.conf``:: [libdefaults] default_realm = TEST.LEDOIAN.CZ [realms] TEST.LEDOIAN.CZ = { admin_server = localhost kdc = localhost } I left the definition of other realms in place in the Debian deployment, they should not interfere. Creating the realm (with various prompts, just commands):: # kdb5_util -r TEST.LEDOIAN.CZ create -s # systemctl enable --now krb5-kdc krb5-kadmind # kadmin.local kadmin.local: addprinc krbuser@TEST.LEDOIAN.CZ kadmin.local: addprinc -randkey host/localhost@TEST.LEDOIAN.CZ kadmin.local: ktadd host/localhost@TEST.LEDOIAN.CZ # kinit krbuser # klist The ``klist`` command should show that we got a valid ticket. Thus is the deployment tested as working. Note that the keytab used is by default ``/etc/krb5.keytab``, which is probably not meant for FreeRADIUS to use. And in fact, the ``freerad`` user does not have permissions to read it, yet… Also, we are abusing the host principal. That is probably bad, but for PoC deployment it will suffice (sadly)… Hooking Kerberos into FreeRADIUS ================================ Now this part is really awful, ugly, nasty, etc. You have been warned… We will need to get quite creative, since most other people just use either ntlm_auth, or LDAP, since the use-case is connecting to a Windows Active Directory[^Well, in the end this *will* be connected to an Active Directory, but I do not want to try deploying an ad hoc instance of neither LDAP or AD, so I stick to hacking Kerberos into my setup. Hopefully, the switch is not too hard. (Also, EAP-GSSAPI would be nice, but I am not aware of it being implemented.)]. The `wiki page for rlm_krb5 `__ is quite unhelpful, really, and the documentation `at networkradius `__, while being a bit longer, only contains fragments of hints of what needs to be done, like > In order to use Kerberos authentication, the administrator must manually set ``control:Auth-Type := krb5``. Where the hell should I put that? The simple part --------------- Enabling the krb5 module seems like a good idea:: cd mods-enabled ln -s ../mods-available/krb5 The module has no configuration in ``mods-config``, so we edit ``mods-available/krb5`` directly. The comments `there `__ are quite helpful luckily, so we just change two lines to semi-reasonable values:: krb5 { # … keytab = /etc/krb5.keytab service_principal = host/localhost # … } Also, as the wiki page said, we tweak ``authenticate`` block in the default site:: authenticate { Auth-Type Kerberos { krb5 } # … the rest of values } But it did not work, no Auth-Type was found when authorizing, so no running of ``authenticate`` block. And there was that ``control:Auth-Type`` stuff, which should be somewhere? The awful hacks --------------- So, naturally, I ended up permuting various options ("rlm_krb5" vs. just "krb5" vs. "Kerberos", different options in the ``users`` file), most of which either did not load (``radiusd`` refused to start) or did not use the intended paths, so the Kerberos database was not contacted. I kind of knew that I need to force FreeRADIUS to set Auth-Type to krb5 (not really, read on), but I had no idea how. I found `a stub wiki page about Auth-Type `__ which mostly says that I should not want to set it directly, so I disregarded that. What helped, was a snippet on `deploying FreeRADIUS with FreeIPA `__, which was setting the ``control:Auth-Type`` for LDAP. Long story short: the following nasty trick worked: The ``users`` file can specify action for fallback (``DEFAULT``), which I succesfully abused just by putting ``DEFAULT`` by itself towards the end of the file (line 200 for me). No options, no filters, just nothing, catch-all. Now we have a known module that will match, so we use the FreeIPA snippet in the ``authorize`` section by putting the following just after the ``files`` directive:: # authorize { … files if ((ok || updated) && User-Password) { update { control:Auth-Type := Kerberos } } # … } If I understand this, when the flow finishes processing ``files`` module, it checks whether the current status is ``ok`` or ``updated`` and the request contained a password, and if so, updates Auth-Type to be Kerberos. While this means that we will not be able to use any other users from the ``users`` file, we reach the correct ``authenticate`` block and use ``krb5`` module! I think that in the end, if you only intend to authenticate Kerberos users, this hack does not get much in the way. And you still can restrict the ``DEFAULT`` rule to only match specific suffixes (or maybe other rules) to limit its reach. Progress: the error message has changed --------------------------------------- Of course at this point I was oblivious to the fact that ``radiusd`` setuid's to ``freerad`` user, since the documentation said about "user running radiusd", so now I got a Permission denied in the ``krb5`` module. One ``strace`` and ``cat /proc/XXXX/status`` later, I became aware, so the last bit was:: apt install acl setfacl -m u:freerad:r /etc/krb5.keytab Did I say this was a really hacky PoC deployment in a VM? Isn't this just awful? But it works, at least with ``radtest``… EAP and a few more tweaks in configurations =========================================== The last issue is that EAP does not specify PAP as valid mechanism, so we cannot use that directly. So while ``radtest`` happily uses that and hands the password over to FreeRADIUS (PAP is the default method), which can do Kerberos stuff to authenticate it, with EAP we need to get a bit creative. EAP-PWD looks promising by the abbreviation, but alas – see `Caveat 5 `__. (If FreeRADIUS knows the password, as is the case with Cleartext-Password, it can do many of the challenge-response methods of verifying shared knowledge of password without sharing it. That is the reason this worked on my bare-metal configuration, but does not work with Kerberos anymore.) Luckily, my Android phone has died and at this point I only had a laptop with IWD to test my setup, so the only thing I could do was reading ```iwd.network`` manpage `__. I noticed that when using TTLS[^`Not to be confused with Twinkle, Twinkle, Little Star `__ :-)], it is allowed to use ``Tunneled-PAP`` in the second phase. I think I needed to do similar tweaks to ``sites-enabled/inner-tunnel`` on the FreeRADIUS side, add the external hostapd to ``clients.conf`` and set correct IP in ``hostapd.conf``, and my final config for IWD (``/var/lib/iwd/testX.8021x``) is like this:: [Settings] AutoConnect=false Hidden=false [Security] EAP-Method=TTLS EAP-Identity=anonymous EAP-TTLS-Phase2-Method=Tunneled-PAP EAP-TTLS-Phase2-Identity=krbuser EAP-TTLS-Phase2-Password=pw IWD complains in a log a bit, but it succeeds eventually:: EAP server tried method 4 while client was configured for method 21 EAP completed with eapSuccess It worked, so I didn't bother to read ``radiusd`` output. Final deployment differences from stock config (patches or whole configs) ========================================================================= Again, a disclaimer: it is PoC deployment. It should work, you should not use it. It is insecure, horrible and may kill your puppy. Do not apply directly to a production server, this is for your information only; yada yada yada… Also, the patches may contain extraneous information and have been tweaked to not contain any real IP addresses. Please use your brain (or at least eyes and read `above `__ for the rationale). These files do not describe complete deployment (else this post would not be needed). Sometimes, the patches are created against a template (if Debian generates the final config from it). Here goes. (Patches are ended with ``.patch``) .. TODO: create links, add patches - FreeRADIUS (on Debian): - ```/etc/freeradius/3.0/clients.conf`` <>`__ - ```/etc/freeradius/3.0/sites-enabled/default`` <>`__ - ```/etc/freeradius/3.0/sites-enabled/inner-tunnel`` <>`__ - ```/etc/freeradius/3.0/mods-enabled/krb5`` <>`__ (symlinked from ``../mods-available/krb5``) - ```/etc/freeradius/3.0/users`` <>`__ (or wherever that symlink leads) - Kerberos (on Debian) - ```/etc/krb5.conf`` <>`__ - hostapd (on Arch) - ```/etc/hostapd/hostapd.conf`` <>`__ - IWD config file (on an Arch WiFi device) - ```/var/lib/iwd/testX.8021x`` <>`__ Caveats ============================= 1. Differences between distros: while FreeRADIUS configuration resides at ``/etc/raddb`` in Arch, it is located at ``/etc/freeradius/3.0`` in Debian. The directory structure seems to be the same. Likewise, the server binary is ``radiusd`` on Arch and ``freeradius`` on Debian. 2. Debian packaging shenanigans: while the configuration for FreeRADIUS's rlm_krb module is bundled in the ``freeradius-config`` package (pulled by ``freeradius``), the module itself is in ``freeradius-krb5``. So you can enable the config even without having the module. Arch bundles everything in the ``freeradius`` package. 3. RADIUS realms have nothing to do with Kerberos realms. 4. The ``users`` file allows you to specify ``Auth-Type``, but I either do not understand its syntax (which might not be unlang), or it just does nothing – it did not assign ``Auth-Type`` for me with the default user. 5. EAP-PWD does not send password, it is a cryptographic way to just prove knowledge of it. This makes it unusable for us, since FreeRADIUS does not know any passwords – Kerberos does. Security considerations and improvement tips ============================================ A few tips on how to make the deployment more useful, secure, production-ready etc. - Use reasonable secrets, not the defaults. This is true for all parts: Kerberos realm, RADIUS clients, user passwords. - Use reasonable permissions for files. - Authenticate the RADIUS by pinning the TLS certificate in the WiFi network configuration. (Both IWD and wpa_supplicant can definitely do that.) Otherwise, if someone creates a rogue AP, they get to know the passwords in clear, because we are tunnelling PAP. - Use a dedicated Kerberos keytab and principal for FreeRADIUS, do not abuse the host one. (hint: ``kadmin.local -t keytab``) - Secure access to FreeRADIUS as much as possible. AFAIK Kerberos expects that passwords are only known to Kerberos AS and the other party. - Think whether you really cannot avoid handling and sending cleartext passwords (possibly by not using Kerberos in the first place). If you can find a way to avoid that with Kerberos, please let me know. Also, `solve the real issue `__. - Try understand yourself, what is happening and what is security-critical. Understand, how this relates to your threat model, what are the risks etc. - Read the documentation, do not be like me, especially if you are deploying this to production. (I only need a PoC and will not be the one deploying this for real.) - Do not use that horrible hack with default user. Find some better way of matching requests that should be authenticated using Kerberos. - Do not blindly believe a random guy on the internet. (I might be wrong, misguided and/or malicious, and you would never know; in all cases I would present this as trying to help you.) References ========== Other links =========== - Somehow, I avoided unlang documentation before writing this post: ``__