From 87eb1c13003f277973ee27e9f30eb4b46470a6fb Mon Sep 17 00:00:00 2001 From: Pavel 'LEdoian' Turinsky Date: Tue, 28 Feb 2023 20:36:50 +0100 Subject: [PATCH] WIP: kerberos tutorial (rst version) --- content/wifi-kerberos.rst | 633 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 633 insertions(+) create mode 100644 content/wifi-kerberos.rst diff --git a/content/wifi-kerberos.rst b/content/wifi-kerberos.rst new file mode 100644 index 0000000..cc121d1 --- /dev/null +++ b/content/wifi-kerberos.rst @@ -0,0 +1,633 @@ +A PoC Guide To Authenticating WiFi Clients Using Kerberos +@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ + +:language: en +:date: 2023-02-11 +:tags: poc, network, guide +:slug: kerberos-wifi-poc-rst + +.. FIXME: remove "-rst" from slug + TODO: I hate myself for letting this indent using spaces. + FIXME: - links with ``…`` + - multiline links + - anchors + - footnotes + +I have been recently helping someone setting up a WiFi which would authenticate +users using a centrally deployed Kerberos. It turned out to be a bit painful +with lots of dead ends, so this is meant to save time anyone reading this.[^If +anyone can even find this post, that is…] + +.. TODO: video link + +Disclaimer: I definitely cannot claim in-depth understanding of what is +happening. Please take anything here with a grain of salt, I might be wrong. +Also, as the title says, this is just a PoC setup. It works, but is very much +hacked together, not nice, probably not secure, etc. I discuss security +improvements at the end of the post. + +I also do not inherently say that you need all the steps written below. I only +document, what ended up working for me in the end, so there might be some +extraneous settings that are not really used. + +While this post contains a few storytelling parts and feels like being written +chronologically, it was written post hoc and occasionally explains stuff I did +not know when experimenting with this. I tried to help the reader undestand +what is happening more than to describe how I got there.[^Also, I pretty much +just randomly searched the web and tried snippets, the important part is how I +eventually understood what I had done.] + +In other words, this should be understood as "RADIUS and Kerberos setup for the +impatient by an impatient". I didn't find anything, so I wrote it myself. + +What we want in the end +======================= + +Obviously, the goal is to have some WiFi end devices[^I will try to avoid the +word "client" in order not to confuse it with a RADIUS client, which is the +WiFi access point.]. They connect to an access point (AP) and authenticate with +their username and password using IEEE 802.1X (a.k.a WPA-Enterprise). The +credentials then get verified using the central Kerberos, and if they are +correct, the end device is allowed to use the network. + +The 802.1X standard (or maybe only its WPA variation) uses authentication using +RADIUS, which is unfortunately different from Kerberos, so there is a RADIUS +server in the middle which forwards the credentials to Kerberos. (This has an +unfortunate security deficiency of RADIUS server handling cleartext +passwords). + +.. TODO: overall architecture scheme + +I do not dive into the rest of the network configuration like IP addressing +scheme and assignment, DNS etc. While in my testing a DHCP setup was involved, +it is out of scope for this post. + +Terminology +----------- + +.. TODO: radius server, what secrets do we use, …, what authentication/links are between the components. + +Technologies used +----------------- + +- A laptop with a WiFi adapter which supports AP mode (look for "Supported + interface modes" in ``iw phy``) +- Arch Linux (various versions), Debian bullseye +- hostapd for AP serving +- FreeRADIUS for RADIUS server +- MIT Kerberos (krb5) as a test kerberos setup +- Random WiFi devices + +In my PoC deployment most of this runs on localhost, but it should not be hard +to distribute services to different machines. + +.. this feels so wrong… + +.. _Setup: + +A note about setup and testing +-------------------------------------------------- + +I decided to migrate to VM halfway through, when I realized I would need to +deploy my own Kerberos realm. I had the hostapd + cleartext in RADIUS working +on bare metal Arch and tested with a few devices, so that seemed trustworthy +enough to do the rest of the testing only using ``radtest`` against FreeRADIUS +and either cleartext passwords or Kerberos in the VM. + +I first set it up on Arch, but it did not work, so I tried whether Debian would +have more-working configuration out-of-box. It didn't, but now I know the +differences between the systems. So the final deployment is with a Debian VM. + +In the end, I ended up pointing the bare-metal hostapd on Arch to the +virtualized RADIUS server to test it end-to-end. + +I am stripping my IP addresses, both for security reasons and because my +network setup is actually more complicated than this, so it would be confusing. +If you can run everything on localhost (either bare-metal, or with some kind of +passing your WiFi adapter to the VM), I believe it should run fine. + +Hostapd: The easy part of the setup +=================================== + +Installing hostapd is simple, it is probably just ``hostapd`` in the +repositories. (Yes, one package even for Debian…) + +Setting up hostapd is quite straight forward: set WiFi SSID and let it +forward request to the RADIUS server. Its `default configuration +file `__ is quite well +documented in comments, my only changes to ``hostapd.conf``. are:: + + ssid=testX + auth_algs=3 + ieee8021x=1 + eapol_version=2 + eap_message=hello + wpa=2 + wpa_key_mgmt=WPA-EAP + wpa_pairwise=TKIP CCMP + + # RADIUS config + auth_server_addr=127.0.0.1 + auth_server_port=1812 + auth_server_shared_secret=testing123 + # Optionally also set acct_server_{addr,port,shared_secret} + +Arch has also changed some paths compared to the official tarball, but that +seems unimportant. + +Side note: Authenticating against hostapd directly +-------------------------------------------------- + +Hostapd is actually capable of performing basic authentication itself. While it +AFAIK cannot forward any credentials to other services, for basic testing this +is sufficient. + +The simpler option is classic pre-shared key:: + + auth_algs=1 + wpa_key_mgmt=WPA-PSK + wpa_passphrase=TheVerySecurePreSharedKey + +If we want to authenticate by name and password, it can also do that. Instead +of setting ``auth_server_*`` settings, generate a CA key, a private key and a +certificate for it (so that the CA certifies the server certificate), set paths +and direct hostapd to the user file:: + + # This should probably be different file. I have no idea what I did and whether this is any secure. PoC stuff :-) + ca_cert=/etc/hostapd/server.pem + server_cert=/etc/hostapd/server.pem + private_key=/etc/hostapd/server.key + + eap_user_file=/etc/hostapd/hostapd.eap_user + +The key generation is out of scope for this post, mostly because I struggled +with it too much and then stole the keys from the FreeRADIUS anyway. You can +try running various commands from +`ArchWiki `__ and maybe be more +successful.[^Learning OpenSSL syntax seemed like too much work to me, even +though it is quite useful :-)] + +The +```hostapd.eap_user`` `__ +file contains some pre-defined credentials, they probably work, I do not know +half the authentication mechanisms. For authenticating as ``hello:world`` I ended +up with this (I found this somewhere on the internet, sorry, I cannot probably find source):: + + "hello" PEAP [ver=0] + "hello" MSCHAPV2 "world" [2] + +At this point, the WiFi should be running, visible and it should be possible to connect. + +FreeRADIUS basic deployment +=========================== + +Installing the base of FreeRADIUS is also simple, in both distributions it is +sufficient to install the ``freeradius`` package and it pulls any dependencies. +You also need to bootstrap certificates, which Debian does during the +installation, on Arch you need to run the ``/etc/raddb/certs/bootstrap`` script +yourself (which is about `the only thing ArchWiki tells +you `__). + +You could now run the bundled systemd service (and on Debian it runs by +default), but for debugging it is +`recommended `__ to run +``radiusd -X`` in console and see its output. (Also see `Caveat 1 `__.) + +To test RADIUS, you can just follow the `Getting +Started `__ guide, but I'll try +to sum it up: First, add your user in ``/etc/raddb/users`` (I put it somewhere around +the line with the example "bob" user):: + + clear Cleartext-Password := "password" + +Again, this file is quite heavily commented, which helps a bit. + +Then we need to tell FreeRADIUS to allow connecting from the outside. The +default configuration in ``/etc/raddb/clients.conf`` probably has it uncommented:: + + client localhost { + ipaddr = 127.0.0.1 + secret = testing123 + # Possibly other settings, but e.g. localhost_ipv6 does not have anything more configured, so this might be sufficient. + } + +Note that the secret is shared with the "client", i.e. the authenticator, hostapd for us. + +You can now test the setup with ``radtest clear password 127.0.0.1 0 testing123`` +– it should end with the line "Received Access-Accept". (The "0" parameter +should be a "nas-port-number", zero worked for me… Given the comments for port +in ``/etc/raddb/clients.conf``, it might just look up the 1812 port in +``/etc/services``.) + +With this and hostapd setup, the ``clear:password`` credentials should also work for accessing the WiFi. + +What is happening +----------------- + +At this point, I think that it is reasonable to understand a few bits of +``radiusd -X`` output and configuration. + +Configuration basics +```````````````````` + +The configuration is split across multiple files, but really the main one is +just ``radiusd.conf``, which contains core configuration and then includes the +rest: + +- The configuration of each "site" (instance) is in ``sites-enabled/`` +- Each module has its configuration in ``mods-enabled/``, but for some of them + (e.g. files) it is split also to ``mods-config/`` +- Known clients are defined in ``clients.conf`` +- The ``users`` file is just a symlink to ``mods-config/files/authorize`` +- I did not need to care for the other files included + +How a user gets authenticated depends on the site configuration, and is +performed in several phases: + +1. Authorize phase (and the ``authorize { }`` block) describes, which modules + should try matching the request. If the request matches for some module, it + gets an ``Auth-Type`` assigned, which seems to be an abstract way of + describing a method of authentication. + + If no module matches, FreeRADIUS falls back to rejecting the client. (I do + not know whether this can be changed.) + + In our case, we want ``files`` to try to match. + +2. Depending on the Auth-Type, the relevant module from ``authenticate`` block is + used to perform the authentication. + +Modules in both sections can return one of several results like ``ok``, +``notfound``, ``reject``, ``fail``, etc. (for example see +`rlm_krb5 `__ +description).[^Sometimes, the modules are prefixed with ``rlm_``. As I understand it, the +module is really called like ``rlm_files`` and has a corresponding ``.so`` in +``/lib/freeradius``, but for most of the configuration the prefix is dropped.] + +Some modules do not implement ``authorize`` section (rlm_krb5), some do not +implement ``authenticate`` (files). If I understand this correctly, some modules +cannot match requests and some cannot verify the passwords. The files module is +quite interesting in this manner, since it seems to only add the password as a +hint and leaves the verification itself to PAP (or maybe some other module +depending on authentication method, e.g. ``radtest -t …``) + +There are other sections relevant for proxying and accounting, I did not care +about those. + +The configuration itself is written in "unlang" which seems to be a DSL to +describe the sequences outlined above. + +By default, there are two sites enabled: the default one, and an +"inner-tunnel". The latter is used for EAP and similar protocols, which tunnel +the request to another server. I think the configuration should be quite +similar for both sites, but it allows for differences. + +The radiusd output +`````````````````` + +At the start, radiusd just reads its config and dumps it to console. The +semi-important parts of the dumped configuration: + +- ``main`` → ``security`` contains user and group which FreeRADIUS setuid's into, + so it does not run as root. +- At one point, all the ``Auth-Type``'s are created. +- Sometime later, the ``files`` module gets loaded:: + + # Loaded module rlm_files + # Loading module "files" from file /etc/freeradius/3.0/mods-enabled/files + files { + filename = "/etc/freeradius/3.0/mods-config/files/authorize" + acctusersfile = "/etc/freeradius/3.0/mods-config/files/accounting" + preproxy_usersfile = "/etc/freeradius/3.0/mods-config/files/pre-proxy" + } + +- And then the module gets "instantiated":: + + # Instantiating module "files" from file /etc/freeradius/3.0/mods-enabled/files + reading pairlist file /etc/freeradius/3.0/mods-config/files/authorize + reading pairlist file /etc/freeradius/3.0/mods-config/files/accounting + reading pairlist file /etc/freeradius/3.0/mods-config/files/pre-proxy + +We will later see similar lines for the krb5 module, but it is not enabled +yet. + +After the initialization, FreeRADIUS is "Ready to process requests". The +handling of each request has its sequence number in parentheses at the start of +each line. + +When handling the request, we can read: + +- The request itself:: + + (0) Received Access-Request Id 109 from 127.0.0.1:46187 to 127.0.0.1:1812 length 75 + (0) User-Name = "clear" + (0) User-Password = "password" + (0) NAS-IP-Address = 127.0.1.1 + (0) NAS-Port = 0 + (0) Message-Authenticator = 0x01fc48c3e3a30442c018651db62ddcce + +- The processing of the ``authorize`` phase:: + + (0) # Executing section authorize from file /etc/freeradius/3.0/sites-enabled/default + … + (0) files: users: Matched entry clear at line 91 + (0) [files] = ok + … + (0) [pap] = updated + (0) } # authorize = updated + (0) Found Auth-Type = PAP + + We see that the ``Auth-Type`` got set to PAP (Password authentication protocol) +- The processing of ``authenticate`` (Why it is described as "group" I do not know):: + + (0) # Executing group from file /etc/freeradius/3.0/sites-enabled/default + (0) Auth-Type PAP { + (0) pap: Login attempt with password + (0) pap: Comparing with "known good" Cleartext-Password + (0) pap: User authenticated successfully + (0) [pap] = ok + (0) } # Auth-Type PAP = ok + + Auth-Type was PAP, so the relevant section matched, the only module there matched the password and we are in. +- The response:: + + (0) Sent Access-Accept Id 109 from 127.0.0.1:1812 to 127.0.0.1:46187 length 0 + +- Inbetween some sections which I did not need to care about. + +If the credentials in our setup are not correct, either the PAP rejects a bad +password, or files do not even match a bad name and an Auth-Type is not set. + +Kerberos +======== + +Here starts the fun and ugliness. I did not have any working Kerberos realm, +nor was in a position to persuade admins of one to trust my machine with +handling user's credentials. (I do not know, whether I really need to handle +the passwords in cleartext, but given how Kerberos tickets are created, I think +I do. And AFAIK EAP cannot tunnel Kerberos protocol.) So we end up deploying +our custom realm. + +Installation: ``krb5`` package on Arch, ``krb5-kdc`` and ``krb5-admin-server`` on +Debian. Debian's installation scripts ask some questions and then generate the +config file, but I ended up following +`ArchWiki `__ anyway. + +My ``/etc/krb5.conf``:: + + [libdefaults] + default_realm = TEST.LEDOIAN.CZ + [realms] + TEST.LEDOIAN.CZ = { + admin_server = localhost + kdc = localhost + } + +I left the definition of other realms in place in the Debian deployment, they should not interfere. + +Creating the realm (with various prompts, just commands):: + + # kdb5_util -r TEST.LEDOIAN.CZ create -s + # systemctl enable --now krb5-kdc krb5-kadmind + # kadmin.local + kadmin.local: addprinc krbuser@TEST.LEDOIAN.CZ + kadmin.local: addprinc -randkey host/localhost@TEST.LEDOIAN.CZ + kadmin.local: ktadd host/localhost@TEST.LEDOIAN.CZ + # kinit krbuser + # klist + +The ``klist`` command should show that we got a valid ticket. Thus is the +deployment tested as working. Note that the keytab used is by default +``/etc/krb5.keytab``, which is probably not meant for FreeRADIUS to use. And in +fact, the ``freerad`` user does not have permissions to read it, yet… + +Also, we are abusing the host principal. That is probably bad, but for PoC +deployment it will suffice (sadly)… + +Hooking Kerberos into FreeRADIUS +================================ + +Now this part is really awful, ugly, nasty, etc. You have been warned… + +We will need to get quite creative, since most other people just use either +ntlm_auth, or LDAP, since the use-case is connecting to a Windows Active +Directory. The `wiki page for +rlm_krb5 `__ is quite unhelpful, +really, and the documentation `at +networkradius `__, +while being a bit longer, only contains fragments of hints of what needs to be done, like + +> In order to use Kerberos authentication, the administrator must manually set ``control:Auth-Type := krb5``. + +Where the hell should I put that? + +The simple part +--------------- + +Enabling the krb5 module seems like a good idea:: + + cd mods-enabled + ln -s ../mods-available/krb5 + +The module has no configuration in ``mods-config``, so we edit +``mods-available/krb5`` directly. The comments +`there `__ +are quite helpful luckily, so we just change two lines to semi-reasonable values:: + + krb5 { + # … + keytab = /etc/krb5.keytab + service_principal = host/localhost + # … + } + +Also, as the wiki page said, we tweak ``authenticate`` block in the default site:: + + authenticate { + Auth-Type Kerberos { + krb5 + } + # … the rest of values + } + +But it did not work, no Auth-Type was found when authorizing, so no running of +``authenticate`` block. And there was that ``control:Auth-Type`` stuff, which +should be somewhere? + +The awful hacks +--------------- + +So, naturally, I ended up permuting various options ("rlm_krb5" vs. just "krb5" +vs. "Kerberos", different options in the ``users`` file), most of which either +did not load (``radiusd`` refused to start) or did not use the intended paths, so +the Kerberos database was not contacted. I kind of knew that I need to force +FreeRADIUS to set Auth-Type to krb5 (not really, read on), but I had no idea +how. I found `a stub wiki page about +Auth-Type `__ which mostly says that +I should not want to set it directly, so I disregarded that. + +What helped, was a snippet on `deploying FreeRADIUS with +FreeIPA `__, +which was setting the ``control:Auth-Type`` for LDAP. + +Long story short: the following nasty trick worked: The ``users`` file can +specify action for fallback (``DEFAULT``), which I succesfully abused just by +putting ``DEFAULT`` by itself towards the end of the file (line 200 for me). No +options, no filters, just nothing, catch-all. + +Now we have a known module that will match, so we use the FreeIPA snippet in +the ``authorize`` section by putting the following just after the ``files`` directive:: + + # authorize { … + files + if ((ok || updated) && User-Password) { + update { + control:Auth-Type := Kerberos + } + } + # … } + +If I understand this, when the flow finishes processing ``files`` module, it +checks whether the current status is ``ok`` or ``updated`` and the request +contained a password, and if so, updates Auth-Type to be Kerberos. While this +means that we will not be able to use any other users from the ``users`` file, we +reach the correct ``authenticate`` block and use ``krb5`` module! + +I think that in the end, if you only intend to authenticate Kerberos users, +this hack does not get much in the way. And you still can restrict the +``DEFAULT`` rule to only match specific suffixes (or maybe other rules) to limit +its reach. + +Progress: the error message has changed +--------------------------------------- + +Of course at this point I was oblivious to the fact that ``radiusd`` setuid's to +``freerad`` user, since the documentation said about "user running radiusd", so +now I got a Permission denied in the ``krb5`` module. One ``strace`` and ``cat +/proc/XXXX/status`` later, I became aware, so the last bit was:: + + apt install acl + setfacl -m u:freerad:r /etc/krb5.keytab + +Did I say this was a really hacky PoC deployment in a VM? Isn't this just +awful? But it works, at least with ``radtest``… + +EAP and a few more tweaks in configurations +=========================================== + +The last issue is that EAP does not specify PAP as valid mechanism, so we +cannot use that directly. So while ``radtest`` happily uses that and hands the +password over to FreeRADIUS (PAP is the default method), which can do Kerberos +stuff to authenticate it, with EAP we need to get a bit creative. EAP-PWD looks +promising by the abbreviation, but alas – see `Caveat 5 `__. + +(If FreeRADIUS knows the password, as is the case with Cleartext-Password, it +can do many of the challenge-response methods of verifying shared knowledge of +password without sharing it. That is the reason this worked on my bare-metal +configuration, but does not work with Kerberos anymore.) + +Luckily, my Android phone has died and at this point I only had a laptop with +IWD to test my setup, so the only thing I could do was reading ```iwd.network`` +manpage `__. +I noticed that when using TTLS[^`Not to be confused with Twinkle, Twinkle, +Little Star `__ :-)], it is allowed to use +``Tunneled-PAP`` in the second phase. + +I think I needed to do similar tweaks to ``sites-enabled/inner-tunnel`` on the +FreeRADIUS side, add the external hostapd to ``clients.conf`` and set correct IP +in ``hostapd.conf``, and my final config for IWD (``/var/lib/iwd/testX.8021x``) is +like this:: + + [Settings] + AutoConnect=false + Hidden=false + + [Security] + EAP-Method=TTLS + EAP-Identity=anonymous + EAP-TTLS-Phase2-Method=Tunneled-PAP + EAP-TTLS-Phase2-Identity=krbuser + EAP-TTLS-Phase2-Password=pw + +IWD complains in a log a bit, but it succeeds eventually:: + + EAP server tried method 4 while client was configured for method 21 + EAP completed with eapSuccess + +It worked, so I didn't bother to read ``radiusd`` output. + +Final deployment differences from stock config (patches or whole configs) +========================================================================= + +Again, a disclaimer: it is PoC deployment. It should work, you should not use +it. It is insecure, horrible and may kill your puppy. Do not apply directly to +a production server, this is for your information only; yada yada yada… + +Also, the patches may contain extraneous information and have been tweaked to +not contain any real IP addresses. Please use your brain (or at least eyes and +read `above `__ for the rationale). These files do not describe complete +deployment (else this post would not be needed). + +Sometimes, the patches are created against a template (if Debian generates the +final config from it). + +Here goes. (Patches are ended with ``.patch``) + +.. TODO: create links, add patches + +- FreeRADIUS (on Debian): + - ```/etc/freeradius/3.0/clients.conf`` <>`__ + - ```/etc/freeradius/3.0/sites-enabled/default`` <>`__ + - ```/etc/freeradius/3.0/sites-enabled/inner-tunnel`` <>`__ + - ```/etc/freeradius/3.0/mods-enabled/krb5`` <>`__ (symlinked from ``../mods-available/krb5``) + - ```/etc/freeradius/3.0/users`` <>`__ (or wherever that symlink leads) +- Kerberos (on Debian) + - ```/etc/krb5.conf`` <>`__ +- hostapd (on Arch) + - ```/etc/hostapd/hostapd.conf`` <>`__ +- IWD config file (on an Arch WiFi device) + - ```/var/lib/iwd/testX.8021x`` <>`__ + +Caveats +============================= + +1. Differences between distros: while FreeRADIUS configuration resides at + ``/etc/raddb`` in Arch, it is located at ``/etc/freeradius/3.0`` in Debian. The + directory structure seems to be the same. Likewise, the server binary is + ``radiusd`` on Arch and ``freeradius`` on Debian. +2. Debian packaging shenanigans: while the configuration for FreeRADIUS's + rlm_krb module is bundled in the ``freeradius-config`` package (pulled by + ``freeradius``), the module itself is in ``freeradius-krb5``. So you can enable + the config even without having the module. Arch bundles everything in the + ``freeradius`` package. +3. RADIUS realms have nothing to do with Kerberos realms. +4. The ``users`` file allows you to specify ``Auth-Type``, but I either do not + understand its syntax (which might not be unlang), or it just does nothing – + it did not assign ``Auth-Type`` for me with the default user. +5. EAP-PWD does not send password, it is a cryptographic way to just prove + knowledge of it. This makes it unusable for us, since FreeRADIUS does not + know any passwords – Kerberos does. + +Security considerations and improvement tips +============================================ + +A few tips on how to make the deployment more useful, secure, production-ready etc. + +- Use reasonable secrets, not the defaults. This is true for all parts: Kerberos realm, RADIUS clients, user passwords. +- Use reasonable permissions for files. +- Authenticate the RADIUS by pinning the TLS certificate in the WiFi network configuration. (Both IWD and wpa_supplicant can definitely do that.) Otherwise, if someone creates a rogue AP, they get to know the passwords in clear, because we are tunnelling PAP. +- Use a dedicated Kerberos keytab and principal for FreeRADIUS, do not abuse the host one. (hint: ``kadmin.local -t keytab``) +- Secure access to FreeRADIUS as much as possible. AFAIK Kerberos expects that passwords are only known to Kerberos AS and the other party. +- Think whether you really cannot avoid handling and sending cleartext passwords (possibly by not using Kerberos in the first place). If you can find a way to avoid that with Kerberos, please let me know. Also, `solve the real issue `__. +- Try understand yourself, what is happening and what is security-critical. Understand, how this relates to your threat model, what are the risks etc. +- Read the documentation, do not be like me, especially if you are deploying this to production. (I only need a PoC and will not be the one deploying this for real.) +- Do not use that horrible hack with default user. Find some better way of matching requests that should be authenticated using Kerberos. +- Do not blindly believe a random guy on the internet. (I might be wrong, misguided and/or malicious, and you would never know; in all cases I would present this as trying to help you.) + +References +========== + +Other links +=========== + +- Somehow, I avoided unlang documentation before writing this post: ``__