|
|
Do not forget about IPv6 DNS
|
|
|
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
|
|
|
|
|
:slug: forgetting-dns6
|
|
|
:date: 2023-10-28 23:31
|
|
|
:tags: ipv6-only, dns
|
|
|
:category: networking
|
|
|
:keywords: dns, ipv6, deployment, bug
|
|
|
:lang: en
|
|
|
:translation: false
|
|
|
:status: draft
|
|
|
|
|
|
Do you think IPv6-only internet works OK? I am going to tell you that it does
|
|
|
not, but it is not immediately visible. TL;DR: The internet can be broken also
|
|
|
by forgetting to add AAAA records of the *nameservers*. This creates IPv4
|
|
|
requirement for the resolving even when the target is reachable using IPv6.
|
|
|
|
|
|
Quick recap
|
|
|
===========
|
|
|
|
|
|
Connecting to a website is easy, right? You type in the name, you get the front page.
|
|
|
|
|
|
.. figure:: {static}/images/forgetting-dns6/image1.svg
|
|
|
:width: 50%
|
|
|
|
|
|
This is a very naïve idea of connecting to a server.
|
|
|
|
|
|
OK, it is a bit harder: the computer needs an IP address, so we need to use
|
|
|
this magic box called DNS. The flow looks something like this:
|
|
|
|
|
|
.. figure:: {static}/images/forgetting-dns6/image2.svg
|
|
|
:width: 50%
|
|
|
|
|
|
Slightly better, now we at least know the machine-readable address.
|
|
|
|
|
|
And for IPv6-only, everything on the picture has to have IPv6 connectivity and AAAA DNS records.
|
|
|
|
|
|
Reaching IPv4 land from IPv6-only
|
|
|
---------------------------------
|
|
|
|
|
|
There are :s:few many sites that still only support IPv4. To reach them, we
|
|
|
need someone, who can reach both the IPv4- and IPv6-land, to go there on our
|
|
|
behalf – a proxy. This proxy can be ad-hoc (I often use ``ssh -D``), or there
|
|
|
are well-known protocols like NAT64 with DNS64 to do that in a standard and
|
|
|
lightweight manner. [#nat44]_
|
|
|
In that case, the connection looks like this:
|
|
|
|
|
|
.. figure:: {static}/images/forgetting-dns6/image3.svg
|
|
|
:width: 100%
|
|
|
|
|
|
And now we can reach the whole internet.
|
|
|
|
|
|
You might already know that you need some workaround like this to reach GitHub.
|
|
|
What I think you didn't know, you need similar workaround to reach the Wikipedia.
|
|
|
|
|
|
Disclaimer: While I am sad that GitHub lives in the past and it is stupid that
|
|
|
they do not have IPv6, I do not want to shame Wikipedia in particular.
|
|
|
It is just an example I found out recently. I am aware of several other
|
|
|
sites suffering from the same problem, including at least one IPv6 test. [#test-aaaa]_ (It would
|
|
|
be nice if they added the missing piece in the puzzle, though.)
|
|
|
|
|
|
Enter DNS
|
|
|
=========
|
|
|
|
|
|
Our picture has one unexplored magic box: the DNS. As per the definition (which
|
|
|
I just made up and was not bothered to even fully formulate):
|
|
|
|
|
|
> yada yada distributed database of records attached to the strings – domain
|
|
|
names. The records hold various information about the domain, depending on the type.
|
|
|
|
|
|
There are three interesting types of records: A records give IPv4 addresses,
|
|
|
AAAA give IPv6 addresses, and NS give names of servers who know about the
|
|
|
particular subtree of the database. And to actually resolve the final AAAA
|
|
|
record, the (recursive) resolver starts at the *root zone* and tries to find
|
|
|
the answer. [#dns-simplification]_ The resolution algorithm can be visualised like this:
|
|
|
|
|
|
.. figure:: {static}/images/forgetting-dns6/image4.svg
|
|
|
:width: 100%
|
|
|
|
|
|
Yeah, it's a mess.
|
|
|
|
|
|
There is one extra tricky bit: the NS records contain *names*, not addresses,
|
|
|
so when resolving, we need *two* queries for each layer (very simplified):
|
|
|
first we ask for the final domain (``blog.ledoian.cz``) and get a NS record
|
|
|
(when the server does not have the answer) and then we need to ask for the A or
|
|
|
AAAA record of the name from that record, so that we can connect to the server
|
|
|
mentioned in the NS record.
|
|
|
|
|
|
You might start to see the issue. When the DNS was just a black box, we could
|
|
|
paint the whole picture green and call it a day. And from the regular user's
|
|
|
point of view, that is the case, just use some public DNS like 1.1.1.1, 8.8.8.8
|
|
|
or 9.9.9.9. Oh, right, I meant these easy-to-remember addresses:
|
|
|
2606:4700:4700::1111, 2001:4860:4860::8888 and 2620:fe::fe, respectively. The
|
|
|
point is, they will give you the answer, because they are dual-stack, not IPv6-only.
|
|
|
|
|
|
In a way, those servers (or other dual-stack resolvers) act like another proxy,
|
|
|
similar to the SSH, NAT64 and NAT44 ones mentioned earlier. This may not be
|
|
|
much of a problem for many people. But if you have any reason to use your own
|
|
|
recursive DNS server (privacy reasons, DNSSEC validation, ISP provides bad
|
|
|
service, you are the ISP, …) *inside* an IPv6-only network, you *will* have
|
|
|
issues. [#dns-behind-nat64]_
|
|
|
|
|
|
Example: Wikipedia
|
|
|
==================
|
|
|
|
|
|
Let's now see this in action. You know Wikipedia, right? And you can reach
|
|
|
Wikipedia on IPv6, right? It has an AAAA record (don't mind the CNAME, that
|
|
|
means that the server is really called some other way)::
|
|
|
|
|
|
$ dig en.wikipedia.org AAAA
|
|
|
[…]
|
|
|
en.wikipedia.org. 18737 IN CNAME dyna.wikimedia.org.
|
|
|
dyna.wikimedia.org. 323 IN AAAA 2a02:ec80:600:ed1a::1
|
|
|
|
|
|
And this record does work::
|
|
|
|
|
|
$ ncat --ssl 2a02:ec80:600:ed1a::1 443 <<GO
|
|
|
GET /wiki/Main_Page HTTP/1.1
|
|
|
Host: en.wikipedia.org
|
|
|
|
|
|
GO
|
|
|
HTTP/1.1 200 OK
|
|
|
[…]
|
|
|
content-language: en
|
|
|
content-type: text/html; charset=UTF-8
|
|
|
content-length: 98078
|
|
|
|
|
|
<!DOCTYPE html>
|
|
|
[…]
|
|
|
|
|
|
But we can dig deeper: let's see what servers we are really asking::
|
|
|
|
|
|
$ dig en.wikipedia.org AAAA +trace
|
|
|
|
|
|
; <<>> DiG … <<>> en.wikipedia.org AAAA +trace
|
|
|
;; global options: +cmd
|
|
|
. 78918 IN NS e.root-servers.net.
|
|
|
. 78918 IN NS f.root-servers.net.
|
|
|
. 78918 IN NS g.root-servers.net.
|
|
|
. 78918 IN NS h.root-servers.net.
|
|
|
. 78918 IN NS i.root-servers.net.
|
|
|
. 78918 IN NS j.root-servers.net.
|
|
|
. 78918 IN NS k.root-servers.net.
|
|
|
. 78918 IN NS l.root-servers.net.
|
|
|
. 78918 IN NS m.root-servers.net.
|
|
|
. 78918 IN NS a.root-servers.net.
|
|
|
. 78918 IN NS b.root-servers.net.
|
|
|
. 78918 IN NS c.root-servers.net.
|
|
|
. 78918 IN NS d.root-servers.net.
|
|
|
;; Received 525 bytes from … in 0 ms
|
|
|
|
|
|
org. 172800 IN NS c0.org.afilias-nst.info.
|
|
|
org. 172800 IN NS a2.org.afilias-nst.info.
|
|
|
org. 172800 IN NS a0.org.afilias-nst.info.
|
|
|
org. 172800 IN NS b0.org.afilias-nst.org.
|
|
|
org. 172800 IN NS b2.org.afilias-nst.org.
|
|
|
org. 172800 IN NS d0.org.afilias-nst.org.
|
|
|
;; Received 788 bytes from 202.12.27.33#53(m.root-servers.net) in 24 ms
|
|
|
|
|
|
wikipedia.org. 3600 IN NS ns0.wikimedia.org.
|
|
|
wikipedia.org. 3600 IN NS ns1.wikimedia.org.
|
|
|
wikipedia.org. 3600 IN NS ns2.wikimedia.org.
|
|
|
;; Received 658 bytes from 2001:500:48::1#53(b2.org.afilias-nst.org) in 20 ms
|
|
|
|
|
|
en.wikipedia.org. 86400 IN CNAME dyna.wikimedia.org.
|
|
|
;; Received 94 bytes from 208.80.153.231#53(ns1.wikimedia.org) in 132 ms
|
|
|
|
|
|
Hey, there are IPv4 addresses in there! I know, this is cheating, the output is
|
|
|
run from a dual-stack machine. But we can still simulate IPv6-only resolution
|
|
|
by adding ``-6`` flag::
|
|
|
|
|
|
$ dig en.wikipedia.org AAAA +trace -6
|
|
|
|
|
|
; <<>> DiG … <<>> en.wikipedia.org AAAA +trace -6
|
|
|
;; global options: +cmd
|
|
|
. 78915 IN NS d.root-servers.net.
|
|
|
. 78915 IN NS e.root-servers.net.
|
|
|
. 78915 IN NS f.root-servers.net.
|
|
|
. 78915 IN NS g.root-servers.net.
|
|
|
. 78915 IN NS h.root-servers.net.
|
|
|
. 78915 IN NS i.root-servers.net.
|
|
|
. 78915 IN NS j.root-servers.net.
|
|
|
. 78915 IN NS k.root-servers.net.
|
|
|
. 78915 IN NS l.root-servers.net.
|
|
|
. 78915 IN NS m.root-servers.net.
|
|
|
. 78915 IN NS a.root-servers.net.
|
|
|
. 78915 IN NS b.root-servers.net.
|
|
|
. 78915 IN NS c.root-servers.net.
|
|
|
;; Received 525 bytes from … in 0 ms
|
|
|
|
|
|
org. 172800 IN NS d0.org.afilias-nst.org.
|
|
|
org. 172800 IN NS c0.org.afilias-nst.info.
|
|
|
org. 172800 IN NS b2.org.afilias-nst.org.
|
|
|
org. 172800 IN NS a0.org.afilias-nst.info.
|
|
|
org. 172800 IN NS b0.org.afilias-nst.org.
|
|
|
org. 172800 IN NS a2.org.afilias-nst.info.
|
|
|
;; Received 816 bytes from 2001:500:2::c#53(c.root-servers.net) in 8 ms
|
|
|
|
|
|
wikipedia.org. 3600 IN NS ns0.wikimedia.org.
|
|
|
wikipedia.org. 3600 IN NS ns1.wikimedia.org.
|
|
|
wikipedia.org. 3600 IN NS ns2.wikimedia.org.
|
|
|
couldn't get address for 'ns0.wikimedia.org': not found
|
|
|
couldn't get address for 'ns1.wikimedia.org': not found
|
|
|
couldn't get address for 'ns2.wikimedia.org': not found
|
|
|
dig: couldn't get address for 'ns0.wikimedia.org': no more
|
|
|
|
|
|
|
|
|
Some of those IPv4 addresses were benign – the respective servers are reachable
|
|
|
both using IPv4 and IPv6 address, or there is an alternative server that is
|
|
|
reachable using IPv6. That is the case for the root nameserver – in the second
|
|
|
case, we used C, which has IPv6 address (2001:500:2::c). In fact, the M server
|
|
|
also has IPv6 address, but dig chose the IPv4 one (it should not matter)::
|
|
|
|
|
|
$ dig m.root-servers.net AAAA
|
|
|
[…]
|
|
|
m.root-servers.net. 77991 IN AAAA 2001:dc3::35
|
|
|
|
|
|
But the latter case is the bigger issue. For the domain ``wikipedia.org`` there
|
|
|
are three nameservers::
|
|
|
|
|
|
$ dig wikipedia.org NS -6
|
|
|
[…]
|
|
|
wikipedia.org. 86400 IN NS ns0.wikimedia.org.
|
|
|
wikipedia.org. 86400 IN NS ns1.wikimedia.org.
|
|
|
wikipedia.org. 86400 IN NS ns2.wikimedia.org.
|
|
|
|
|
|
This resolution is the last one that worked in IPv6-only mode, because none of
|
|
|
these three servers has AAAA record (some of them may have IPv6, which we do not learn about)::
|
|
|
|
|
|
$ dig ns0.wikimedia.org AAAA
|
|
|
[…]
|
|
|
;; Got answer:
|
|
|
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59468
|
|
|
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
|
|
|
|
|
|
The NOERROR status says the domain name exists, but we got zero answers for
|
|
|
AAAA records. This is the case for all three nameservers. And here is the
|
|
|
ultimate picture of what is happening and what goes wrong.
|
|
|
|
|
|
.. figure:: {static}/images/forgetting-dns6/image5.svg
|
|
|
:width: 100%
|
|
|
|
|
|
The breakage in action
|
|
|
|
|
|
Also note that the connection from the laptop to the DNS resolver may in fact
|
|
|
consist of a chain of several (caching, non-recursive) DNS resolvers, so that
|
|
|
the final DNS resolver can have dual-stack connectivity.
|
|
|
|
|
|
The problems with this state
|
|
|
============================
|
|
|
|
|
|
So, what is the deal. We *just* need to have a dual-stack DNS resolver
|
|
|
somewhere, and that's it, no? Well, yes but actually no.
|
|
|
|
|
|
There are two problems with this: First, this means that any new ISP needs to
|
|
|
have *at least some* IPv4 address, even if they intend to just use IPv6
|
|
|
services. IPv4 addresses are scarce, `expensive
|
|
|
<https://blog.apnic.net/2021/12/16/opinion-ipv4-address-markets/>`__ and small
|
|
|
blocks `don't route well
|
|
|
<https://labs.ripe.net/author/stephen_strowes/visibility-of-ipv4-and-ipv6-prefix-lengths-in-2019/>`__,
|
|
|
which is not great both from the
|
|
|
new ISP's and from overal routing's point of view. It also hinders IPv6
|
|
|
deployment and postpones IPv4 abandonment, needlessly.
|
|
|
|
|
|
The second issue is that this is not very visible. We are building IPv6 world,
|
|
|
but deep inside, it still relies on IPv4, which might lead to great surprise
|
|
|
when we start cutting off IPv4 internet. And it might lead to false sense of
|
|
|
having IPv6 deployed, which is not true to the whole extent.
|
|
|
|
|
|
Insert "It was DNS" meme here.
|
|
|
|
|
|
Solution
|
|
|
========
|
|
|
|
|
|
The solution of this state is simple: get IPv6 connectivity to your
|
|
|
authoritative DNS server (or use another) and do not forget to add an AAAA
|
|
|
record for it in DNS. If the DNS server already has IPv6, it is probably just
|
|
|
adding a single line to the zone file (and a second one for the DNSSEC
|
|
|
signature), which should not be a big deal.
|
|
|
|
|
|
Unfortunately, this needs to be done for the whole DNS chain.
|
|
|
Especially domain names at universities are infamous for very nested domains.
|
|
|
A domain name may looks like
|
|
|
``machine.department.location.faculty.university.some-common.suffix``. That
|
|
|
tree is deep, and so is the resolution of this problem.
|
|
|
|
|
|
Amusing bug of almost good deployment
|
|
|
=====================================
|
|
|
|
|
|
We have seen there may be multiple NS records for a domain, and thus
|
|
|
multiple nameservers. This is good for redundancy. But this does not mean that
|
|
|
the servers will have the same records – they are only supposed to give
|
|
|
equivalent answers.
|
|
|
|
|
|
I have come across a silly misconfiguration: a domain which has several
|
|
|
nameservers, which serve a *slightly* different set of NS records for its
|
|
|
subdomain. Specifically, the servers which were only reachable using IPv4 were
|
|
|
*exactly* the servers that knew about one additional nameserver for the
|
|
|
subdomain, which, incidentally, was the *only* one that was IPv6-capable.
|
|
|
|
|
|
So, while all the correct records were present in DNS (somewhat/somewhere), this still
|
|
|
meant that IPv6-only resolution was doomed to fail, because the IPv6 nameserver
|
|
|
chain was broken.
|
|
|
|
|
|
-----
|
|
|
|
|
|
.. [#nat44] This is very much the same as when you try to reach the
|
|
|
IPv4-public-land from IPv4-private-land, that is, from a private range of IP
|
|
|
addresses. This is called either just NAT, or NAT44, meaning IPv4-to-IPv4 NAT.
|
|
|
|
|
|
.. [#test-aaaa] There are several more tests that do not even have the AAAA
|
|
|
record, lol.
|
|
|
|
|
|
.. [#dns-simplification] In my example, there is a recursive DNS resolver external to my machine,
|
|
|
in order not to complicate it too much. Yes, the real deployment is often
|
|
|
trickier.
|
|
|
|
|
|
.. [#dns-behind-nat64] I have not yet tried to run a recursive DNS in a network
|
|
|
with DNS64 and NAT64. Could be fun :-D My wild guess is that I would need
|
|
|
CLAT (i.e. the full 464XLAT deployment) to make that work, since the
|
|
|
resolver is connecting directly to IPv4 addresses and would need to learn to
|
|
|
use NAT64 to resolve them. (The CLAT could be built right into the resolver,
|
|
|
though).
|