DnsName::try_from_str incorrectly disallows domains of length 254 that have a trailing .

[zacknewman]

As the below code shows, DnsName::try_from_str disallows domains of length 254 that have a trailing .:

use rustls_pki_types::DnsName;
fn main() {
    // This proves that `DnsName::try_from_str` allows and even retains trailing `.`.
    assert!(DnsName::try_from_str("example.com.").map_or(false, |dom| dom.as_ref() == "example.com."));
    let long_label =
        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    assert_eq!(long_label.len(), 63);
    // The maximum length of a domain is 254 if a trailing `'.'` exists; otherwise the max length is 253.
    let mut long_domain = format!("{long_label}.{long_label}.{long_label}.{long_label}");
    long_domain.pop();
    long_domain.pop();
    long_domain.push('.');
    assert_eq!(long_domain.len(), 254);
    // This incorrectly `panic`s despite being a valid domain.
    assert!(DnsName::try_from_str(long_domain.as_str()).is_ok());
}

DnsName::try_from_str allows for trailing dots; therefore the maximum length of a DnsName is 254, not 253, when there is a trailing dot. The maximum length is 253 only when there isn't a trailing dot. This can easily be explained by the fact that the maximum length of a domain in wire format is 255 since domains in wire format are serialized such that each label is preceded by one byte that represents its length. This is necessary since DNS is an octet protocol that allows all octets to exist in a label; furthermore, the only label that is allowed a length of 0 is the root label. This root label is necessary so a resolver knows when there are no more labels in a domain.

When domains are in a representation format—which is what DnsName::try_from_str is based on—that disallows "escape characters", then it's one of two forms:

1. <label_1>.<label_2>...<label_n>
2. <label_1>.<label_2>...<label_n>.

When both 1. and 2. are transformed into wire format, they produce the same output:

[label_1_len, label_1, label_2_len, label_2, ..., label_n_len, label_n, 0] where label_i_len is the one byte value inclusively between 1 and 63 that represents the number of bytes that make up label_i and label_i is shorthand for each byte that it consists of. From this, we see that the length of such a domain is the sum of lengths of each label plus the total number of labels plus 1.

Now we need to derive a function that maps the length of 1. and 2. to the length of the wire-format value since it is the wire format value that matters. We are assuming that the representation format is "simple" (e.g., there are no "escape characters" and each "character" is encoded in 1 byte (e.g., ASCII)).

1. The total length of the domain is equal to the sum of the lengths of each label plus n - 1. This is 2 less than the actual domain in wire format. Since the max length of a domain in wire format is 255, this means the max length is 255 - 2 = 253.
2. The total length of the domain is equal to the sum of the lengths of each label plus n. This is 1 less than the actual domain in wire format. Since the max length of a domain in wire format is 255, this means the max length is 255 - 1 = 254.

If the above is all too technical and "math-y", then here are "proofs by example":

Using domain which is a library for DNS libraries (i.e., it conforms to RFC 9499 and related RFCs). It is maintained by the same company that maintains popular DNS software like Unbound and NSD:

use domain::base::Name;
fn main() {
    let long_label = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    assert_eq!(long_label.len(), 63);
    let mut long_domain = format!("{long_label}.{long_label}.{long_label}.{long_label}");
    long_domain.pop();
    long_domain.pop();
    long_domain.push('.');
    assert_eq!(long_domain.len(), 254);
    assert!(Name::<Vec<u8>>::from_chars(long_domain.chars()).is_ok());
}

Using idna which is a library for IDNA according to WHATWG URL Standard:

use idna::uts46::{AsciiDenyList, DnsLength, Hyphens, Uts46};
fn main() {
    let long_label = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    assert_eq!(long_label.len(), 63);
    let mut long_domain = format!("{long_label}.{long_label}.{long_label}.{long_label}");
    long_domain.pop();
    long_domain.pop();
    long_domain.push('.');
    assert_eq!(long_domain.len(), 254);
    assert!(Uts46::new()
        .to_ascii(
            long_domain.as_bytes(),
            // This is a proper superset of characters that are allowed by [`DnsName::try_from_str`].
            // This is not relevant though since we are simply showing that domains of length 254 with
            // a trailing `.` are valid.
            AsciiDenyList::URL,
            // Even though `DnsName::try_from_str` disallows hyphens in certain positions, we don't here.
            // Again, this is irrelevant for this example since we are dealing with a domain that only contains
            // the letter a.
            Hyphens::Allow,
            DnsLength::VerifyAllowRootDot
        )
        .is_ok());
}

Using ascii_domain which is a library maintained by me that deals strictly with "representation format" domains:

use ascii_domain::{char_set::PRINTABLE_ASCII, dom::Domain};
fn main() {
    let long_label = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    assert_eq!(long_label.len(), 63);
    let mut long_domain = format!("{long_label}.{long_label}.{long_label}.{long_label}");
    long_domain.pop();
    long_domain.pop();
    long_domain.push('.');
    assert_eq!(long_domain.len(), 254);
    // `PRINTABLE_ASCII` is a proper superset of what `DnsName::try_from_str` allows, but that is
    // irrelevant for this example since we are dealing with a domain that only contains
    // the letter a.
    assert!(Domain::try_from_bytes(long_domain.as_bytes(), &PRINTABLE_ASCII).is_ok());
}