CNAME not displayed with -ports flag

[JoshuaMart]

HTTPX Version : v1.2.1

Hi,
When the -ports option is used with -cname it is not displayed anymore

Current Behavior:

root@5131dca14ebb: echo "www.dnsimple.com" | httpx -silent -cname
https://www.dnsimple.com [dnsimple.com]

root@5131dca14ebb: echo "www.dnsimple.com" | httpx -silent -cname -ports 80,443
http://www.dnsimple.com
https://www.dnsimple.com

Expected Behavior:

That the cname is also displayed with the -ports option

Steps To Reproduce:

echo "www.dnsimple.com" | httpx -silent -cname -ports 80,443

Regards

[jimen0]

The difference comes from the following snippet:

+	gologger.Info().Msgf("[-] About to get DNS data of domain=%s", domain)
	ips, cnames, err := getDNSData(hp, domain)
	if err != nil {
		ips = append(ips, ip)
	}
+	gologger.Info().Msgf("[-] IPs=%#+v, cnames=%#+v, err=%v", ips, cnames, err)

$ echo "www.dnsimple.com" | httpx -cname -ports 80,443
[INF] [-] About to get DNS data of domain=https://www.dnsimple.com:443
[INF] [-] About to get DNS data of domain=https://www.dnsimple.com:80
[INF] [-] IPs=[]string{}, cnames=[]string(nil), err=<nil>
http://www.dnsimple.com
[INF] [-] IPs=[]string{}, cnames=[]string(nil), err=<nil>
https://www.dnsimple.com

...

$ echo "www.dnsimple.com" | httpx -cname              

[INF] [-] About to get DNS data of domain=www.dnsimple.com
[INF] [-] IPs=[]string{"104.245.210.170"}, cnames=[]string{"dnsimple.com"}, err=<nil>
https://www.dnsimple.com [dnsimple.com]

The difference between valid and empty results comes from the argument with which fastdialer's GetDNSData method is being called. The hostnames I used below are the ones httpx passes to it as you can see in the above debug logs I added to httpx. Here is a minimal reproducer:

$ go run ./cmd/fastdialer/main.go
[] []
[104.245.210.170] []

package main

import (
	"fmt"

	"github.com/projectdiscovery/fastdialer/fastdialer"
)

func main() {
	fd, err := fastdialer.NewDialer(fastdialer.DefaultOptions)
	if err != nil {
		panic(err)
	}

	data, err := fd.GetDNSData("https://www.dnsimple.com:443")
	if err != nil {
		panic(err)
	}

	fmt.Println(data.A, data.AAAA)

	data, err = fd.GetDNSData("www.dnsimple.com")
	if err != nil {
		panic(err)
	}

	fmt.Println(data.A, data.AAAA)
}

[jimen0]

Hi @Mzack9999 @ehsandeep - based on the above, is fastdialer's GetDNSData(hostname string) expected to be able to work with strings such as https://www.dnsimple.com:443 or should it always receive just DNS names such as www.dnsimple.com? Once you answer that, I can work on a patch and submit it.

If we go with the last one, which is the one that sounds right to me, the fix goes in httpx by cleaning up the hostname before passing it to fastdialer.

Thanks!

[Mzack9999]

@jimen0 Unfortunately, I've seen your comment only now, and meanwhile, a solution was already worked at #625. Your suggestion is 100% correct and should be implemented shortly. Thanks for your help!