Nslookup Reported "Can't Resolve '(Null)': Name Does Not Resolve" Though It Successfully Resolved The Dns Names

nslookup reported can't resolve '(null)': Name does not resolve though it successfully resolved the DNS names

This is a bug/oddity in nslookup. The "can't resolve" message is actually about the DNS server in use, not the site you are trying to look up.

For example this query (which tells nslookup to lookup google.com using the 8.8.8.8 DNS server) has no error message:

nslookup google.com 8.8.8.8
Server:    8.8.8.8
Address 1: 8.8.8.8 dns.google

Name:      google.com
Address 1: 172.217.164.110 sfo03s18-in-f14.1e100.net
Address 2: 2607:f8b0:4005:80b::200e sfo03s18-in-x0e.1e100.net

But this query (in which the DNS server is "null") does show the "error":

UAP-AC-LR1-BZ.v4.0.42# nslookup google.com
nslookup: can't resolve '(null)': Name does not resolve

Name:      google.com
Address 1: 172.217.164.110 sfo03s18-in-f14.1e100.net
Address 2: 2607:f8b0:4005:80b::200e sfo03s18-in-x0e.1e100.net

Admittedly this is misleading/confusing, and really should be fixed in nslookup.

Why dig does not resolve K8s service by dns name while nslookup has no problems with it?

Answer

Dig does not complete the query by default with the search path. The search path is set in /etc/resolv.conf. The +search flag enables the search path completion.

From the Man Pages

+[no]search

Use [do not use] the search list defined by the searchlist or domain directive in resolv.conf (if any). The search list is not used by default.

https://linux.die.net/man/1/dig

Demonstration

I have created a scenario for katacoda which goes through the same example interactively https://www.katacoda.com/bluebrown/scenarios/kubernetes-dns

First create and expose a pod, then start another pod interactively with dnsutils installed, from which DNS queries can be made.

kubectl create namespace dev
kubectl run my-app --image nginx --namespace dev --port 80
kubectl expose pod my-app --namespace dev
kubectl run dnsutils --namespace dev --image=bluebrown/netutils --rm -ti

Nslookup resolves the service OK

$ nslookup my-app
...
Name:   my-app.dev.svc.cluster.local
Address: 10.43.52.98

But dig didn't get an answer, why?

$ dig my-app
...
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
...

In order to understand why dig doesn't find the service, let's take a look at /etc/resolv.conf

$ cat /etc/resolv.conf
search dev.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.43.0.10
options ndots:5

This file contains a line with the following format.

search <namespace>.svc.cluster.local svc.cluster.local cluster.local

That means, when providing an incomplete part of the fully qualified domain name (FQDN), this file can be used to complete the query. However, dig doesn't do it by default. We can use the +search flag in order to enable it.

dig +search my-app
...
;; QUESTION SECTION:
;my-app.dev.svc.cluster.local.  IN      A

;; ANSWER SECTION:
my-app.dev.svc.cluster.local. 5 IN      A       10.43.52.98

Now the service-name has been correctly resolved. You can also see how the query has been completed with the search path by comparing the question section of this command with the previous one without +search flag.

We can get the same service without +search flag when using the FQDN. The +short flag isn't required, but it will reduce the output to only the IP address.

$ dig +short my-app.dev.svc.cluster.local
10.43.52.98

However, the benefit of using the search method it that queries will automatically resolve to resources within the same namespace. This can be useful to apply the same configuration to different environments, such as production and development.

The same way the search entry in resolv.conf completes the query with the default name space, it will complete any part of the FQDN from left to right. So in the below example, it will resolve to the local cluster.

$ dig +short +search my-app.dev
10.43.52.98

DNS resolve failing for specific domains. Domains work with nslookup

"you can try and use a different library, the DnsClient (nuget) instead, see dnsclient.michaco.net. The domain names in question seem to work just fine " -
@MichaC

The problem was in fact the Arsoft.Tools.Net library I was using. Switching to DnsClient.Net fixed the problem.

Kubernetes DNS lookg not working from worker node - connection timed out; no servers could be reached

First of all, according to the docs - please note that Calico and kubeadm support Centos/RHEL 7+.

In both Calico and kubeadm documentation we can see that they only support RHEL7+.

By default RHEL8 uses nftables instead of iptables ( we can still use iptables but "iptables" on RHEL8 is actually using the kernel's nft framework in the background - look at "Running Iptables on RHEL 8").

9.2.1. nftables replaces iptables as the default network packet filtering framework

I believe that nftables may cause this network issues because as we can find on nftables adoption page:

Kubernetes does not support nftables yet.

Note: For now I highly recommend you to use RHEL7 instead of RHEL8.

---

With that in mind, I'll present some information that may help you with RHEL8.

I have reproduced your issue and found a solution that works for me.

• First I opened ports required by Calico - these ports can be found
  here under "Network requirements".
  
  As workaround:
• Next I reverted to the old iptables backend on all cluster
  nodes, you can easily do so by setting FirewallBackend in
  /etc/firewalld/firewalld.conf to iptables as described
  
  here.
• Finally I restarted firewalld to make the new rules active.

I've tried nslookup from Pod running on worker node (kworker) and it seems to work correctly.

root@kmaster:~# kubectl get pod,svc -o wide
NAME      READY   STATUS    RESTARTS   AGE    IP           NODE      NOMINATED NODE   READINESS GATES
pod/web   1/1     Running   0          112s   10.99.32.1   kworker   <none>           <none>

NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE     SELECTOR
service/kubernetes   ClusterIP   10.99.0.1    <none>        443/TCP   5m51s   <none>
root@kmaster:~# kubectl exec -it web -- bash
root@web:/# nslookup kubernetes.default
Server:         10.99.0.10
Address:        10.99.0.10#53

Name:   kubernetes.default.svc.cluster.local
Address: 10.99.0.1

root@web:/#

MySQL is not resolving IP address, but nslookup does resolve

There's forward lookups, like resolving an A record, and there's reverse lookups, which involve finding a PTR record. They're often paired together on public networks, but remember that one or more A records can point to the same IP but there should be only one reverse PTR record. Additionally, while the PTR record is supposed to be a resolvable address, this may not be the case, the reverse entry might be invalid or resolve to an entirely different address.

Unless you have a resolver for 10.10.in-addr.arpa, which is not normally the case since that's a reserved address space, you cannot reverse lookup the IP back to the hostname.

With nslookup you can test reverse mapping:

nslookup 10.10.1.1

Where that address is whatever you're trying to reverse.

---

Related Topics