Bulk RDAP

ben

ben

Well-known member
Joined
Jul 8, 2024
Posts
826
Reaction score
524
Trophy points
94
Built this for my own internal tooling, as I couldn't find a decent reliable one focused on UK domains.

Features:
  • Bulk .UK RDAP lookups
  • Current holding TAG
  • Registrar name
  • Original registering TAG, where available
  • Last transfer date, where available
  • Registration status
  • Registry statuses
  • Expiry date
  • Expected drop date and exact UTC drop time
  • Pending delete detection
  • Registered / no registration found highlighting
  • Search, filter and sort results
  • Copy results
  • Custom CSV export with selectable columns
  • Mobile-friendly results view
  • Runs within Nominet’s rate limits, on JS so you get 1000 queries a day per your IP.
Basically, a lightweight quick tool to get the info you need.

The export to CSV is nice, you can select which info you need on the CSV too. Give it a whirl and see if it's useful for you.

Bulk RDAP tool
 
Nice ... I finished a python script a few days and had it processing a massive city, town & village list and checking to see if was registered in either .co.uk or .uk
It processed another list with about 65'000 lines in (1 domain = 2 lines ).. processed that in under 8 hours :D
 
As a little tip, depending on what you coded it in one way to massively reduce a list without affecting your limit just to see if a domain exists or not is to perform a dig query... a DNS resolver will return NXDOMAIN if the domain isn't registered.
That way you can perform a hell of a lot of simultaneous queries by using a list of IP's like google, cloudflare etc DNS resolvers and just doing a round robin for each domain in the list.
 
  • Like
Reactions: ben
ILoveDotUK said:
As a little tip, depending on what you coded it in one way to massively reduce a list without affecting your limit just to see if a domain exists or not is to perform a dig query... a DNS resolver will return NXDOMAIN if the domain isn't registered.
That way you can perform a hell of a lot of simultaneous queries by using a list of IP's like google, cloudflare etc DNS resolvers and just doing a round robin for each domain in the list.
Click to expand...
Genuinely great advice. This was more for internal tooling for us, we want to see which domains were moving to what tags etc. but this is interesting solution to the problem you described.
 
I can send a few variants of my script if you want...
I've just checked 13490 girls names in both extensions in less than 2 hours lol
 
  • Like
Reactions: ben
ILoveDotUK said:
I can send a few variants of my script if you want...
I've just checked 13490 girls names in both extensions in less than 2 hours lol
Click to expand...
Did you find anything good
 
Quite a few lol

Heres a couple ( still free )
emmanuelle.uk
immy.uk
kaitlyn.uk
 
Kaitlyn is a good one
 
they are 3 I just pulled at random lol there is some awesome names available
Just running a benchmark test at the moment .. Not too shabby 😂
1784825331231.png
 
  • Love
Reactions: ben
About to test it with 500k & 1M queries .. the above finished within no time at all and only 20 workers 😂
 
Kaleido said:
This is immense. Top work kidda 👏

Is there any chance the results window can be expanded to fit rather than the current scrollable letterbox?
Click to expand...
Yeah, I’m gonna work on a better layout. I think I’ll probably get rid of the input section and replace it with the results section dynamically.

You can export all the info though, which is sometimes easier to look at. I might also all export to PDF which would but a lot easier to read
 
ILoveDotUK said:
a DNS resolver will return NXDOMAIN if the domain isn't registered.
Click to expand...
DNS will return NXDOMAIN if there is no zone, has no (direct) relationship to whether or not it is registered.

You can get a result for a name that doesn't exist - because it has a zone/cache on that resolver ( and a result that is incorrect )
- both common issues with CloudFlare.

You can also get nxdomain for a registered name for numerous reasons like
If it has no nameservers at the registry
is on certain hold types
the nameservers afecdown/firewalled/rate-limiting
the zone is cortipt/missing

Plus there are domains not registered, and also not available.

So dns can give you a rough-guide, but is not as reliable as an available checkat a registry to see if a domain is registerable and not registered.

DIG at the root may get you closer than asking a resolver :)
 
yeah I know but that's my point regarding bulk domain availability checks ...
A DIG removes a massive amount of checks on domains that would otherwise waste RDAP queries..
Makes sense to push all domains through a simple DIG +short first and any that return a DNS record are registered and can be added to a registered list and therefore don't require an RDAP query.
Any that return NXDOMAIN can then the queried via RDAP ..
I've checked a LOT of domains over the last week and wouldn't have been able to check anywhere near to 300k domains if I were simply querying Nominet for the status of them all
 
Available to members and non-members.

The .UK zone files – Registrar Resources

Introduction The zone file is available to all and contains files for the .uk, .co.uk, .org.uk, .me.uk, .ltd.uk, .plc.uk, .net.uk and .sch.uk zones, subject to standard licence terms. We will also provide a list of all domains currently registered with Nominet. This will include domains that are...
registrars.nominet.uk registrars.nominet.uk

"We will also provide a list of all domains currently registered with Nominet. This will include domains that are suspended, or do not have nameservers attached and are therefore not included in zone files."
 
Darren said:
Available to members and non-members.

The .UK zone files – Registrar Resources

Introduction The zone file is available to all and contains files for the .uk, .co.uk, .org.uk, .me.uk, .ltd.uk, .plc.uk, .net.uk and .sch.uk zones, subject to standard licence terms. We will also provide a list of all domains currently registered with Nominet. This will include domains that are...
registrars.nominet.uk registrars.nominet.uk

"We will also provide a list of all domains currently registered with Nominet. This will include domains that are suspended, or do not have nameservers attached and are therefore not included in zone files."
Click to expand...

😮 Even easier then lol.. nice
 
You must log in or register to reply here.
Back
Top