Competitive intelligence from domain expiry dates

The client ran a portfolio of web properties in a competitive niche where domains themselves were strategic assets. Competitors would register domains containing valuable keywords, build minimal sites to hold rankings, and sometimes let them expire when they lost interest. The client wanted two things: a way to know exactly what their competitors were doing with their domains, and a way to catch valuable domains the moment they became available. Both required constant monitoring, and both were being done by a person checking WHOIS records manually a few times a week — which meant they were always late.

competitor monitoring

We built two interlocking systems. The first is a competitor domain monitor. It takes a list of competitor domains — usually a few hundred — and runs daily checks via the WHOIS XML API. For each domain, it tracks registration dates, expiry dates, nameserver changes, DNS record shifts, and registrar transfers. Any change triggers an alert. If a competitor switches nameservers, that might mean a site migration. If a domain's expiry date passes without renewal, that's an acquisition opportunity. The system maintains a full history for each domain, so the client can see patterns — like a competitor consistently letting domains lapse in a particular keyword category.

discovery and buyout

The second system handles domain discovery and buyout intelligence. The client defines brand keywords — terms they want to own across the domain space. The system then monitors domain expiration feeds, matching expired or expiring domains against those keyword patterns. When a match is found, it checks availability in real time, scores the domain based on keyword relevance and historical metrics, and surfaces it to the team with a priority flag. The goal is to shrink the window between "this domain just dropped" and "we registered it" from days to minutes.

filtering noise

The biggest challenge was noise. Brand keywords like common industry terms match hundreds of irrelevant domains daily. A keyword like "crypto" would flag domains about cryptocurrency, cryptography, cryptozoology, and someone's cat named Crypto. We built a multi-layer filtering pipeline: first a pattern match, then a relevance scorer that considers domain length, TLD, hyphenation patterns, and whether the surrounding words make sense together. This brought false positives down from roughly 80% to under 15%, though the team still reviews flagged domains before acting.

whois and api limits

WHOIS data itself turned out to be unreliable in ways we hadn't fully anticipated. Privacy protection services mask ownership details for a growing number of domains. Some registrars return stale data. Expiry dates occasionally shift without explanation — a domain that looked like it was expiring next week suddenly shows a renewal two years out. We built the system to treat WHOIS data as probabilistic rather than authoritative: changes are flagged, but the system explicitly marks confidence levels and recommends manual verification for high-value targets.

Staying within API budgets required careful orchestration. Monitoring hundreds of domains daily, plus running expiration feeds and availability checks, could easily consume the entire monthly API allocation in a week. We implemented priority-based scheduling: tier-one competitors and high-value keyword domains get daily checks, tier-two gets every three days, and the long tail runs weekly. The system also deduplicates API calls — if a domain appears in both the competitor list and the keyword match feed, it only gets checked once.

results

The result is a competitive intelligence layer that runs continuously and has already caught several high-value domain acquisitions the client would have missed entirely. The honest lesson: domain intelligence is fundamentally a signal-to-noise problem. The automation handles the noise; the value comes from the team knowing which signals matter.

Frontend & API: Next.js 14 (Route Handlers), Prisma, PostgreSQL

Domain Intelligence: WHOIS XML API (registration, DNS, expiry monitoring)

Scheduling: cron-based monitoring jobs with priority tiers