CrawlCheck vs URX: Which Is Better in 2026?

CrawlCheck is a free GEO (generative engine optimization) audit tool that shows exactly how AI crawlers see your website. Paste any domain and get an instant report: which of 61 AI crawlers are allowed, blocked, or ignored by your robots.txt, whether your llms.txt file exists and is well formed, and whether your sitemap is healthy and discoverable.

AI search is becoming a real traffic source. Chatbots like ChatGPT, Claude, Perplexity and Google AI Overviews fetch pages before they answer, and they obey very different rules from Googlebot. A robots.txt that looks fine for classic search can silently lock out GPTBot, ClaudeBot, PerplexityBot or Google-Extended, which means your content never enters the index behind those answers.

CrawlCheck checks all of this in one scan. You get a per-crawler breakdown that separates allowed, blocked and ignored agents, a check on your llms.txt and sitemap, and a short list of concrete fixes ordered by impact. Every scan produces a shareable report link you can send to a client, a developer or your team, so the fix list turns into a work item instead of a vague warning.

The report flags the exact user-agent lines that are missing or disallow rules that are too broad, points out a missing or malformed llms.txt, catches sitemap errors such as blocked entries or a stale lastmod, and gives copy-paste snippets for the most common setups, including Cloudflare, Next.js, WordPress and plain static hosting. Each finding is labelled with a severity level so you can fix the blockers first and leave the cosmetic items for later.

The tool is built for SEO teams, content marketers, developer-facing SaaS founders and indie makers who want their pages to be cited by AI search engines, not just ranked by classic search. It runs on Cloudflare Workers, needs no account, and returns results in a few seconds. Visit crawlcheck.dev to run a free audit on your own site.

What problem it solves

A robots.txt that looks fine for Googlebot can silently lock out GPTBot, ClaudeBot, PerplexityBot and Google-Extended, so your content never enters the index behind AI answers.

Alternatives & competitors

Manual robots.txt checkers, Screaming Frog, and generic SEO audit tools that do not model individual AI crawler user-agents.

URX gives app makers and marketers one Smart Link to share across iPhone, Android and web audiences. It sends iPhone visitors to the App Store, Android visitors to Google Play, and everyone else to a website. Basic store routing works without modifying the app. Use one address on launch pages, social profiles, emails, posters and other places where sharing several download links would be awkward.

Destinations remain editable while the shared link stays the same. A printed QR code can therefore stay useful when the app store or web destination changes. URX includes QR code exports and a Bio Page for presenting links together. Click analytics show device, country and source information so makers can understand where traffic comes from. These are click reports, not promises of installs or sales.

The free account includes 25 links, two projects, one Bio Page, 1,000 monthly clicks and 30 days of analytics. After the free allowance, visitors may see a short forwarding page containing ads or sponsors before continuing. The free tier is available without a credit card, but creating and managing links requires an account.

Pro starts at $9 per month and adds deep linking, routing rules and a longer analytics history. Growth starts at $29 per month and adds custom domains, team seats and API access. Android App Links require an owned domain on Growth. Deferred deep linking requires a first-launch request from the app; this integration requirement is separate from basic device-based store routing. Current pricing is at https://urx.si/pricing.

URX also provides a Growth Plan with marketing Missions and Visibility tools for directory discovery and checking mentions in AI answers. These tools support planning and monitoring without guaranteeing listings, rankings or AI recommendations. The product serves app makers, indie developers, founders, agencies and creators.

The public demo at https://urx.si/demo shows a routing decision, including matching rule and destination, and stops before redirecting. Visitors can explore that demo before signing up. URX is operated by Apexmart LLC; its public contact is hello@urx.si.