Skip to content
Meirra

About MeirraBot

Our web crawler for SEO analysis

What is MeirraBot?

MeirraBot is Meirra's web crawler. It visits websites that are registered on our SEO platform to analyse them for technical issues, content quality, performance, and search engine compliance.

  • Only crawls websites registered by their owners on our platform
  • Does not scrape, index, or redistribute any content
  • Respects robots.txt directives and crawl-delay settings
  • Uses HTTP requests by default — no headless browser for most pages

How to Identify MeirraBot

You will see this user agent in your server access logs when MeirraBot visits your site.

User-Agent string

MeirraBot/1.0 (+https://meirra.com/bot)

How to Control MeirraBot

MeirraBot fully respects robots.txt. You can allow, throttle, or block it using standard directives.

Allow with crawl delay

User-agent: MeirraBot
Allow: /
Crawl-delay: 2

Block completely

User-agent: MeirraBot
Disallow: /

Block specific paths

User-agent: MeirraBot
Allow: /
Disallow: /admin/
Disallow: /api/

You can also email support@meirra.com to request removal or adjust crawl behaviour.

Crawl Behaviour

Fetch methodHTTP GET (no JavaScript rendering by default)
Crawl rateAdaptive — slows down based on server response times
Rate limitingStops after 3 consecutive 429 responses and saves partial results
SitemapReads sitemap.xml first to discover pages — no blind spidering
robots.txtChecked before every crawl — honours Disallow and Crawl-delay
Timeout30 seconds per page — will not hammer unresponsive servers

Verify MeirraBot Requests

To confirm a request is genuinely from MeirraBot and not a spoofed user agent:

  1. Check the User-Agent header matches: MeirraBot/1.0 (+https://meirra.com/bot)
  2. Perform a reverse DNS lookup on the source IP — it should resolve to *.meirra.com
  3. Compare the IP against our published crawler IPs listed below

Crawler IP Addresses

MeirraBot requests originate from the following IP addresses. You can add these to your firewall allowlist.

107.173.130.79

107.172.58.185

Last updated: May 2026

Questions?

If you have questions about MeirraBot or need assistance with crawler configuration, contact us at support@meirra.com.