Meet SaaSCubesBot

The crawler that reads your homepage, and how to let it through

What it is

An honest crawler with a name

SaaSCubesBot fetches pages for SaaS Cubes, a curated directory of software products. It identifies itself on every request rather than pretending to be a browser, so you can recognise it, allow it, or block it — deliberately, either way.

It sends this User-Agent:

SaaSCubesBot/1.0 (+https://saascubes.com/bot)

It only ever issues ordinary GET requests for HTML. It does not log in, submit forms, follow links beyond the page it was given, or collect personal data.

When it visits

Only when there is a reason

SaaSCubesBot does not crawl the web looking for sites. It reads a page only when someone has submitted that product to the directory, which means one of four things is happening:

01

When a product is submitted

We read the homepage once to draft the listing — the name, the description, the logo and the colours. This happens while the submitter is still filling in the form.

02

To verify a backlink

Free listings are published in exchange for a link back to us. We check the homepage for that link before approving, and again whenever the owner asks us to.

03

When an editor refreshes a listing

If a listing looks out of date, we re-read the page to pick up a new logo, a new description or a changed brand colour.

04

While a listing is live

We re-read the homepage a day, a week and a month after a listing goes live, then once a month, to confirm the product is still up and, for a free listing, that the link is still there. Each read also asks for /llms.txt. A read that fails is tried again 3 and 7 days later before anything changes. The date of the last good read shows on the listing as "Checked live", and a found llms.txt is linked there too.

In practice a live listing gets six requests in its first month (three homepage reads, each followed by one for /llms.txt), then two a month — never a sustained crawl, and never more than one page at a time.

If we are being blocked

How to let us read your page

Some firewalls challenge any request that is not a real browser. If yours does, we cannot confirm your backlink — and a free listing that we cannot verify does not get approved. The fix is to allow the User-Agent above through your WAF:

  • Cloudflare — add a WAF custom rule that skips the managed challenge when the User-Agent contains SaaSCubesBot.
  • Vercel — exempt the same User-Agent from Attack Challenge Mode.
  • Anything else — allow the User-Agent, or ask us for the address we fetch from via contact.

Would rather we did not read your site at all? Block the User-Agent, or ask us to remove your listing — both are fine, and neither needs an explanation.