Home › AI for small firms › Putting it to work
Putting it to work
How to let ChatGPT find your Webflow website
Set and verify Webflow robots rules, distinguish staging from the production domain, and check CMS content without treating new content signals as guarantees.
To make a Webflow page available to ChatGPT search, publish it on the intended public domain and check the effective crawler rules. Save and publish robots changes before testing the production response.
A Designer preview or staging page can differ from the live site. Keep the production domain, published content and current access policy together in the test record.
Where do I edit Webflow robots rules?
The Webflow robots guide documents the following route:
- Open Site settings, then SEO, then Indexing.
- Add or review the intended robots.txt rules.
- Save and publish the site.
- Inspect /robots.txt on the production HTTPS domain.
Webflow normally includes a sitemap link in the file. Its Remove sitemap.xml from robots.txt option controls that inclusion. Review the served output before changing another setting to fix what might only be an unpublished edit.
Keep the previous configuration and the reason for each custom restriction. A wildcard allow rule does not automatically resolve the result of a more specific crawler group.
Am I checking staging or production?
Record the primary customer-facing domain and a real published URL. Test it while signed out. Confirm that any redirect ends on the intended address and the visitor receives the current page.
A staging domain may be deliberately excluded from discovery to avoid duplicate or unfinished pages appearing in results. Preserve that decision while checking the live business website. The right remedy can be correcting the test URL rather than opening staging to crawlers.
| Test target | Question to answer |
|---|---|
| Primary domain | Is this the address customers should use? |
| Published CMS item | Is the intended item actually available? |
| Final redirected URL | Does the visitor reach the correct page? |
| Root robots file | What rules are served after publishing? |
| Sitemap | Does it list the public URLs intended for discovery? |
Send those observations to the site manager when a request fails. They make the issue easier to reproduce than a general instruction to improve AI visibility.
What should I inspect on CMS pages?
Choose a representative item, not only a collection listing. Read the rendered page with no editor session. Check whether the heading, body text and important facts appear in the published document.
A card may have the right title while its destination is unpublished, empty or missing a key CMS field. Inspect the destination separately. If a template relies on an interaction to reveal essential information, verify that the useful answer is still available to ordinary readers.
Run the answer-readiness checker on that page. Use the general access guide to distinguish a content problem from an access restriction.
Does a Content-Signal instruction guarantee compliance?
Webflow's documentation describes content-usage signals and notes that this is a proposed extension rather than an established web standard accepted by every crawler. Do not treat a newly added signal as proof that all services will act on it.
Keep the intended search and training policy explicit. Check the crawler's own documentation and the response actually served. A descriptive signal, a robots rule and authentication have different roles.
Webflow also warns that robots.txt is public and is not protection for sensitive information. Put confidential material behind appropriate access controls; do not rely on a listed disallowed path to keep it secret.
What should a UK service page answer?
Give customers the facts they need to decide whether to enquire. For a UK design studio, that might include the type of work, actual project scope, location or remote coverage, how estimates are prepared and the route to request one.
Avoid publishing a price without explaining what it covers. Keep portfolio claims and results supported by the firm's own records. A page written around a genuine customer question is more useful than adding a paragraph that repeatedly asks an assistant to recommend the business.
The crawler checker can establish whether a technical rule needs attention. It cannot establish that a particular service page is the best answer to every question.
How should I verify a change?
Recheck the same production URL after publishing. Keep the before-and-after outputs, date and changed setting. Confirm that another important public page still works and that private or staging content remains under its intended policy.
Then record actual citations and referrals over time. A successful response means that request worked at that time. Search inclusion, recommendations and customer enquiries need their own evidence.
Which questions come up most often?
Must I publish after changing Webflow robots rules?
Webflow documents saving and publishing the site. Verify the file served by the production domain afterwards.
Should staging be opened to fix a production crawler warning?
Check the target URL first. Staging may intentionally remain excluded while the production site is public.
Can robots.txt protect confidential pages?
No. It is a public policy file, not authentication. Sensitive content needs proper access controls.
Where this comes from: Webflow Help, Set robots.txt rules, current publishing route and proposed content-signal limitations; first-party CMS verification workflow; checked 30 October 2026. Worked examples are illustrative unless a source is named.
Put a number on it: AI readiness score: is your business ready for AI? · How many hours can AI save me each week? Free, in your browser, no sign-up.
AEO Lookup editorial team
We write practical guides for people who run small UK firms, and we check every fact against its source on the date shown. Read how we review, or tell us if something has changed.