Charlotte med spa websites are open to AI search crawlers: as of September 2026, all 20 practice homepages we measured serve OAI-SearchBot, Claude-SearchBot, PerplexityBot and Applebot, and 18 of 20 admit all six AI crawlers we tested. The gaps are inside: one homepage delivers nine words of raw HTML, and 3 of 19 pass the first-100-words test.
The headline findings
Twenty Charlotte-area med spa homepages, measured September 4, 2026: 19 of 20 allow every AI crawler in robots.txt, 19 of 20 serve HTTP 200 to all six crawler user agents, 0 of 20 refuse OAI-SearchBot, 18 of 20 carry JSON-LD, 16 of 20 name a credentialed provider, and 18 of 20 show a booking link in raw HTML.
| Measure | Count | What it means |
|---|---|---|
| robots.txt allows all six AI crawlers | 19 of 20 | The exception disallows only the training crawlers |
| HTTP 200 to all six crawler user agents | 19 of 20 | One firewall refuses ClaudeBot only |
| Returns an error to OAI-SearchBot | 0 of 20 | All eligible for ChatGPT search answers |
| Raw HTML under 500 words as GPTBot | 3 of 20 | One is a nine-word client-rendered shell |
| First-100-words test: pass | 3 of 19 | Most open with a welcome, not an answer |
| JSON-LD present | 18 of 20 | 15 declare a local-business type |
| Named MD, DO, NP or PA on homepage or About page | 16 of 20 | 3 name nobody; 1 unreadable in raw HTML |
| Booking link in homepage raw HTML | 18 of 20 | 13 lead to a third-party booking platform |
Crawler access: who gets in
Access is close to universal. Nineteen of 20 robots.txt files allow all six crawlers; the twentieth allows every search crawler and disallows only GPTBot and ClaudeBot, the training crawlers, through Cloudflare's managed content-signal block. At the server, 19 of 20 return HTTP 200 to all six user agents, and no practice turns away a search crawler.
The single server refusal is a 403 to ClaudeBot alone, confirmed on a second fetch twenty minutes later while robots.txt allows it: a hosting firewall rule, not a decision, and it costs nothing in search, since Anthropic documents Claude-SearchBot as its search indexer. Its policy page states that "Anthropic's Bots respect 'do not crawl' signals by honoring industry standard directives in robots.txt."
Two practices wrote explicit permissions. Genevieve & Co. publishes an allowlist that names GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot and Google-Extended and allows each; The Skin Bar's Squarespace file lists the AI crawlers and allows them. The remaining files carry no AI rules and allow everything by default.
What an AI crawler sees in the raw HTML
Nineteen of 20 homepages deliver real text in raw HTML. Fetched as GPTBot, the median homepage carries 974 words, range 9 to 2,775; nine exceed 1,000 words and three fall under 500. One of the three is client-rendered: nine words of raw HTML around a JavaScript bundle, with the booking buttons, provider names and address a browser shows absent.
That page is the most expensive finding here. Lantern's measurement is unambiguous: "As of June 2026: none of the major AI crawlers render JavaScript." Nine words give a crawler nothing to cite, however generous the robots.txt; the fix is server-side rendering or prerendering.
| Practice | Raw-HTML words as GPTBot | AI crawlers allowed | JSON-LD | Booking link |
|---|---|---|---|---|
| Genevieve & Co. | 9 | 6 of 6 | yes | no |
| Infinity MedSpa and Wellness | 1,749 | 6 of 6 | yes | yes |
| Jon'Ric Charlotte Medical Spa | 1,006 | 6 of 6 | yes | yes |
| Eden MedSpa | 1,317 | 5 of 6 | yes | yes |
| Voci MedSpa | 681 | 6 of 6 | yes | yes |
| Prestige Medical Spa | 900 | 6 of 6 | no | yes |
| The Modern Aesthetic | 492 | 6 of 6 | yes | no |
| Fanous MedSpa | 842 | 6 of 6 | yes | yes |
| Aesthetica Med Spa | 2,358 | 6 of 6 | yes | yes |
| Miramae Studio | 908 | 6 of 6 | yes | yes |
| Skindulgence Medspa | 839 | 4 of 6 | yes | yes |
| Med Boutique | 639 | 6 of 6 | yes | yes |
| Plumped | 1,821 | 6 of 6 | yes | yes |
| Evolve Medical Associates | 2,775 | 6 of 6 | yes | yes |
| Urban Medspa & Weight Loss Center | 1,243 | 6 of 6 | yes | yes |
| M2 Medical Spa and Wellness | 1,730 | 6 of 6 | yes | yes |
| Essential Aesthetics and Laser | 1,340 | 6 of 6 | no | yes |
| The Skin Bar | 973 | 6 of 6 | yes | yes |
| Skyn Elegant Aesthetics | 975 | 6 of 6 | yes | yes |
| Legacy Health & Wellness | 478 | 6 of 6 | yes | yes |
Structure and the first 100 words
Structure is the weakest layer. On the extractability script, which grades raw HTML as AI retrieval reads it, the 19 scorable homepages score a median 44 of 100, range 9 to 79. Three pass the first-100-words check, ten are partial, six fail. Skindulgence Medspa scores 79, the highest; Essential Aesthetics and Laser and M2 Medical Spa and Wellness also pass.
The check asks whether the first 100 words say what the practice is and where, with a specific fact. Surfer's June 2026 analysis of 100,000 citation placements across 10,000 AI Overviews found that "38% of all AI citations are pulled from the first 100 words of a page."
Schema and the named medical director
Structured data is common and mostly generic: 18 of 20 homepages carry JSON-LD, 15 declare a local-business type, 5 use any Medical type, one uses MedicalSpa, two carry a Person node and none carries FAQPage. Google's guide is explicit that "Structured data isn't required for generative AI search, and there's no special schema.org markup you need to add."
The medical-director signal is stronger than the schema. Sixteen of 20 practices name a medical director or credentialed provider (MD, DO, NP or PA) on the homepage or an About or team page, in plain text a crawler can read; 3 name nobody and 1 is unreadable in raw HTML. Only 2 back the name with a Person node.
Booking links
Eighteen of 20 homepages carry a visible booking link in raw HTML. Thirteen lead to a third-party booking platform (Zenoti, Aesthetic Record, Meevo, Pabau, Mangomint, Vagaro, Jane and AestheticsPro appear), one to the practice's own booking subdomain, and four to an on-site scheduling page or embedded form. Fifteen of 20 have both a booking link and a named provider.
That split settles the privacy question: the marketing page carries a link, and the calendar, intake form and patient record live on the booking platform behind a login, on a domain the crawler never enters. The two without a link are the client-rendered page and one with no booking control on its homepage.
What to check on your own site this week
Five checks, none needing a vendor: fetch your homepage as OAI-SearchBot and confirm a 200, count the words in your raw source outside script tags, read your first 100 words aloud, confirm your medical director's name and credential appear as text, and follow your booking button from the raw HTML.
- 1Crawler access: fetch your homepage with curl under the OAI-SearchBot user agent and confirm HTTP 200; repeat for Claude-SearchBot, PerplexityBot and Applebot. A 403 is usually a firewall default.
- 2Raw HTML: view source and count the words outside script tags. Under 500 is a shell; under 50 is nothing.
- 3First 100 words: what you are, where you are and one specific fact. A welcome line is 100 wasted words.
- 4Named provider: your medical director's name and credential as plain text on the homepage or About page, plus a Person node if you publish JSON-LD.
- 5Booking link: a real anchor in raw HTML pointing at your booking flow, not a JavaScript-only widget.
Method
The study measured 20 homepages on September 4, 2026 (UTC), with the stdlib Python script ai_readiness_study.py from Mirastart's SEO toolkit: one request per second, raw HTML only, no JavaScript rendering. The sample is 20 independent med spa and aesthetics practices with a Charlotte-area address and their own site, 17 in Charlotte and one each in Matthews, Huntersville and Fort Mill.
- Sample: web searches for med spa, Botox and aesthetics practices in Charlotte plus published Charlotte roundups; national chains' corporate location pages excluded. One candidate's site was a maintenance page that day and was replaced; one host did not resolve.
- Crawler access: robots.txt parsed with Python's urllib.robotparser for six tokens (no file counts as allowed), then the homepage fetched under each vendor's documented user-agent string; the one non-200 was re-fetched 20 minutes later. Only the string was tested; real crawlers arrive from published IP ranges a firewall may treat differently.
- Words, structure and schema: homepage fetched as GPTBot with script, style, noscript, template and SVG blocks removed; extractability.py scored the same HTML, which also supplied JSON-LD, title and canonical. Booking link and named provider were read from the raw HTML of the homepage and the About or team pages linked from it: a Book, Schedule or Appointment control pointing at a booking platform, an on-site scheduling page or an embedded form; a person named with MD, DO, NP or PA.
- Limits: homepages only, one measurement per site, user-agent strings only, as of September 2026. Raw data: docs/data/charlotte-med-spa-ai-search-readiness-2026.csv in the Mirastart repository and at the public URL in Sources, 20 rows, CC BY 4.0.
Every practice in the sample can request its own row and the evidence behind each cell privately: write to support@mirastart.com with the practice name. Practices outside the sample can ask for the same measurement.
Sources
- Raw data: per-practice measurements (CSV, CC BY 4.0) - every number in the tables above, one row per practice, with the evidence behind the booking and provider columns
- OpenAI, Overview of OpenAI Crawlers - Documents OAI-SearchBot (search), GPTBot (training), ChatGPT-User and OAI-AdsBot as separate crawlers; sites opted out of OAI-SearchBot are not shown in ChatGPT search answers.
- Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler? - Names ClaudeBot (training), Claude-User and Claude-SearchBot (search indexing); source of the quoted sentence on robots.txt.
- Apple, About Applebot - Published June 8, 2026. Applebot powers Spotlight, Siri and Safari search; Applebot-Extended is a training opt-out that does not crawl webpages.
- Google Search Central, Optimizing your website for generative AI features on Google Search - Last updated July 10, 2026. Source of the quoted sentence on structured data.
- Surfer, Why Almost 40% of AI Citations Come from Your First 100 Words - 100,000 citation placements across 10,000 AI Overviews, data collected June 2, 2026; source of the quoted sentence.
- Lantern, AI Crawlers Do Not Render JavaScript - Updated June 2026; source of the quoted sentence. Gemini, riding Googlebot's rendering service, is the one exception.
- Charlotte Charm, Best Med Spa in Charlotte: Complete Guide (2026) - Fetched September 2026; a 14-entry list with street addresses and websites, used with web searches to assemble the sample.