> ## Documentation Index
> Fetch the complete documentation index at: https://www.commercengine.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deployment & Indexability

> Choose server or static SEO serving, configure preview noindex behavior, and validate robots, sitemaps, Markdown mirrors, and deep links.

The decisive SEO question is not “Which framework?” It is:

> **Does an application server receive requests after deployment?**

That determines whether Commerce Engine should generate discovery responses on demand or write them as files at build time.

## Decision matrix

| Production shape         | Use                                                 | Typical examples                                                                      |
| ------------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------- |
| Server receives requests | framework server adapter / portable request handler | Next.js server, TanStack Start, Astro server output, SvelteKit node/edge adapters     |
| Files/CDN only           | `writeCommerceSeoAssets()` prebuild                 | Vite SPA, Astro static output, SvelteKit `adapter-static`, fully exported Next output |

Both modes produce the same discovery model. The difference is only where it is materialized.

## Server mode

Use one framework handler and enable robots + sitemap:

<CodeGroup>
  ```typescript Next.js theme={"theme":{"light":"github-light","dark":"github-dark"}}
  createNextjsSeoProxy(seo, { robots: true, sitemap: true });
  ```

  ```typescript TanStack Start theme={"theme":{"light":"github-light","dark":"github-dark"}}
  createTanStackStartSeoMiddleware(seo, { robots: true, sitemap: true });
  ```

  ```typescript Astro theme={"theme":{"light":"github-light","dark":"github-dark"}}
  createAstroSeoMiddleware(seo, { robots: true, sitemap: true });
  ```

  ```typescript SvelteKit theme={"theme":{"light":"github-light","dark":"github-dark"}}
  createSvelteKitSeoHandle(seo, { robots: true, sitemap: true });
  ```
</CodeGroup>

The portable `createCommerceSeoRequestHandler()` from `@commercengine/seo/server` provides the same contract for other runtimes that use standard Web `Request` and `Response` objects.

A request handler always owns:

* HTML/Markdown content negotiation for product/category/search routes
* explicit `.md` mirrors
* `/llms.txt`
* `/sitemap.md`

It owns `/robots.txt` only when `robots: true` is enabled, and `/sitemap.xml` plus `/sitemap/{id}.xml` shards only when `sitemap: true` is enabled. Both server-mode options default to `false` so the package cannot silently shadow existing application routes.

### Why `Vary: Accept` matters

When the same URL can return HTML or Markdown, the HTML response must vary on `Accept`; otherwise a shared cache can serve the wrong representation to the next client. Commerce Engine adapters scope this header only to URLs that actually negotiate Markdown so unrelated CDN cache keys do not fragment.

Explicit `.md` URLs remain the primary discovery links and work even when an AI client cannot set custom `Accept` headers.

## Static mode

A static host cannot inspect `Accept`, run catalog queries, or build arbitrary search results after deployment. Generate real files before the framework build:

Keep site identity and public routes in the same pure `commerceSeo` config the application and AI tools already import. A TypeScript prebuild can reuse that module directly:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pnpm add -D tsx dotenv
```

```typescript scripts/generate-seo-assets.ts theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { existsSync } from "node:fs";
import { createCommerceSeo } from "@commercengine/seo";
import { writeCommerceSeoAssets } from "@commercengine/seo/build";
import { createStorefront, Environment } from "@commercengine/storefront";
import { config as loadEnvFiles } from "dotenv";
import { commerceSeo } from "../src/lib/commerce-seo.config";

// Prebuilds run before the framework loads its env files. Read local production
// files explicitly; dotenv does not overwrite variables already injected by CI.
const envFiles = [
  ".env.production.local",
  ".env.local",
  ".env.production",
  ".env",
].filter(existsSync);
if (envFiles.length) loadEnvFiles({ path: envFiles });

const storeId = process.env.CE_STORE_ID;
const apiKey = process.env.CE_API_KEY;
if (!storeId || !apiKey) {
  throw new Error("[seo] missing CE_STORE_ID / CE_API_KEY");
}

const seo = createCommerceSeo({
  ...commerceSeo,
  storefront: createStorefront({
    storeId,
    apiKey,
    environment:
      process.env.CE_ENV === "production"
        ? Environment.Production
        : Environment.Staging,
  }),
});

await writeCommerceSeoAssets(seo, { outDir: "public" });
```

Leave `indexable` unset in the shared build script. The package then uses Vercel, Netlify, Cloudflare, or generic Node deployment signals: preview builds remain non-indexable, while proven production builds become indexable automatically.

Run that script before the framework build, for example:

```json package.json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "scripts": {
    "build": "tsx scripts/generate-seo-assets.ts && vite build"
  }
}
```

Use the framework's normal build command after `&&` (`astro build`, `next build`, and so on).

### Output directory

| Framework/deployment       | Directory |
| -------------------------- | --------- |
| Vite / React SPA           | `public/` |
| Astro static               | `public/` |
| Static Next.js             | `public/` |
| SvelteKit `adapter-static` | `static/` |

Generated files are artifacts and should generally be gitignored.

### Why physical files instead of catch-all routes

Framework routers prioritize concrete product routes. A `/product/[slug]` route can claim `/product/shoe.md` before a catch-all Markdown route sees it. Files placed in the publish directory bypass the application router entirely.

## Static assets generated

By default the static generator can produce:

```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
robots.txt
sitemap.xml
sitemap/{id}.xml        # only when sharding is needed
llms.txt
sitemap.md
<productBase>/*.md
<categoryBase>/*.md
```

The package defaults are `/products` and `/category`; the examples in these docs configure `/product` explicitly to match the Commerce Engine starter route convention.

Optional variant mirrors can be generated when variant public URLs use distinct paths. Query-string variants usually reuse the base product `.md` file rather than multiplying the file count.

The static generator builds `sitemap.md` and XML sitemap output from the assets it actually emits, so intentionally unroutable catalog entities are not advertised as dead links.

## Production versus preview indexability

`createCommerceSeo()` resolves one `indexable` value and applies it consistently across head metadata, robots behavior, dynamic responses, and static output.

`indexable` is an explicit override:

```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
indexable: true   // force indexability
indexable: false  // force noindex
// omitted        // detect the deployment; only proven production is indexable
```

If it is omitted, the package detects common deployment environments:

| Platform signal          | Production                                            | Preview/non-production                   |
| ------------------------ | ----------------------------------------------------- | ---------------------------------------- |
| Vercel `VERCEL_ENV`      | `production`                                          | `preview`, `development`                 |
| Netlify `CONTEXT`        | `production`                                          | `deploy-preview`, `branch-deploy`, `dev` |
| Cloudflare Pages/Workers | configured production branch, default `main`/`master` | other branches                           |
| Generic Node             | `NODE_ENV=production`                                 | `NODE_ENV=development`                   |
| No reliable signal       | not proven production                                 | safe non-indexable default               |

<Warning>
  Preview platforms often build with `NODE_ENV=production`. Do not use `NODE_ENV` alone to decide whether a branch deployment should be indexed.
</Warning>

## Static noindex protection

HTML pages can carry robots metadata. Markdown and text files cannot. On a non-indexable static build, the package can emit a `_headers` file with a site-wide `X-Robots-Tag: noindex, nofollow` rule for hosts such as Netlify and Cloudflare Pages.

If your host does not understand `_headers`, apply equivalent response headers in that host's configuration.

## Robots policy

Production robots should make public storefront pages crawlable while keeping operational/session URLs out of discovery. Use package policy unless your application has a specific reason to override it.

A non-indexable deployment intentionally emits an allow-all robots policy with no sitemap while its pages/responses carry `noindex, nofollow`. Do not replace that with `Disallow: /`: a crawler blocked from fetching the page cannot read the noindex directive.

Never use `robots.txt` as the only preview protection. `robots.txt` controls crawling; page/header directives control indexability.

## Sitemap limits

XML sitemaps are limited to 50,000 URLs per file. Commerce Engine automatically shards larger catalogs and produces a sitemap index.

You can reduce the maximum during testing:

```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
createNextjsSeoProxy(seo, {
  robots: true,
  sitemap: true,
  maxUrlsPerSitemap: 10_000,
});
```

Do not silently truncate the catalog to keep one sitemap file small.

## Build credentials

Static SEO generation runs before the app build and needs public catalog credentials. Local development may use `.env.local`; CI usually injects environment variables directly and has no local env file.

A robust prebuild script:

1. reads the local file only when it exists
2. overlays/accepts CI environment variables
3. fails explicitly if Store ID or API key is missing
4. never emits an empty “valid-looking” sitemap after an authentication failure

The Commerce Engine starters implement this pattern.

## Existing-file conflicts

Before adding the package, remove overlapping files/routes:

* static `robots.txt`
* static or hand-built `sitemap.xml`
* custom `llms.txt`
* old Markdown route handlers
* duplicate JSON-LD generators

A physical `public/robots.txt` can shadow a perfectly correct dynamic `robots.txt` with no warning.

## Deep-link hosting checks

SEO creates links to real pages, so production host routing must resolve them.

### SPA

Configure a catch-all rewrite to `index.html` **after** static-file matching. Hard-refresh a product/category URL as part of deployment validation.

### SvelteKit adapter-static on Vercel

Use:

```json vercel.json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "cleanUrls": true
}
```

If `/privacy-policy.html` works but `/privacy-policy` 404s, the problem is host routing rather than the Svelte build.

## Validate the deployed site

Check the deployed production origin, not only localhost:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -I https://acme.example/product/example
curl -H 'Accept: text/markdown' https://acme.example/product/example
curl https://acme.example/product/example.md
curl https://acme.example/robots.txt
curl https://acme.example/sitemap.xml
curl https://acme.example/llms.txt
curl https://acme.example/sitemap.md
```

Then verify:

* canonical URLs use the real production domain
* sitemap URLs hard-load successfully
* preview builds return noindex protection
* product/category Markdown links resolve
* only one product structured-data entity is present per page
* social preview tags use product/category values rather than generic site defaults

<CardGroup cols={2}>
  <Card title="SEO package" icon="magnifying-glass" href="/docs/seo-ai/seo">
    Structured data and discovery API.
  </Card>

  <Card title="Framework guides" icon="code" href="/docs/seo-ai/overview#framework-map">
    Framework-specific wiring.
  </Card>
</CardGroup>