The short answer to what is technical SEO: it is the work on a website's code, server and structure that lets Google crawl, render and index your pages, and lets AI search tools read them. A technical SEO audit checks that work across about 12 areas, from robots.txt and indexing to page speed, redirects, mobile display and structured data.
Content and links decide how high a page can rank. Technical SEO decides whether the page is in the race at all. A service page blocked by one line in robots.txt, or a product range hidden behind JavaScript that crawlers never run, earns nothing no matter how well it is written.
On this page
- What technical SEO covers
- Why technical SEO matters for AI search in 2026
- Crawling: can search engines reach your pages
- Rendering: can Google see content built with JavaScript
- Indexing: which pages Google keeps
- Page experience: speed, mobile and security
- Site structure, redirects and status codes
- Structured data and international signals
- Technical SEO on WordPress, Shopify and JavaScript sites
- Signs your site has a technical SEO problem
- What a technical SEO audit checks
- Which other checks complete the audit
- How to prioritize audit findings
- How often to run an audit and what it costs
What technical SEO covers
Google's guide to how Search works describes three stages: crawling, indexing and serving results. It also states that not every page makes it through each stage, that Google does not guarantee it will crawl, index or serve any page, and that it does not accept payment to crawl a site more often. Technical SEO is the work that gets your pages through the first two stages cleanly.
Type of SEO | What it covers | Who usually does it | Example task |
|---|---|---|---|
Technical SEO | Crawling, rendering, indexing, speed, mobile, security, redirects, structured data | Developer with an SEO lead | Remove a noindex tag left on 40 product pages after launch |
On-page SEO | Titles, headings, page copy, internal links, images | Writer or SEO lead | Rewrite a service page to answer the questions buyers ask |
Off-page SEO | Links, mentions, reviews, business listings | SEO lead or PR | Earn a link from a trade association directory |
The three overlap. Internal links are both a technical signal and an on-page task, and a slow page is a technical problem that hurts every piece of content on it. A useful rule: if fixing it needs access to the code, the server or the CMS settings, it belongs in technical SEO.
Why technical SEO matters for AI search in 2026
AI Overviews and AI Mode draw on the same index as regular results. Google's documentation on AI features says there are no additional requirements to appear in them, and that you do not need to create new machine-readable files, AI text files or markup. A page has to be indexed and eligible to show with a snippet. The best practices Google lists are technical ones:
- Crawling allowed in robots.txt, and by any CDN or hosting firewall.
- Content findable through internal links.
- A good page experience.
- Main content available as text, supported by images and video where useful.
- Structured data that matches the visible text on the page.
- Up-to-date Merchant Center and Business Profile information.
Other AI tools add their own crawlers. OpenAI's crawler documentation says OAI-SearchBot is for search, GPTBot collects data for training, and each setting is independent. A site that blocks GPTBot can still appear in ChatGPT search answers, while a site that blocks OAI-SearchBot cannot. On the Google side, the Google-Extended token controls Gemini training and grounding and, per Google, does not affect inclusion or ranking in Google Search.
Check your firewall as well as robots.txt. Security tools and CDNs can block AI crawlers by default, and a block there never shows up in the robots.txt file you are reviewing. The guides to showing up in Google AI Overviews and getting recommended by ChatGPT cover the content side of AI visibility.
Crawling: can search engines reach your pages
Crawling problems are the most expensive technical issues, because a page Google cannot reach cannot rank. Google's robots.txt documentation says the file is used mainly to manage crawler traffic and is not a mechanism for keeping a page out of Google. To keep a page out of results, use a noindex tag or put the page behind a password.
That distinction causes real damage. A robots.txt rule that blocks /services/ hides every page in that folder from crawlers, but if other sites link to those URLs, Google can still list them with no description. The fix for an unwanted page is noindex; the fix for a page you want ranked is to remove the block.
Links are how Google finds most new pages, which makes internal links a crawling tool. Sitemaps help too, and Google says a small site of about 500 pages or fewer, with good internal links, might not need one. New sites with few external links, large sites and sites with lots of video or news content should have one.
Crawl budget gets more attention than it deserves. Google's crawl budget guide is written for sites with 1 million or more unique pages that change weekly, or 10,000 or more pages that change daily. A 60-page service business does not have a crawl budget problem; it has a crawl access problem or no problem at all.
Rendering: can Google see content built with JavaScript
Google processes JavaScript pages in three phases: crawling, rendering and indexing. Its JavaScript SEO guide says a page may sit in the render queue for a few seconds, but it can take longer. The same guide calls server-side rendering or pre-rendering a great idea, because not all bots can run JavaScript.
That last point matters more in 2026, with AI search tools sending their own crawlers. A crawler that reads only the HTML your server sends will see a React or Vue storefront that loads product names, prices and reviews in the browser as an empty page. Server-side rendering, static generation or pre-rendering puts that content in the first HTML response.
To test a page, open it in Search Console's URL Inspection tool and view the crawled page. Then compare it with your browser's view-source output. If your main content, links or prices appear in the rendered version only, any crawler that skips JavaScript misses them. A faster check needs no tools: turn off JavaScript in Chrome's developer settings and reload the page, and whatever disappears is what those crawlers never see.
Indexing: which pages Google keeps
Crawled does not mean indexed. The Page indexing report in Search Console lists every URL Google knows about and why each one is or is not indexed. Google's own help page says not to expect every URL on your site to be indexed, and that duplicates and pages without meaningful content may be left out. Your job is to make sure the pages that bring in customers are the ones that make it.
Status in Search Console | What it means | What to do |
|---|---|---|
Discovered, currently not indexed | Google found the URL but has not crawled it yet | Add internal links from strong pages; check server speed |
Crawled, currently not indexed | Google crawled the page and chose not to index it for now | Improve or merge thin pages; no need to resubmit |
Duplicate without user-selected canonical | Google picked another URL as the main version | Add a canonical tag to the version you want |
Soft 404 | The page says not found but returns a success code | Return a real 404, or restore the content |
Excluded by noindex | A noindex directive stopped indexing | Remove it if the page should rank |
Duplicate URLs split signals across copies of the same page: http and https versions, www and non-www, trailing slashes, and URL parameters from filters or tracking. Google's canonicalization guidance ranks the signals: a redirect and a rel="canonical" tag are strong signals, and listing a URL in a sitemap is a weak one. Use redirects for versions nobody should visit, and canonical tags for variants that need to stay live, such as filtered category pages.
Page experience: speed, mobile and security
Google measures page experience through several signals, and its page experience documentation says there is no single signal. The self-assessment it gives asks four things: good Core Web Vitals, secure delivery over HTTPS, good display on mobile devices and no intrusive interstitials.
Metric | What it measures | Good threshold |
|---|---|---|
Largest Contentful Paint (LCP) | Loading: when the main content appears | 2.5 seconds or less |
Interaction to Next Paint (INP) | Responsiveness to taps, clicks and typing | Less than 200 milliseconds |
Cumulative Layout Shift (CLS) | Visual stability while the page loads | Less than 0.1 |
The thresholds come from Google's Core Web Vitals page, which says good scores align with what its core ranking systems seek to reward. Measure them in Search Console's Core Web Vitals report, which uses real Chrome user data, and use PageSpeed Insights to find the cause on a single page. The Core Web Vitals explainer walks through the usual fixes.
Mobile comes first in indexing. Google uses the mobile version of a site, crawled with a smartphone agent, for indexing and ranking, and asks that the mobile site carry the same content as the desktop site. A product description hidden on mobile to save space is a description Google may never index.
Site structure, redirects and status codes
Structure tells crawlers which pages matter. Keep revenue pages within three clicks of the home page, use short descriptive URLs such as /services/ac-repair/, and add breadcrumbs so the hierarchy is visible to people and crawlers alike. Orphan pages, with no internal links pointing to them, are a structural fault almost every audit turns up.
Status codes need to say what is true:
- 200 for a live page.
- 301 for a permanent move, pointing straight to the final URL.
- 302 only for a move you will reverse.
- 404 or 410 for a page that is gone with no replacement.
- 5xx errors fixed fast, because repeated server errors slow crawling.
Redirect chains, where URL A sends visitors to B and B sends them to C, waste crawl time and slow pages down. Point every old URL straight at its final destination. For site moves, Google advises keeping redirects for at least one year, and says a small to medium site can take a few weeks for most pages to move in its index.
Redesigns and platform changes are the riskiest moments for site structure. Before launch, crawl the old site and export every URL, then map each one with traffic or links to its new address. After launch, check the 404 report daily for the first two weeks. A 150-page site that skips this step can lose every ranking those old URLs held, and the cause takes weeks to spot because Google recrawls gradually.
Structured data and international signals
Structured data is code, usually JSON-LD, that labels what a page contains: a product with a price, a local business with hours, an article with an author, a set of FAQs. It qualifies pages for rich results and makes facts easier for machines to read. Google is clear that no special schema is needed to appear in AI features, so treat structured data as accuracy work, not a ranking trick.
The rule an audit checks is consistency. The price, rating, hours and address in your markup must match what a visitor sees on the page, and the business details should match your Google Business Profile. Mismatches show up after a price change that updated the page but not the template that writes the markup, so a product marked up at $49 still shows $49 to Google while the page says $59.
Sites in more than one language or country need hreflang tags, which tell Google which version to show to which audience. A US company with a separate Canadian French site needs each of its pages to list both versions, so 200 pages means 200 matched pairs. Single-market sites can skip this part of the audit.
Technical SEO on WordPress, Shopify and JavaScript sites
Each platform handles part of technical SEO for you and leaves a different part exposed. Knowing which part saves hours in an audit.
WordPress. The setting to check first sits under Settings, then Reading. The Search engine visibility option asks search engines not to index the site, and WordPress notes it is up to search engines to honor the request. Developers tick it on staging sites, and it sometimes survives launch. Plugin conflicts, outdated PHP and heavy page builders cause most WordPress speed problems; the guides to updating your PHP version and WordPress 7.1 cover the upkeep side.
Shopify. Every Shopify store gets an automatic sitemap.xml at the root of its domain, and Shopify's default robots.txt blocks admin, cart, checkout and account pages plus filtered and sorted collection URLs. Merchants can edit the file through a robots.txt.liquid template, but Shopify calls that an unsupported customization and warns that incorrect use can result in loss of all traffic. On Shopify, apps are the usual speed problem, since each one can add scripts to every page.
Custom JavaScript sites. React, Vue and other framework builds carry the most technical risk, because content that loads in the browser may never reach crawlers that skip JavaScript. Ask for server-side rendering or static generation from the start; retrofitting it after launch costs far more than building it in. The explainer on how headless commerce works covers the same trade-off for online stores.
Signs your site has a technical SEO problem
Technical problems rarely announce themselves. These symptoms are the usual first clue, and each one points to a specific check:
Symptom | Likely cause | First check |
|---|---|---|
New pages still missing from Google after four weeks | Crawl block, noindex tag or no internal links | URL Inspection on one missing page |
Not-indexed count jumped after a launch or update | Template change added noindex or broke canonicals | Page indexing report, sorted by reason |
A result shows your URL with no description | Page blocked by robots.txt but linked from elsewhere | robots.txt rules for that folder |
Wrong version ranking, such as http or a filter URL | Missing redirects or canonical tags | URL Inspection, Google-selected canonical |
Traffic fell the week a redesign went live | Old URLs not redirected, content removed | Redirect map and 404 report |
Mobile pages feel slow and jump while loading | Large images, heavy scripts, late-loading ads | Core Web Vitals report, mobile tab |
The robots.txt row surprises most owners. Google's documentation says a blocked URL can still appear in results if other pages link to it, with no description shown, because Google indexed the address without visiting the page. A month of new pages that never appear is the strongest sign of all, since Google says crawling takes a few days to a few weeks.
What a technical SEO audit checks
The audit is a structured review of every item above, with a pass standard for each one and a list of fixes in priority order. A complete audit covers 12 areas. The first six decide whether Google can reach, render and keep your pages:
Area | What the audit checks | Where to check | Pass standard |
|---|---|---|---|
Crawl access | robots.txt rules, firewall and CDN blocks, login walls | robots.txt file, URL Inspection, server logs | No revenue page blocked; AI search crawlers allowed if you want AI visibility |
Indexing | Indexed vs excluded pages, noindex tags, soft 404s | Page indexing report | Every page meant to rank is indexed |
Canonicals and duplicates | One live version per page; canonical tags on variants | Site crawler, URL Inspection | Google-selected canonical matches yours |
Sitemaps | Only live, indexable 200 URLs listed | Sitemaps report | No redirected, blocked or 404 URLs in the file |
Rendering | Content and links present without JavaScript | URL Inspection, view-source | Main content in the first HTML response |
Core Web Vitals | LCP, INP and CLS on real user data | Core Web Vitals report, PageSpeed Insights | Good on mobile for the main page templates |
Which other checks complete the audit
The other six cover mobile display, security, site structure and markup:
Area | What the audit checks | Where to check | Pass standard |
|---|---|---|---|
Mobile parity | Same content and links on mobile and desktop | Mobile browser, URL Inspection | No content or links hidden on mobile |
HTTPS | Valid certificate, no mixed content, http redirects to https | Browser, site crawler | One secure version only |
Status codes and redirects | 404s, 5xx errors, chains and loops | Site crawler, server logs | No chains; internal links point to 200 URLs |
Site structure | Click depth, orphan pages, internal links, breadcrumbs | Site crawler | Revenue pages within three clicks |
Structured data | Valid markup that matches visible content | Rich Results Test, Search Console rich result reports | No errors; prices and hours match the page |
International | hreflang pairs and return tags | Site crawler | Each language version references the others |
Search Console is free and covers most of the list. A desktop crawler fills the gaps: Screaming Frog's SEO Spider crawls up to 500 URLs on its free version, with a paid annual license for larger sites. Larger sites usually add server log analysis to see what Googlebot and AI crawlers actually request.
How to prioritize audit findings
An audit that hands you 300 issues in one spreadsheet is a data export. A useful audit sorts findings by revenue at risk and effort to fix:
- Clear anything that stops a money page from being crawled or indexed: robots.txt blocks, stray noindex tags, firewall blocks, broken canonicals.
- Repair server errors, redirect chains and internal links pointing at 404s.
- Speed up the templates that carry most traffic, usually the home page, service or category pages and product pages, until Core Web Vitals pass.
- Merge or improve duplicate and thin pages, and link the orphans.
- Clean up structured data errors and mismatches, then the remaining warnings.
For example, say a 400-page Shopify store in Texas gets an audit with three main findings: 38 collection pages set to noindex by a theme setting, 120 internal links pointing through redirect chains, and a mobile LCP of 4.1 seconds on product pages. The noindex fix takes an hour and brings 38 pages back into the index, so it goes first. The redirect clean-up takes about a day. The LCP work, which means compressing images and cutting app scripts, takes a week and comes third.
Ask for each finding to carry three fields: the URLs affected, the expected impact and the hours to fix. Those three fields turn an audit into a work plan your developer can schedule.
How often to run an audit and what it costs
Check Search Console monthly for new indexing errors and Core Web Vitals changes. Run a full technical audit once a year. Run one before and after any redesign, platform change or domain move as well, since a migration changes URLs, templates and internal links at once.
Audit prices follow site size and depth. In Ahrefs' survey of 439 SEO providers, the most common per-project price band was $2,501 to $5,000. Freelancers averaged $2,348.63 per project and agencies $9,507.84. A 30-page service site sits at the low end; a 20,000-product store with log file analysis sits at the top. The guide to how much SEO costs per month breaks down retainer and project pricing in more detail.
Fixes take time to show. Google says crawling can take a few days to a few weeks after a change. The post on how long SEO takes to show results covers what to expect after technical fixes ship.
Your next step is to open the Page indexing report in Search Console and check that every service, product or location page you sell from is marked as indexed. If the list has gaps, or you want a full audit with fixes ranked by revenue at risk, talk to the Techno Mantra SEO team. It runs technical audits and has the developers to ship the fixes, from robots.txt and redirects to Core Web Vitals and structured data.
Send your site through the contact page or to info@technomantraa.com, and every inquiry gets a reply within one business day.
Frequently asked questions
Part of it. Search Console, robots.txt edits, sitemap submission and most CMS settings need no code. Fixing JavaScript rendering, server errors, redirect rules and slow page templates usually does, so most small businesses pair an SEO lead who finds the issues with a developer who fixes them.
No. A tool report lists every issue a crawler finds, which can run to hundreds, with no sense of which ones cost you customers. A technical SEO audit checks those findings against Search Console data, removes false alarms and ranks the fixes by revenue at risk and hours to fix.
Yes, but less of it. A 30-page site rarely has crawl budget or hreflang issues, yet it can still have a stray noindex tag, a robots.txt block, slow mobile pages or broken redirects. A yearly audit and a monthly look at Search Console cover most small sites.
Not for Google. Google states that you do not need new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode. Crawl access, indexable text, internal links and accurate structured data matter more than any extra file.
Google Search Console, PageSpeed Insights and the Rich Results Test are free and cover indexing, speed and structured data. Add a site crawler such as Screaming Frog, which is free up to 500 URLs, to find redirects, broken links, orphan pages and duplicate titles.