Preferences

Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website. More information

Accept all cookies

Technical SEO: The Only Guide You Need

Master technical SEO: crawlability, indexability, Core Web Vitals, XML sitemaps, robots.txt, and site speed. The complete technical SEO guide for Google rankings.

Man with dark hair and beard wearing a light brown shirt speaks in front of a microphone on a podcast or recording setup.Portrait of a man with short dark hair wearing a white shirt and dark jacket, looking directly at the camera with a neutral expression.Man with short dark hair, beard, and clear glasses wearing a black t-shirt with a white circular logo, standing in front of a stone wall.Celio fabianoSmiling young woman with long brown hair wearing a red top and necklace, outdoors in a tree-filled background.photo de profil du client Xavier Breull
+ 9'000 subscribers
A developer testing site crawl structure using robots.txt and XML sitemap to ensure proper indexing by Google.
Upload UI element
Thibault Besson-Magdelain fondateur de Sorank

About Author

Thibault Besson-Magdelain

Founder of Sorank, 5+ years of experience in SEO, GEO enthusiast.
Share on

Summary: Technical SEO is the process of optimizing your website's infrastructure, code, and performance to improve crawlability, indexability, and Core Web Vitals, making it possible for Google to rank your pages effectively.

No matter how good your content is or how authoritative your backlinks are, if your technical SEO is broken, you will not rank. Technical SEO is the foundation every other SEO effort sits on. It makes sure Google can crawl your pages, index them properly, load them quickly, and understand their structure. Without solid technical SEO, you're building a castle on sand.

Technical SEO covers four main areas: crawlability (can Google access your pages?), indexability (will Google add them to its index?), Core Web Vitals (do your pages load fast and respond quickly?), and structured data (does Google understand your content?). This guide covers each area in depth and provides the steps to fix the most common issues.

Crawlability: Make Your Site Accessible to Google

Crawlability is Google's ability to access and read the pages and resources on your website. If a page isn't crawlable, Google can't index it. Common crawlability issues include: pages blocked by robots.txt, broken redirects, pages that require login, pages served over HTTP (not HTTPS), and resources (CSS, JavaScript) that are blocked from Googlebot.

Google Search Console's Crawl report shows which pages have crawl errors. Monitor that report monthly. Common errors include "404 Not Found" (broken links to non-existent pages), "Server Error" (5XX errors), and "Timeout" (pages that take too long to load).

Check your robots.txt file. The file tells Google which parts of your site to crawl and which to skip. A misconfigured robots.txt can block important pages. Google's robots.txt documentation defines the correct syntax. Common mistakes include blocking entire directories or blocking CSS/JavaScript files, which prevents Google from rendering pages correctly.

Test crawlability using PageSpeed Insights or Lighthouse. These tools show whether your resources load correctly and whether your page is renderable. Resources that fail to load block Google's ability to understand your page structure.

Indexability: Getting Pages Added to Google's Index

Indexability is whether Google actually adds your page to its search index. A page can be crawlable (Google can access it) but not indexable (Google won't add it to search results). Common indexability issues include: pages blocked by meta robots noindex tags, pages served with X-Robots-Tag noindex headers, duplicate content without canonical tags, and pages that don't meet quality standards.

Audit your meta robots tags. A page with `` will be crawled but not indexed. The tag is useful for staging environments and private content, but applying it to public pages blocks indexing. Check your most important pages to ensure they have the correct robots tags (or no robots tag, which defaults to index).

Use canonical tags for duplicate content. If you have multiple pages with similar or identical content, specify which version is canonical using a canonical tag implementation. Canonical tags tell Google which page to index and which to treat as duplicates. Without proper canonicalization, Google indexes multiple versions and fragments your link authority.

Google Search Console's Coverage report shows which pages are indexed and which are blocked. Pages marked "Excluded" are not indexed. Common exclusion reasons include "Duplicate without user-selected canonical," "Blocked by robots.txt," or "Discovered, currently not indexed." For each exclusion, Search Console explains why and suggests fixes.

Site Structure and Internal Linking

A clear site structure helps Google understand your content hierarchy. Use descriptive URL slugs and organize pages into logical categories. Avoid deeply nested URL structures. A page at `/products/category/subcategory/sub-subcategory/product` is harder for Google to crawl than a page at `/products/product`.

Internal linking architecture matters. Pages linked from your home page and main navigation are crawled more often and carry more authority. Build a logical link hierarchy where important pages are 1-2 clicks from the home page. Use descriptive anchor text (not "click here") so Google understands what your links point to.

Build an XML sitemap listing your important pages. Sitemaps help Google discover pages that might not be found through internal links. Submit your sitemap to Google Search Console. Sitemaps are particularly valuable for sites with thousands of pages or dynamic content that changes frequently.

Core Web Vitals and Page Experience

Core Web Vitals are three metrics that measure user experience on your website: Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), and Interaction to Next Paint (INP). Core Web Vitals are confirmed Google ranking factors. Poor vitals hurt your rankings; good vitals help.

Largest Contentful Paint (LCP) measures how long it takes for the main content on a page to load. Google recommends LCP under 2.5 seconds. Slow LCP is often caused by slow server response time, render-blocking CSS, unoptimized images, or large JavaScript files. Optimize LCP by improving server speed, deferring non-critical JavaScript, and compressing images.

Cumulative Layout Shift (CLS) measures unexpected layout shifts that happen during page load. A high CLS score means your page jumps around as it loads, frustrating users. Common CLS issues include images without specified dimensions, ads that load asynchronously, and fonts that swap suddenly. Fix CLS by specifying image dimensions in HTML, reserving space for ads, and using font-display: swap in @font-face rules.

Interaction to Next Paint (INP) measures responsiveness to user interactions (clicks, taps, keyboard). Google recommends INP under 200 milliseconds. Slow INP indicates JavaScript that takes too long to execute. Optimize INP by breaking up long JavaScript tasks, reducing third-party script impact, and using modern rendering techniques.

Test your Core Web Vitals using Google's PageSpeed Insights tool. The tool provides real user data (when available) and lab measurements, plus specific recommendations for improvement.

Page Speed and Performance Optimization

Page speed directly affects Core Web Vitals and user experience. Fast pages rank better, load better, and convert better. Slow pages frustrate users and hurt your business. Optimize page speed by:

Reducing image file size through compression and modern formats (WebP). Unoptimized images often account for 50-80% of page weight. Use an image CDN if possible. Lazy-load below-the-fold images so they don't block page load. Defer loading of non-critical resources (videos, below-the-fold images) until the user scrolls near them.

Minimize and defer JavaScript. Render-blocking JavaScript delays page paint. Defer non-critical JavaScript or load it asynchronously. Use code splitting to load only the code needed per page. Remove unused JavaScript libraries and dependencies.

Optimize CSS. Large CSS files block rendering. Inline the critical CSS needed for above-the-fold content. Defer non-critical CSS to load asynchronously. Remove unused CSS rules.

Enable GZIP compression. Compressing HTML, CSS, and JavaScript reduces file size by 60-80%. Web.dev's performance learning path provides detailed optimization guidance.

Use a Content Delivery Network (CDN). A CDN serves content from servers near your users, reducing latency. CDNs also cache static content, improving load speed globally.

HTTPS and Security Signals

HTTPS is a ranking factor. Google prioritizes sites with SSL certificates over insecure HTTP sites. Every page should be served over HTTPS. Get a free SSL certificate from Let's Encrypt or your hosting provider. Redirect all HTTP pages to HTTPS using 301 redirects.

After enabling HTTPS, update your site URL in Google Search Console to use the HTTPS version. That ensures Google crawls and indexes your secure site. Watch for mixed content (pages that load both HTTPS and HTTP resources), which browsers may warn users about.

Mobile-First Indexing

Google now indexes the mobile version of your site as the primary version. If your mobile site is slow, poorly formatted, or missing content, your rankings suffer. Make sure your mobile site is responsive, loads fast, and includes all the content from your desktop site.

Test mobile-first indexing using PageSpeed Insights in mobile mode. Verify that buttons, forms, and navigation work properly on touch devices. Test on real mobile devices, not just browser emulation.

Structured Data and Schema Markup

Schema.org structured data helps Google understand your content. Adding schema markup to your pages increases the chance of appearing in featured snippets, rich snippets, and knowledge panels. Schema markup isn't a ranking factor in itself, but it unlocks rich SERP features that improve CTR.

Identify the schema types that apply to your content (Article, Product, Recipe, Event, Organization, LocalBusiness). Implement JSON-LD schema for each content type. Validate schema using Google's Rich Results Test. Fix every validation error before publishing.

Continuous Technical SEO Monitoring

Build a technical SEO monitoring routine. Check Google Search Console weekly for new crawl errors and indexing issues. Monitor Core Web Vitals monthly. Run PageSpeed Insights tests on key pages quarterly. Track crawl budget and crawl statistics to confirm Google is crawling your site efficiently.

Set up alerts for ranking drops. Sudden ranking declines sometimes signal technical issues like indexing problems or crawl errors. Diagnose immediately to prevent prolonged traffic loss.

Conclusion

Technical SEO is the invisible infrastructure that enables every other SEO effort. Without proper crawlability, indexability, performance, and structure, your content and links cannot deliver results. Master the fundamentals: audit crawlability, ensure proper indexing, improve Core Web Vitals, optimize page speed, and implement schema markup. Those technical foundations are non-negotiable for modern SEO success.

Use our GEO SEO audit tool to identify technical issues across your entire site. Our platform crawls your site the way Google does, identifies crawl errors, indexing problems, and Core Web Vitals issues, and prioritizes fixes by impact. Get started with a free technical audit today.

Frequently questions asked

What happens if your site isn't crawlable by Google?

If Google can't crawl your site, it can't index your pages, and no indexing means no rankings. Common issues include blocked resources (CSS, JavaScript files), robots.txt rules that block Googlebot, redirect loops, and broken internal links. Audit your site with Google Search Console to spot crawl errors.

Do Core Web Vitals directly affect rankings?

Yes. Core Web Vitals are confirmed Google ranking factors. Sites with poor Core Web Vitals (slow LCP, high CLS, high INP) rank lower than competitors with better vitals, all else being equal. However, highly relevant content with poor vitals can still outrank less relevant content with perfect vitals.

How often should you audit technical SEO?

Run a full technical SEO audit at least quarterly. Monthly monitoring of Core Web Vitals and crawl errors is essential. After any major redesign, migration, or site update, a post-launch technical audit is critical. Technical issues can appear suddenly, so continuous monitoring is recommended.

Our Blog for Ambitious Company