Some of the most consequential SEO directives never appear in the HTML. They travel in the HTTP response headers, invisible in the page source and in a browser's rendered view. A single X-Robots-Tag: noindex sent by a server can remove a page from search results while the HTML looks perfectly healthy.
The Sorank HTTP Headers Checker requests any URL and shows the full response, grouped so you can read it without knowing every header by heart.
The headers that decide whether a page ranks
- X-Robots-Tag: the header version of the robots meta tag. It can carry noindex, nofollow, noarchive and more, and it applies to non-HTML files too, which is how PDFs and images get excluded.
- Status code: a page that answers 200 is indexable, a 301 passes signals to its target, a 302 declares a temporary move, and a 404 or 410 removes the page. Sites regularly serve the wrong one by accident.
- Location: the destination of a redirect, which tells you whether the chain ends where you think it does.
- Canonical link header: a canonical can be declared in a header rather than in the head, and the two sometimes disagree.
Caching and speed
Headers also decide how much of your site a returning visitor has to download again:
- Cache-Control sets how long a response may be reused. Static assets deserve a long max-age, HTML usually does not.
- ETag and Last-Modified let a browser ask whether anything changed instead of downloading the file again.
- Content-Encoding shows whether the response was compressed with gzip or brotli. An uncompressed HTML response is an easy win left on the table.
Security headers
They do not rank a page, but they protect the people on it, and their absence shows up in audits:
- Strict-Transport-Security forces HTTPS for future visits.
- Content-Security-Policy limits which scripts and styles may run.
- X-Content-Type-Options stops browsers from guessing a file's type.
- Referrer-Policy controls what your site leaks to the pages it links to.
When to run this check
- A page disappeared from search and the HTML shows nothing wrong. Check X-Robots-Tag first.
- After a migration or a CDN change, when redirect rules and cache policies are rewritten in bulk.
- On a staging server that leaked into production, where a blanket noindex header often survives the move.
- Before a performance push, to see what is cached, what is compressed, and what is neither.

























