User-Agent and handshake disagree

The loudest signal in bot detection is not any single fingerprint. It is a client whose header claims one thing while its handshake demonstrates another — and it is almost always unintentional, because the two are set in completely different places.

Why headers cannot fix it

A User-Agent is a string your code chooses, sent in the request, after the connection is already established. The TLS fingerprint falls out of the library performing the handshake, which happened first and which your header dictionary has no access to. Changing the former does nothing to the latter:

import requests

r = requests.get(
    "https://kittens.sh/api/clean",
    headers={"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
                           "AppleWebKit/537.36 (KHTML, like Gecko) "
                           "Chrome/120.0.0.0 Safari/537.36"},
)
print(r.json())

The response comes back with the Chrome User-Agent echoed faithfully and a JA4 that is unmistakably Python over OpenSSL. From the server’s side that is not an ambiguous signal: browsers do not have that handshake, and nothing that has that handshake is a browser.

What a detector does with it

The rule is cheap to run and hard to argue with, which is why it is everywhere. In outline:

Parse the claimed browser family and major version out of the User-Agent.
Look up the JA4 against a table of fingerprints that family is known to produce.
Cross-check the HTTP/2 fingerprint, which is a separate layer and defeated separately.
Score the disagreement rather than blocking on it, because the false positives below are real.

This site runs a version of the same comparison and says so plainly on the front page: the cat is unimpressed when the handshake and the User-Agent tell different stories.

The false positives

Anyone writing this rule for real hits these within a day. Each produces a genuine mismatch from a client that is not lying:

Corporate TLS interception. Managed devices route through a middlebox that terminates TLS and re-originates it. The handshake you measure belongs to the appliance; the User-Agent belongs to the genuine browser behind it. Whole enterprises look like bots this way.

Anti-virus and VPN clients. The same mechanism on consumer machines, with a much wider spread of appliance fingerprints.

Embedded browsers and webviews. Electron, CEF, in-app browsers and mobile webviews present Chrome-derived User-Agents on top of a build whose TLS configuration is its own.

Anything you put in front of your own origin. A CDN or load balancer that terminates TLS means your origin measures the CDN, for every visitor, always. If your fingerprints look identical across all traffic, this is why — and it is the reason this service terminates TLS itself.

Stale tables. A new browser release ships a new fingerprint. Until your table knows about it, every early updater is a mismatch. The tables need maintaining, and a rule that blocks outright will block real users on release day.

Closing the gap from the client side

If you are the one being flagged and the traffic is legitimate — testing your own service, a permitted integration, a crawler operating within a site’s terms — the fix is not a better header. It is a client that controls its own handshake: curl-impersonate and curl_cffi ship browser-matched builds, and tls-client and rnet expose the cipher and extension ordering directly.

Whichever you choose, check it rather than trusting it — verifying an impersonation library walks through the comparison, including the HTTP/2 layer that most of them leave stock.

See your own

The inspector shows both sides and says whether they agree. From a script, /api/clean returns the fingerprints and the User-Agent together.