For example, the user agent string might be used by a web server to choose variants based on the known capabilities of a particular version of client software.
The user agent string is one of the criteria by which Web crawlers may be excluded from accessing certain parts of a website using the Robots Exclusion Standard (robots.txt file).
The format of the user agent string in HTTP is a list of product tokens (keywords) with optional comments.
[5] Other browsers were considered to be older products such as Mosaic, Cello, or Samba, and would be sent a bare bones HTML document.
Automated web crawling tools can use a simplified form, where an important field is contact information in case of problems.
Web browsers created in the United States, such as Netscape Navigator and Internet Explorer, previously used the letters U, I, and N to specify the encryption strength in the user agent string.
Websites often include code to detect browser version to adjust the page design sent according to the user agent string received.
[9][10] Other HTTP client programs, like download managers and offline browsers, often have the ability to change the user agent string.
[15] Starting with Firefox 110 released in February 2023,[16] Mozilla announced it would temporarily freeze portions of the browser's user agent string at version 109.